15:00:32 <bswartz> #startmeeting manila
15:00:33 <openstack> Meeting started Thu Feb  1 15:00:32 2018 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:36 <openstack> The meeting name has been set to 'manila'
15:00:40 <bswartz> hello all
15:00:42 <amito> hello o/
15:00:44 <ganso> hello
15:00:45 <tpsilva> hey
15:00:49 <zhongjun> Hi
15:00:57 <tbarron> hi
15:01:11 <toabctl> hi
15:01:23 <bswartz> courtesy ping: gouthamr xyang markstur vponomaryov cknight
15:01:35 <markstur> hi
15:02:21 <vkmc> o/
15:02:27 <bswartz> I'll skip announcements today and go right into topics because the topics cover the announcements (if that makes sense)
15:02:38 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:02:55 <bswartz> #topic Rocky PTL
15:03:03 <bswartz> #link http://lists.openstack.org/pipermail/openstack-dev/2018-January/126767.html
15:03:21 <bswartz> So all of you who read the ML know that I'm stepping down at PTL
15:03:38 <bswartz> I'm not a candidate for Rocky
15:03:58 <bswartz> I think I've been PTL 7 (maybe 8) terms and it's long enough
15:05:30 <bswartz> I guess the most important thing I can say is thank you to everyone in the community who has supported the project, and hopefully my leadership has been okay
15:05:42 <bswartz> :-)
15:06:09 <tbarron> well your leadership has been awesome
15:06:14 <zhongjun> But you still will continue to work on manila?
15:06:48 <bswartz> So my term should continue until the end of Queens, and I'll keep running the meetings, finish the release, and make sure the PTG is organized
15:06:55 <xyang> hi
15:06:59 <zhongjun> Thanks for all your work over 5 years
15:07:04 <bswartz> And I'd like to help teach the new PTL the ropes, whoever that may be
15:07:21 <bswartz> zhongjun: yes I intend to remain involved
15:07:50 <toabctl> bswartz, thanks for being the PTL so long. Well done!!
15:08:15 <bswartz> The main think I wanted to mention at this meeting today is that I encourage anyone interested in leadership to nominate themselves for the Rocky term
15:08:21 <toabctl> bswartz, will manila still be you fulltime job what you get paid for?
15:08:57 <bswartz> Last time I checked, there weren't any nominees, and if the nominating period ends with no nominees, we enter a murky territory where the TC chooses someone
15:09:26 <markstur> tc can make bswartz lifetime appointment?
15:09:29 <bswartz> toabctl: I'm still paid to work on manila, but not full time -- I'm also doing kubernetes work now
15:09:42 * dustins slides in late
15:09:50 <ganso> bswartz: the TC may choose anybody or only manila contributors?
15:10:21 <bswartz> To nominate yourself, add a file to this directory: https://github.com/openstack/election/tree/master/candidates/rocky/Manila
15:10:48 <bswartz> ganso: honestly I'm not sure how the TC makes that decision -- in the cases I've been in the past, it's been messy
15:11:21 <bswartz> s/been/seen/
15:11:35 <bswartz> Better to avoid that and follow the normal nomination/election process
15:11:57 <bswartz> I'm very committed to helping the new PTL come up to speed, because I care about the fate of the project
15:12:33 <bswartz> But I'm not a huge fan of the lifetime appointment thing
15:12:52 <xyang> bswartz: thank you for being a great PTL for so many years!
15:13:13 <amito> bswartz: thank you!
15:13:16 <bswartz> Thanks for the kind words
15:13:48 <bswartz> Please PM me offline if you're interested in leading the project but have any questions or reservations
15:13:58 <bswartz> Getting back to business...
15:14:06 <bswartz> #topic RC1
15:14:15 <bswartz> #link https://launchpad.net/manila/+milestone/queens-rc1
15:14:47 <bswartz> We're a week away from the RC1 target date and we don't really have many bugs to speak of
15:15:27 <bswartz> Is anyone working on bugs, and they simply haven't been targeted yet?
15:15:55 <ganso> bswartz: I am
15:16:00 <bswartz> If not, we could cut the RC as soon as ganso's bug is fixed
15:16:07 <zhongjun> I have a simple one
15:16:09 <bswartz> ganso: I see 1 in your name -- are there others?
15:16:19 <dustins> ganso: They must be the two I added to the etherpad this morning :)
15:16:27 <ganso> yes, but I would like to discuss the controversy of one before targetting the others
15:16:34 <bswartz> Okay we can cover specific bugs in the next topic
15:16:39 <tbarron> https://review.openstack.org/#/q/owner:junbo85.li%2540gmail.com+status:open
15:16:48 <tbarron> ^^^ soem of those are in master
15:16:50 <zhongjun> We also have the bug that it can let ci break
15:17:22 <bswartz> But I wanted to make it clear that this page is how I decide when to cut the RC, and if there are no open bugs on it, I have to assume it's safe to cut the RC immediately
15:17:23 <amito> I work on one related to our driver (infinidat), don't know if that counts
15:17:49 <bswartz> amito: yes, driver bugs are covered by the RC deadline
15:18:06 <bswartz> either the bug must be fixed before RC1, or we punt it to Rocky
15:18:35 <bswartz> No bugs may be fixed between RC1 and final release except for bugs discovered after the RC1 tag
15:19:09 <amito> bswartz: ok. the patch is already out there
15:19:25 <bswartz> amito: what matters is the milestone target in LP
15:19:36 <bswartz> Post a link to the LP bug, and I'll target it right now
15:20:05 <bswartz> Obviously every change we merge must have a bug number in it at this point (except docs)
15:20:22 <amito> https://bugs.launchpad.net/manila/+bug/1746439
15:20:23 <openstack> Launchpad bug 1746439 in Manila "INFINIDAT share driver fails to delete shares with snapshots" [Undecided,In progress] - Assigned to Amit Oren (amito)
15:20:37 <bswartz> Any others to target right now?
15:21:41 <bswartz> Okay
15:21:46 <zhongjun> https://review.openstack.org/#/c/539406/ ?
15:22:06 <bswartz> zhongjun: thanks
15:22:31 <tbarron> https://review.openstack.org/#/c/535611/
15:22:53 <bswartz> So reviewers should focus review on the bugs on that list, and as we get closer to the deadline, we either need to get the fix merged, or untarget the bug if the fix needs more time for whatever reason
15:23:15 <bswartz> tbarron: already got that one
15:23:26 <tbarron> bswartz: cool, sorry
15:23:29 <bswartz> The client bugs will be trickier
15:23:36 <bswartz> I have yet to create a rc1 milestone for the client
15:23:52 <bswartz> but there is time to merge fixes to our client too
15:24:33 <bswartz> Anyone who owns a bug targeted at RC1 can expect pings from me next week as long the bugfixes aren't merged
15:24:59 <bswartz> Based on history, this list doesn't look very bad
15:25:38 <bswartz> I'm optimistic we can get everything reviewed and merged by early next week, as long as the authors are responsive to review feedback
15:25:57 <bswartz> Let's move on to dustins' list in case he has some bugs the required discussion
15:26:09 <bswartz> #topic Let's Go Over New Bugs
15:26:17 <bswartz> #link https://etherpad.openstack.org/p/manila-bug-triage-pad
15:26:22 <dustins> It's a short one this week, actually, both ones opened in the last day or so
15:26:33 <dustins> #link https://bugs.launchpad.net/manila/+bug/1746725
15:26:34 <openstack> Launchpad bug 1746725 in Manila "LVM driver is unable to remove addresses in different IP versions belonging to the same interface properly" [Undecided,New]
15:26:40 <ganso> dustins: I opened them both today
15:26:48 <ganso> and one of them is the one I would like to discuss
15:26:54 <bswartz> Ooh they're in red
15:26:58 <dustins> ganso: Is it this one or the other one?
15:27:00 <bswartz> They must be very important
15:27:10 <bswartz> lol
15:27:10 <ganso> it is the first one
15:27:27 <dustins> Okay, cool, take it away, ganso :)
15:27:46 <ganso> ok so let's talk about https://bugs.launchpad.net/manila/+bug/1746725
15:27:48 <openstack> Launchpad bug 1746725 in Manila "LVM driver is unable to remove addresses in different IP versions belonging to the same interface properly" [Undecided,New]
15:28:04 <ganso> I identified a problem in exportfs -u command from nfs library
15:28:18 <bswartz> Oh yes
15:28:19 <ganso> or if it is isn't a problem, it is something that the LVM driver is not prepared to handle
15:28:27 <bswartz> s/nfs library/nfs-utils/
15:28:33 <ganso> as it can leave the manila DB in an inconsistent state
15:29:02 <ganso> in summary: the driver uses exportfs -u to remove rules
15:29:16 <bswartz> I've been aware of problems with the NFS helper used by the generic and LVM drivers since ocata
15:29:33 <bswartz> That's the reason I proposed a new helper (NfsExportsHelper)
15:29:35 <ganso> and when there are 2 rules, one being IPv4 and the other being IPv6, and both relate to the same physical network interface, the exportfs -u removes both rules with when attempting to remove one or the other
15:29:55 <bswartz> #link https://review.openstack.org/#/c/476239/
15:30:20 <ganso> so, I am already working on a fix, and the approach I am taking to fix this bug is pretty much the same as bswartz in his NFSHelper
15:30:39 <ganso> I am using https://github.com/openstack/manila/blob/81e708281f6bdeea2dba12eb08cf9c33265e7050/manila/utils.py#L700
15:30:39 <bswartz> The main issue with my new helper is that I never found a way to migrate users from one helper to another, making upgrades a possible nightmare
15:31:15 <ganso> to write back the file with all the rules, except the one removed, after reading it with a cat
15:32:00 <ganso> bswartz pointed out a possible disadvantage
15:32:05 <ganso> of using this approach
15:32:29 <ganso> if you look closely, this command processes input from the execute command
15:32:36 <bswartz> ganso's fix might work, but it builds on an unstable foundation
15:32:47 <ganso> bswartz: precisely
15:33:11 <bswartz> I'd like to work out a solution to the upgrade problem and then move everyone to the new helper I proposed
15:33:23 <bswartz> Unfortunately that can't really happen until rocky
15:34:11 <ganso> I don't have enough field experience to know all possible consequences of this approach, bswartz feels uneasy about it. I would like to know other community members' take on this or if someone can share more knowledge on the caveats of this approach
15:34:12 <bswartz> So do we want to commit ganso's fix for queens, or leave this bug unfixed (it only occurs in pretty rare circumstances)
15:34:14 <bswartz> ?
15:34:53 <ganso> if we as a community agree this is not the right solution, and I cannot come up with a better one, I will  prefer to punt this to Rocky
15:35:02 <tbarron> +1
15:35:23 <bswartz> tbarron: +1 for not fixing?
15:35:27 <tbarron> if we had lvm driver users broken right now complaining I'd think differently
15:35:30 <tbarron> bswartz: ack
15:35:32 <bswartz> k
15:35:57 <bswartz> ganso: can you live with that?
15:36:21 <ganso> bswartz: yes, then I would like to discuss the fix for the share migration bug that we wouldn't be able to test in the gate
15:36:41 <bswartz> ganso: is that the next bug in the list?
15:36:55 <ganso> bswartz: no, but this decision impacts that bug
15:37:10 <dustins> bswartz: Doesn't look like it, I only grabbed bugs that were new and unassigned
15:37:25 <bswartz> ganso: can you link that bug?
15:37:36 <ganso> https://bugs.launchpad.net/manila/+bug/1745436
15:37:38 <openstack> Launchpad bug 1745436 in Manila "Host assisted Share Migration breaks on IPv4+IPv6 backends" [High,In progress] - Assigned to Rodrigo Barbieri (rodrigo-barbieri2010)
15:37:53 <bswartz> Oh this one that's already targeted
15:37:59 <ganso> so, while fixing that bug, I stumbled across the LVM bug because that's what we use to test in the gate
15:38:25 <bswartz> As long as the core code works with non-buggy drivers, I think that's good enough for the release
15:38:50 <bswartz> If we have to mark some tests as skip temporarily, that's okay with me
15:39:08 <bswartz> (only skip on the buggy drivers)
15:39:24 <ganso> bswartz: I'd prefer to not take that approach
15:39:59 <bswartz> ganso: what do you have in mind?
15:40:32 <ganso> bswartz: I like any other alternative as well, such as having an ipv6-only job, which would still not allow to test IPv4+IPv6 configuration
15:40:59 <ganso> bswartz: but it would improve coverage and allow this bug to be fixed and 3rd party vendors to be able to use IPv6 in migration (which is blocked right now)
15:41:32 <bswartz> Okay I'm confused
15:41:44 <ganso> right now vendors can only use share migration with IPv4 only. They could use it with IPv6-only before we blocked it
15:41:56 <bswartz> I want to fix the core issue
15:42:07 <ganso> bswartz: the LVM driver cannot handle IPv4+IPv6 as it is, so we cannot have a gate job for it
15:42:30 <bswartz> That doesn't stop us from fixing the underlying problem
15:42:45 <ganso> bswartz: I don't feel happy about disabling all upstream share migration tests so we can commit this fix, I'd rather stay as we are than do that
15:43:13 <bswartz> Me neither -- why couldn't we disable just the versions of the test that will run into trouble?
15:43:26 <bswartz> Like only the mixed v4/v6 flavors of migration
15:43:55 <ganso> bswartz: ok, we would only need to disable it in the LVM driver job
15:44:01 <ganso> bswartz: scenario job and generic would still run it
15:44:09 <ganso> (I need to double check that)
15:44:11 <bswartz> would generic pass or fail?
15:44:30 <ganso> in theory will pass, as the job  is already configured with IPv4 only
15:44:32 <bswartz> Generic uses the same nfs helper code
15:44:35 <bswartz> Ok
15:44:36 <ganso> but I need to double check if it is not disabled
15:45:07 <ganso> because, when we moved all the important backend jobs to run on the LVM job, we may have disabled them in the other jobs for performance reasons
15:45:17 <bswartz> We can always verify the fix using yours (or my) fix to the helper, even if neither of those changes are ready to merge for queens
15:45:52 <ganso> ok
15:45:58 <ganso> seems like we reached an agreement
15:46:04 <bswartz> Okay
15:46:11 <bswartz> there's another bug on dustins' list
15:46:40 <ganso> https://bugs.launchpad.net/manila/+bug/1746723
15:46:41 <openstack> Launchpad bug 1746723 in Manila "LVM driver does not handle IPv6 addresses in recovery mode" [Undecided,New]
15:46:44 <ganso> that is an easy one
15:46:57 <ganso> but there is no point in working on that we if don't address the previous one
15:47:06 <ganso> the recovery mode path is used only by share migration at this moment
15:47:19 <bswartz> Okay but what is the impact for not fixing this?
15:47:29 <ganso> and it is not handling IPv6 well. We don't see it in the gate because we are dodging that
15:47:33 <bswartz> Does it cause problems in ipv6-only scenarios (which should work in theory)?
15:48:06 <ganso> bswartz: that IPv6-only scenario has been blocked by our previous IPv6 patch, which allows only IPv4 export locations to be used in share migration
15:48:32 <bswartz> But that one will be fixed won't it?
15:48:32 <ganso> bswartz: so unless we unblock that, nobody will hit be able to run IPv6-only migration and will never hit that bug on LVM
15:48:39 <bswartz> Ugh
15:49:09 <bswartz> I wish we'd identified these issues earlier, and we could have spend time sorting out the upgrade issues around my new helper patch
15:49:56 <bswartz> Does anyone else have heartburn about potentially not testing a bunch of ipv6 code paths in the gate?
15:50:39 <dustins> I mean, IPv6 isn't going away anytime soon, we'll have to test it sooner than later
15:50:43 <ganso> bswartz: I can fix that bug and unblock IPv6, which will allow IPv6-only scenarios, and we wouldn't be running the tests in the LVM job. So all should work fine in a production cloud, in theory. We wouldn't be preventing anyone from running IPv6-only jobs anymore
15:50:58 <ganso> s/IPv6-only jobs/IPv6-only migrations
15:51:11 <bswartz> Okay I like that idea
15:51:26 <ganso> ok
15:51:30 <ganso> sounds good
15:51:54 <bswartz> Any other bugs?
15:52:07 <ganso> not from me
15:52:16 <bswartz> #topic Open Discussion
15:52:17 <dustins> None from me today
15:52:33 <bswartz> Anything else for today?
15:52:36 <markstur> bswartz  Thank you, Ben! for all you've done as PTL and the making of manila
15:52:47 <tbarron> +1000
15:53:00 <bswartz> I should mention that this is the time in each release when we usually update docs related to changes during the release
15:53:57 <bswartz> We should make sure that the various API changes that went in have corresponding docs changes
15:54:19 <bswartz> It's mostly small stuff
15:54:37 <bswartz> We can revisit that again next week
15:55:00 <bswartz> Oh and please keep adding topics to the Manila PTG etherpad
15:55:10 <bswartz> I'll add the topic about the NFS export helper and upgrade issues
15:55:28 <bswartz> That's all for today
15:55:37 <bswartz> Thanks everyone
15:55:41 <bswartz> #endmeeting