15:00:11 <bswartz> #startmeeting manila
15:00:14 <openstack> Meeting started Thu Oct  6 15:00:11 2016 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:17 <openstack> The meeting name has been set to 'manila'
15:00:18 <bswartz> hello all
15:00:19 <cknight> Hi
15:00:23 <vponomaryov1> Hello
15:00:32 <ganso> hello
15:00:35 <toabctl> hi
15:00:41 <ravihandran> hi
15:00:50 <gouthamr> hello o/
15:01:12 <dustins> \o
15:01:18 * bswartz wonders where everyone is
15:01:54 <bswartz> markstur xyang tbarron: courtesy ping
15:02:07 <markstur___> hi
15:02:08 <gouthamr> you need to add a couple "_"s
15:02:09 <gouthamr> :P
15:02:14 <bswartz> >_<
15:02:33 <markstur___> I did.  It would let me back in with 1 or 2 or 3
15:02:51 <bswartz> markstur: maybe try 4 _s?
15:03:08 <bswartz> lol
15:03:11 <bswartz> #topic announcements
15:03:23 <bswartz> Newton shipped today!
15:03:24 <xyang2> hi
15:03:34 <markstur___> release party!
15:03:37 <kaisers> hi
15:03:50 <ravichandran> hi
15:03:59 <bswartz> The final newton bits ended up being the same as the RC1 bits because we didn't have any critical bugs
15:04:03 <tbarron> hi
15:04:11 <vkmc> o/ hi
15:04:17 <bswartz> We did push a bugfix release of manila-ui earlier this week
15:04:59 <bswartz> This means that the quiet period on the stable branches is over and master is TRULY open for Ocata development
15:05:18 <dustins> Devstack isn't gonna know what hit it
15:05:22 <bswartz> there's no reason not to merge features if they're ready at this point
15:05:51 <gouthamr> dustins: usually its the other way round
15:06:01 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:06:17 <bswartz> #topic Ocata Design Summit
15:06:23 <bswartz> #link https://etherpad.openstack.org/p/manila-ocata-design-summit-topics
15:06:40 <bswartz> okay so I'd like to make some decisions about the agenda for the design summit today
15:07:43 <bswartz> the etherpad doesn't have the topics divided into fishbowls or working sessions currently
15:08:20 <bswartz> but please vote (using +1 on etherpad) on the topics you'd like to cover right now
15:09:24 <tbarron> gouthamr: :)
15:09:46 <gouthamr> bswartz: to understand correctly, elimination of race conditions is not the same as DLM and our road to HA? :)
15:10:03 <bswartz> gouthamr: correct
15:10:32 <bswartz> gouthamr: we can fix all of our race conditions without a DLM and not have HA support
15:10:53 <bswartz> we cannot have HA support without fixing our race conditions and without a DLM
15:11:38 <gouthamr> bswartz: okay.. yes.. but i'm interested in the DLM support because of an existing feature breaking when deployed according to the recommended way of deploying manila services
15:11:46 <gouthamr> bswartz: s/an existing feature/share replication
15:12:19 <bswartz> gouthamr: I'm sure that topic will get some coverage
15:12:37 <gouthamr> bswartz: i think this can be done in parallel, i'd revive the DLM patch when im done prepping for summit
15:12:50 <gouthamr> bswartz: nice, thank you..
15:13:23 <bswartz> gouthamr: share replication probably needs another pass after we agree on solution for locking and states
15:13:59 <gouthamr> yes, its possible that we might use fine grained state transitions even with replication
15:14:13 <bswartz> I don't want to rewrite everything but we should consider that if necessary
15:14:52 <gouthamr> we have gigantic critical sections so we can avoid the state transitions, you convinced me that's untenable.. so i'm in agreement :P
15:14:59 <bswartz> I notice that "share groups" is absent from this list
15:15:40 <gouthamr> so is ameade
15:15:51 <bswartz> If interest in that feature has waned then perhaps it's time to delete the much-maligned "consistency groups" feature
15:15:58 <cknight> bswartz: I'll ping ameade about it
15:16:03 <bswartz> okay
15:16:45 <bswartz> I don't want to carry around consistency groups forever if we know we don't want to keep it as is
15:18:18 <tbarron> I'm surprised more people aren't voting for getting the generic driver to work better
15:18:44 <bswartz> tbarron: it may be that we just do that and it doesn't need much discussion
15:19:11 <bswartz> tbarron: I'm willing to lead that effort
15:20:11 <ganso> bswartz: I am not aware of the fact that the container driver has been a disappointment. I am not fond of the idea of improving the generic driver, we wanted to deprecate it with the container driver
15:20:31 <bswartz> ganso: well that is what the discussion would be about
15:20:40 <markstur___> ganso, I don't see your -1 on that subject
15:20:44 <bswartz> not everyone is up to date with the status of the generic driver
15:21:02 <bswartz> some of these things we can cover in weekly meetings between now and barcelona
15:21:06 <tbarron> bswartz: I think we need a status report at least, and then maybe common direcitoin
15:21:11 <tbarron> direction
15:21:12 <bswartz> we have today and 2 more weeks before the summit
15:22:00 <tbarron> yeah, not everything has to be at summit, and some things need to be async as well and involve people who can't attend summit
15:22:13 <bswartz> tbarron: good point
15:22:24 <bswartz> who will attend the summit and who won't?
15:22:30 * bswartz will attend
15:22:34 <ganso> 'me will
15:22:38 * tbarron will
15:22:38 <ganso> oops
15:22:41 <cknight> I'll be there
15:22:43 * vponomaryov1 will
15:22:51 * markstur___ will be there!
15:22:56 <tbarron> vponomaryov1: +1000
15:22:58 <bswartz> yay markstur!
15:23:02 <ganso> markstur___: :D
15:23:07 <tbarron> markstur___: awesome
15:23:24 <markstur___> vponomaryov1, Finally get to meet you
15:23:28 <bswartz> Will it be markstur or markstur______?
15:23:38 <gouthamr> i'll be there
15:23:39 <vponomaryov1> ))
15:23:42 <markstur___> I'll fill in the blank next week
15:23:48 <bswartz> markstur: see if you can get them to put the extra underscores on your badge
15:23:55 <tbarron> markstur___: hee hee
15:24:47 <bswartz> okay well based on the etherpad voting I'm seeing some clear preference for certain topics
15:24:53 <markstur___> Seriously.  I got kicked out with all shorter attempts (nick in use race)
15:25:45 <kaisers> oops, will be on the summit, too
15:26:02 <bswartz> kaisers: cool
15:26:17 <bswartz> I didn't see anyone who says they can't attend
15:26:29 <bswartz> but not everyone answered
15:26:42 <vponomaryov1> whre is xyang?
15:26:44 <ganso> xyang2 ?
15:26:58 <vponomaryov1> ganso: sooo , slooow )
15:27:17 <gouthamr> #shamelessplug https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/16280/manila-share-data-does-not-simply-move-and-protect-itself-oh-wait-it-does
15:27:22 <bswartz> I know xyang will be there from the cinder meeting
15:27:29 <bswartz> although we have more conflicts with cinder than we have in the past
15:27:33 <xyang2> sorry.  I"m in a call
15:27:37 <xyang2> I"ll be at the summit
15:27:45 <bswartz> so people will have to make choices about which sessions to attend
15:28:00 <ganso> xyang2: cool :)
15:28:18 <bswartz> okay I have enough information that I cna take a pass at scheduling sessions this afternoon
15:28:32 <tbarron> vkmc and rraja will be there
15:28:47 <bswartz> it's good to see that people are focused on incremental improvements rather than huge new things given the extremely short schedule
15:29:16 <bswartz> wow I don't know if I've seen rraja since the hong kong summit
15:29:16 <vponomaryov1> bswartz: owners of such features are just hiding  from us )
15:30:03 <bswartz> vponomaryov1: that's fine but they're missing their chance to bring up topics in Barcelona
15:30:29 <bswartz> okay we better move on
15:30:34 <bswartz> next up:
15:30:43 <bswartz> #topic Microversion-ing addition or removal of transitional ('micro') states
15:31:00 <bswartz> #link https://review.openstack.org/#/c/369668/
15:31:09 <bswartz> gouthamr: you're up
15:31:38 <gouthamr> thanks bswartz okay, a little background -> we've had bad race conditions in manila since iirc share instances appeared
15:32:03 <gouthamr> in mitaka, we cleaned up access rules a little bit, added a new driver interface update_access and removed the per instance access rule state
15:32:35 <gouthamr> and subsequently fixed a bunch of bugs with different approaches..
15:32:45 <bswartz> yes and a mistake was made during mitaka
15:32:56 <gouthamr> and one was left that we know of... https://bugs.launchpad.net/manila/+bug/1626249
15:32:58 <openstack> Launchpad bug 1626249 in Manila "Concurrency issues with bulk access rule updates" [High,In progress] - Assigned to Goutham Pacha Ravi (gouthamr)
15:33:08 <gouthamr> originally reported by tpsilva
15:33:17 <bswartz> when we introduced new access rule states, we should have hidden the ones users don't need to know about
15:33:41 <gouthamr> so, the solution after much brainstorming with ganso tpsilva vponomaryov1 bswartz cknight + has been restoring the per rule access state
15:33:42 <bswartz> in particular "updating_multiple" should have been mapped to simply "updating" at the view builder level
15:34:09 <vponomaryov1> bswartz: that is the problem, how do we map states and microversions?
15:34:18 <gouthamr> so, that patch https://review.openstack.org/#/c/369668/ does that
15:34:31 <gouthamr> vponomaryov1 had a question about the approach though and i'm open to ideas
15:34:36 <bswartz> vponomaryov1: this is the point from the review that I think requires discussion here
15:34:51 <gouthamr> i'm knocking two legit states from the share instance's access_rules_status
15:34:58 <bswartz> so the general rule is that any user-visible change should be microversioned
15:35:06 <gouthamr> with no microversion change (oh the horror)
15:35:11 <bswartz> there are exceptions to that rule though
15:35:38 <gouthamr> :) so wanted to bring that up for discussion here hoping we can reason out the approach
15:36:07 <bswartz> in cases where preserving backwards compatibility is impossible or undesirable (such as when fixing a bug), then microversioning the change is pointless
15:36:31 <gouthamr> bswartz: i kind of agree, because users may not have expected that behavior per se...
15:36:40 <gouthamr> but i'm wary of saying that for certain
15:37:07 <gouthamr> the scripts that we write 'usually' only care about two states: available or error
15:37:17 <bswartz> one of the main points of microversioning is to introduce new behavior while preserving old behavior for clients that expect the old behavior
15:37:19 <gouthamr> for example, in tempest
15:37:36 <bswartz> when fixing a bug, it's hard to argue that anyone "expected" the old buggy behavior
15:38:01 <ganso> bswartz: there may be tools dependent on that state
15:38:09 <ganso> bswartz: or scripts
15:38:17 <bswartz> ganso: yes that's where the tension comes from
15:38:24 <markstur___> there are sometimes bug vs feature arguments
15:38:30 <ganso> bswartz: microversioning allows the users to go back to the previous microversion until their scripts are fixed
15:38:48 <bswartz> ganso: however consider when the nature of the bugfix makes it impossible to emulate the old (buggy) behavior for those clients who expect it
15:38:56 <gouthamr> ganso: what kind of scripts do you expect that expect a second rule to transition state to "updating_multiple"?
15:39:06 <bswartz> ganso: cinder faces this conundrum with their attachment APIs
15:39:17 <ganso> bswartz: oh I see your point... backwards compatbility :\
15:39:22 <gouthamr> these transitional (micro) states are so brief and unpredictable to be relied upon
15:39:38 <gouthamr> expected to be brief*
15:39:46 <tbarron> yeah in this case it's a theoretical possibility but the actual liklihood is minimal i think
15:40:01 <ganso> gouthamr: true, it is highly unlikely
15:40:24 <markstur___> gouthamr, brief as in permanent?
15:40:38 <markstur___> like bad tattoo
15:40:48 <bswartz> so my stance is that unless we really see value in supporting both the old behavior (for backward compatibility) and the new behavior, then microversioning the change achieves almost nothing
15:40:59 <gouthamr> markstur___: yes.. the true states for that field are 'active', 'out_of_sync' or 'error'
15:41:08 <bswartz> markstur___: you have experience in that area?
15:41:09 <gouthamr> markstur___: haha
15:41:57 <gouthamr> some transitional states make sense, some don't imho
15:42:14 <gouthamr> so, we should have always mapped the states for consistent user experience
15:42:52 <gouthamr> with that said, it's going to be difficult to maintain backwards compatibility
15:43:01 <bswartz> transitional states have value to the user insofar as they signal "don't do anything else to this share right now"
15:43:38 <gouthamr> bswartz: i consider 'out_of_sync' as a transitional state then
15:43:51 <bswartz> GUI clients should ideally suppress actions on shares in transitional states
15:44:24 <bswartz> gouthamr: that's a bit different because it's the replica state not the share state
15:44:41 <bswartz> and transitions into and out of that state can happen with NO user interaction
15:44:41 <ganso> bswartz: it is the instance, in fact
15:44:43 <gouthamr> bswartz: nope, overloaded in "access_rules_status"
15:45:11 <bswartz> oh you're talking about that access rule status
15:46:24 <ganso> gouthamr: but bswartz stated a good example of what could be relying on the states... fortunately our vanilla manila-ui does not do anything special
15:46:33 <bswartz> I'll say again the main value of states at the API level is to give the user a clue about what he can/can't do with that share at that time
15:46:34 <gouthamr> my proposal is that we microversion this, but don't care about breaking clients that expect these transitional states, 'updating', 'updating_multiple'
15:46:52 <gouthamr> and do things right from here on out when introducing any states
15:47:18 <bswartz> from a user's perspective, all they need to know is whether the access rules they've set on the share are active or not, so they can wait for them to become active
15:47:38 <gouthamr> i.e, microversion and/or map to existing states taht users should rely upon when we expect a state transition only for the sake of internal logic
15:47:39 <gouthamr> bswartz: +1
15:47:52 <vponomaryov1> gouthamr: I agree with bswartz here, if we cannot keep backwards compat then no need in microversion
15:48:24 <gouthamr> vponomaryov1: anything that changes in behavior in the server should ideally bump the microversion
15:48:30 <ganso> gouthamr: +1
15:48:31 <bswartz> gouthamr: why do you propose adding a microversion?
15:48:35 <gouthamr> vponomaryov1: that patch also adds 'applying' and 'denying'
15:48:44 <ganso> gouthamr: we could make that adjustment in the view only, it would not affect internal logic
15:48:45 <gouthamr> as per instance access rule state
15:49:01 <bswartz> ganso: +1
15:49:06 <vponomaryov1> gouthamr: addon of microversion means you should support ol dstatuses for old microversions, that is my point
15:49:12 <ganso> gouthamr: the internal logic is being fixed, it could be seen as a mistake to expose so many statuses to the API
15:49:27 <bswartz> we can have as many states as we want internally and the viewbuilder should map those to a small number of user visible states
15:49:33 <ganso> gouthamr: since for the user, the only important states would be "active" or "out_of_sync"
15:49:56 <gouthamr> yes... so we're in agreement. i could write up a spec if anyone wants for these changes
15:50:14 <vponomaryov1> in agreement for what? ))
15:50:32 <bswartz> gouthamr: I feel a spec is required here -- there's too many details to argue about
15:50:35 <gouthamr> (i intend to modify the associated ORM models (add backrefs) and the DB APIs as well, in line with share instances and share snapshot instances)
15:50:54 <bswartz> it would be better to argue about those in the spec and then just implement the spec, avoiding all the arguments in the code review
15:51:09 <gouthamr> bswartz: okay.. if you aren;t tired of my writing :D
15:51:18 <gouthamr> sure +1
15:51:40 <ganso> or we could not have the spec (as we currently don't) and discuss it at the summit, there is a topic for that proposed
15:51:41 <markstur_> gouthamr, Your writing is excellent
15:51:48 <ganso> unless we agree we do not need that topic anymore
15:51:49 <bswartz> yes specs are going to be more important for ocata
15:52:00 <bswartz> I need to update my own spe
15:52:04 <bswartz> my spec about specs
15:52:18 <bswartz> #topic open discussion
15:52:21 <gouthamr> markstur_: :) thanks...
15:52:27 <bswartz> okay anyone have something else?
15:52:28 <ganso> "Access rules fixes (patch proposed by goutham) +1+1+1+1+1+1"
15:52:31 <tbarron> i think having this written down will be very helpful
15:52:32 <markstur_> gouthamr, It's your artwork that is subpar  :)
15:52:39 <bswartz> a few things came up earlier during the meeting...
15:52:44 <gouthamr> markstur_: i don't do the ascii diagrams! :P
15:52:51 <bswartz> ganso asked about the container driver
15:53:00 <bswartz> tbarron asked about the generic driver service image
15:53:34 <bswartz> gouthamr: I use plantuml for diagrams
15:53:53 <bswartz> ganso: so quick update on the container driver
15:53:58 <gouthamr> bswartz: yes.. and i've had it in my post it notes to plantuml our thin provisioning logic #someday
15:54:30 <bswartz> after working on the container driver for about a year, I feel it's about as good as it will get given the existing limitations in docker and ganesha, which are significant
15:55:17 <vponomaryov1> bswartz: don't forget to say that ganesha is not used yet at all because of such limitations ))
15:55:20 <bswartz> our options with the container driver are to either wait for docker and ganesha to get better, or to rearchitect the driver significantly
15:55:29 <ganso> bswartz: so it will not be good enough to be a generic driver replacement?
15:55:43 <ganso> bswartz: would rearchitecting solve these issues? has this been researched?
15:55:46 <gouthamr> ganso: loopback, i'll add the spec link to the etherpad when done.. we can possibly integrate it into the micro-states discussion.. its all intertwined
15:55:58 <ganso> gouthamr: +1
15:56:04 <bswartz> ganso: as vponomaryov points out, the existing driver doesn't even support NFS because ganesha is so bad at basic things which we need
15:56:49 <bswartz> I had high hopes for ganesha after hearing how it was being used for various production use cases
15:57:10 <tbarron> do we have written down somewhere what is needed of ganesha?  has anyone involved themselves in the upstream ganesha community?
15:57:14 <bswartz> but while working with aovchinnikov on the driver it became clear that ganesha is still missing some essential things
15:57:30 <bswartz> tbarron: I wrote in their mail list what we want from them
15:57:42 <tbarron> we aren't just consumers of other upstream projects, sometimes we need to participate
15:57:50 <bswartz> tbarron: it's also on the etherpad
15:58:24 <tbarron> i have a more vital interest in this subject now so want to catch up again :)
15:58:31 <bswartz> tbarron: yes I suspect that the ganesha team won't address our use cases unless we join them and do the work ourselves
15:59:15 <bswartz> tbarron: but because we work in python and they work in C++ my personal preference is to put container driver on the back burner and pursue approaches which we can more directly help with
15:59:33 <tbarron> so i want to discover how much the needs for container driver overlap with needs for, say, nfs gateway to native cephfs
15:59:49 <bswartz> tbarron: that's certainly worth looking into
15:59:52 <tbarron> bswartz: k, i'll listen about that ...
16:00:11 <bswartz> we're out of time for today
16:00:14 <bswartz> thanks all
16:00:20 * xarses waves
16:00:21 <bswartz> tbarron I'll ping you about the other thing
16:00:33 <bswartz> #endmeeting