15:01:04 <bswartz> #startmeeting manila
15:01:04 <openstack> Meeting started Thu Nov 10 15:01:04 2016 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:07 <openstack> The meeting name has been set to 'manila'
15:01:11 <bswartz> hello all
15:01:12 <cknight> Hi
15:01:17 <ganso> hello
15:01:18 <vponomaryov> Hello
15:01:20 <xyang1> hi
15:01:24 <markstur> hi
15:01:26 <ravichandran> hello
15:01:32 <zengyingzhe_> Hi
15:01:41 <zhonghua> hi
15:01:48 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:01:53 <tbarron> hi
15:01:59 <toabctl> hi
15:02:10 <bswartz> #topic announcements
15:02:21 <bswartz> we're 1 week away from ocata-1
15:02:46 <bswartz> as agreed, that's when we will freeze low priority specs
15:02:56 <bswartz> more about that later in the agenda
15:03:32 <TommyLikeHu_> thanks ganso
15:03:37 <bswartz> first up we have a hopefully small issue raised by one of ganso's specs
15:03:38 <ganso> TommyLikeHu_: np
15:03:46 <bswartz> #topic Decide defaults for migration options
15:03:56 <bswartz> #link https://review.openstack.org/#/c/392291
15:04:10 <bswartz> I thought we had reached an agreement in BCN about this
15:04:15 <bswartz> but apparently not
15:04:47 <bswartz> my preference is to default all of the share-migration related optional flags to False
15:05:00 <bswartz> because False is a more obvious default than True
15:05:28 <vponomaryov> bswartz: it is situation when default should be the most strictest variant
15:05:38 <bswartz> and I also feel that a migration operations with no special args should always succeed (so it should be compatible with the fallback migration)
15:06:16 <vponomaryov> bswartz: the most strictest because it is fool-proof approach
15:06:21 <bswartz> the problem I see is that we're adding more and more options over time
15:06:23 <ganso> vponomaryov: +1
15:06:32 <bswartz> if we default eveything to true than a command that works last release won't work next release
15:06:41 <bswartz> it's a backwards compatibility thing
15:07:06 <ganso> bswartz: we are not committed to keep backwards compatibility on experimental features
15:07:14 <vponomaryov> ganso: +1
15:07:17 <bswartz> ganso: I'm talking about the future not the past
15:07:32 <bswartz> suppose we ship ocata with migration not experimental
15:07:36 <bswartz> and everything defaults to true
15:07:56 <ganso> bswartz: in that case, yes, we should not change it anymore in the future
15:07:57 <bswartz> then in pike, we add a --preserve-fairy-dust migration flag
15:08:07 <bswartz> and it defaults to true
15:08:35 <bswartz> then people will observe that commands that used to work no longer too
15:08:37 <toabctl> do we have real user feedback for the share-migration API? I outside of the manila-core group ?
15:08:39 <bswartz> s/too/do/
15:09:03 <bswartz> toabctl: SAP has provided significant feedback
15:09:18 <ganso> bswartz: at that point, we may break the consistency of "all options TRUE" or "all options FALSE", but if we retain the assumption that the API parameters default value follows the strictest default set idea then we should be fine IHMO
15:09:28 <ganso> bswartz: s/IHMO/IMHO
15:09:51 <bswartz> ganso: if we default everything to false then adding new options in the future is easy
15:10:03 <toabctl> bswartz, I didn't read the feedback. was it possitive? or are there any concerns about the current API design? just asking.
15:10:31 <bswartz> toabctl: many of these optional flags were inspired by feedback from SAP in Austin
15:10:35 <vponomaryov> toabctl: concerns are listed in appropriate spec with migration API improvements
15:10:40 <ganso> toabctl: most part of the feedback came from Goutham's interaction with some customers he was presenting to
15:10:42 <toabctl> thx
15:10:46 <TommyLikeHu_> what is SAP
15:10:58 <bswartz> SAP is a large software company in germany
15:11:07 <TommyLikeHu_> thanks
15:11:20 <bswartz> #link http://go.sap.com/index.html
15:11:24 <ganso> yup, mkoderer___  gave some feedback from SAP as well, that the options should be the strictest
15:11:53 <vponomaryov> ganso: what a coincidence ))
15:11:55 <bswartz> ganso: my interpretation was that he wanted a way to guarantee that migrations were lossless
15:12:07 <bswartz> that guarantee didn't need to be on by default
15:12:19 <bswartz> as long as there's a way to do it, users should be happy
15:12:48 <vponomaryov> bswartz: lossless by default, and lossy only when "yes I know what I am doing"
15:12:55 <markstur> I thought there should be no loss unless the flag said it's ok to be lossy
15:13:04 <ganso> bswartz: IIRC, he said that if there was any risk of losing data accidentally (like starting a migration that may lose data because the admin forgot the use a set of parameters), they would just disable migration in their cloud
15:13:06 <bswartz> argh
15:13:23 <ganso> vponomaryov: exactly
15:13:32 <bswartz> the problem with that approach is that it won't work with most backends most of the time
15:13:37 <markstur> I think others like disruptive we have more leeway with.
15:14:01 <bswartz> migration will become a feature that fails all the time unless you either specific a bunch of flags or have a very special backend
15:14:05 <ganso> markstur: +1, that's exactly what I believe we had agreed to in the Newton midcycle
15:14:09 <bswartz> that seems anti-cloud to me
15:14:28 <vponomaryov> bswartz: it is safe
15:14:42 <vponomaryov> bswartz: after getting API error, admin will use proper set of options
15:14:47 <vponomaryov> bswartz: no data loss
15:14:57 <bswartz> but remember we want to open up migration to normal users
15:15:00 <markstur> bswartz: I agree that it is unfortunate and inconvenient, but with data loss we seem to have a real need for "are you sure?" type of inconvenience
15:15:01 <bswartz> not just admins
15:15:22 <bswartz> normal users can't possibly know what will and won't work ahead of time
15:15:39 <bswartz> therefore they should specify what they want and find out if it works
15:16:04 <markstur> normal users would probably need to fail and then decide to try again w/ non-defualt options
15:16:13 <markstur> hopefully user messages can help with that
15:16:14 <vponomaryov> tbarron, cknight, xyang1, what do you think?
15:16:29 <bswartz> markstur: but how would they know which options to make false and which ones to leave true?
15:16:39 <bswartz> it will depend heavily on which backends they have
15:16:44 <bswartz> which they won't know
15:16:48 <markstur> trial and error / admin help / ... ?
15:16:49 <vponomaryov> bswartz: following "fool-proof" approach
15:16:54 <ganso> bswartz: using the UI it is less likely that the admins will ignore the feature, he may just tick everything (if it is not ticked by default) because they want to have the "best possible migration", but my main concern is the python-manilaclient
15:16:56 <xyang1> vponomaryov: sorry, I'll have to get back to this later.  in another meeting at the same time:(
15:17:54 <bswartz> my main concern is the REST API
15:18:13 <bswartz> I want to make sure that it's sane
15:18:23 <tbarron> vponomaryov: I don't have a settled view on this.  Would of course like to provide a good user experience for the end user with no "surprises".
15:18:30 <bswartz> I think we've already agreed that any option added in the future would have to default to false
15:18:55 <bswartz> if we're okay with future options defaulting to false then why aren't we okay with existing options defaulting to false
15:19:23 <bswartz> I don't have this same paranoia you guys do about users accidentally migrating their data and being unhappy about the result
15:19:30 <vponomaryov> bswartz: because the point is safety, not false/true values
15:19:53 <bswartz> migration will be clearly documented and I prefer the default to be: it always works, but we don't preserve anything
15:20:40 <ganso> users may have a hard time understand why there was data loss
15:20:42 <bswartz> I hate the idea that migration will be another feature that works with a tiny fraction of backends (like replication)
15:21:00 <ganso> when they open a ticket for their admin to migrate their data, they do not expect their data to have missing metadata
15:21:23 <bswartz> ganso: the admins job is to preserve it for them
15:21:34 <bswartz> it's not hard to check the boxes in the UI
15:21:38 <tbarron> bswartz: +1, and that's why I oscillate
15:22:09 <ganso> bswartz: yes, being fool-proof prevents the admin from losing their data by mistake when they could have just inputted the parameters in python-manilaclient
15:22:13 <bswartz> vponomaryov: if safety is all we care about, why implement fallback approach at all?
15:22:39 <ganso> bswartz: the fallback approach currently is the only way to migrate across vendors
15:22:40 <bswartz> why not force all migrations to be "safe"?
15:22:41 <vponomaryov> bswartz: for case when it is enough
15:22:53 <vponomaryov> bswartz: but it should not be default, IMHO
15:23:03 <bswartz> it seems to me that the user has to know what he's doing no matter which approach we take
15:23:10 <markstur> To make defuult false, we could change preserve_metadata to opposite.  Like no_preserve_metadata
15:23:23 <bswartz> markstur: that's not my point
15:23:26 <ganso> bswartz: it is the desired mechanism to use and it should be proper documented that there will be losses when migrating across vendors
15:23:36 <bswartz> I want a migration with no optional args specified to work with fallback approach
15:24:03 <bswartz> and I want users to ask for any special flags they care about (like snapshots, lossless, no-disrupt, etc)
15:24:11 <ganso> markstur: in that case, yes, that's one alternative
15:24:18 <markstur> but it won't "work" if damaging your ACLs is considered a major fail
15:24:51 <markstur> "just works" assumes the result will be OK and not in major need of repair (if possible)
15:24:58 <bswartz> okay so I feel like I'm the only one arguing for my approach
15:25:06 <bswartz> not sure how to break the impasse
15:25:12 <bswartz> I don't want to eat up the whole meeting on this topic
15:25:22 <bswartz> can we settle it with a vote?
15:25:29 * markstur will wish I agreed with bswartz everytime I forget the option
15:25:40 <ganso> bswartz: your point on having a rule that defaults everything to false to be easier to add new parameters is valid
15:25:42 <TommyLikeHu_> cool
15:25:47 <ganso> bswartz: but then I would go with markstur's idea
15:25:56 <ganso> bswartz: of inverting all parameters
15:26:07 <ganso> bswartz: so we still have the strictest combination defaulting to false
15:26:31 <bswartz> ganso:  so you're okay with all flags default to false, but only TRUE is compatible with fallback migration
15:26:46 <ganso> bswartz: yes
15:26:50 <bswartz> except for options we add in the future, where false will be compatible with fallback
15:27:00 <bswartz> that seems equally confusing
15:27:10 <ganso> bswartz: I mean, for any options we introduce, we would be following 2 rules
15:27:16 <ganso> bswartz: defaulting to false, and being strict
15:27:34 <bswartz> then you have the same problem I mentioned
15:27:37 <ganso> bswartz: so --no-preserve-fairy-dust if the admin wants to allow fallback
15:27:52 <bswartz> a command that works in Ocata won't work in puke
15:27:54 <bswartz> pike
15:28:04 <tbarron> :)
15:28:21 <bswartz> because the admin will need to add --no-preserve-fairy-dust for the command to succeed
15:28:25 <markstur> lol on puke
15:28:50 <ganso> bswartz: well, yes
15:28:57 <ganso> bswartz: that does not really solve the problem
15:28:59 <bswartz> it seems as bad as the other proposal
15:29:06 <ganso> bswartz: indeed it does
15:29:59 <bswartz> so the basic issue here is that our desire to protect admins and users from mistakes is conflicting with our desire to have a forwards compatible API with sane semantics
15:30:08 <bswartz> s/sane/consistent/
15:30:36 <vkmc> wouldn't the first be easily addressed with proper docs?
15:30:39 <bswartz> I don't see how we can have both
15:30:51 <vponomaryov> vkmc: who reads docs? ))
15:31:04 <vkmc> vponomaryov, docs! dooooocs!
15:31:05 <vkmc> haha
15:31:06 <bswartz> and honestly IDK care about protecting users from mistake at the API level -- I feel that's the responsibility of a high level abstraction
15:31:30 <vkmc> bswartz++
15:31:47 <bswartz> the unlink() system call doen't have a (bool are_you_sure) paramer
15:31:56 <bswartz> parameter
15:32:33 <TommyLikeHu_> lol
15:33:00 <vponomaryov> bswartz: not sure it is good example, it does not have two ways of work, like migration does
15:33:25 <ganso> bswartz: if we had the possibility of allowing access to the destination share to check the data before issuing migration-complete I could lean towards your proposal, but we agreed to not allow that in a previous meeting
15:33:38 <vkmc> lol
15:33:49 <bswartz> my only point is that APIs should do exactly what you ask them to do -- if we assume users calling our APIs don't know what they're asking for then all hope is lost
15:34:10 <vkmc> bool are_you_sure sounds like a windows interface to me
15:34:14 <ganso> s/hope/data
15:34:50 <vponomaryov> vkmc: housewife's mode happens to be really useful ))
15:34:53 <toabctl> looks like the problem is that we provide a mgration where data can be lost. that's imo the problem
15:35:03 <cknight> bswartz: you can always have safe defaults in manila client, even if the API defaults all arguments to false.
15:35:24 <cknight> bswartz: the client *is* a thin abstraction atop the API
15:35:33 <bswartz> cknight: that's something I'd be more interested in considering
15:36:06 <vponomaryov> cknight: manilaclient will be used almost always
15:36:12 <bswartz> ganso: what if the API did what I want, but the CLI client explicitly defaulted to true (opposite of what the API does)
15:36:15 <vponomaryov> cknight: to talk to server
15:36:34 <cknight> ganso: +1 That's what I'm suggesting.
15:36:35 <ganso> bswartz: that looks weird, but acceptable I would say
15:37:06 <tbarron> vponomaryov: housewife's role?  what are you talking about?
15:37:16 <bswartz> that would result in safety for people who use python-manilaclient, but users who don't want a fisher-price interface can just calls the REST APIs and get sane behavior
15:37:19 <vponomaryov> tbarron: "mode"
15:37:32 <tbarron> vponomaryov: same question
15:37:56 <bswartz> tbarron vponomaryov: let's not go down a rathole with our limited time in this meeting
15:37:57 <vponomaryov> tbarron: I mean extremely high level of fool-proof design
15:38:00 <ganso> bswartz: wait, why we would like the python-manilaclient to have insane behavior while the REST API is sane? there is no consistency in that
15:38:09 <bswartz> ganso: no
15:38:24 <bswartz> I suggest the API defaults to false, and false is compatible with fallback
15:38:47 <bswartz> the CLI client can default to explicit true values when it sends the API unless the users says not to
15:39:26 <vponomaryov> bswartz: why we use defaults in API then? wouldn't it be better to require setting it explicitly?
15:39:50 <bswartz> vponomaryov: because microversions
15:39:53 <vponomaryov> bswartz: and set safe variants in our client?
15:40:04 <vponomaryov> bswartz: it is experimental now
15:40:17 <ganso> bswartz: that easily leads to another mistake, where user of python-manilaclient assumes the default is TRUE, while it is not, so the user issues a curl command and migrates data losing metadata
15:40:40 <bswartz> vponomaryov: it would be nice to make everything explictly required but when we add the next value in the future it will need to have a default for backwards compatibility
15:41:24 <vponomaryov> bswartz: it is ok for old microversions and it is not the point of currect concern
15:42:07 <bswartz> okay I'm going to table this subject in the interest of time
15:42:18 <bswartz> #topic Spec review & prioritization
15:42:36 <bswartz> spec review have been going great
15:42:41 <TommyLikeHu_> sorry to disturb, but did we go throgh all of the specs last week?
15:42:50 <vponomaryov> TommyLikeHu_: no
15:42:53 <bswartz> the review focus specs are getting reviews exactly as we'd hoped
15:43:11 <TommyLikeHu_> ok~
15:43:12 <bswartz> I have started writing the spec about race conditions
15:43:20 <bswartz> #link https://etherpad.openstack.org/p/manila-ocata-spec-review-focus
15:43:24 <bswartz> #link https://review.openstack.org/#/q/status:open+project:openstack/manila-specs
15:43:37 <vponomaryov> TommyLikeHu_: not all of desired specs were even created
15:43:45 <TommyLikeHu_> lol
15:44:02 <bswartz> I would like to propose that we mark the eliminate race conditions spec high priority
15:44:06 <bswartz> #link https://review.openstack.org/396255
15:44:12 <cknight> bswartz: +1
15:44:22 <vponomaryov> +1
15:44:22 <bswartz> I'm working on completing it in parallel with reviewing other sepcs
15:44:23 <TommyLikeHu_> +1
15:44:24 <vkmc> +1
15:44:35 <ganso> +1
15:44:38 <bswartz> TommyLikeHu_: we reviewed the specs that existing at the time last week
15:44:49 <TommyLikeHu_> how about the share backup
15:44:52 <bswartz> any other new specs to consider
15:44:58 <TommyLikeHu_> https://review.openstack.org/#/c/330306/
15:45:15 <bswartz> TommyLikeHu_: we briefly touched on it, but we can revisit it today
15:45:36 <markstur> +1 for ending race conditions hi-pri
15:45:40 <bswartz> I'm in favor of leaving it low priority, because it's a significant feature and this is a short release
15:46:15 <TommyLikeHu_> if it's a significant feature
15:46:22 <bswartz> that's sort of what we agreed to last week, but with minimal discussion because we ran out of time
15:46:24 <TommyLikeHu_> can we keep on this
15:46:37 <TommyLikeHu_> and maybe merge it in Ocata
15:46:37 <TommyLikeHu_> ?
15:46:43 <TommyLikeHu_> only specs
15:46:54 <bswartz> TommyLikeHu_: if the spec merges by next week then the code could be considered in ocata
15:47:01 <bswartz> if not, then it's automatically pushed to pike
15:47:11 <ganso> TommyLikeHu_: I believe there is no point in merging the spec if we do not want the feature in ocata
15:47:20 <TommyLikeHu_> cool~
15:47:27 <cknight> ganso: +1
15:47:39 <bswartz> yeah and as ganso says we may decide not to merge the spec just so we can reserve bandwidth for other things
15:47:43 <vponomaryov> TommyLikeHu_: spec can be provided in "pike" subdirectory
15:47:58 <vponomaryov> TommyLikeHu_: specifying by it desired release target
15:48:02 <bswartz> time is short in ocata so we have to jealously guard core reviewer time
15:48:14 <bswartz> okay any other specs to consider for ocata
15:48:23 <TommyLikeHu_> https://review.openstack.org/#/c/365617/
15:48:30 <bswartz> gouthamr had said something about an access rule spec....
15:48:31 <vponomaryov> bswartz: HA?
15:48:39 <vponomaryov> bswartz: do we have spec about HA?
15:48:56 <TommyLikeHu_> no~
15:49:03 <ganso> bswartz: I am looking forward to seeing the access rules refactor spec
15:49:09 <bswartz> vponomaryov: it's touched on in the race condition spec, but HA is something we assume works for api/scheduler and assume it doesnt work for m-shr
15:49:28 <bswartz> and we don't plan to fix the m-shr problems for HA in ocata
15:49:36 <vponomaryov> bswartz: ok
15:50:15 <tbarron> bswartz: +1
15:50:22 <bswartz> gouthamr is on vacation today and tomorrow unfortunately
15:50:33 <bswartz> so if he has a spec forthcoming it might not be until next week
15:50:45 <TommyLikeHu_> enjoy his berlin beer
15:50:57 <bswartz> however he's at a conference next week so I expect he'll have other stuff to focus on
15:51:13 <bswartz> let me suggest this
15:51:42 <bswartz> if the spec materializes and people want to designate it high priority, I'll push a patch to do that and we can use gerrit to decide it
15:51:43 <ganso> bswartz: if we designate his spec hi-pri while it is not submitted yet, would that work?
15:51:48 <bswartz> ganso: no
15:52:22 <bswartz> the high-priority designation requires a gerrit change anyways, so that can be the voting interface
15:52:36 <bswartz> we'll just avoid merging it until we have enough +2s
15:52:56 <bswartz> any other specs to consider today?
15:53:12 <TommyLikeHu_> https://review.openstack.org/#/c/390052/
15:53:57 <bswartz> TommyLikeHu_: we decided low priority for that one
15:54:08 * bswartz goes to the review focus etherpad
15:54:08 <TommyLikeHu_> sure
15:54:37 <TommyLikeHu_> I am ok if this can be reviewed~
15:54:51 <bswartz> looks like still only one of the 8 merged
15:55:02 <bswartz> we need to merge more specs soon
15:55:15 <bswartz> and remember we're looking for 6 or 7 +2s on the specs on the etherpad
15:55:38 <vponomaryov> bswartz: 6 when author is another core member?
15:55:46 <cknight> bswartz: revert-to-snapshot is ready, just need 1-2 more reviews.  https://review.openstack.org/#/c/391269
15:55:58 <TommyLikeHu_> I can do a help for the +2
15:56:10 <bswartz> vponomaryov: it's approximate, but yes if the author is a core then they shouldn't +2 their own spec
15:56:37 <bswartz> cknight: what's with Erlon?
15:56:58 <cknight> bswartz: no idea, he just posted a few things I disgree with
15:57:03 <bswartz> let's see if we can get him to remove his -1
15:57:13 <bswartz> but if he's not around I'm happy to override him
15:57:44 <bswartz> #topic open discussion
15:58:00 <bswartz> we're still carrying a backlog of design summit topics we didn't get to in BCN
15:58:14 <bswartz> not sure if we'll cover any of those next week
15:58:21 <bswartz> top priority for everyone is specs
15:59:01 <tbarron> bswartz: cknight TommyLikeHu_ zhonghua erlon is also reviewing the corresponding cinder spec.
15:59:12 <bswartz> we still need to resolve the issue in ganso' spec about migration defaults
15:59:22 <bswartz> I guess I'll work with ganso offline on that
15:59:35 <bswartz> and we can have an ML thread about it maybe
15:59:47 <bswartz> thank everyone
15:59:51 <bswartz> #endmeeting