15:00:37 <bswartz> #startmeeting manila
15:00:38 <openstack> Meeting started Thu Jun  4 15:00:37 2015 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:39 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:41 <openstack> The meeting name has been set to 'manila'
15:00:45 <cknight> Hi
15:00:47 <bswartz> hello all
15:00:48 <zhongjun2> hi
15:00:48 <ganso_> hello
15:00:49 <rraja> hi
15:00:50 <xyang> hi
15:00:50 <lpabon> hi all
15:00:53 <vponomaryov> hello
15:00:54 <u_glide2> hi
15:01:21 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:01:41 <bswartz> okay I'm pretty behind this week
15:01:58 <bswartz> as some of you know my daughter was born on Monday
15:02:15 <bswartz> there's a few urgent topics though and then I'll turn the meeting over to open discussion
15:02:32 <xyang1> Congrats!
15:02:40 <bswartz> xyang1: thank you
15:02:51 <bswartz> #topic Manila Midcycle Meetup
15:03:27 <bswartz> so as promised I started an ML thread about the midcycle date options
15:03:45 <bswartz> I've only heard back from a few people, but so far neither week is ideal for everyone
15:04:18 <vponomaryov> I am ok with either of proposed
15:04:30 <ganso_> either for me is fine
15:04:42 <u_glide2> +1
15:05:09 <cFouts> o/
15:05:13 <bswartz> is there anyone who wants to travel, who can only make one of those weeks (other than xyang)?
15:05:22 <markstur> hi
15:05:35 <bswartz> or anyone who can participate but only on one of those weeks?
15:05:51 <markstur> I could only travel one of the weeks.
15:06:01 <bswartz> markstur I know about your vacation dates -- any chance you would travel the other week>
15:06:05 <markstur> But I can't verify that I'll get approval until after the date is set
15:06:29 <bswartz> yeah I know about travel budget issues, believe me
15:06:38 <lpabon> bswartz: i am not sure if I will be going, but if I do, most likely it will be on July 20th
15:06:46 <markstur> I was told "likely" for August.  I haven't heard if we spent all our July money in Vancouver.  I still think travel is "likely".
15:07:17 <lpabon> i'm checking with csaba and rraja to see if they are going.  We will let you know
15:07:28 <bswartz> okay cool
15:07:38 <bswartz> I will wait a few more days for responses to the ML thread
15:07:51 <bswartz> you can mail me directly instead of responding to the thread if you like
15:08:33 <bswartz> but I will set a date before the next meeting, so let me know today or tomorrow about your preferences
15:09:23 <bswartz> okay next topic
15:09:31 <bswartz> #topic 3rd party vendor CI
15:09:38 <bswartz> #link https://etherpad.openstack.org/p/manila-driver-maintainers
15:09:50 <bswartz> thanks to those who filled this out
15:09:55 <bswartz> some are still missing info
15:10:16 <bswartz> I'm guessing they just didn't come to the weekly meeting
15:10:35 <bswartz> I'll go ping people starting tomorrow for the missing entries
15:11:08 <bswartz> driver maintainers should be paying attention to these meetings though...
15:11:56 <bswartz> #topic open discussion
15:12:07 <bswartz> okay those were the 2 most pressing issues
15:12:14 <markstur> was migration on the topic list today?
15:12:26 <ganso_> we can discuss it
15:12:32 <bswartz> we have some technical discussions going on in the background
15:12:41 <bswartz> yeah migration is one
15:12:47 <bswartz> I'll offer my thoughts
15:13:12 <bswartz> there are a few ways forward on the migration-changing-the-share-ID issue
15:13:24 <bswartz> read ganso's ML thread if you're not familair with them
15:13:54 <bswartz> one thing I think will help decide that issue is if we can figure out how the various options will affect share replication
15:14:20 <bswartz> the current proposal for share replication is to use a single row and a single share ID for all copies of a given share
15:15:02 <bswartz> migration basically requires 2 IDs (at least temporarily) so if we can update the replication proposal with new options it might help us decide
15:15:25 <bswartz> that will help me decide at least -- I haven't been thinking about replication these last few days...
15:16:58 <bswartz> the other related issue is availability zones
15:17:10 <bswartz> which I brought up last week
15:17:26 <ganso_> bswartz: by what you mentioned about replication approach, is it including a list of all other replicated share IDs in a field in the single row entry?
15:17:40 <bswartz> I'm now pretty confident that AZs are the right abstraction to allow end users to request data locality between their shares and their nova instances
15:18:07 <bswartz> but if we support AZs than that will affect both replication and migration
15:18:27 <bswartz> because there is a use case to "change AZ" on a share which would be a migration by default
15:18:50 <bswartz> so these 3 feature proposals are all tangled up now
15:19:04 <cknight> ganso_: No, the original replication proposal had only a single ID for a share, regardless of its number of replicas.
15:19:14 <bswartz> ganso_: yes what cknight said
15:19:30 <bswartz> we could update the proposal -- no code has been written for that feature yet
15:19:34 <ganso_> cknight: how are you tracking the replicas?
15:20:03 <cknight> ganso_: That's up to the driver based on its conf file input.
15:20:18 <cknight> ganso_: We wanted to be minimally prescriptive to allow more flexibility for drivers.
15:20:47 <ganso_> cknight: so I can assume the driver may make use of Private Driver Storage for that, right?
15:20:56 <cknight> ganso_: Yes, definitely.
15:21:26 <bswartz> yes -- the apparent advantage to the single ID proposal is that it's maximally flexible for backends, and should minimize the set of implementations that can't meet the replication API's contract
15:21:47 <bswartz> however flexibility has its own drawbacks too, which I why I'm willing to reconsider
15:22:19 <bswartz> I just haven't made any progress with my thinking
15:22:29 <bswartz> hopefully next week will be better
15:22:59 <bswartz> anyone else have thoughts on that stuff?
15:23:12 <bswartz> or any other topics for today?
15:23:13 <ganso_> I see two categories here. We have been discussing about trying to do all we can in the Core code to minimize driver code effort, but as far as I understood, replication is taking a different approach
15:23:32 <u_glide2> bswartz: I need your approve on https://review.openstack.org/#/c/187953/ (manila-image-elements project)
15:23:34 <bswartz> ganso_: that's true
15:23:52 <ganso_> so if the driver does not implement how to track the IDs, and how to manage its own replication, then replication is not working for that backend at all
15:24:28 <ganso_> That's a bit similar to Proposal 1 from my email.
15:24:51 <markstur> u_glide2, Won't you need a pypi project for manila-image-elements?
15:24:57 <bswartz> ganso_: replication is intended to be completely optional -- driver can support it or not
15:25:30 <bswartz> migration is supported for all backends whether they implement it or not (at least that's what you've proposed) which means we need to have a universal fallback
15:25:34 <u_glide2> markstur: I need to, but its not required on current step
15:25:35 <vponomaryov> markstur: yes, we need it
15:25:43 <ganso_> bswartz: you are right
15:26:02 <ganso_> bswartz: I understand now the direction we should follow
15:26:16 <ganso_> bswartz: may we scratch Proposal 1 off the list then?
15:26:29 <bswartz> the question is whether it's advantageous to the backends that do wish to implement replication if we let them store multiple share "instances" in the DB
15:27:12 <ganso_> bswartz: it can be a guideline for them
15:27:19 <bswartz> ganso_: I think proposal 1 is still attractive for some reasons
15:27:44 <bswartz> cknight likes it best (I think) because it's the smallest possible change
15:28:06 <vponomaryov> ))
15:28:10 <cknight> bswartz: +1
15:28:24 <xyang> ganso_: have you looked at cinder's internal tenant proposal?
15:28:46 <bswartz> xyang: I believe we will end up needing something similar regardless of what proposal we go with
15:28:48 <ganso_> xyang: not yet
15:30:11 <bswartz> we will need to use some mechanism to prevent end users from seeing 2 shares while a migration is in progress, yet the admin does need to see both shares in case the migration fails and needs fixing
15:31:24 <vponomaryov> bswartz: in case of "proposal 3" admin will be able to track all instances
15:31:31 <ganso_> bswartz: if it needs fixing from admin side then the error state cleanup is not properly implemented
15:31:39 <vponomaryov> bswartz: but he can not track it using proposal 1
15:31:46 <ganso_> vponomaryov: +1
15:31:52 <bswartz> I disagree
15:32:04 <bswartz> well perhaps my idea of proposal 1 is slightly different
15:32:17 <bswartz> my idea of proposal 1 is:
15:32:35 <bswartz> we create a new share with a new ID as the migration destination
15:32:51 <bswartz> that share is owned by an "internal" tenant to hide it while the migration is in progress
15:33:09 <ganso_> vponomaryov: good point, but there are other mechanisms for that still. Proposal 1 is the one that complements the current prototype with the smallest changes, we would need only to filter out the temporary share in a way that only the admin sees, but not the user
15:33:12 <bswartz> when the migration completes, the old share is deleted, and the new share
15:33:22 <bswartz> the new share's ID is changed to match the old one
15:34:01 <bswartz> backends may have to use driver private share data to perform the ID-change operation because many drivers are unable to actually rename their underlying share object
15:34:07 <vponomaryov> bswartz: ok, but 1 still does not satisfy future "replication". replication requires separation of "representation" of a share and its "instances"
15:34:32 <vponomaryov> bswartz: proposal 3 does satisfy it
15:34:59 <ganso_> bswartz: there is more than one way to achieve that last step, in my prototype, I never delete the original DB row. In your idea, you delete it, and change the ID, I did not it was possible to change a UUID in sql alchemy
15:35:03 <vponomaryov> bswartz: and satisfies "migration" too
15:35:34 <bswartz> I believe it's possible to do 1 and make replication work, since the driver is free to store whatever it needs in driver private share data
15:35:41 <bswartz> however I'm not certain it's preferrable
15:36:31 <ganso_> the way I see it, if we implement proposal 3, then drivers can use it, but they also can use proposal 1 if they want
15:36:45 <ganso_> because Private Driver Storage is there
15:36:45 <vponomaryov> ganso_: +1
15:37:04 <ganso_> not sure if this is redundant
15:37:11 <ganso_> like, wasted effort
15:37:35 <bswartz> ganso_: what you say is true, but I would want to do a design and see if it actually helps or not
15:38:06 <vponomaryov> helps what? only migration? or also AZs and replication?
15:38:09 <bswartz> because the downside to 3 is that it's a major change which introduces complexity to the project for possibly very little gain
15:38:50 <bswartz> vponomaryov: helps with implementing migration for one or more backends
15:39:05 <ganso_> is proposal 2 helpful for replication or AZ at all?
15:39:36 <bswartz> AZs are not affected by the instances/multiple IDs
15:40:56 <bswartz> I'll see if I can compose a response on the ML which summarizes my thoughts
15:41:25 <bswartz> cknight: I know you have a point of view on this, perhaps you can respond on the ML too
15:41:34 <cknight> bswartz: ok
15:42:05 <bswartz> I'm really happy that we're thinking about all of these issues now during the first milestone
15:42:30 <bswartz> I'm optimistic we will find a good solution and have time to implement all of the proposed features during liberty
15:43:00 <vponomaryov> bswartz: in interface, but implementation in drivers?
15:43:19 <bswartz> vponomaryov: in some drivers
15:43:49 <bswartz> for replication I'm still looking for a volunteer to implement it in the generic driver
15:44:09 <bswartz> but we may need to get the design farther along before someone can actually start that
15:44:17 <u_glide2> we also should schedule API migration to micro-versions framework
15:44:55 <cknight> u_glide2: +1
15:44:58 <vponomaryov> u_glide2: +1
15:45:20 <cknight> u_glide2: I've been waiting for the API WG guidelines on microversions.  I'll ping them about that.
15:45:34 <cknight> u_glide2: They promised to do that at the summit.
15:45:39 <u_glide2> it's big chunk of work, which will affect all user-facing features
15:45:41 <bswartz> migration doesn't NEED any implementations
15:45:55 <u_glide2> cknight: sounds good
15:45:57 <bswartz> although having 1 is implrtant for tesiing the optimized case
15:46:48 <ganso_> bswartz: in fact, if you take a look at the prototype and the proposal, some method implementation is currently required from drivers, like the network path and mount command. Maybe we can improve on that, I do not know how yet
15:47:21 <bswartz> ganso_: I haven't looked yet
15:48:08 <bswartz> ganso_: what about the data copy servive?
15:48:57 <ganso_> bswartz: it will help
15:49:30 <ganso_> bswartz: I need to give it more thought
15:50:49 <ganso_> bswartz: I am not sure if drivers will still have to provide the network path and mount command to the data copy service or we can include it in it somehow
15:52:38 <bswartz> I'll want to know more about that
15:52:47 <bswartz> I just haven't had time to look
15:53:25 <bswartz> hopefully I'll be doing more code reviews when I get home from the hospital
15:53:47 <bswartz> okay if that's all I think we're done for today
15:53:53 <rraja> bswartz: a related question of sorts. is there already a list available of APIs a driver must support to be included in Liberty?  this question was raised in the openstack-manila channel as well.
15:54:03 <ganso_> bswartz: what was the other pressing issue?
15:54:18 <ganso_> rraja: I am compiling this list
15:54:25 <rraja> ganso_: thanks!
15:54:31 <bswartz> rraja: it's smething we need
15:54:52 <bswartz> the developer docs should make this clear
15:54:54 <vponomaryov> also, we need to decide what to do with drivers that already exist but do not satisfy all required features criteria?
15:54:55 <ganso_> rraja: on monday I will post a preliminary list based on etherpad and everyone will be able to collaborate before we move it to the wiki
15:55:01 <rraja> bswartz: it would help us prioritize our BPs for liberty.
15:55:25 <bswartz> and the functional test suite should validate whether driver actually implement the APis
15:55:56 <bswartz> ganso_ I covered them both -- midcycle meetup date and driver maintainers for CI
15:56:05 <bswartz> okay I think we're done
15:56:14 <bswartz> thanks everyone and sorry I've been mostly AFK
15:56:34 <bswartz> #endmeeting