15:00:37 #startmeeting manila 15:00:38 Meeting started Thu Jun 4 15:00:37 2015 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:39 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:41 The meeting name has been set to 'manila' 15:00:45 Hi 15:00:47 hello all 15:00:48 hi 15:00:48 hello 15:00:49 hi 15:00:50 hi 15:00:50 hi all 15:00:53 hello 15:00:54 hi 15:01:21 #agenda https://wiki.openstack.org/wiki/Manila/Meetings 15:01:41 okay I'm pretty behind this week 15:01:58 as some of you know my daughter was born on Monday 15:02:15 there's a few urgent topics though and then I'll turn the meeting over to open discussion 15:02:32 Congrats! 15:02:40 xyang1: thank you 15:02:51 #topic Manila Midcycle Meetup 15:03:27 so as promised I started an ML thread about the midcycle date options 15:03:45 I've only heard back from a few people, but so far neither week is ideal for everyone 15:04:18 I am ok with either of proposed 15:04:30 either for me is fine 15:04:42 +1 15:05:09 o/ 15:05:13 is there anyone who wants to travel, who can only make one of those weeks (other than xyang)? 15:05:22 hi 15:05:35 or anyone who can participate but only on one of those weeks? 15:05:51 I could only travel one of the weeks. 15:06:01 markstur I know about your vacation dates -- any chance you would travel the other week> 15:06:05 But I can't verify that I'll get approval until after the date is set 15:06:29 yeah I know about travel budget issues, believe me 15:06:38 bswartz: i am not sure if I will be going, but if I do, most likely it will be on July 20th 15:06:46 I was told "likely" for August. I haven't heard if we spent all our July money in Vancouver. I still think travel is "likely". 15:07:17 i'm checking with csaba and rraja to see if they are going. We will let you know 15:07:28 okay cool 15:07:38 I will wait a few more days for responses to the ML thread 15:07:51 you can mail me directly instead of responding to the thread if you like 15:08:33 but I will set a date before the next meeting, so let me know today or tomorrow about your preferences 15:09:23 okay next topic 15:09:31 #topic 3rd party vendor CI 15:09:38 #link https://etherpad.openstack.org/p/manila-driver-maintainers 15:09:50 thanks to those who filled this out 15:09:55 some are still missing info 15:10:16 I'm guessing they just didn't come to the weekly meeting 15:10:35 I'll go ping people starting tomorrow for the missing entries 15:11:08 driver maintainers should be paying attention to these meetings though... 15:11:56 #topic open discussion 15:12:07 okay those were the 2 most pressing issues 15:12:14 was migration on the topic list today? 15:12:26 we can discuss it 15:12:32 we have some technical discussions going on in the background 15:12:41 yeah migration is one 15:12:47 I'll offer my thoughts 15:13:12 there are a few ways forward on the migration-changing-the-share-ID issue 15:13:24 read ganso's ML thread if you're not familair with them 15:13:54 one thing I think will help decide that issue is if we can figure out how the various options will affect share replication 15:14:20 the current proposal for share replication is to use a single row and a single share ID for all copies of a given share 15:15:02 migration basically requires 2 IDs (at least temporarily) so if we can update the replication proposal with new options it might help us decide 15:15:25 that will help me decide at least -- I haven't been thinking about replication these last few days... 15:16:58 the other related issue is availability zones 15:17:10 which I brought up last week 15:17:26 bswartz: by what you mentioned about replication approach, is it including a list of all other replicated share IDs in a field in the single row entry? 15:17:40 I'm now pretty confident that AZs are the right abstraction to allow end users to request data locality between their shares and their nova instances 15:18:07 but if we support AZs than that will affect both replication and migration 15:18:27 because there is a use case to "change AZ" on a share which would be a migration by default 15:18:50 so these 3 feature proposals are all tangled up now 15:19:04 ganso_: No, the original replication proposal had only a single ID for a share, regardless of its number of replicas. 15:19:14 ganso_: yes what cknight said 15:19:30 we could update the proposal -- no code has been written for that feature yet 15:19:34 cknight: how are you tracking the replicas? 15:20:03 ganso_: That's up to the driver based on its conf file input. 15:20:18 ganso_: We wanted to be minimally prescriptive to allow more flexibility for drivers. 15:20:47 cknight: so I can assume the driver may make use of Private Driver Storage for that, right? 15:20:56 ganso_: Yes, definitely. 15:21:26 yes -- the apparent advantage to the single ID proposal is that it's maximally flexible for backends, and should minimize the set of implementations that can't meet the replication API's contract 15:21:47 however flexibility has its own drawbacks too, which I why I'm willing to reconsider 15:22:19 I just haven't made any progress with my thinking 15:22:29 hopefully next week will be better 15:22:59 anyone else have thoughts on that stuff? 15:23:12 or any other topics for today? 15:23:13 I see two categories here. We have been discussing about trying to do all we can in the Core code to minimize driver code effort, but as far as I understood, replication is taking a different approach 15:23:32 bswartz: I need your approve on https://review.openstack.org/#/c/187953/ (manila-image-elements project) 15:23:34 ganso_: that's true 15:23:52 so if the driver does not implement how to track the IDs, and how to manage its own replication, then replication is not working for that backend at all 15:24:28 That's a bit similar to Proposal 1 from my email. 15:24:51 u_glide2, Won't you need a pypi project for manila-image-elements? 15:24:57 ganso_: replication is intended to be completely optional -- driver can support it or not 15:25:30 migration is supported for all backends whether they implement it or not (at least that's what you've proposed) which means we need to have a universal fallback 15:25:34 markstur: I need to, but its not required on current step 15:25:35 markstur: yes, we need it 15:25:43 bswartz: you are right 15:26:02 bswartz: I understand now the direction we should follow 15:26:16 bswartz: may we scratch Proposal 1 off the list then? 15:26:29 the question is whether it's advantageous to the backends that do wish to implement replication if we let them store multiple share "instances" in the DB 15:27:12 bswartz: it can be a guideline for them 15:27:19 ganso_: I think proposal 1 is still attractive for some reasons 15:27:44 cknight likes it best (I think) because it's the smallest possible change 15:28:06 )) 15:28:10 bswartz: +1 15:28:24 ganso_: have you looked at cinder's internal tenant proposal? 15:28:46 xyang: I believe we will end up needing something similar regardless of what proposal we go with 15:28:48 xyang: not yet 15:30:11 we will need to use some mechanism to prevent end users from seeing 2 shares while a migration is in progress, yet the admin does need to see both shares in case the migration fails and needs fixing 15:31:24 bswartz: in case of "proposal 3" admin will be able to track all instances 15:31:31 bswartz: if it needs fixing from admin side then the error state cleanup is not properly implemented 15:31:39 bswartz: but he can not track it using proposal 1 15:31:46 vponomaryov: +1 15:31:52 I disagree 15:32:04 well perhaps my idea of proposal 1 is slightly different 15:32:17 my idea of proposal 1 is: 15:32:35 we create a new share with a new ID as the migration destination 15:32:51 that share is owned by an "internal" tenant to hide it while the migration is in progress 15:33:09 vponomaryov: good point, but there are other mechanisms for that still. Proposal 1 is the one that complements the current prototype with the smallest changes, we would need only to filter out the temporary share in a way that only the admin sees, but not the user 15:33:12 when the migration completes, the old share is deleted, and the new share 15:33:22 the new share's ID is changed to match the old one 15:34:01 backends may have to use driver private share data to perform the ID-change operation because many drivers are unable to actually rename their underlying share object 15:34:07 bswartz: ok, but 1 still does not satisfy future "replication". replication requires separation of "representation" of a share and its "instances" 15:34:32 bswartz: proposal 3 does satisfy it 15:34:59 bswartz: there is more than one way to achieve that last step, in my prototype, I never delete the original DB row. In your idea, you delete it, and change the ID, I did not it was possible to change a UUID in sql alchemy 15:35:03 bswartz: and satisfies "migration" too 15:35:34 I believe it's possible to do 1 and make replication work, since the driver is free to store whatever it needs in driver private share data 15:35:41 however I'm not certain it's preferrable 15:36:31 the way I see it, if we implement proposal 3, then drivers can use it, but they also can use proposal 1 if they want 15:36:45 because Private Driver Storage is there 15:36:45 ganso_: +1 15:37:04 not sure if this is redundant 15:37:11 like, wasted effort 15:37:35 ganso_: what you say is true, but I would want to do a design and see if it actually helps or not 15:38:06 helps what? only migration? or also AZs and replication? 15:38:09 because the downside to 3 is that it's a major change which introduces complexity to the project for possibly very little gain 15:38:50 vponomaryov: helps with implementing migration for one or more backends 15:39:05 is proposal 2 helpful for replication or AZ at all? 15:39:36 AZs are not affected by the instances/multiple IDs 15:40:56 I'll see if I can compose a response on the ML which summarizes my thoughts 15:41:25 cknight: I know you have a point of view on this, perhaps you can respond on the ML too 15:41:34 bswartz: ok 15:42:05 I'm really happy that we're thinking about all of these issues now during the first milestone 15:42:30 I'm optimistic we will find a good solution and have time to implement all of the proposed features during liberty 15:43:00 bswartz: in interface, but implementation in drivers? 15:43:19 vponomaryov: in some drivers 15:43:49 for replication I'm still looking for a volunteer to implement it in the generic driver 15:44:09 but we may need to get the design farther along before someone can actually start that 15:44:17 we also should schedule API migration to micro-versions framework 15:44:55 u_glide2: +1 15:44:58 u_glide2: +1 15:45:20 u_glide2: I've been waiting for the API WG guidelines on microversions. I'll ping them about that. 15:45:34 u_glide2: They promised to do that at the summit. 15:45:39 it's big chunk of work, which will affect all user-facing features 15:45:41 migration doesn't NEED any implementations 15:45:55 cknight: sounds good 15:45:57 although having 1 is implrtant for tesiing the optimized case 15:46:48 bswartz: in fact, if you take a look at the prototype and the proposal, some method implementation is currently required from drivers, like the network path and mount command. Maybe we can improve on that, I do not know how yet 15:47:21 ganso_: I haven't looked yet 15:48:08 ganso_: what about the data copy servive? 15:48:57 bswartz: it will help 15:49:30 bswartz: I need to give it more thought 15:50:49 bswartz: I am not sure if drivers will still have to provide the network path and mount command to the data copy service or we can include it in it somehow 15:52:38 I'll want to know more about that 15:52:47 I just haven't had time to look 15:53:25 hopefully I'll be doing more code reviews when I get home from the hospital 15:53:47 okay if that's all I think we're done for today 15:53:53 bswartz: a related question of sorts. is there already a list available of APIs a driver must support to be included in Liberty? this question was raised in the openstack-manila channel as well. 15:54:03 bswartz: what was the other pressing issue? 15:54:18 rraja: I am compiling this list 15:54:25 ganso_: thanks! 15:54:31 rraja: it's smething we need 15:54:52 the developer docs should make this clear 15:54:54 also, we need to decide what to do with drivers that already exist but do not satisfy all required features criteria? 15:54:55 rraja: on monday I will post a preliminary list based on etherpad and everyone will be able to collaborate before we move it to the wiki 15:55:01 bswartz: it would help us prioritize our BPs for liberty. 15:55:25 and the functional test suite should validate whether driver actually implement the APis 15:55:56 ganso_ I covered them both -- midcycle meetup date and driver maintainers for CI 15:56:05 okay I think we're done 15:56:14 thanks everyone and sorry I've been mostly AFK 15:56:34 #endmeeting