15:00:35 #startmeeting manila 15:00:35 Meeting started Thu Jun 23 15:00:35 2016 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:36 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:38 The meeting name has been set to 'manila' 15:00:46 hello all 15:00:46 \o/ 15:00:47 hello 15:00:47 Hi 15:00:48 hi 15:00:49 hello 15:00:49 hi 15:00:51 hello 15:00:54 hi 15:00:57 \o 15:00:57 Hello 15:01:02 hi 15:01:05 hi. 15:01:07 * bswartz notes that ameade had his caffeine today.... 15:01:09 o/ 15:01:15 hi 15:01:19 * ameade is at a coffee shop 15:01:28 #agenda https://wiki.openstack.org/wiki/Manila/Meetings 15:01:43 DaiDV: your on the agenda btw 15:01:52 hi 15:02:04 #topic announcements 15:02:18 reminder that midcycle meetup is next week, tuesday/wednesday 15:02:32 from 1200 UTC to 2000 UTC 15:03:07 bswartz: google hangout, right? 15:03:12 apologies to US west coast and east asia people for the times, we cannot make everyone happy 15:03:20 hi 15:03:21 webex? 15:03:23 Hi everyone, my proposing is "Implement extend and shrink features for glusterfs driver" 15:03:30 #link https://etherpad.openstack.org/p/newton-manila-midcycle 15:03:58 I haven't decided on webex or google hangouts, although we will use one of those 15:04:18 in the past the limited number of people in google hangouts has been a deal breaker 15:04:19 I'd vote for hangouts, webex sucks in Linux 15:04:28 though hangouts does limit participants... 15:04:29 dustins: +1 15:04:37 does anyone know if hangouts has increased the person limit? 15:05:03 I don't think so 15:05:07 ganso: dustins: +1 15:05:19 10 for normal, 15 if you have a google apps for business account 15:05:32 * dustins was expecting the Google Apps one to be higher than that 15:05:43 I heard that bluejeans works for Linux Osx and windows 15:05:43 I hate webex as much as anybody, but it actually works and it scales way better than hangouts 15:05:46 what about bj? 15:05:47 yeah 15:06:05 Bluejeans is indeed an option 15:06:10 Bluejeans works for me but I don't have an account 15:06:11 bswartz: works, but not on linux 15:06:22 I think the RH ppl could host it 15:06:26 can make it work, but then it breaks on upgrades, etc 15:06:36 I keep a windows VM around expressly for webex, MS outlook, and powerpoint 15:06:56 don't mean to rathole 15:06:57 I have my personal macbook that I could use, but I'd rather not 15:07:01 bluejeans has the distinction of being the only app ever to cause a kernel panic on my laptop 15:07:08 (and don't tell the other Hatters that I have a mac) 15:07:41 * mkoderer doesn't propose to host the meeting with Micosoft Lync 15:07:59 That DEFINITELY wouldn't work on Linux :P 15:08:31 Let's start with hangout and switch to webex if needed 15:08:33 heh, lync 15:08:34 if we did hangouts we'd need to setup a phone bridge and many people would be stuck using that due to the 10 person limit 15:08:35 bswartz: I join webex on my Mac. I also do powerpoint on Mac. outlook is a pain. I’m using the web version 15:09:00 bswartz: not fair to anyone stuck with phone only 15:09:06 bswartz: they wouldn't see the demos 15:09:17 cknight: demos? 15:09:20 is there anyone who can't manage at all with webex? 15:09:27 mkoderer: screen sharing 15:09:29 mkoderer: your demo, forgot? 15:09:31 isn't it possible to use both hangouts and webex? 15:09:48 ganso: indeed it is, but only audio would go across 15:09:51 ganso: +1 we could start hangouts and broadcast the webex 15:10:07 vponomaryov: :) 15:10:11 oh god are you volunteering to do that gouthamr? 15:10:23 only broadcast through hangouts though, no conferencing :) 15:10:36 that sounds hideous and like it might cause your laptop to catch on fire 15:10:51 yeah, use a mac bswartz 15:11:21 macs aren't open enough 15:11:28 bswartz: +1 :) 15:11:29 i can play with it and see if it works.. 15:11:34 i don't have windows or mac but guess i'll figure something out 15:11:47 mkoderer: it would be really big lost not seeing demo of HPB feature 15:11:56 vponomaryov: I can do 15:11:59 * tbarron works on an open source project 15:12:11 tbarron: We can huddle around my macbook :P 15:12:14 mkoderer: cool! 15:12:16 we should schedule a session at the beginning of midcycle to just make the conference work 15:12:25 tbarron: there are rumors that webex can be made to work on Fedora 15:12:28 ganso: +1 15:12:44 tbarron: it's much less stressful to create a windows VM however and run webex there 15:12:57 no license 15:13:01 but licenses... 15:13:04 yeah 15:13:07 tbarron: Windows is free for 90 days or something like that 15:13:20 ah, true, windows 10 i guessa 15:13:50 it only starts complaining about activation if you continue to use it for a long time 15:14:09 okay enough on this topic 15:14:14 So "borrow" a copy of windows, got it 15:14:20 :) 15:14:23 Personally I'm leaning towards webex because it will allow enough people 15:14:34 anyone can join in audio only mode with no software at all -- just a phone 15:14:36 how many people are for Manila usually? 15:14:43 tbarron: you don’t have a laptop? 15:14:53 vkmc: more than hangouts allow 15:14:55 vkmc: ~15 15:15:01 xyang: fedora on laptop 15:15:04 I think the last one had 15-20 15:15:08 we'll see if we can bridge a google hangout in with some kind of magic 15:15:26 xyang: that's what vkmc dustins tellesnobrega and I all use 15:15:38 tbarron: ok 15:15:50 I'm not going to do the whole google business account thing to get 15 people though -- that was a ridiculous hassle the one time I tried it 15:15:53 * tbarron subtly points out that we're bringing more folks in to work on manila 15:16:12 besides the ceph driver people 15:16:31 okay let's move on 15:16:38 #topic Implement extend and shrink features for glusterfs driver 15:16:49 DaiDV: you're up 15:17:01 yes. 15:17:40 I wanna implement two features for glusterfs driver: extend and shrink. 15:18:02 DaiDV: do you have a spec already? 15:18:04 everyone remembers that "extend" is required feature )) 15:18:17 yes, extend is a required feature 15:18:26 if it's not implemented, how is glusterfs not failing CI? 15:18:35 mkoderer: a spec isn't needed for a minor driver feature 15:18:38 bswartz: extend tests can be disabled 15:18:41 yes. 15:18:44 cknight: ok 15:18:45 * bswartz groans 15:18:55 someone please remove the option to disable extend tests 15:19:13 CI extend is not working on glusterfs backend. 15:19:28 bswartz: don't think that this is so easy .. 15:19:36 bswartz: options are not only for CI, so, I disagree to remove it 15:19:51 DaiDV: i think this is a small feature, and does not need a spec 15:19:51 what is the point of the option then? 15:20:07 DaiDV: you'd have to just modify quota limits, right? 15:20:14 yes. 15:20:15 rraja: we are not requiring specs 15:20:16 disable things that I do not want to test 15:20:29 any driver that doesn't implement extend needs to be removed, and failing CI is a good way to find out which drivers that is 15:20:30 and we also manager some exception 15:20:49 such as: shrink_size smaller than data size is used 15:21:11 and extend_size exceeds volume size. 15:21:20 DaiDV: have you looked at existing implementations in other drivers? 15:21:38 DaiDV: you are not first who implements these features 15:21:48 yes, I see it is implemented on 15:22:05 some special driver 15:22:25 extend should be fairly trivial to implement unless the underlying file system on the backend is horribly inflexible 15:22:32 But glusterfs is not supported. 15:22:50 we ran into issues with extend in the generic driver due to issues with how cinder handles extend 15:23:13 sorry, what's not supported? 15:23:26 sorry, I mean 15:23:37 glusterfs driver is not support them. 15:23:42 DaiDV: it is unclear what is your concern that you need help with 15:24:05 DaiDV: ok. can you please email in openstack-dev or gluster-dev and we'll happy to help you out. 15:24:07 ? 15:24:17 yeah I agree, it seems like DaiDV should just go ahead and implement the features in the driver 15:24:25 +1 15:24:27 I'm not sure what we need to discuss 15:24:54 although I do feel like the option to disable extend tests should be removed 15:25:14 bswartz: +1 15:25:15 I can't think of any good reason not to run those tests 15:25:21 bswartz: it is easy to workaround by other means 15:25:29 bswartz: +1 15:25:49 bswartz: agree with remove the option to disable extend test 15:25:49 vponomaryov: yea, like a regex in MANILA_TESTS env_var 15:25:55 vponomaryov: you can always check the CI logs, right? 15:26:07 not sure how this should be "technically" possible.. 3rd party ci systems cannot be controlled.. they could simply delete the test files 15:26:16 rraja: I also can return fake "success" 15:26:16 we could check CI logs but that's labor intensive 15:26:21 vponomaryov: :) 15:26:48 vponomaryov is sneaky 15:26:55 bswartz: we should do for new drivers, but in this case, we accepted a driver before extend existed and was considered a minimum feature 15:27:14 markstur_: ti catch a cheater think as cheater 15:27:19 s/ti/to/ 15:27:19 ronald reagan always said "trust but verify" -- that's my philosophy when it comes to 3rd party CI 15:27:56 okay I think this topic is done 15:28:08 #topic Do all drivers have to inherit from driver.ShareDriver? 15:28:20 mkoderer: not sure where you're going with this topic 15:28:46 I came across this topic while reviewing https://review.openstack.org/#/c/312423/ 15:28:46 bswartz: he dislikes EMC drivers implementation 15:29:14 I see it dangergous if all drivers do not share the same base class 15:29:31 yeah... I agree 15:29:43 what is going on in the EMC driver that you don't like? 15:29:44 if we introduce a new functioanltity these functions are not inheriteed to all the drivers 15:29:48 mkoderer: +1 15:30:25 mkoderer: +1 15:30:27 it looks to me like the driver does inherit from sharedriver 15:30:31 bswartz: basically it has it's one base with ABC 15:30:34 class EMCShareDriver(driver.ShareDriver): 15:30:37 mkoderer: actually, they inherit base class here - https://github.com/openstack/manila/blob/a97d2bd7/manila/share/drivers/emc/driver.py#L62 15:30:38 so what am I missing? 15:30:50 EMCShareDriver subclasses Driver, then proxies calls into plug-ins. What's wrong with that? 15:30:55 We have one commone EMC Share Driver 15:31:11 Isilon and VNX and Unity have plugins to this common driver 15:31:23 https://review.openstack.org/#/c/312423/17/manila/share/drivers/emc/plugins/unity/connection.py 15:31:40 mkoderer: that is the plugin 15:31:45 it seems that the EMC driver is following the same model the NetApp driver uses with a single driver class and plugins for different platforms 15:31:47 sah ok 15:31:49 the plugin to EMCShareDriver 15:31:57 bswartz: you are right 15:32:01 ok maybe I am wrong.. than I am fine 15:32:10 there are a number of advantages to that kind of design 15:32:35 I just wanted to ask if we want to stick that every driver needs to inherit from base driver 15:32:38 bswartz: It's a common pattern in Cinder where there are driver subclasses for different protocols. 15:32:43 if this is the case I am good 15:32:55 mkoderer: answer is "it is already so" 15:32:59 Sorry? I wonder what blueprint/spec or any introduction indicate "all drivers have to inherit from driver.ShareDriver"? 15:33:18 DaiDV: I believe there is a document in devref 15:33:19 vponomaryov: ok sry.. seems i misread the code 15:33:27 so mkoderer I think we all agree that drivers do need to inherit from sharedriver, and there aren't any that don't, so there's no problem 15:33:40 bswartz: +1 next topic :) 15:33:44 cool, that was an easy one to resolve 15:33:47 mkoderer: EMC and NetApp drivers just do additional branching 15:33:51 #topic open discussion 15:33:56 I have a topic 15:34:05 oh 15:34:23 what's the state of the contaienr driver? :) 15:34:26 ganso: what is it? 15:34:42 bswartz: additional instances in optimized migration 15:34:45 mkoderer: waiting for 2 +2s 15:34:53 #topic additional instances in optimized migration 15:34:54 mkoderer: its doing great! 15:35:01 oh yes this is an important one 15:35:15 bswartz and I were discussing this week about an approach of creating an additional share instance in migration, similar to what the fallback approach does. Conceptually I think it is not accurate because I believe that for some vendors there may be only one instance at all times 15:35:32 ^ this is for driver optimized migration, forgot to mention 15:35:39 I have always been against the plan to do driver-optimized migration without creating a second share instance 15:36:03 bswartz: +1, I am surprised it is not so 15:36:16 ganso: share instances were designed for it 15:36:27 I would like to input from other vendors present here if there are in fact two instances in every migration, and that the source instance can be restored to at any time 15:36:30 regardless of what the driver does internally, I think the share API and manager should still create a new share instance for the "destination" 15:36:59 ganso: ZFSonLinux will have two 15:37:25 ganso: so, it "should" have two share instances 15:37:45 so for netapp, we have a technology that lets us move a flexvol (share) from one aggr (pool) to another by copying the data internally to our array 15:38:29 bswartz: you are planning to implement driver-optimized migration in NetApp driver in Newton, right? 15:38:30 however its no problem for us to simply rename the flexvol to match the new share instance ID after it's moved, or to store a mapping from the new ID to the old name, if we choose not to rename i 15:38:34 rename it* 15:38:51 xyang bswartz: hacking on that right now.. 15:38:56 xyang: I don't work on the netapp driver so I don't know what the plans are 15:39:10 gouthamr, bswartz: ok, thanks 15:39:15 I just know how I would do it if I was 15:39:16 bswartz: if moving fails after some data is moved, can you restore the original? 15:39:25 though, no, no official plans on getting it done. i just want to verify that we can support it 15:39:37 one of the advantages in the fallback approach is that up to the last step in migration, you have the source instance intact 15:39:57 gouthamr: do you know the answer to ganso's question? 15:40:07 * gouthamr reading up 15:40:10 when using vol move, can you abort and go back to the pre-migration state? 15:40:36 ganso: i think it makes sense in giving the driver two instances.. 15:40:55 I'm concerned the admin may indeed be confused on what to do if he sees two instances when the driver is migrating, and believes he can delete one of them to fix it up 15:41:11 ganso: i did think of updating export_locations and all that, so it may just work for the NetApp driver to have just one migration instance 15:41:37 ganso: s/migration instance/share instance 15:41:57 ganso: couldn't we solve that by having the driver implement a check if we got a deletion request for a share instance that was the source or destination of an actively migrating share? 15:42:22 yeah, you have the TASK_STATE don't you? 15:42:24 bswartz: that is an unexpected snceario 15:42:31 ganso: when you do "mv foo bar" you have two while operation not completed 15:42:39 ganso: are you supposed to delete the original one after migration is complete? 15:42:40 ganso: you make it sound like it's valid for the admin to do that 15:42:48 ganso; so, admin won't nr confused 15:42:51 xyang: yes 15:42:51 ganso: if we don't want to allow the admin to do that then we can lock it out at the API layer 15:42:58 bswartz: +1 15:43:49 either it's okay for the admin to meddle with share instances that are part of a migration, in which case the driver needs to handle it, or it's not okay, in which case the API service should prevent it 15:44:04 I am planning on manila itself cleaning the new instance if migration fails 15:44:46 manager code will create the new instance db entry, and feed the driver with it as "migrating_share", so driver interface will have "source_share" and "migrating_share" 15:45:03 if optimized migration fails, we should clean it up 15:45:22 why would we clean up in that case when we don't clean up in other cases? 15:45:23 this should not be a dangerous operation at all, so the source_instance should be clean 15:45:39 when we had only one instance, we always expect data to be intact whenever it fails 15:45:46 it's sometimes better to leave an object in an error state so the admin can investigate and figure out what went wrong 15:46:00 ganso: i think we can leave it behind if it fails.. administrator can be told to worry about that instance.. 15:46:16 bswartz: we can leave the new instance in error state, but data has to be intact 15:46:41 yes I agree 15:46:59 in fact the source instance doesn't need to remain in an error state 15:47:12 bswartz: yes, that's what I expect 15:47:12 the source instance could go back to a normal state in a failed migration 15:47:34 but the destination instance should hang around in whatever state it was in when we failed 15:47:44 so the admin can go look at what went wrong 15:47:53 ok so, I will leave the new instance in error state 15:48:11 there is a second topic that wasn't brought up before 15:48:28 so we should tell admins to check for an accumulation of errored instances? 15:48:51 markstur_: we would document this 15:48:54 I am considering now, after talking with Valeriy, that we want to allow share_network_id to be changed in a migration, if it is specified in API parameter 15:49:22 migrate across share networks? 15:49:24 markstur_: yeah I think there needs to be a way for the admin to find those 15:49:35 gouthamr: yes 15:49:36 ganso: do we report migration fail in this case? the original share becomes available, admin may think things are wroking fine 15:49:37 gouthamr: it similar to the force-retype 15:49:58 xyang: we have the field task_state that would show the value "migration_error" in the main share model 15:50:05 ganso: the share network would only matter if there was a share server though 15:50:13 bswartz: yes 15:50:26 bswartz: this is another thing that is going to be taken care of 15:50:28 how to you migrate to a share server? 15:50:39 oh a new share server would get created 15:50:59 or an existing one would get reused 15:51:04 bswartz: there will be a new method that validates driver compatibility to perform optimized migration, if driver agrees, and check if there is a share server in the destination host + share_network, 15:51:05 * bswartz considers 15:51:24 if there isn't, we invoke setup_server in the destination host 15:51:51 okay but I can't think of a case where migration would fail if we don't allow changing the share network 15:52:08 bswartz: when you create a share, you tie it to a share_network 15:52:16 bswartz: that share_network may not span across other AZs 15:52:23 the problem here is that an admin-triggered migration should be unknown to the user (aside from any outages) 15:52:37 changing the share network would definitely be visible to the end user 15:52:39 bswartz: the host the admin chooses to migrate the share to may be in another AZ 15:52:45 bswartz: your question like "if I created new share instance, can we reused share server" ? right? 15:53:08 ganso: so, if there is no share server, you create a new one, today.. don't you? 15:53:17 gouthamr: fallback does that, easily 15:53:22 DaiDV: share server belongs to host, it always will be new share server 15:53:22 we need to add new user-visible change-AZ and change-share-network APIs that let the user change those things and trigger migrations on the backend if needed 15:53:34 gouthamr: but within driver.migration_start, the driver cannot do that 15:53:45 bswartz: s/ created /create 15:54:01 bswartz: but wouldn't then migration become a user feature? 15:54:08 * bswartz thinks more 15:54:19 okay I see it now 15:54:22 bswartz: then migration becomes tenant action? 15:54:30 xyang: yea, it would 15:54:43 yes the force-retype style of admin-migration would cause a user-visible change too, and for good reason 15:54:47 my concern is, there are several variables that bind a share and prevent it from being moved 15:54:55 therefore there's no reason not to do something similar for AZs or share networks 15:55:10 okay I'm on board 15:55:33 to answer xyang's question, migration won't ever be a tenant action, because tenants don't know about backends, but... 15:55:51 there is an alternative 15:56:01 ...tenant should be able to share-retype, share-change-az, and share-change-share-network, which might trigger migrations indirectly 15:56:40 allow only fallback migration to do this kind of stuff 15:56:49 oh and don't forget everyone's favorite: share-change-protocol 15:56:54 bswartz: lol 15:57:05 :D 15:57:10 :-( 15:57:20 :) 15:57:24 not cknight's favorite :) 15:57:27 vponomaryov: btw, why not we shared share server for two share instances? 15:58:09 ganso: One more question about migration… 15:58:12 DaiDV: it will mean we refer to single object twice 15:58:13 ganso: we can certainly limit driver-assisted-migration to as small a subset of use cases as we want 15:58:46 ganso: If I'm running a single-vendor Manila cloud, and that vendor can do optimized migration, is there a way to disable the fallback migration entirely? 15:59:01 cknight: yes, latest patch includes that config option 15:59:09 cknight: it is not merged yet 15:59:12 ganso: great, thanks. 15:59:17 cknight: I think we agreed that it should be disable-able on a request by request basis 15:59:38 the user who calls the API should be allowed to veto fallback migrations 15:59:43 bswartz: such an admin might want to know that "lossy" migration will never occur in his cloud 15:59:56 time check 16:00:08 I don't see why the admin would care -- the it's the user's data not her's 16:00:20 bswartz: he does that by default, lossy default value is false, preventing fallback migration 16:00:23 fuel was canceled 16:00:23 the owner of the data should make the decision 16:00:28 so go ahead 16:00:39 xarses: ty 16:01:26 well I don't want to keep you all late 16:01:31 bswartz: I guess I am done with my topic, unless there are more questions 16:01:37 topic to possibly continued at midcycle 16:01:39 ... 16:01:43 thanks everyone 16:01:46 bye 16:01:52 #endmeeting