15:00:46 #startmeeting manila 15:00:47 Meeting started Thu Dec 8 15:00:46 2016 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:48 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:50 The meeting name has been set to 'manila' 15:00:52 hello all 15:00:55 hello 15:00:57 hello 15:00:57 hi 15:00:57 \o 15:01:00 hi 15:01:01 hi 15:01:08 hello o/ 15:01:18 hello 15:01:34 cknight xyang1: courtesy ping 15:01:45 hi 15:02:17 hi 15:02:22 okay we have a packed agenda today so I won't waste time 15:02:27 * gouthamr looks at agenda, clears desk. 15:02:28 #agenda https://wiki.openstack.org/wiki/Manila/Meetings 15:02:34 #topic announcements 15:02:41 Ocata-2 is next week! 15:02:50 :( 15:02:58 yeah time is flying by 15:03:14 not much we can do but try to focus on priorities 15:03:38 Ocata-2 is the merge deadline for high priority specs 15:03:53 and also we hope to see significant progress on implementation for high priority specs by then 15:04:10 also, Monday is the new driver proposal deadline 15:04:28 bswartz: 12/19? 15:04:32 bswartz: monday dec 19th, correct? 15:04:42 * bswartz looks confused 15:04:48 did I get the date wrong? 15:04:59 bswartz: yes, it is 12/19 on the wiki 15:05:00 bswartz: this next monday is dec 12th 15:05:07 #link https://releases.openstack.org/ocata/schedule.html 15:05:14 you're right! 15:05:26 sorry, driver proposal freeze is 2 mondays away 15:05:36 12/19 is the correct date 15:05:47 #topic PTG 15:06:06 so I need a show of hands for who has registered for the PTG and plans to come 15:06:27 I have registered for it 15:06:29 bswartz: I registered. 15:06:34 registered and requested 15:06:38 I'm pretty sure we won't get any time/space in Boston, so Atlanta is all we'll get for Pike 15:06:51 bswartz: I am waiting for travel support response 15:06:51 not registered 15:06:55 bswartz: I've registered 15:07:13 ganso? mkoderer? 15:07:17 I registerd, plan to come 15:07:18 bswartz: ^ 15:07:23 oh missed your reply 15:07:58 anyone definitely not able to come? 15:08:15 zhongjun 15:08:24 wait, when will it be? 15:08:33 concrete dates? 15:08:43 vponomaryov: R+0 15:08:49 Feb 20-24 15:08:59 ok 15:09:15 vponomaryov: you should register -- we'll talk later 15:09:24 sure 15:09:38 bswartz: register = pay the entry ticket, correct? 15:09:41 for those of you who may/may not come, tickets are limited so try to get that decision made soon 15:10:30 and if you think it's likely just go ahead and get a ticket, they're only $100 15:10:30 tickets are $100 but I think it says they are refundable (not sure how to un-reimburse) 15:11:10 I will be standing outside selling them for $300 if you register late 15:11:21 tbarron: heh 15:11:35 #topic Default share type 15:11:39 lol 15:11:42 tbarron: You're up 15:11:49 now that we have customers we discovered that people deploying manila don't always do what devstack does - in particular 15:11:49 one can deploy manila without defining an default share type in manila.conf 15:11:49 if one is willing to have users always specify a share type when they create shares 15:11:49 this doesn't sit well with manila traditionalits :) 15:12:06 traditionalists 15:12:25 this issue goes back to the time when we decided to add DHSS extra spec and make it non-optional 15:12:33 so do we want to somehow insist that a default share type always be defined? 15:12:40 the install guide asks for you to create one as part of the installation 15:12:49 because the value MUST be set to true or false there must exist a share type 15:13:18 actually I got that backwards 15:13:20 tbarron: what is the use case to force users specify one? 15:13:37 there must exist a share type no matter what -- all shares have types 15:13:42 the share type must exist in order to create a share, but a default doesn't have to exist 15:13:50 however because the API doesn't require you to specify one, there must exist a default one 15:14:08 and manila can't magically create a default one for you because manila doesn't know if you want DHSS=true or false 15:14:25 it could just throw an error instead 15:14:56 vponomaryov: the use case would be if a cloud operator has both DHSS=True and False backends and wants all creates to be explicit 15:14:57 markstur: that not too different from what we do 15:15:00 if shares cannot be created without a share type, one will need to be created eventually... so can't we just assigned the oldest one created as default if nothing is set in manila.conf? 15:15:13 ganso: again that's basically what we do 15:15:34 do we? 15:15:38 nope 15:15:43 if default isn't set, it isn't set 15:15:45 I think tbarron is suggesting that somehow the admin shouldn't have to do ANYTHING and it should still just work 15:16:02 tbarron: am I wrong? 15:16:28 I guess, we should just require it in UI, that's all 15:16:35 and leave server side as is 15:16:44 bswartz: we don't magically pick the last defined share-type as default - maybe devstack does that in setup if that's what you mena 15:16:45 mean 15:17:02 vponomaryov: that doesn't address the problem when there are no share types in the datagbase 15:17:05 tbarron, bswartz: then maybe we could do it to solve the immediate issue 15:17:08 I think UI should auto-select the default if there is a default and otherwise make you pick one (or change it) 15:17:22 Is the "default_share_type" config value listed as a required option in manila.conf? 15:17:27 bswartz: when there is no such, then it is not completed setup 15:17:38 vponomaryov: that's what I told tbarron 15:17:53 dustins: manila.conf sets it to None 15:18:14 sample manila.conf shows it None, which leads some to think None is OK 15:18:17 bswartz; in case of tbarron's situation there are share types 15:18:18 tbarron would you be okay with a solution that picks an arbitrary share type as the default if the config file doesn't specify one? 15:18:29 dustins: chicken and the egg problem there.. you need manila API service to run before you can create the default share type 15:19:04 gouthamr: it is not chiken-egg problem, one service creates other uses 15:19:16 So the very first thing a Manila admin should do is to create the default share type and then add it to the Manila configuration? 15:19:19 strict linear dependency 15:19:22 bswartz: I really don't know the right answer here, I just ran into the problem (well, Dustin ran me into the problem) 15:19:24 (and then restart manila)? 15:19:27 vponomaryov: if you require something in the DEFAULT section, how can the manila-api service start? 15:19:35 dustins: actually we expect that deployment tools do that for you 15:19:56 gouthamr: it seems we don't actually require 15:20:05 bswartz: Which is why DevStack does it, yeah? 15:20:11 ganso: yes.. we can't require ... 15:20:11 if cust has types (dhss options for example) and doesn't want a default, but wants to explicitly choose for each share -- that could be a good feature 15:20:12 gouthamr: but it will become chicken-egg if we arbitrary select 15:20:22 gouthamr: easy, you define config first start service and then just create share type with name you already set up in config 15:20:22 what makes this problem thorny is that we made a decision to punt this issue to the deployment tools, but the only such tool that existed at the time was devstack 15:20:31 gouthamr: unless, in this scenario, as there will not be any share types in the first run, we don't select 15:20:34 vponomaryov: :P yeah, like we do in devstack 15:20:48 so we weren't able to tell redhat/suse/mirantis that this is something they needed to deal with in their tools 15:21:04 markstur: yeah, I think that's a valid line of reasoning as we have left things. We might want to close it off, but as things are currently ... 15:21:10 this decision was part of kilo 15:21:24 and ideally the default would only be needed and used if share-type is omitted or as a UI pre-select 15:21:30 vponomaryov: actually, the installation guide suggests the exact thing.. configure the default type, start services and create it 15:21:54 okay so hopefully we all understand the status quo now 15:22:03 the question is can we do anything to make it better 15:22:14 gouthamr: how can configure it before starting the service? 15:22:30 if somehow the admin forgets or the deployment tool doesn't create a default, should we still barf with an error when a user creates a share? 15:22:58 or could we do something magical like create a share type out of thin air and guess the value of DHSS? 15:23:05 only if the user creates a share w/o specifying a share-type 15:23:08 ganso: set ``default_share_type=xyzzy``, start manila.. create the share type xyzzy 15:23:13 ganso: it's just a string in the config file 15:23:37 So I suppose we could create a default default 15:23:39 gouthamr: oh ok, if that does not lead to errors like not finding the share type xyzzy, then ok 15:23:41 not a required config opt, for a reason.. 15:23:45 can we magically create a default share type if it's set in the config file? 15:23:48 ganso: yes, we don't check 15:24:05 but expecting admin to pick one or require explicit would be an option 15:24:08 tbarron: we could, but what about the value of DHSS? 15:24:41 bswartz: we'd have to have config for what that would be too 15:24:57 tbarron: that was my original proposal during kilo! but it was shot down 15:25:00 then we could faiil startup if those aren't in the config file 15:25:08 tbarron: creating the default share type is a one time setup activity, no? 15:25:26 tbarron: we gonna have config for config 15:25:28 gouthamr: yes, I just don't want the support calls when people don't do it. 15:25:35 we believed it was better to no add multiple config options, and to leave it to the deployment tools, because they ought to know this stuff 15:25:58 tbarron: we documented it is necessary... 15:26:17 so I'd like to wrap up this topic and move on 15:26:20 bswartz: and that may be the outcome of this discussion, but as you said in kilo the "deployment tools" were just devstack so this directive didn't get fully communicated 15:26:22 what can we do as a next step? 15:27:03 gouthamr: probably our deployment didn't read the doc, guess we'll need to fix that 15:27:08 maybe we can log a warning saying the default share type doesn't exist 15:27:08 would that help? 15:27:10 but we 15:27:10 tbarron: I take responsibility for the communication failure -- I was even involved with the redhat deployment tool work and I failed to mention this requirement 15:27:42 but we've pushed the hard problem of deciding *what* to set as the default downstream 15:27:42 it's easy to take it for granted when devstack does it for you 15:27:59 tbarron: it's not hard if you're also configuring one or more drivers 15:27:59 devstack sets up a very contrained environment 15:28:21 it's hard if you are configuring more than one and they vary on DHSS setting 15:28:34 otherwise we would have just decided the issue upstream 15:28:41 the point is that the values for the default share type should be informed by the driver configuration 15:28:55 tbarron: what is wrong with multi-mode setup? 15:29:27 vponomaryov: customer has two backends, one DHSS=true, one DHSS=false, what shoudl default share type be? 15:29:38 tbarron: we can't have multiple defaults... 15:29:39 tbarron: so, just decide 15:29:42 should be none. Require selection 15:29:42 tbarron: in that case the admin should decide 15:29:45 tbarron: so, they'd have to decide 15:29:45 tbarron: and make one 15:29:45 throw a coin 15:29:54 if we have to decide a coin flip would be the best way 15:29:54 I have to say half an hour passed 15:30:24 if there is no default it just needs to be chosen for each create share 15:30:34 markstur: +1 15:30:38 we could leave it to the scheduler 15:30:39 tbarron: http://docs.openstack.org/project-install-guide/shared-file-systems/draft/install-share-rdo.html -> maybe we need to change this.. and add a section on default share types 15:30:41 tbarron: is there any change you would like to see upstream on this? or are you just trying to figure out what changes need to be made downstream? 15:30:59 the scheduler knows which backends are there, and which modes they are operating 15:31:02 let's close this out for today 15:31:06 ganso: no there's a reason that won't work 15:31:24 the API service needs to throw an error if the share network is not present and the share type is DHSS=true 15:31:34 and also it needs to throw and error in the reverse case 15:31:35 bswartz: oh right 15:31:53 okay let's move on 15:32:06 I suspect something better could be done but I'm not sure what 15:32:12 #topic migration-start new command syntax 15:32:16 ganso: you're up 15:32:25 ok so the spec merged without deciding on the new migration-start CLI syntax 15:32:31 I have coded a few possibilities 15:32:41 and I would like to present this so we waste no time 15:32:57 1) All parameters fixed positional 15:32:59 ganso: which spec? limk? 15:33:06 manila migration-start share_1 ubuntu@generic1#GENERIC1 True True True True 15:33:14 :( 15:33:15 2) All parameters with optional syntax, any order 15:33:20 #link https://github.com/openstack/manila-specs/blob/master/specs/ocata/ocata-migration-improvements.rst 15:33:26 manila migration-start share_1 ubuntu@generic1#GENERIC1 --writable True --nondisruptive False --preserve-metadata True --preserve-snapshots False 15:33:32 3) key=value style, comma separated, any order 15:33:40 manila migration-start share_1 ubuntu@generic1#GENERIC1 writable=True, preserve-metadata=True,preserve-snapshots=True,nondisruptive=True 15:33:50 4) key=value style, space separated, any order 15:33:59 manila migration-start share_1 ubuntu@generic1#GENERIC1 writable=True preserve_metadata=True preserve_snapshots=True nondisruptive=True 15:34:39 ganso: are we voting? 15:34:46 IMO (2) 15:34:46 gouthamr: I suggest we do 15:34:59 no don't vote until we make arguments 15:35:08 (2) +1 15:35:10 bswartz: #2 feels like the parameters are optional, but they are not 15:35:10 (1) is hard to understand 15:35:11 nicelist 15:35:16 but valud 15:35:17 I like #2 15:35:18 #3 and #4 are positional 15:35:30 (3) and (4) would be a significant variation from existing CLI style 15:35:38 (1) is very unclear 15:35:40 i think 2 is best because it is most like the API style we are using 15:35:46 Having the explicit options does help 15:35:56 It's easy to make mistakes with #1 15:35:57 bswartz: #3 and #4 are similar to how extra-specs are used 15:35:59 ganso: CG commands have required args be specified in "optional" style 15:36:10 1 - is the most fun to read. #trutrutrutru #butitisjibberish 15:36:12 the only downside to (2) is that it's verbose as hell, but that's no different than the rest of openstack 15:36:28 also, there's nova boot --nic net-id=xxx,... 15:36:49 variant (2) making named options required 15:37:01 and requiring them on server side too 15:37:10 Can we do #2 with shortcut -w 1 -d 0 -m1 -s 0 or is that too cryptic picking letters? 15:37:16 ganso: any idea what nova CLI does with those comma-separated stings? does it ship them as a single json string or does it break them up and send separate json? 15:37:25 markstur: cryptic 15:37:26 I mean the shortcut is an alias not a replacement for long names 15:37:38 like -v or --verbose 15:37:46 bswartz: I read the code, it parses and separates the commands 15:37:51 bswartz: I already do it in my code as well 15:37:51 because people want verbose, but don't want to be verbose 15:38:08 what would osc do? 15:38:10 i wonder what the openstack-sdk guys prefer for this 15:38:13 hah 15:38:23 ganso: and in the shift to opestack client, did they keep that gross syntax or did they clean it up? 15:38:30 tbarron: WWOD <-- is that a thing now? 15:38:39 bswartz: regarding openstack client I have no idea 15:38:47 ganso: so, your statement in variant 2 is a bit false, because "named arg != optional arg" 15:38:50 it's a joke but the serious point is that integrating with user expectations across projects is desirable 15:39:06 okay I don't see any argument against (2) 15:39:08 +1 15:39:12 so this feels decided 15:39:14 anything else? 15:39:15 vponomaryov: syntax is similar to optional 15:39:25 agree with tbarron let's make it osc ready 15:39:30 ganso: "proposal" shoudl be strict 15:39:56 ok #2 is decided 15:40:02 ganso: we already agreed that these parameters will have no defaults -- that was requested by you for safety reasons 15:40:19 bswartz: yes, in the proposed code there are no defaults 15:40:23 tbarron: +1 for OSC "compliance" 15:40:36 so users will have to type out all that crap no matter what 15:40:45 or use a GUI 15:41:01 okay moving on 15:41:09 #topic remove access-update fallback to access_allow/deny 15:41:13 tbarron: back to you 15:41:42 sorry, was pinged ... back 15:41:59 ganso raised the question in a code review whether 15:42:23 we'll be removing the fallback in the manager from access-update to the old allow/deny methods 15:42:29 so a few drivers haven't gotten around to implementing the new method 15:42:42 soon, either as a part of gouthamr work, or independently 15:42:47 xyang1 mentioned at least one EMC driver is in that boat 15:42:54 we decided at the contributor's meetup that fallback will be removed, and drivers that don't implement update_access will be tagged as deprecated with their CIs failing 15:43:05 GPFS is in that boat, but will fix that leak soon 15:43:24 so I talked to the Isilon team about this. because Ocata is so short, they couldn't get it done in Ocata. They can get it implemented in Pike 15:43:35 markstur: you mean abandon that ship 15:43:44 so my view is that this kind of cleanup should have a deadline, but that it should be after feature freeze 15:43:48 this is tech debt 15:44:00 i think we decided in the summit to round up the remaining stragglers and warn them and get it done -- (or was that a different topic?) 15:44:14 deadline for this was newton, we weren't very strict about this 15:44:19 ganso: if you mark something as deprecated in O, then it should be removed in P 15:44:21 well the other thing we agreed to in the contributor meetup is that throwing out drivers is a user-unfriendly way to deal with unmaintained drivers 15:44:22 right now we should concentrate on other stuff 15:44:26 otherwise the actual deadline was probably a release or 2 ago 15:44:30 There are many drivers in that boat 15:44:32 #link https://etherpad.openstack.org/p/ocata-manila-access-rules 15:44:35 xyang1: we decided at contributor's meetup that we would not remove drivers anymore 15:44:39 xyang1: even deprecated drivers 15:44:52 we need another mechanism to deal with unmaintained drivers and whatever that is should be used in this case 15:44:57 ganso: I mean deprecate allow access and deny access 15:45:24 I'm okay with just breaking the drivers that haven't updated yet, but only if we have a good way to communicate with end users about it 15:45:28 xyang1: then indeed it should have been removed in newton 15:45:32 deprecate is usually a warning that something will go away but it isn't necessarily a deadline 15:45:37 xyang1: allow/deny was deprecated in mitaka 15:45:49 ok 15:46:03 bswartz: +1 on end users, it's they who should be prioritized 15:46:04 I want it to be easy to find out what the status of all the drivers is w.r.t. how up to date they are and whether they're continually tested 15:46:05 the deadline has been communicated but we either have communication issues or just schedule/resource issues. 15:46:34 it's okay to have drivers in tree that are broken if they have some value to real users even in their broken state 15:46:59 bswartz: they will lose their value if allow/deny cannot be used 15:47:01 but we need to make sure that users don't accidentally try to use a broken driver expecting it to work well 15:47:02 bswartz: Yes. bigger picture a grading-scheme or whatever to report what is known-good/bad would be a good answer 15:47:02 ganso: Is the deprecated message printed out? 15:47:14 no 15:47:18 xyang1: 15:47:21 xyang1: no 15:47:41 ganso: it should be in logs, or written somewhere 15:47:41 xyang1: we failed at creating the deprecation message :\ 15:47:53 ganso: so it is not deprecated 15:47:58 ganso:( 15:48:14 ganso: also now we have release notes. it should be in release notes now 15:48:16 xyang1: nice trick )) 15:48:17 we have a deprecation comment in code 15:48:26 xyang1: deprecation messages are typically for config opts or something admin facing 15:48:27 vponomaryov::) 15:48:30 so the goal posts are deprecation keep moving 15:48:31 ganso: not good enough for support 15:48:36 tbarron: indeed 15:48:45 gouthamr: this needs it too 15:48:47 xyang1: unless we want the admins to call developers, that would be a bad way to deprecate driver interfaces 15:48:48 :P 15:48:55 when we decided to do this, all that was required was a ML post 15:49:14 I'm hearing argument for a little more time 15:49:15 gouthamr: I wouldn't mind they call developers to get their attention:) 15:49:15 the world has changed, there are customers 15:49:33 anyway, I think we need better document 15:49:41 if someone would like to add log spam and release notes about this issue and wait one more release, that's okay with me 15:49:47 if update_access is required, it should be documented 15:49:52 not just in meeting notes 15:50:16 however no matter how long we wait, something will come crying when we actually pull the trigger 15:50:40 bswartz: do you know what other drivers that have not implemented this? 15:50:53 xyang1: they were listed in that etherpad 15:51:08 EMC VNX 15:51:08 EMC Unity 15:51:10 EMC Isilon 15:51:10 HDFS Native 15:51:12 IBM GPFS 15:51:12 Oracle ZFSSA 15:51:15 * bswartz doesn't have the link 15:51:22 some of which are in review 15:51:27 You can take VNX and Unity out 15:51:28 #link: https://etherpad.openstack.org/p/ocata-manila-access-rules 15:51:36 gouthamr: ty 15:51:57 did we have a volunteer to do hdfs? 15:52:22 markstur: no 15:52:32 ideally this kind of thing will be easier to discern if reliable CI exists, because tests will simply start failing 15:52:56 hdfs is listed as unmaintained on my list 15:53:34 redhat has kindly implemented some fixes for it, but hasn't offered to take it over 15:53:44 Is someone from Oracle working on it? 15:53:45 okay let's try to squeeze in another topic 15:53:54 #topic decision of deadline for driver config docs in devref removal 15:53:59 ganso: back to you 15:54:16 ok so, last meeting we decided to remove all driver config-related docs from devref 15:54:26 yes 15:54:41 but as tbarron pointed out, it is bad user experience for first party drivers if that is removed and it is not yet somewhere else 15:54:54 and tbarron came up with a deadline strategy 15:55:10 ganso: it's not hard to put those somewhere else is it? 15:55:18 so, we would like to establish a deadline, and then remove all the config-related driver docs from devref 15:55:24 I think it'll look weird if only 1st-party drivers are in the doc but we'll see 15:55:37 ganso/tbarron: do you have a list of drivers which need docs work in the config ref? 15:55:42 markstur: indeed, that's not what we are suggesting 15:55:47 yeah ok 15:56:02 bswartz: I don't have a list 15:56:20 * markstur thinks either he needs to adjust his highlight settings or ganzo was yelling at him 15:56:33 bswartz: I'd say that we have a deadline and remove them when the deadline is met regardless 15:56:35 ganzo ^_^ 15:57:06 I haven't had time to audit them all, but there's stuff in zfs-linux, ganesha, cephfs, generic, at least that is non-config I think 15:57:35 oh yea, there's that... there are stuff related to admin-ref, user-guide, etc 15:57:37 tbarron: yes, generic one has list of requirements for image 15:57:46 but there is no devref stuff 15:58:03 devstack is not even mentioned in these docs 15:58:07 My thought is that a deadline could be set for the latter part of ocata, and re-asserted around feature freeze. We have other stuff to work on now. 15:58:19 sorry my phone rang 15:58:35 ganso: are you suggesting we block the devref removal patch? 15:58:45 ganso: yeah, but we *should* have devstack related stuff in devref for these 15:58:49 ganso: we need to make the docs more devstack-y.. i can help rewrite after feature freeze 15:58:52 bswartz: temporarily, yes, until that deadline we establish 15:58:54 tbarron: The problem with doc deadlines is that while we can work on that stuff late, we are at the mercy of doc folks to actually merge it 15:59:11 markstur: true that 15:59:12 ganso: suppose we set the deadline to O-3, then what? we go ahead and merge your change? 15:59:13 gouthamr: still, content that is there is not at all for devs 15:59:18 gouthamr: at this moment at least 15:59:32 removing our devref manila-controlled stuff before we are happyish with the official doc stuff is a problem 15:59:34 ganso: the docs exist in stable branches and in git history if people really want to find them 15:59:37 gouthamr: so we would need new instructions for devs, but still move the existing ones to admin-ref and config-ref 15:59:50 I say as long as we get all this done before stable/ocata is cut there is no problem 16:00:13 ganso: agree.. dustins and i wanted to audit it one point of time.. that work just got de-prioritized with our schedules 16:00:29 yeah... :( 16:00:49 can we revisit at/after PTG? 16:00:57 time's up! 16:00:58 gouthamr: work schedules or life? 16:01:05 back to manila irc 16:01:08 i still believe we don't think we should chuck anything before we move the docs 16:01:08 sorry we didn't make it through the whole agenda 16:01:08 markstur: both 16:01:32 we'll carry over topics to next week or deal with them before next week if necessary 16:01:34 #endmeeting