12:00:17 #startmeeting nova api 12:00:18 Meeting started Tue Aug 25 12:00:17 2015 UTC and is due to finish in 60 minutes. The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:00:19 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 12:00:22 The meeting name has been set to 'nova_api' 12:00:39 hello, who is here today? 12:00:40 o/ 12:00:52 o/ 12:00:54 hi 12:00:59 hi 12:01:04 oomichi: hey, long time no see :) 12:01:23 yeah :) 12:02:12 not sure johnthetubaguy and sdague will join the meeting 12:02:25 o/ 12:02:32 o/ 12:02:37 ok, let's run the meeting 12:03:00 #topic Actions from last meeting 12:03:30 one for sdague: check sdague his action in next week meeting 12:03:57 yeah, I remember he was out for two meetings, I guess this is the second one he is out for 12:04:18 but sdague sent email 12:04:21 #link https://github.com/stackforge/gerrit-dash-creator/blob/master/dashboards/nova-api.dash 12:04:33 he add create dashboard for api patches 12:04:40 yea 12:04:47 I generated one 12:04:49 #link 12:04:51 https://review.openstack.org/#/dashboard/?foreach=%28project%3Aopenstack%2Fnova+OR+project%3Aopenstack%2Fpython%252Dnovaclient+OR+project%3Aopenstack%2Fnova%252Dspecs%29+%28%28file%3A%5E.%2Anova%2Fapi.%2A+OR+file%3A%5E.%2Aapi_samples.%2A%29+OR+message%3Aapiimpact%29+status%3Aopen+NOT+owner%3Aself+NOT+label%3AWorkflow%3C%3D%252D1+label%3AVerified%3E%3D1%252cjenkins+NOT+label%3ACode%252DReview%3E%3D% 12:04:53 252D2%252cself+branch%3Amaster&title=Nova+API&Proposed+API+changing+Specs=project%3Aopenstack%2Fnova%252Dspecs+NOT+label%3ACode%252DReview%3E%3D%252D2%252cself&Needs+Feedback+%28Changes+older+than+5+days+that+have+not+been+reviewed+by+anyone%29=NOT+label%3ACode%252DReview%3C%3D2+age%3A5d&Your+are+a+reviewer%252c+but+haven%27t+voted+in+the+current+revision=reviewer%3Aself&Needs+final+%2B2=label%3ACo 12:04:55 de%252DReview%3E%3D2&Passed+Jenkins%252c+No+Negative+Feedback=NOT+label%3ACode%252DReview%3E%3D2+NOT+label%3ACode%252DReview%3C%3D%252D1+limit%3A50&Down+voted+changes=label%3ACode%252DReview%3C%3D%252D1 12:04:57 oops... 12:05:09 max line length FTL 12:05:17 OK, whats the dashboard trying to include? 12:05:21 anyway I tried, it's great 12:05:37 ah, I see now, the link tells me 12:05:50 \o 12:05:54 * bauzas waves late 12:06:01 johnthetubaguy: yea, let say everything include 12:06:07 bauzas: welcome 12:06:12 (lurking mostly) 12:06:17 another action is for me and gmann_ 12:06:34 gmann_ and alex_xu take a look at tempest failure of https://review.openstack.org/#/c/214085/3 12:06:36 morning folks. 12:06:54 sdague: morning 12:06:56 stupid jetlag from west coast, sorry I'm late 12:06:59 this is related to next topic 12:07:07 sdague: morning! 12:07:10 sdague: eh I'm not alone \o/ 12:07:19 let's jump to next topic directly 12:07:39 #topic v2.0 on v2.1 12:08:11 #link https://review.openstack.org/#/c/214085/ 12:08:20 johnthetubaguy: thanks 12:08:25 so I guess the api-paste.ini changes are the big one here 12:08:36 johnthetubaguy: yeh, I was wondering where those all stood 12:08:39 well, actually not really, just the test failures they highlight 12:09:02 #link https://review.openstack.org/215436 12:09:12 we try to fix that failure in above patch 12:09:14 it looks like 1 tempest test failed that flip 12:09:20 sdague: so we have a bug in v2.1's legacy mode , this deals with some of that: https://review.openstack.org/#/c/215436 12:09:42 we plan to strip the extra parameter out 12:09:55 that is different with our original spec saying 12:09:55 so relax broke patternProperties, but alex_xu has fixed that now :) 12:10:03 sdague: need for extra param strip out, then tempest is fine 12:10:11 yeah, the next bit is do we strip out the bad properties 12:10:30 gmann_: we still need skip the host update testcase for compatible mode I guess? 12:10:39 gmann_: well we still have a tempest test failure, but its unexpected 2xx rather than 5xx, when 4xx was expected 12:10:45 I thought that if they are requesting v2 that we leave in the extra props? 12:10:58 edleafe: they were, but it creates big problems for the API 12:10:59 johnthetubaguy: ahhh right. 12:10:59 Because some might be relying on them 12:11:13 ok, https://review.openstack.org/#/c/215436 seems mostly good, though I agree with johnthetubaguy's -1 comments there. If we can get that respun that would be great 12:11:13 one point I want to metion, the host update with extra parameter return 400 before, with strip out extra params, the api will return 2xx, hope this ok for everyone 12:11:22 edleafe: I am voting we strip them, and add back in the ones folks need (I think scheduler hints) 12:11:30 there is still a hosts API failure on johnthetubaguy's patch 12:11:42 I can look into that one today 12:11:51 sdague: so that the issue we are discussing right now 12:12:11 johnthetubaguy: how do we know which ones are needed? 12:12:13 sdague: to strip or not to strip the extra properties, although either way we get a tempest failure right now 12:12:38 johnthetubaguy: sdague that will be difficult to handle on tempest side. different expected code for v2.1 and v2onv2.1 12:12:47 johnthetubaguy: agreed with you (re: hints) 12:13:05 edleafe: using master, we are OK, its was out of tree stuff that got broke, I believe, which makes me less sympathetic 12:13:11 johnthetubaguy: but I would like to see alaski yelling or accepting 12:13:11 gmann_: so is that just a bug? 12:13:24 something screwed up on the initial v2.1 ? 12:13:36 sdague: its an issue with how we are doing the compatiblity 12:13:43 ok 12:13:47 v2 actually had some validation, tempest tests some of it 12:13:56 now v2.1 compat mode, drops that validation 12:14:11 yea, few api have some validation 12:14:29 sdague: not actually, fir v2 on v21 we skip the additional property validation so we need to do something for extra param which can cause 5xx error as v21 do not have python code valisdation as v2 has 12:14:49 we either let the prams through (causes 5xx errors, in this case), or we drop possible bad params (cases 2xx success by ignoring stuff), or we try add the odd validation from v2 back into v2.1 12:15:11 ok, I'll probably need to walk through the code once I'm more awake to fully understand. I trust you folks. 12:15:17 johnthetubaguy: yea, those are all cases :) 12:15:28 that means too relax validation for extra parameters on v2 comp api? 12:16:00 well, the thing is, we want bad requests to do no halm to the system 12:16:09 we want all requests that were accepted before to be accepted now 12:16:23 if we strip the params, we get the above two cases sorted 12:16:28 johnthetubaguy: yea, that's the goal 12:16:29 it feels like keeping the v2 validation around for this case 12:16:42 yea, because some extra param might having success cases on v2 12:16:42 even though it will be a little gross 12:16:51 sdague: so that would make us pass the tempest test, I just worry about the other stuff we missed 12:17:18 sdague: if we choice that way, we need check all the api 12:17:40 johnthetubaguy: sdague : poython code one might be little dificult as we do nto have tests for all extra param for other API 12:18:01 so there is another way, I guess, which is add a validation decorator that re-instates full validation for v2.1 compat mode, for certain APIs? 12:18:01 yea we need to walk all v2 code and put validation.. 12:18:09 sdague: v2 on v21 is probably gonna be a little gross :) 12:19:00 so... this is for all the APIs? 12:19:08 sorry, I only noticed a single failure here 12:19:14 so I was leaning towards, strip the params, and let some things that fail succeed, because we get the saftey of avoiding some bad 5xx 12:19:27 sdague: its more that we assume a lack of tempest coverage for the other APIs 12:19:27 sdague: yea, v2 might have lot of extra param validation in python code 12:19:36 ok 12:20:02 sdague: johnthetubaguy yea that was test with extra param in tempest 12:20:12 ok, right. 12:20:32 ok, so I think I'm with you johnthetubaguy then, stripping seems the next best option 12:20:36 we do not have other extra param cases cover under tempest 12:20:43 ok, cool 12:20:48 sdague: 12:20:51 +1 12:20:55 looks like we like this 12:20:58 sdague: I am tempted to also add extra validation for that call, to make tempest happy, but yeah, that seems the safer general appraoch 12:21:29 sdague: how to handle Tempest for v2onv21 where this test will pass 12:21:32 ok, let's move on, afraid we run out of time 12:21:33 (notes what he said is actually quite complicated to implement) 12:21:43 ok, before we move on, who's doing what? 12:21:58 and sorry test will fail in v2onv21 and pass on v2 and v21 12:22:04 who is going to rev - https://review.openstack.org/#/c/215436 to fix the comments. And who is going to do this additional patch? 12:22:13 probably need to skip the tempest test, or delete it 12:22:15 I will 12:22:20 alex_xu: great 12:22:24 who is taking the tempest test bit? 12:22:30 I can take the tempest bit 12:22:36 swett 12:22:37 oops 12:22:39 sweet 12:22:46 alex_xu: extra param stripout can be done on same patch 12:23:07 #action alex_xu to create updated validation patch which strips out extra params 12:23:19 did we agree the paste-api.ini changes? 12:23:23 its worth a review 12:23:24 sdague: thanks, I'm typing slow... 12:23:28 I drop /v3 12:23:31 #action sdague to address any additional tempest issues 12:23:56 #topic Test collapse of v2.0 and v2.1 12:23:57 johnthetubaguy: I much prefer the direction 12:24:00 johnthetubaguy: looks right 12:24:05 gmann_: you turn 12:24:12 johnthetubaguy: I thought we were going to drop v1.1 at the same time 12:24:15 alex_xu: thanks 12:24:19 sdague: oomichi: cool thanks, there is this bit too: https://review.openstack.org/#/c/214592/ 12:24:37 sdague: seemed easier to leave it there till we drop /v2 12:24:40 before that - v2.1 default on gate 12:24:59 sdague: regarding v21 deafult on gate 12:25:11 #link https://review.openstack.org/#/c/163718/ , https://review.openstack.org/#/c/207183/ 12:25:21 gmann_: my paste-api.ini changes mess up the tempest stuff I guess? we need the v21 job testing v2 instead, or something 12:25:33 sdague: can i merge these two patches to move forward? 12:25:53 sdague: i like your approach which makes catalog in much better shape 12:25:59 gmann_: so johnthetubaguy's changes will make it v2.1 on the gate 12:26:15 oh, not quite actually 12:26:19 and we still need job for legacy v2? 12:26:25 sdague: actually, thinking about it, it makes it the v2.1 v2 legacy mode by default 12:26:34 johnthetubaguy: yea not actually 12:26:46 alex_xu: yes, how about I take an action to sort out our eventual test matrix here 12:26:51 so we need three tests: v2_legacy, v2.1_compat and v2.1 I guess? 12:26:55 alex_xu: I feel we need to test legacy v2 now 12:26:58 sdague: that will cool 12:27:06 cools 12:27:10 sdague: it will have v2onv21 as deafult 12:27:20 #action sdague will sort out our eventual test matrix here 12:27:42 I'm not really thrilled with the idea of putting computev2.1 into tempest's service catalog for us, because I feel like we want to not do things that way 12:27:56 I'll have a proposal by end of day 12:28:05 sdague: cool. Thanks 12:28:12 sdague: cool, I guess we need that test the edges plan in there too 12:28:26 sdague: what is the main concern? 12:28:31 (well, top bottom and interesting) 12:28:43 johnthetubaguy: may be all for v21 and one job for each v2 and v2onv21 12:28:47 gmann_: I am +2 on all those unit test tidy ups now 12:28:59 gmann_: I think it should be, probably 12:29:10 sdague: #link https://review.openstack.org/#/c/163718/ , https://review.openstack.org/#/c/207183/ , https://review.openstack.org/#/c/163731/ 12:29:19 adiantum_: those are all patches for gate v21 thing 12:29:30 ops 12:29:44 gmann_: ok yeh, once I'm more awake here I'll dive through them all 12:29:53 sdague: Thanks 12:29:58 NExt Topic - Test collapse of v2.0 and v2.1 12:30:11 #link https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/test-collapse-v2-and-v21,n,z 12:30:19 #topic Test collapse of v2.0 and v2.1 12:30:45 all patches are up for review. most of them merged. oomichi helped there :) 12:31:09 gmann_: yeah, I will check them later :) 12:31:10 cool, thanks gmann_ oomichi 12:31:16 ok, will take a quick run through those post meeting 12:31:26 looks like johnthetubaguy is +2 on them all, so hopefully we can land them today 12:31:37 yeah, sounds promising 12:31:41 sdague: yeah, agree 12:31:44 johnthetubaguy: Thanks i did not see 12:31:55 I will rebase https://review.openstack.org/#/c/214985/ too 12:32:16 after meeting 12:32:21 cool, looks everything good at here. 12:32:25 let's move on? 12:32:31 alex_xu: yea 12:32:39 #topic Removal of v3 naming from source tree 12:32:52 All the file moving is done 12:33:05 just left some class renaming 12:33:09 we did the tests above 12:33:12 anything left? 12:33:15 edleafe: do you still have any trouble? 12:33:49 alex_xu: no, I have to clean up some things oomichi noted 12:33:57 I saw there still have a lot of comments in the code metion 'v3', those should be cleanup, but not hurry I think 12:33:57 oh, so I have a bit, in the config 12:33:59 but otherwise it's ok 12:33:59 gmann_: https://review.openstack.org/#/c/214985/ seems in merge conflict 12:34:00 https://review.openstack.org/#/c/214592/ 12:34:23 sdague: yea, i will rebase soon 12:34:47 so, just to be clear, we have about 4 working days left before the -2 procedural hammer falls, so we might want to pick our fights here 12:34:49 johnthetubaguy: yea, that is important step for remove extension in M 12:34:51 alex_xu: about the v3 comments: should they be all changed to 'v21', or something else (like 'microversions')? 12:35:30 edleafe: not sure, I few there may have some todo become useless 12:35:32 johnthetubaguy: for bug fix too? 12:35:39 I think we just do v3 -> v21 for now 12:35:48 johnthetubaguy: so, I don't think we should say that the extensions configs are removed in M 12:35:54 alex_xu: ok, I'll check on the TODOs 12:35:59 gmann_: bug fixes are OK, assuming there are no string changes, or dock impact 12:36:09 I think we need to give a full cycle there 12:36:15 johnthetubaguy: ok 12:36:24 edleafe: if we think those are not hurry, let me take care of them, I guess those need some history background 12:36:30 sdague: I was being a bit aggressive there, true 12:36:33 because people don't know it's coming unless they were paying attention 12:36:42 I'd just say they are deprecated, and will be removed in the future 12:36:49 but don't say when 12:36:51 sdague: yeah, that makes more sense 12:36:57 alex_xu: sounds good. I wasn't around for that history 12:37:00 othewise, I'm +2 12:37:07 sdague: it was really just a cut and paste from another comment nearby 12:37:12 yep, no worris 12:37:18 cool 12:37:44 sdague: and when we will plan for removal in N? 12:37:50 gmann_: probably 12:37:58 ohk 12:38:03 but it's a big move, and we need to give people time to absorb it 12:38:20 true 12:38:21 sdague: +1 12:38:27 sdague: +1 12:38:48 anymore question on this? 12:38:49 johnthetubaguy: so, I'd also suggest that we actually add a v2.1 config group, even though all it's opts are deprecated, and deprecate the v3 group as a whole as well 12:38:55 just for cleanliness 12:39:03 the v3 group gets deleted in M 12:39:18 the v2.1 has all deprecated options which will go away ~N 12:39:32 because configuring v2.1 via a v3 group is going to be confusing 12:39:33 sdague: oh, good idea, I assume I can just set a deprecated group name 12:39:38 yeh, I think so 12:39:53 johnthetubaguy: if you don't have time for the config thing, I can run with it today 12:40:03 sdague: good idea, resolve the confuse 12:40:19 sdague: I can tidy that after this meeting, keep you clear for the test stuff 12:40:29 ok 12:40:32 Will labeling this stuff 'v21' be confusing? We all know that that means 'microversions', but will most people understand that shorthand? 12:41:00 edleafe: you access it via the /v2.1 URL, so I am OK with that 12:41:13 or just under the api group 12:41:14 but do we want to run v21 based on CONF.osapi_v3(21).enabled? 12:41:30 edleafe: we can rename it once we drop v2 code 12:41:41 gmann_: right, so that option should not get moved 12:41:48 and just be removed in M 12:41:56 johnthetubaguy: ok, that sounds good. Maybe we'll get some feedback from ops 12:42:02 that was just a massive oversight 12:42:29 edleafe: ops? which bit do you want feedback on? 12:42:33 sdague: or we just keep that in config and run v21 as deafult 12:42:39 gmann_: right 12:42:44 johnthetubaguy: if they're setting config options, etc. 12:42:51 for L it's the switch for v2.1 12:42:54 enabled by default 12:42:57 it's the external-facing stuff that I wonder about 12:43:10 in M we delete the option so you can't turn off v2.1 12:43:30 sdague: +1 12:43:34 sdague: ok, cool 12:43:43 any more question then move on? 12:44:00 I added on more topic 12:44:05 #topic microversions client 12:44:07 edleafe: I am not sure what they will say will stop us removing the options ASAP 12:44:07 #link https://github.com/openstack/python-novaclient/commit/39739158b0cf10a775fd3899e35df7f53b4c9336 12:44:30 johnthetubaguy: not about removing; more about naming 12:44:38 I saw we just bump to the newest version, I guess there are a lot of middleversion not implement yet, how can we push that? 12:44:51 ask each bp owner? 12:45:24 alex_xu: novaclient doesn't implement every bit of the nova api 12:45:24 johnthetubaguy: when we freeze python-novaclient? 12:45:36 sdague: why? 12:45:45 alex_xu: history 12:45:54 sdague: ok... 12:45:57 mostly, I mean, novaclient has never been 100% api coverage 12:46:05 alex_xu: unsure, honestly 12:46:27 so it's kind of ok if exposure of the API features ends up happening when people need them. 12:46:45 so given python-novaclient is kinda dying off now, I am OK with us not worrying too much about that either 12:46:47 libraries should freeze with the rest of the code 12:47:03 important thing, folks have a pattern to follow now, if they want to add things, which is great 12:47:03 just so we don't disrupt heat / trove / etc releases 12:47:08 johnthetubaguy: ++ 12:47:29 sdague: true, dependency freeze hits, and stable branches now, I think 12:47:44 ok, cool 12:47:58 so we needn't worry about that, can save some time for us 12:48:02 let's move on 12:48:15 #topic API Documentation Improvement 12:48:25 #link https://etherpad.openstack.org/p/nova-v2.1-api-doc 12:48:49 I check the v2.1 api doc, to ensure the status 12:48:59 looks like we have a lot of problem 12:49:00 alex_xu: how did the discussions go with annegentle on those? 12:49:38 yeh, honestly, until we get some of these pre freeze bits wrapped up, I haven't spent the mental time on this issue. 12:49:47 johnthetubaguy: annegentle mention whether we have time to implement the generate swagger from code now 12:50:07 oh, based on the API sample generation code? 12:50:36 I take look into that today, the generate may not hard, but after we generate the swagger, we still missing the parameter explain text. those doc string not in our code 12:50:48 johnthetubaguy: based on wsgi stack I guess 12:50:57 alex_xu: right, we should figure out where to put those docstrings in our code 12:51:16 if we have the pattern for one, we can start adding more in that model 12:51:17 * johnthetubaguy wonders about extending the json validation... 12:51:32 so I think we jump to generate swagger from code for this time looks like impossible 12:51:44 alex_xu: yeh, it seems like not a lot of time 12:51:49 I'd say make that an M goal 12:51:57 sdague: yea 12:52:06 johnthetubaguy: yeah, I have the same thinking.. 12:52:08 I think realistic goal for L is an updated concept guide 12:52:23 and any prototyping for swagger that will help us have a plan for M 12:52:23 yea, that need lot of manual effort 12:52:30 we can create sample json files from json-schema. 12:52:54 I'd like to see enough of a proto type that we can have a summit session in Tokyo on the swagger parts and develop the plan for the cycle. 12:52:57 I do wonder about also adding stubs in for the missing API docs, in the existing docs? 12:53:10 #link https://github.com/elmiko/pecan-swagger somebody work on generate swagger from pecan 12:53:12 sdague: +1 for prototype at tokyo 12:53:22 but very initial status of those code 12:53:34 alex_xu / oomichi / gmann_ is a prototype for tokyo realistic? 12:53:49 sdague: yeah, I can :) 12:53:57 sdague: I'm cool 12:53:59 great 12:54:08 sdague: sure we can try 12:54:11 sounds good 12:54:12 sdague: that was my first goal when I started validation works 2.5 years ago 12:54:20 oomichi: :) 12:54:29 oomichi: :) 12:54:34 yeh, it's been a long road here, but things seem to be coming together now 12:54:44 oomichi: I remember liking the direction, its great to see us getting close! 12:54:45 oomichi: i remember your spec :) 12:55:06 so do we still need fix the wadl for now? 12:55:28 alex_xu: honestly, I wouldn't bother 12:55:31 the descision is just focus on concept doc? 12:55:38 it does seem like a distraction 12:55:42 yea, that need lot of work 12:56:03 yea, and will become useless soon 12:56:04 so, how about the following: we aim to have a doc plan discussion in 2 week's time at this meeting 12:56:06 what can we do, quick wins to the existing docs for L? 12:56:18 things should have calmed down on the freeze and bug fix by then 12:56:23 yeah, lets table that for then 12:56:33 and we figure out what we can get through pre release at that point. 12:56:37 yeah, and we need to discuss it across projects 12:56:41 I'll commit to having a base plan at the start of the meeting 12:56:54 and we can sort it out together 12:56:54 johnthetubaguy: remove v2.1 doc, ask people reference to v2. because v2.1 compatible with v2 12:57:06 sdague: we have ideas here: https://etherpad.openstack.org/p/nova-v2.1-api-doc 12:57:23 oh, well that's what I get for being out for a week :) 12:57:30 sdague: do you want an action? 12:57:30 sorry for not being caught up on that 12:57:48 alex_xu: we need to stop the confusion, thats for sure 12:57:56 alex_xu: lets wait until next week to set the action 12:58:05 sdague: ok 12:58:13 #info detailed API doc plan discussion in 2 weeks time 12:58:23 that's probably enough for the agenda 12:58:40 let's move on, left 2 mins for open? 12:58:49 cool 12:58:55 #topic open 12:59:22 one review requestion in the agenda 12:59:24 #link https://review.openstack.org/#/c/209917/ 12:59:38 zhengyu, are you here? 12:59:45 yes 13:00:01 Kevin_Zheng_: cool 13:00:06 it is bug fix 13:00:14 do we have time for this in L? 13:00:20 added also user_id this afternoon according to gmann's segestion 13:00:34 I think the code are quite ready 13:00:39 it feels a bit like a feature that should wait for M? 13:00:43 alex_xu: it feels very close to the wire 13:00:43 Kevin_Zheng_: yea IMO user_id would be good also 13:00:50 if it was two weeks ago, maybe 13:00:54 Kevin_Zheng_: its really a question of review bandwidth 13:01:03 yea, a little late 13:01:04 I'm happy to get that in as soon as we reopen master 13:01:09 I like the change, but it feels too late 13:01:11 sdague: +1 13:01:18 let's end the meeting, and we move the discussion to openstack-nova 13:01:24 oops, yes 13:01:28 #endmeeting