00:01:15 <cyeoh> #startmeeting nova api
00:01:16 <openstack> Meeting started Fri Oct 17 00:01:15 2014 UTC and is due to finish in 60 minutes.  The chair is cyeoh. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:01:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:01:20 <openstack> The meeting name has been set to 'nova_api'
00:01:30 <eliqiao> hi
00:01:32 <cyeoh> jaypipes: sorry started it with the wrong name :-)
00:01:35 <jaypipes> cyeoh: good evening :)
00:01:40 <jaypipes> no probs
00:01:46 <alex_xu> hi, good evening and morning
00:01:52 <cyeoh> morning! Who else is on?
00:02:13 <cyeoh> oomchi mentioned he'd be on a plane today so won't be able to make it
00:02:33 <cyeoh> ok, lets get started
00:02:40 <cyeoh> #topic v2.1 on v3
00:03:00 <eliqiao> yeah, seems he went to German ?
00:03:21 <cyeoh> So the patches seem to be merging in ok. Does anyone have any issues with what is currently in review?
00:03:29 <cyeoh> eliqiao: yea I think he went to LinuxCon
00:03:50 <eliqiao> not yet, I have some patches waiting for merging..
00:04:01 <alex_xu> just think we should merge porting patch first, then begin to improvement the v2.1 api code. right?
00:04:24 <jaypipes> cyeoh: unfortunately, I've been swamped with resource tracker and scheduler specs, patches, and reviews. have not reviewed any 2.1 api patches :(
00:04:30 <eliqiao> agreed, it will be easy for review.
00:04:31 <cyeoh> alex_xu: yes, port first in one patch then cleanup patches
00:04:46 <cyeoh> jaypipes: np
00:05:11 <cyeoh> so in terms of further work - we do need to:
00:05:36 <cyeoh> - verify nothing merged into the v2 version at the end of Juno that wasn't in v2.1 (and fix if it has happened)
00:05:54 <cyeoh> - keep an eye out for anything that touches v2 from now on and ensure that they also fix v2.1
00:06:23 <cyeoh> please -1 any patch that you see does that (sometimes the v2.1 code is different enough that a fix is not required though)
00:06:43 <alex_xu> ok
00:06:50 <eliqiao> agreed.
00:07:05 <cyeoh> Also we need to finish all the json schmea for v2.1 - looks like probably gmann has put that on the worklist etherpad? https://etherpad.openstack.org/p/v2_1_WorkList
00:07:23 <alex_xu> emm...I put that
00:07:41 <eliqiao> yeah, 1 update on that one.
00:07:43 <cyeoh> alex_xu: ah ok, cool - thanks for that!
00:07:48 <alex_xu> cyeoh, np :)
00:08:20 <cyeoh> the other thing we need to look at when people have some time is tempest testing. That will get easier as more patches merge
00:09:00 <alex_xu> also we have some unittest didn't share between v2 and v2.1 yet, those should be complex thing
00:09:08 <cyeoh> Once we have v2.1 passing all the tempest tests that v2 do I think we go to the CD operators  and ask them if they could do some testing
00:09:48 <cyeoh> alex_xu: yes I'd like us to spend some time reducing that sort of technical debt - eg servers tests. Some of them just need a lot of cleaning too. Lots of duplicated code etc
00:10:17 <cyeoh> where you have spare time it'd be great to work on that sort of thing.
00:10:38 <alex_xu> cyeoh, yes, I will try to list those work on our workinglist first
00:10:48 <cyeoh> alex_xu: thanks!
00:10:55 <alex_xu> cyeoh, np
00:11:07 <cyeoh> #topic microversions
00:11:28 <cyeoh> I unfortunately haven't had time to update the spec this week. Am hoping to get to it today.
00:11:50 <cyeoh> It's blocking things like the tasks api so its very important for kilo
00:12:13 <cyeoh> I appreciate all the feedback we've had so far.
00:12:28 <cyeoh> jaypipes: if you get a chance would you mind having a look at it too?
00:12:50 <cyeoh> #link https://review.openstack.org/127127
00:13:25 <cyeoh> its something I think other projects should consider too. Given as their APIs grow its going to be harder for them to do big v2->v3 like transitions
00:14:24 <jaypipes> cyeoh: yup, on it.
00:14:33 <cyeoh> jaypipes: thanks!
00:14:42 <cyeoh> #topic api policy
00:14:46 <jaypipes> np! again, sorry for being so busy :(
00:14:58 <cyeoh> jaypipes: thats fine I totally understand :-)
00:15:21 <cyeoh> alex_xu: so I've read through your first two api policy specs and they look pretty reasonable (I left some comments)
00:15:29 <alex_xu> cyeoh, thanks
00:15:31 <cyeoh> I haven't had a chance to look at third yet though
00:15:40 <jaypipes> links please?
00:15:52 <cyeoh> #link https://review.openstack.org/127160
00:16:00 <cyeoh> #link https://review.openstack.org/128560
00:16:02 <alex_xu> That's for full view of policy improvement
00:16:05 <jaypipes> thx'
00:16:11 <cyeoh> #link https://review.openstack.org/127863
00:16:41 <alex_xu> I mean 128560 is the full view
00:16:53 <cyeoh> alex_xu so I'm not really familiar with oslo policy.d stuff. Does it give the override like feature that John was looking for?
00:17:06 <alex_xu> jaypipes, cyeoh, if you want to review, I suggestion begin from 128560
00:17:14 <alex_xu> cyeoh, yes
00:17:18 <jaypipes> alex_xu: will do! :)
00:17:35 <alex_xu> jaypipes, thanks
00:17:48 <alex_xu> #link https://review.openstack.org/#/c/104157/
00:17:55 <alex_xu> this is the policy.d spec
00:18:07 <jaypipes> one thing I wanted to note about "backwards incompatibility"...
00:18:11 <cyeoh> I think this one area where we need the feedback from operators and listen to them very carefully. As they're the ones going to feel the pain from the change
00:18:19 * jaypipes should wait for open topic...
00:18:37 <cyeoh> backwards compatibility for api policy or more generally?
00:18:56 <alex_xu> cyeoh, yes, how we can get feedback from them?
00:19:21 <cyeoh> alex_xu: well John is a good rep from the rackspace side. Maybe can get Phil Day to comment too?
00:19:55 <cyeoh> alex_xu: maybe you could post about the specs to the operators list too - just make sure we're clean in the specs the impact on them
00:19:56 <alex_xu> cyeoh, ok, John already on the review list, I will try to mail Phil
00:20:07 <alex_xu> cyeoh, ok
00:20:35 <cyeoh> jaypipes: we can talk about your thing now if you'd like
00:20:41 <cyeoh> #topic backwards compatibility
00:21:32 <jaypipes> cyeoh: ok, cool. so, something that I brought up on a patch from xu-huawei (can't find the patch right now...) was that I disagreed that changing failure HTTP response codes was "OK" to do.
00:22:00 <cyeoh> yep, so the APIGuidelines page has for a very long time said that we can change failure HTTP response codes
00:22:14 <jaypipes> cyeoh: any change to HTTP response codes, IMO, is a backwards incompatible change, because it, by definition, will break clients expecting to translate a code into a message.
00:22:20 <cyeoh> and still consider backwards compatible (even though I agree its not)
00:22:42 <jaypipes> cyeoh: you mean the APICompatibility wiki page, right?
00:23:03 <jaypipes> or rather APIChangeGuidelines :)
00:23:04 <cyeoh> #link https://wiki.openstack.org/wiki/APIChangeGuidelines
00:23:07 <jaypipes> yeah :)
00:23:10 <eliqiao> jaypipes: IMO, if we don't bring backwards incompatible change, then ,why we develop the new api?
00:23:15 <alex_xu> actually I prefer the jaypipes saying, maybe we should adjust the guideline in API working group
00:23:30 <cyeoh> yes, I think we should change it in the longer term, BUT
00:23:34 <jaypipes> alex_xu: for sure, it's something the API WG should look into long term, yes
00:23:49 <cyeoh> we have done so many of these in the last few cycles that I think we should just finish cleaning them up
00:23:54 <eliqiao> even if the http response code is not correct, shouldn't we change it and make it much reasonable ?
00:24:12 <jaypipes> cyeoh: ok, that's fair I guess.
00:24:13 <cyeoh> we can easily put the clamp down on it once we have microversions and can easily make backwards incompatible changes
00:24:16 <alex_xu> eliqiao, that's the reason we bring up micro-version
00:24:22 <jaypipes> cyeoh: as you know, I'm more focused on future APIs :)
00:25:00 <cyeoh> eliqiao: so the difference between success and error codes is that lots of people check for success codes, much fewer developers look closely at error codes
00:25:03 <eliqiao> okay , I get what jaypipes' mean. not change current api. do the 'right' things on new APIs, correct?
00:25:10 <cyeoh> anyway thats why I think they are treated differently
00:25:10 <jaypipes> cyeoh: well, microversions doesn't get us out of all compatibilty issues. but schema and payload discovery will.
00:25:21 <jaypipes> eliqiao: yes.
00:25:34 <cyeoh> jaypipes: I think microversions does allow us to make any backwards incompatible change
00:25:40 <cyeoh> we just bump the major version
00:25:59 <jaypipes> cyeoh: yes. but that's not micro versioning, now, is it? :)
00:26:12 <cyeoh> jaypipes: yea I think it is
00:26:44 <cyeoh> we just don't try to do v2->v3 like transitions in one go
00:27:02 <jaypipes> I guess I've been thinking of microversions as the ability of an API minor version to be incremented along with backwards-compatible changes, and major version numbers saved for backwards-incompatible changes.
00:27:41 <jaypipes> cyeoh: yes, we agree, I just have been using slightly different terms.
00:27:45 <cyeoh> yep thats what I meant. So for X.Y.Z as a version number
00:27:49 <jaypipes> ++
00:28:05 <cyeoh> X - would be bumped for backwards incompatible, Y for minor, Z perhaps for bug fixes (maybe)
00:28:10 <cyeoh> ok cool :-)
00:28:55 <jaypipes> cyeoh: do we have a separate set of BPs/specs *just* for the schema discovery stuff?
00:29:06 <cyeoh> so I think JSON-Home/jsonschema will be great for those who want to do discovery. But frankly a lot of developers out there are going to be lazy and just code against a specific version of the API
00:29:42 <cyeoh> jaypipes: no we do need one. I think the microversions work is more urgent for now because its blocking other API work
00:29:57 <cyeoh> but I've mentioned I'd love to see nova specs for them if someone has the time
00:30:16 <jaypipes> cyeoh: for me, it's not really about developers using the schema discovery with auto-generated code or anything like that. it's about the schemas actually serving as the API documentation itself.
00:30:34 <annegentle> or at least some self-discoverability
00:30:39 <jaypipes> ala Swagger or APIBlueprint.
00:30:41 <cyeoh> jaypipes: yea I *really* want to see documentation available like that
00:30:45 <annegentle> you still need more docs
00:30:52 <jaypipes> annegentle: absolutely true.
00:31:00 <annegentle> and YES to Swagger, meh on API blueprint (It's the etherpad of api docs to me)
00:31:02 <cyeoh> so I watched the openstack hour thing on docs the other day
00:31:12 <jaypipes> annegentle: but it's better than our current situation, which is very repetitive (I think  we can ALL agree on that! ;)
00:31:33 <annegentle> cool cyeoh :)
00:31:34 <cyeoh> and the mention about how we need a human involved in producing the docs too
00:31:47 <annegentle> repetitive meaning across the versions or?
00:31:51 <jaypipes> annegentle: yes, ++ on Swagger vs APIBlueprint, now that I've used both, it's a no-brainer.
00:32:17 <cyeoh> so I'm wondering if the json schema/ json home output is sufficient?
00:32:35 <jaypipes> annegentle: repetitive as in just take a look at the repetitive coding changes needed in a very simple API change: https://review.openstack.org/#/c/114107/5 :)
00:32:37 <cyeoh> for the production of docs I mean. Or do we need to provide more
00:32:38 <annegentle> cyeoh: well, so far, it's not sufficient across all OpenStack projects
00:33:12 <annegentle> cyeoh: for reference info, it might be fine, but we do need more guidelines docs as well, which could be common across all APIs for OpenStack services
00:33:19 <jaypipes> I'd really prefer to see the API specs developed in separate repos from code, frankly.
00:33:33 <cyeoh> ok. I'm just wondering if we'd need to still provide api samples
00:33:34 <jaypipes> ala Keystone's identity API documentation/spec
00:33:40 <annegentle> jaypipes: I'm working on patches that add the api info to <project>-specs
00:33:44 <annegentle> yep exactly jaypipes
00:34:05 <annegentle> cyeoh: ideally, yes, tested samples... but again there may be some automation we can do there
00:34:22 <annegentle> cyeoh: to a point anyway
00:34:29 <cyeoh> annegentle: yea even in our api sample tests that api samples aren't often tested very well anyway :-(
00:34:36 <jaypipes> cyeoh: the API samples are a critical part of API documentation, but if we used something like Swagger, the documentation *is* the JSONSchema documents, so you only need to keep the JSONSchema docs that describe the API up to date, and samples are generated from them.
00:34:53 <annegentle> what I'm telling new projects is that it's fine to automate the first round but then carefully hand-maintain to ensure truth... not that code is truth but that the spec is truth. ya know
00:35:04 <annegentle> jaypipes: yeah I think that's true too
00:35:06 <jaypipes> annegentle: right, ++
00:35:32 <cyeoh> so I'd be fine saying the spec is the truth if we actually wrote the spec before doing the code, but historically we haven't done that
00:35:40 <jaypipes> annegentle: ooh, that's a neat idea (api spec within the <project>-specs repo.)
00:35:48 <cyeoh> so IMO for what we currently have the code is the truth
00:36:00 <annegentle> cyeoh: one note about your patch to set up the api working group repo, might take a page from https://review.openstack.org/#/c/125508/5/gerrit/acls/openstack/openstack-specs.config for the question Anita had.
00:36:04 <jaypipes> cyeoh: there is no spoon (truth) :0
00:36:22 <annegentle> jaypipes: yes! Identity is doing that this week... need to get my butt in gear to get patches for the other projets
00:36:29 <cyeoh> annegentle: thanks!
00:36:47 <annegentle> heh hopefully I'm not completely derailing your meeting :)
00:37:04 <annegentle> ice cream time at my house, later!
00:37:10 <cyeoh> so I will wish again to be able to have patches dependent across repositories .... so we can make sure people have docs before merging code
00:37:11 <jaypipes> cyeoh: but, in all seriousness, I totally agree with you that our current state of affairs in nova is certainly not pretty, and indeed, the code is the source of truth for the v2[.1] APIs.
00:37:40 <cyeoh> jaypipes: yep - I -1 patches that say this makes the code conform to the Nova api specs :-)
00:38:04 <cyeoh> annegentle: thanks - enjoy! :-)
00:38:19 <jaypipes> cyeoh: the trick is.. how do we get to a point where for new versions of the compute API, we do actually have a spec that we write code from, and not the other way around?
00:38:59 <cyeoh> jaypipes: so I am trying to tighten down on what goes into nova specs in the API section (though some stuff is still getting approved a bit soon)
00:39:03 <jaypipes> cyeoh: do we say (once the v3->v2.1 stuff is all done) that we're going to do a v3 API in a totally different way, and for the v3 API, we're gonna do spec first, then code to it?
00:39:16 <cyeoh> and I think we can make nova-specs "the truth"
00:39:40 <cyeoh> tests and docs should be written against nova-specs
00:39:42 <jaypipes> cyeoh: yeah, that's a very good idea about having the api spec in the project-specs repo. +10
00:39:49 <jaypipes> cyeoh: right, xactly.
00:40:05 <cyeoh> so I don't think there will ever be a v3.
00:40:18 <cyeoh> we'll use microversions to slowly iterate towards what we want..
00:41:00 <jaypipes> cyeoh: but I want backwards-incompatible stuff...
00:41:32 <jaypipes> for instance... tasks API, getting rid of flavors, getting rid of admin API operations, etc
00:41:37 <cyeoh> jaypipes: yep, thats fine. We bump the microversion major number for that. But we'll be doing that quite a bit
00:41:46 <cyeoh> we just won't call it v3, v4, v5 etc
00:41:52 <jaypipes> oh, I see
00:42:00 * jaypipes needs to read the spec :)
00:42:01 <cyeoh> it'll just be: 3.2.0 or 4.1.0 etc
00:42:05 <jaypipes> will do that tomorrow morn!
00:42:18 <cyeoh> cool :-)
00:42:39 <cyeoh> #topic API WG
00:42:48 <cyeoh> for those who missed the email,
00:42:51 <cyeoh> #link https://wiki.openstack.org/wiki/API_Working_Group
00:43:21 <cyeoh> if you know people from other projects please encourage them to get involved
00:43:36 <cyeoh> there's little point if this turns out to be just a Nova thing
00:44:06 <jaypipes> yes. lbragstad is in, for sure, as is armax for neutron IIRC
00:44:12 <jaypipes> or maybe it was salvatore
00:44:20 <cyeoh> cool :-)
00:44:31 <jaypipes> I'll check and see if eglynn-office would like to have a ceilo-core participate.
00:44:35 <jaypipes> maybe dina...
00:45:04 <cyeoh> yea that'll be great. I wish I was going to Paris :-(
00:45:27 <jaypipes> cyeoh: :( vancouver it will be :)
00:45:36 <cyeoh> jaypipes: yep!
00:45:40 <jaypipes> awesomesauce
00:45:49 <cyeoh> #topic Open Discussion
00:46:01 <cyeoh> anyone have anything?
00:46:08 <gmann> cyeoh: sorry i missed JSON schema one
00:46:30 <gmann> cyeoh:  whats plan for importing response schema we have in tempest to nova
00:47:02 <cyeoh> gmann: we should do it :-) Do you want to write up a spec? I fully support us checking our responses on the way out
00:47:05 <gmann> cyeoh: i think only https://review.openstack.org/#/c/87191/ is pending. other are done
00:47:23 <gmann> cyeoh: sure
00:47:39 <cyeoh> cool :-)
00:47:58 <cyeoh> though given tempest coverage of Nova is 100% complete we will have more schema to write on the Nova side I think
00:48:09 <cyeoh> I mean tempest coverage is not 100% complete
00:48:20 <gmann> cyeoh: true
00:48:36 <cyeoh> anyway I think its something we should fit into Kilo
00:49:11 <cyeoh> does anyone have anything else?
00:49:55 <cyeoh> ok. lets have an early 10 minutes. Thanks everyone!
00:50:10 <alex_xu> thanks all
00:50:15 <gmann> Thanks all
00:50:21 <cyeoh> #endmeeting