00:00:52 <cyeoh> #startmeeting nova api
00:00:53 <openstack> Meeting started Fri Sep 26 00:00:52 2014 UTC and is due to finish in 60 minutes.  The chair is cyeoh. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:00:54 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:00:56 <openstack> The meeting name has been set to 'nova_api'
00:01:06 <cyeoh> Hi - anyone around for the nova api meeting?
00:01:09 <alex_xu> o/
00:02:19 <cyeoh> heh is it just us today?
00:02:25 <cyeoh> gmann: are you around?
00:02:25 <alex_xu> emm....
00:02:40 <gmann> hi
00:03:04 <cyeoh> hey do we have enough people for a meeting? :-)
00:03:52 <cyeoh> so I guess just a couple of things I had on my mind
00:03:54 <alex_xu> >=2 is enough :)
00:04:02 <oomichi> hi
00:04:04 <cyeoh> #topic v21 on v3
00:04:08 <cyeoh> oomichi: hi :-)
00:04:16 <cyeoh> ooh 4 people now :-)
00:04:20 <alex_xu> cool, enough now
00:04:47 <cyeoh> so I went through and reviewed all the v2-on-v3-api patches yesterday. I think Kilo will be open next week
00:04:57 <cyeoh> would be nice to merge them quickly
00:05:16 <oomichi> cyeoh: agree, I also will review them
00:05:46 <oomichi> cyeoh: but we need to wait for kilo spec again before that?
00:05:47 <cyeoh> I suspect there are a few bug patches for v2.1 which I probably missed. I was wondering if it would be possible for people to tag even the bug fixes with v2-on-v3-api
00:05:57 <cyeoh> which will make them easier to find
00:06:05 <cyeoh> oomichi: I'm hoping not.
00:06:26 <oomichi> cyeoh: I also hope that :)
00:06:51 <cyeoh> I guess we can just resubmit the last one but otherwise just try to merge. After all, nothing has changed. I'll ping mikal and john and ask
00:07:23 <oomichi> cyeoh: that's great!
00:07:25 <cyeoh> there's going to be a bit of a lull in patches while people wait for the nova-specs to be approved so it would be good to take advantage of that spare review bandwidth
00:07:29 <gmann> cyeoh: i have some bug patches, I will tag BP on those.
00:07:37 <cyeoh> gmann: thx!
00:08:14 <cyeoh> #topic microversions
00:08:56 <cyeoh> So good discussion on the mailing list. I guess we should keep the discussion going on there
00:09:17 <cyeoh> I'll start a thread about the client header format and what versions mean so we can talk about that separately
00:09:51 <cyeoh> did anyone have anything wanted to talk about microversions?
00:10:20 <oomichi> so I'd like to know microversions are common thing for other area?
00:10:35 <oomichi> or that is Nova specific?
00:10:48 <cyeoh> oomichi: not in openstack - they've been going for big v2->v3 like upgrades
00:10:50 <alex_xu> note: that blog post can't comment, if you have any comment, you can add at github
00:10:56 <cyeoh> but then their APIs are quite small
00:11:06 * dims peeks
00:11:29 <oomichi> cyeoh: I see, thanks :)
00:11:56 <cyeoh> oomichi: its not specification called microversions, but client version headers are used in some non-openstack API
00:12:10 <cyeoh> so its not just something we're making up ;-)
00:12:28 * cyeoh waves at dims
00:12:53 <cyeoh> #topic open discussion
00:13:08 <cyeoh> I guess most people have seen the thread about the cross project api work
00:13:16 <cyeoh> looks like something might actually happen
00:13:33 <cyeoh> as I mentioned on the thread I'd encourage people to go to the wiki and add/comment
00:13:41 <oomichi> yeah, nice discussing now
00:14:00 <cyeoh> doesn't matter at this stage if we don't have conensus over what is in there, more just gathering what we could have in there
00:14:00 <gmann> how cross project API guidelines will be concluded/approved? any spec etc and then import those on wiki or some other process?
00:14:14 <oomichi> how about using gerrit review process instead of wiki?
00:14:28 <cyeoh> gmann: for the moment we'll just get together potential items I think
00:14:48 <cyeoh> oomichi: I think the problem with gerrit at the moment is it makes it harder for people to add just an item or two
00:14:56 <cyeoh> oomichi: especially for users
00:15:17 <cyeoh> oomichi: but eventually once we have a decent draft I agree gerrit would be good so we can vote
00:15:28 <oomichi> cyeoh: on nova-specs process, users also could join it.
00:15:45 <cyeoh> oomichi: yea, but it doesn't really belong in nova-specs since its cross project
00:16:05 <bknudson1> I think we will wind up having microversions in other apis
00:16:08 <cyeoh> I think there is an openstack/governance repo though
00:16:21 <bknudson1> since the major version changes have been a disaster
00:16:22 <oomichi> cyeoh: that seems nice process. draft: wiki -> review: gerrit
00:16:24 <cyeoh> bknudson1: yea we might be the guniea pigs :-)
00:16:54 <cyeoh> bknudson1: anyway I think we'd all appreciate input from other projects into the design of microversions
00:17:07 <bknudson1> where's the design?
00:17:18 <bknudson1> I assume it's just using Accept header.
00:17:42 <cyeoh> so there is a nova spec and alex_xu also has a blog post (though I don't agree with all of it) that covers the implementation side
00:18:01 <bknudson1> approved spec?
00:18:04 * cyeoh is looking for the spec
00:18:13 <cyeoh> bknudson1: no, we didn't get consensus around it
00:18:53 <bknudson1> really? seems pretty obvious
00:19:25 <alex_xu> #link https://review.openstack.org/#/c/101648/
00:19:32 <alex_xu> cyeoh, is this one?
00:19:37 <bknudson1> abandoned :(
00:19:38 <cyeoh> alex_xu: yes, thanks!
00:19:50 <cyeoh> bknudson1: yea we managed to get v2.1 approved so we concentrated on that first
00:20:02 <oomichi> https://review.openstack.org/#/c/96139/11/specs/juno/api-microversions.rst
00:20:44 <alex_xu> yea, two versions
00:20:47 <cyeoh> oomichi: yep, that is another.
00:21:05 <cyeoh> I don't think there is any real disagreement on the client header side
00:21:35 <cyeoh> but we do need to nail down what the version numbers are (eg for X.Y.Z what the X and Y and Z mean)
00:22:04 <cyeoh> and I think we have a rough consensus over experimental and vendor tags
00:22:28 <cyeoh> most of the remaining bit to resolve is the implementation side
00:22:43 <cyeoh> around how much code we copy and paste when we support multiple versions
00:22:47 <bknudson1> I would expect that the Accept header would be used and qvalue support
00:23:30 <bknudson1> with qvalue support the client could say it support version=1.1 and version=1.2 and that it prefers 1.2
00:23:47 <cyeoh> bknudson1: I can't remember the details at the moment but there was some issue with qvalue support, but perhaps I could look at it again
00:24:17 <bknudson1> it might not work with the specific header format.
00:24:31 <oomichi> bknudson1: that means using the latest version as possible between server and client ?
00:25:08 <cyeoh> I think we were looking at something like:   X-OS-Compute-Version: 2.114, experimental, vnd:rax
00:25:32 <bknudson1> right, is that the min version supported or the max version?
00:25:46 <bknudson1> the version that the client supports
00:25:57 <cyeoh> bknudson1: we were thinking of requesting a specific version rather than a range
00:26:07 <oomichi> bknudson1: that would be a nice idea.
00:26:27 <cyeoh> but a range would be fine as well
00:26:49 <cyeoh> presumably can also query from the server beforehand what version range is supported
00:27:05 <alex_xu> yes, the server will return the min version and max version
00:27:08 <cyeoh> the experimental flag would turn on all experimental features,
00:27:27 <cyeoh> and the vnd: flag enables out of tree features
00:27:56 <bknudson1> typically with the Accept header the server will respond with a Content-Type with the actual match...
00:28:16 <cyeoh> bknudson1: yep
00:28:37 <bknudson1> but if you're only supporting the specific version then I guess there's no need for it.
00:29:10 <cyeoh> we'll be supporting a large range of versions for quite a while.
00:29:25 <alex_xu> so the different is the server choice version for client or the client choice version for server?
00:29:26 <oomichi> for working on many environments, the range is nice.
00:29:42 <bknudson1> it's the whole compute API and not the specific resource?
00:29:43 <cyeoh> everything from v2.1 (initial which is equivalent to v2) up to current and will have to bump for any api change
00:30:26 <cyeoh> the need to support many versions is one of the reasons we need to keep code duplication down to a minimum.
00:30:47 <cyeoh> feedback we're getting is that we can't deprecate the v2 api for a number of years
00:31:25 <cyeoh> bknudson1: yes, the microversion number refers to the API as a whole
00:31:27 <bknudson1> do you have the client changes described in the design?
00:31:51 <cyeoh> bknudson1: do you mean what version has what api changes?
00:32:11 <cyeoh> bknudson1: or just what header the client has to send
00:32:19 <bknudson1> y, how's the client going to work?
00:32:45 <bknudson1> is the novaclient going to support a single version?
00:32:59 <cyeoh> So the client will be able to send this X-Compute-Version header (if it doesn't it just gets v2)
00:33:10 <cyeoh> so I think it would be crazy for novaclient to try to support all versions
00:33:25 <cyeoh> so I'm going to suggest supporting v2 and then the latest version of microversions
00:33:37 <bknudson1> so the server gets a request with a higher version than it supports...
00:33:43 <bknudson1> is the request rejected?
00:33:49 <cyeoh> yup
00:34:11 <cyeoh> but older is fine
00:34:20 <bknudson1> unless it's too old.
00:34:30 <cyeoh> well for us there's nothing too old :-)
00:34:30 <oomichi> cyeoh: yeah, all supports are crazy, but maybe novaclient needs to support the latest and oldest(v2.0 compatible).
00:34:44 <cyeoh> bknudson1: no header just means it acts as V2
00:35:18 <cyeoh> oomichi: yep I agree
00:35:48 <bknudson1> so what version would you expect the latest client to send on a request?
00:35:51 <alex_xu> cyeoh, oomichi emm...why we need support oldest?
00:36:27 <oomichi> alex_xu: because we cannot drop v2.0 environments in real world.
00:36:50 <bknudson1> seems like you'd have the client send the latest version, and then if the server didn't support it, it would respond with the latest that it supports.
00:36:54 <cyeoh> bknudson1: whatever version it has been updated to support
00:37:15 <bknudson1> as long as the server tells the client what version it was then it can accept it or ignore it.
00:37:37 <cyeoh> bknudson1: yes if we can  have novaclient  support multiple versions (and novaclient isn't exactly great code)
00:38:51 <cyeoh> perhaps we could have novaclient support some "key" versions. (say nova releases)
00:39:04 <cyeoh> that may be sufficient to make people happy
00:39:28 <cyeoh> but we'd have to do a reasonable amount of refactoring of novaclient code
00:39:43 <oomichi> oh, I can see very long load for it..
00:39:45 <bknudson1> with microversioning are you going to allow backwards-incompatible changes?
00:39:56 <cyeoh> initially I think we'd just do oldest/newest
00:40:07 <alex_xu> oomichi, ok, thanks
00:40:12 <cyeoh> bknudson1: yes definitely - its the major reason for doing microversions
00:40:52 <bknudson1> using semver, so a major bump for backwards-incompatible?
00:41:01 <cyeoh> bknudson1: yes
00:41:31 <cyeoh> and I expect that initially we'll have quite a few major version bumps
00:42:01 <cyeoh> we for example want to introduce the tasks api which will break lots of stuff and server diagnostics is another example of an interface we want to completely change
00:42:32 <bknudson1> the spec has an example: @api.version(introduced="2.0", deprecated="2.115", removed="2.300")
00:42:40 <bknudson1> but since removal of an api would be backwards-incompat
00:42:48 <bknudson1> that would be @api.version(introduced="2.0", deprecated="2.115", removed="3.0")
00:43:05 <cyeoh> bknudson1: i agree
00:43:23 <cyeoh> we hadn't at that point agreed on what the version numbers meant
00:43:50 <cyeoh> I don't think we should be shy about bumping major version numbers in the microversion model though
00:44:18 <eliqiao> hi all ,  i am late :(
00:44:24 <cyeoh> just need to get users used to the idea that a major version bump in a microversion model does not mean that the whole world changes for them
00:44:52 <cyeoh> that it would most likely just be one interface with backwards incompatible changes (which they may not care about at all)
00:44:58 <cyeoh> eliqiao: hi!
00:45:15 <bknudson1> this is where it might be useful to have a version on every resource
00:45:22 <bknudson1> rather than on the whole api
00:45:33 <oomichi> cyeoh: but we need to avoid backwards incompatible changes as possible, I don't want to see same kind of change which we fixed them before.
00:46:15 <bknudson1> if you're going to change 1 api, up the major number ... you're going to wind up being at a pretty high major version #
00:46:36 <bknudson1> but then I'm not sure how the server would advertise its version support for every resource
00:46:50 <bknudson1> unless using JSON Home, since it's got metadata for every resource.
00:47:00 <cyeoh> bknudson1: yea we did talk about versioning individual resources but some people really want to avoid the very large number of different combinations you can get when that happens (eg we have 70 plugins all individually versioned changing at different rates)
00:47:27 <cyeoh> bknudson1: so I don't think we should be concerned about a really high major version
00:47:40 <cyeoh> bknudson1: i don't think it matters, its just a number......
00:47:56 <oomichi> bknudson1: yeah, json-home is nice for it.
00:48:15 <bknudson1> I'm still concerned about how the client is going to work... it would make sense for it to say it supports version 1 or version 2.
00:48:27 <cyeoh> oomichi: yes, we need to be careful about backwards incompatible changes but if we need to do it I think its ok (we just need to review any backwards incompatible change very carefully)
00:48:32 <bknudson1> but then going to saying it supports multiple major versions gets a little crazy
00:49:06 <cyeoh> bknudson1: only if you assume that a major version bump means lots of changes to the api
00:49:31 <oomichi> bknudson1: re: json-home: can we get metadata on the same URLs just by specifying json-home on accept header?
00:49:32 <cyeoh> bknudson1: so say one major version is a result of the server diagnostics api changing but the app never uses it
00:49:36 <bknudson1> the client doesn't know for a major version bump what changed and what didn't
00:49:51 <cyeoh> bknudson1: the client supports both major versions without any code changes
00:50:14 <cyeoh> bknudson1: so we will have to document very carefully what each version of the API supports
00:50:56 <bknudson1> oomichi: I think that the json-home document is supposed to describe some whole part of the api... that's how keystone is using it anyways.
00:51:00 <eliqiao> I think one of the import thing is we should simple the client usage.
00:51:17 <cyeoh> so discoverability can address some issues with lots of versions but it doesn't address changes in semantics which are also api changes
00:51:18 <bknudson1> cyeoh: as long as the client has a fallback (like to v2) then that should be ok.
00:51:40 <oomichi> bknudson1: thanks, I will check keystone code later :)
00:52:30 <bknudson1> oomichi: curl -H "Accept: application/json-home" http://localhost:5000 | python -m json.tool
00:52:42 <bknudson1> curl -H "Accept: application/json-home" http://localhost:5000/v3 | python -m json.tool
00:52:43 <bknudson1> also works
00:53:08 <oomichi> bknudson1: cool, will do it later:)
00:53:09 <bknudson1> in keystone we don't have any metadata yet, just the path.
00:53:21 <cyeoh> yea so we need to have a think about how to support json-home with multiple versions
00:53:45 <bknudson1> your json-home document could be generated based on the version requested.
00:54:05 <bknudson1> use the same header to say what version of json-home you want
00:54:09 <cyeoh> bknudson1: yep we'll definitely need to generate them on the fly
00:54:36 <bknudson1> that could actually be pretty neat
00:54:48 <cyeoh> so we have 5 mins left. Just wanted to check if anyone had anything else they wanted to bring up otherwise can continue with this...
00:55:31 <eliqiao> bknudson1:will json-home bring some more code changes to the infrasture?
00:56:17 <bknudson1> eliqiao: looking at the spec doc, it looks interesting because you've got @api.version(introduced="2.300")
00:56:25 <bknudson1> so you know what the APIs are...
00:56:43 <bknudson1> for each API there'll be a resource, so then you'd need to know what the path is to the API
00:56:55 <oomichi> bknudson1: re: https://review.openstack.org/#/c/122590/ as this meeting, we will keep using v2.1 endpoint without adding more version endpoints.
00:56:56 <bknudson1> which it doesn't know at that point.
00:57:21 <eliqiao> bknudson1:cool, thx.
00:57:23 <bknudson1> oomichi: ok, I can stop complaining then.
00:57:39 <oomichi> bknudson1: thanks ;)
00:57:59 <cyeoh> eventually we'll get to the point of not having versioned endpoints....
00:58:07 <bknudson1> as long as you've got a better plan like microversioning where you're not going to add more!
00:58:34 <cyeoh> yep and we'll be able to drop the version as part of the url as well
00:58:52 <cyeoh> (when we finally deprecate v2 anyway)
00:58:58 <oomichi> cyeoh: yeah, that is good thing.
00:59:23 <cyeoh> ok I think we're pretty much out of time
00:59:56 <cyeoh> thank you everyone for attending. See ya next week.
01:00:03 <bknudson1> here's some keystone code for registering a resource: http://git.openstack.org/cgit/openstack/keystone/tree/keystone/identity/routers.py#n41
01:00:09 <alex_xu> thanks all, great meeting
01:00:26 <gmann> Thanks all. Good discussion :)
01:00:40 <eliqiao> thank you!
01:00:49 <cyeoh> bknudson1: yep we do something very similar, although with the plugin infrastructure its more abstractly definied
01:01:00 <cyeoh> #endmeeting