00:00:35 <cyeoh> #startmeeting nova-api
00:00:36 <openstack> Meeting started Fri Mar 21 00:00:35 2014 UTC and is due to finish in 60 minutes.  The chair is cyeoh. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:00:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:00:39 <openstack> The meeting name has been set to 'nova_api'
00:00:49 <cyeoh> Hi! Who's here today?
00:00:52 <mikal> Me!
00:00:58 <masayukig> hi
00:01:01 <mikal> Can I sneak something onto the agenda at the end?
00:01:01 <mrda> hi all
00:01:02 <ivanzhu> hi
00:01:10 <cyeoh> mikal: sure, please put it on
00:01:23 <alex_xu> hi
00:01:26 <ken1ohmichi> hi
00:01:26 <mikal> What's the agenda URL?
00:01:34 <cyeoh> #link https://wiki.openstack.org/wiki/Meetings/NovaAPI
00:02:02 <mikal> Ta
00:02:02 <cyeoh> #topic v2 on v3 POC progress
00:02:27 <cyeoh> So I think we've mostly been busy on the tempest validation side or nova bugs this week
00:02:39 <cyeoh> so not much has moved on the POC in the last week
00:02:58 <cyeoh> but I think there's another v2 on v3 case we might need to consider
00:03:14 <cyeoh> and that is where there are semantic not output changes between a v2 and v3 method
00:03:35 <cyeoh> the only case I can think of at this time is potentially the flavor create call currently under discussion
00:04:07 <cyeoh> where we may end up with the v3 flavor create automatically adding the tenant to the access list when the flavor is private
00:04:12 <cyeoh> and the v2 version not doing that.
00:04:49 <cyeoh> we probably have 2 choices here - we have two versions of the call (maybe sitting in the same file but with a decorator to divert to the correct one for v2)
00:04:57 <cyeoh> or we start passing version flags
00:05:12 <cyeoh> personally I don't like the latter because I think the code gets more complicated in the long run
00:05:26 <cyeoh> but the former has more code duplication
00:05:36 <cyeoh> so I'm interested if anyone else has opinions/other ideas?
00:05:36 <mrda> I think it might depend upon _how_ semantically different v2 and v3 are
00:06:14 <cyeoh> mrda: so we just do it on a case by case basis?
00:07:01 <mrda> actually, the decorator diversion would work in either case
00:07:45 <mrda> if we do need both approaches, we should have a preferred one, and only use the 2nd for a fallback
00:08:21 <cyeoh> yea I suspect that the flag is okayish if there is only a tiny semantic difference
00:08:39 <ken1ohmichi> or new v2.1 behavor will be defined as the same as current v3 behavor
00:08:40 <mrda> yes
00:08:42 <cyeoh> but if there's a big difference and the implementation is quite different then we have separate function
00:09:10 <mrda> I think that sounds right cyeoh
00:09:31 <cyeoh> ken1ohmichi: so I suspect we can't do that. It's the easiest for us to implement, but it does mean a bigger impact on users of the V2 api moving to V2.1
00:10:00 <cyeoh> I guess we'd need more feedback depending on the change (for flavor-create specifically it might be ok)
00:10:15 <ken1ohmichi> I guess v3 behavior is because it is useful, right?
00:10:38 <cyeoh> yea its probably what we intended to do for v2, we just got it wrong
00:11:01 <cyeoh> and now people like horizon depend on the wrongness
00:11:46 <ken1ohmichi> ok, I also will consider the implementation.
00:11:56 <cyeoh> that sounds good - so for now I'm happy if we're able to say that we have a solution for these circumstances and I guess we'll just code up an example for the POC.
00:11:56 <ken1ohmichi> for v2 behavior
00:12:02 <cyeoh> ken1ohmichi: thanks!
00:12:09 <alex_xu> one more case, we won't proxy neutron call in v3, so in v3 we won't auto allocate network for instance, but in v2 we should keep auto allocate. So cyeoh is right, we should keep same behavior in v2.1
00:13:08 <cyeoh> alex_xu: yea the auto allocation could well be another example. How we handle nova-network and neutron proxying in v3 is all a bit of a mess unfortunately
00:13:10 <ken1ohmichi> alex_xu: that is a big impact
00:14:07 <cyeoh> at some point we're going to have to decide if nova-network is with us long term or not. And if it is if we have APIs like quotas whether we proxy those for neutron
00:14:27 <cyeoh> or if they're just seen as nova-network specific APIs (this will upset some people)
00:14:41 <ken1ohmichi> cyeoh: agreed, proxy seems necessary again..
00:15:04 <cyeoh> ken1ohmichi: yep, for neutron I suspect so
00:15:26 <cyeoh> and we may have to have auto allocation as well.
00:15:49 <cyeoh> was there anything else people wanted to talk about the v2 on v3 POC?
00:16:42 <cyeoh> #topic API extensions used by public clouds
00:17:13 <cyeoh> so last week we talked about asking the public clouds what API extensions they actually used so we would know what areas to concentrate on for the v2 on v3 POC
00:17:31 <cyeoh> not sure if alaski is around today?
00:17:49 <cyeoh> I've emailed HP privately, but haven't heard anything back yet
00:18:05 <mikal> Did you ping Phil Day?
00:18:23 <cyeoh> mikal: yea thats who I sent an email to earlier this week
00:18:32 <mikal> Cool, I think he's the right person to ask.
00:18:53 <clarkb> is that something you can have novaclient list for you?
00:19:03 * clarkb can give you the output if it is available as a user
00:19:38 <cyeoh> clarkb: hrm I'm not sure. does novaclient have an option to query /extensions?
00:20:45 <cyeoh> so thinking about it now I'd guess we just need someone with access to the cloud that can do a get on /extensions
00:20:56 <cyeoh> and that should give us the full list of what is deployed
00:21:23 <clarkb> looks like there is a list-extensions
00:21:25 * clarkb tries this
00:21:35 <cyeoh> clarkb: ah nice :-)
00:22:55 <cyeoh> ok so I think we should be able to get that same info for rackspace pretty easily too.
00:24:34 <cyeoh> clarkb: if you could put up the output you get somewhere we can get to that'd be great.
00:24:47 <cyeoh> #topic API response validation in tempest
00:25:05 <cyeoh> ken1ohmichi: did you want to talk about this? I think its all making pretty good progress?
00:25:12 <ken1ohmichi> yes
00:25:28 <ken1ohmichi> now good progress
00:25:43 <ken1ohmichi> we have a lot of patches for it: https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/nova-api-attribute-test,n,z
00:25:57 <clarkb> cyeoh: http://paste.openstack.org/show/73956/
00:26:02 <mrda> ken1ohmichi: I would say great progress!  You and the team have achieved a lot!
00:26:14 <cyeoh> clarkb: thanks!
00:26:20 <ken1ohmichi> but nova contains 253 v2 APIs and we need more
00:26:26 <cyeoh> ken1ohmichi: wow! We have lots of reviewing to do :-)
00:26:43 <ivanzhu> cyeoh: ken1ohmichi can we just verify the success respond code in the client?
00:27:13 <ken1ohmichi> if not containing body, that is right
00:27:32 <ken1ohmichi> if containing body, we need to verify body of response
00:27:32 <ivanzhu> in many case the response code is the only difference between v2 and v3.
00:27:37 <ken1ohmichi> with jsonschema
00:27:46 <ivanzhu> ken1ohmichi: yes
00:28:06 <ken1ohmichi> ivanzhu: nice info,
00:28:07 <cyeoh> so where the only difference is the response code we can probalby just have one json schema file and override the response code for v3
00:28:08 <ivanzhu> I mean we don't check the response code in tests again
00:28:24 <cyeoh> ivanzhu: so we only check the response code in the client if its a success response code
00:28:28 <ivanzhu> cyeoh:yes
00:28:54 <mrda> ken1ohmichi: where is the list of v2 apis that aren't currently tested?
00:29:04 <ivanzhu> then we can share the v2 tests code in v3 easily
00:29:24 <masayukig> mrda: https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc&usp=drive_web#gid=2
00:29:24 <cyeoh> ivanzhu: so we can probably just do a generic - is it a success code in the test
00:29:31 <mrda> I was assuming that https://etherpad.openstack.org/p/nova-api-attribute-test was the list, and from that it looked good :)
00:29:33 <ken1ohmichi> masayukig: thanks
00:29:56 <ken1ohmichi> mrda: right, thanks:)
00:30:07 <mrda> masayukig: thanks for the link
00:30:25 <cyeoh> #link  https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc&usp=drive_web#gid=2
00:31:03 <cyeoh> anything else on the response validation?
00:31:08 <ken1ohmichi> this task is not only for v2.1 but also current v2, so better to cover not only the defference.
00:32:24 <cyeoh> yes it is very useful to stop us from accidentally making changes to the v2 API as well
00:32:24 <ken1ohmichi> sorry, please go ahead
00:32:47 <cyeoh> np
00:33:01 <cyeoh> #topic SDK port for v3 API
00:33:44 <cyeoh> so this idea came from mikal - that we try before Atlanta to port some SDKs to use the v3 API
00:33:56 <mikal> Want me to do my thing here?
00:34:03 <cyeoh> mikal: please go ahead :-)
00:34:08 <mikal> Earlier in the week I promised cyeoh that I’d try and get some numbers for SDK popularity from Rackspace public cloud. Noting that these numbers are for nova only, here’s what I got back:
00:34:12 <mikal> - python-novaclient
00:34:14 <mikal> - the Rackspace web dashboard
00:34:17 <mikal> - jclouds
00:34:19 <mikal> - pyrax
00:34:22 <mikal> - libcloud
00:34:24 <mikal> - fog
00:34:27 <mikal> So, that’s the top six SDKs that Rackspace sees. If we’re interested in seeing how hard it is to implement v3 in SDKs, jclouds looks like a good place to start.
00:34:46 <cyeoh> are they both SDKs which have multiple cloud backends?
00:35:00 <cyeoh> oops only one there ;-)
00:35:12 <mikal> python-novaclient, the web dashboard an pyrax are all openstack specific IIRC
00:35:17 <mikal> The others are more generic
00:35:25 <mikal> And probably better examples for us anyways
00:35:35 <mrda> libcloud supports about 20 clouds I think
00:35:55 <cyeoh> yea I'd assume for the generic ones that they would have to abstract away all the openstack specific naming etc
00:36:10 <mikal> I believe so
00:36:14 <mikal> Although I've used none of them
00:36:20 <mikal> But I'm hoping that data is useful to you
00:36:25 <mrda> so libcloud won't mark an interface as production unless there are > 2 providers, so as to enforce generic interfaces
00:36:28 <cyeoh> mikal: yes very much so
00:36:37 <mrda> mikal: good stuff!
00:36:51 <mikal> :)
00:37:07 <cyeoh> mrda: so is jclouds something you think you could start looking at?
00:37:10 <mikal> I need to run to another meeting, so unless people have questions I'm going to bail out
00:37:21 <cyeoh> mikal: thanks for coming!
00:37:26 <mikal> NP!
00:37:28 <mikal> Laters
00:37:44 <mrda> thanks mikal
00:37:46 <mrda> cyeoh: potentially, but it is java :)
00:37:55 <cyeoh> yea sorry ;-)
00:38:14 * cyeoh looks to see what libcloud is in
00:38:22 <mrda> if we wanted to keep it python, libcloud would be the choice
00:39:14 <cyeoh> so I don't think we need to keep it python, but it might be a lot faster for you if we did
00:39:24 <mrda> cyeoh: yup
00:39:37 <cyeoh> if libcloud looks really easy maybe then tackle that first and then look at jclouds?
00:39:42 <mrda> I've been actively purging any java knowledge from my brain :)
00:39:49 <cyeoh> :-)
00:40:03 <mrda> so I might look at libcloud then
00:40:20 <cyeoh> sounds good!
00:40:37 <cyeoh> #topic API stability guidelines
00:41:03 <cyeoh> so I put this on the agenda because of the flavor-create issue which came up yesterday
00:41:41 <cyeoh> #link http://lists.openstack.org/pipermail/openstack-dev/2014-March/030508.html
00:42:00 <cyeoh> there's some more info there, but basically we sort of accidentally made a backwards incompatible change to the V2 API back in October
00:42:24 <cyeoh> and we've only recently realised that its broken Horizon which probably means we'll break client scripts
00:42:55 <cyeoh> but the change has been running on continuous deployment clouds for a few months now
00:43:04 <cyeoh> so a revert will hurt them if we take that route
00:43:32 <mrda> I guess the question is, how big a deal is it if this is reverted before release.  Some pain now, but less pain later?
00:43:33 <cyeoh> I don't think we really have a consensus yet (and I don't think there was one in the Nova meeting)
00:43:50 <cyeoh> mrda: yea that's not clear to me how much it will hurt the CD guys
00:44:14 <cyeoh> I guess its an interesting data point that the bug report came from the horizon people, not someone using a public cloud
00:44:31 <cyeoh> so maybe people who use rackspace/HP public clouds don't really use private flavors?
00:44:56 <mrda> By definition, anyone doing CD is expecting change
00:45:09 <clarkb> I'm not sure we can have private flavors in rax/hp public clouds
00:45:14 <mrda> But anyone moving from H to I would like change minimised
00:45:18 <clarkb> they give us a list of flavors and thats what we get
00:45:52 <cyeoh> ok so only people who are doing CD with private clouds would be affected, and I'd guess the list of people who do that is really short?
00:47:53 <cyeoh> so doing the revert may be the least pain solution after all. Does anyone here think that doing the revert would be really bad?
00:48:54 <mrda> I guess this issue highlights the importance of improving api test coverage. So it validates our tempest approach for the POC.
00:49:10 <cyeoh> so embarassingly tempest did pick this up
00:49:28 <mrda> whoops
00:49:28 <cyeoh> and we knew on the Nova side it was backwards incompatible but thought it was ok
00:49:33 <cyeoh> and we changed the tempest test
00:50:16 <cyeoh> I suspect this is a lesson to us that even when we think a small backwards incompatible change is harmless, that its probably not
00:50:41 <cyeoh> and we just have to take a very hard line on pushing back on those changes
00:51:08 <cyeoh> there have been a few other cases in Icehouse when someone has attempted to do a "bug fix" based on what the documentation said rather than what we did, and fortunately we've said no in those cases
00:51:31 <alex_xu> cyeoh, whether we need promise something before release? everything wrong before release is bug?
00:52:17 <cyeoh> alex_xu: so I think that brings up a good point around whether API interfaces (or changes) should be considered "experimental" for say once cycle
00:52:25 <cyeoh> and communicate that well to users
00:52:44 <cyeoh> so we have time to fix mistakes when we stuff up
00:53:52 <alex_xu> yea
00:54:06 <ivanzhu> agree
00:54:29 <cyeoh> I guess its something we should bring up at summit.
00:54:37 <mrda> cyeoh: +1
00:54:42 <alex_xu> +1
00:54:47 <cyeoh> #topic open discussion
00:55:05 <cyeoh> anyone have anything they want to bring up?
00:56:04 <ken1ohmichi> in v2.1 PoC implementation, "create a server" API has been done. how to do PoC?
00:56:04 <cyeoh> ok then. Thanks everyone for attending!
00:56:12 <mrda> thanks cyeoh!
00:56:13 <cyeoh> ken1ohmichi: ah sorry, go ahead :-)
00:56:51 <ken1ohmichi> If we use Tempest for PoC, many tests fails
00:57:04 <cyeoh> ken1ohmichi: what sort of failures?
00:57:17 <ken1ohmichi> because Poc implementation can cover some APIs.
00:57:22 <ken1ohmichi> only
00:58:12 <ken1ohmichi> we need to check Tempest result more deeply for PoC.
00:58:16 <cyeoh> so I guess you can run it locally with a subset of tempest tests to confirm if your patch works
00:58:34 <ken1ohmichi> yes, right. I am doing it now.
00:59:05 <ken1ohmichi> so how to prove PoC for the other people?
00:59:13 <cyeoh> we have an experimental job queue - not sure if its meant for this sort of thing?
00:59:28 <cyeoh> but if it is perhaps we could have one with a subset of tempest tests enabled
00:59:51 <ken1ohmichi> good idea, will talk about it deeply later.
00:59:54 <cyeoh> so we can demonstrate tempest runs that pass and explain why its only a subset
01:00:02 <cyeoh> ken1ohmichi: sounds good
01:00:10 <cyeoh> perhaps you could ping sdague about it?
01:00:12 <ken1ohmichi> the time:-)
01:00:22 <ken1ohmichi> ok, I will do it.
01:00:26 <cyeoh> yep, we're out of time. ok thank you everyone!
01:00:40 <ken1ohmichi> thanks all
01:00:41 <masayukig> thanks
01:00:46 <alex_xu> thanks all
01:00:46 <cyeoh> #endmeeting