13:00:01 <alex_xu> #startmeeting nova api
13:00:02 <openstack> Meeting started Wed Apr  6 13:00:01 2016 UTC and is due to finish in 60 minutes.  The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:06 <openstack> The meeting name has been set to 'nova_api'
13:00:11 <alex_xu> who is here today?
13:00:29 <tojuvone> Hi
13:00:37 <gmann_> hi
13:01:08 <sdague> o/
13:01:12 <sfinucan> o/
13:01:13 <johnthetubaguy> o/
13:01:31 <jichen> o/
13:01:33 <alex_xu> hello everyone, and let's start the meeting
13:01:48 <alex_xu> #topic API Priorities
13:02:11 <alex_xu> For API ref, I think the framework already merged
13:02:28 <alex_xu> sdague: any more update for api ref, I didn't see anything more yet
13:02:38 <cdent> o/
13:02:45 <sdague> we're still working on the converter and the initial dump
13:03:28 <sdague> I'm hoping we'll have a new dump of files this week
13:03:32 <sdague> that will be close enough to run with
13:03:43 <sdague> auggy and karen are doing the various converters there
13:03:56 <alex_xu> converters should a hard work
13:03:59 <sdague> and the sphinx extension will probably need some updates to match the new data format
13:04:06 <johnthetubaguy> thats the WADL to rst?
13:04:07 <sdague> yeh, but they are quite close
13:04:11 <sdague> yes
13:04:15 <johnthetubaguy> cool
13:04:16 <sdague> it's actually 2 bits
13:04:27 <sdague> there is a wadl2rst project that is making the rst stubs
13:04:41 <alex_xu> cool
13:04:42 <sdague> then a different bit of modified fairy slipper which builds the parameters.yaml
13:05:15 <johnthetubaguy> ah, OK, got it
13:05:30 <alex_xu> after that, then we probably can try some microversion doc
13:05:38 <alex_xu> sounds cool
13:05:53 <sdague> #link https://review.openstack.org/#/c/300264/
13:06:01 <sdague> as an example of some of the parameter dumps
13:06:09 <gmann_> yea, that will be nice but have we decided the doc structure for microversion?
13:06:26 * bauzas lurks behind the door
13:06:44 <gmann_> sdague: that is autogenerated ?
13:06:52 <sdague> gmann_: those files are
13:06:59 <sdague> from the wadl sources
13:07:06 <gmann_> sdague: ohh, cool
13:07:35 <sdague> so, on microversions, the current POC is that parameters introduced at certain versions are tagged as such
13:08:04 <sdague> and that adds some css that makes it easy to hide them
13:08:10 <johnthetubaguy> I guess we can do a similar tag for removals?
13:08:16 <sdague> right
13:08:31 <alex_xu> we also have new api and removed api, but i guess that won't be hard
13:08:45 <sdague> honestly, I have some vague ideas here but it's hard to really POC that until we get more of our content into a maleable format
13:09:05 <oomichi> sdague: as the doc, will we write URLs with project-id or without project-id?
13:09:11 * johnthetubaguy is excited he can ee glimmers of light at the end of the tunnel
13:09:32 <sdague> oomichi: for right now, project-id is still in
13:09:52 <oomichi> sdague: oh, ok.
13:09:57 <sdague> lets decide about when we stop having it there after we get all the content in place
13:10:14 <oomichi> sdague: yeah, +1
13:10:17 <alex_xu> +1
13:10:51 <gmann_> sdague: oomichi but doc for v 2.18 onwards we can hide
13:11:10 <gmann_> yea, that we can decide later.
13:11:17 <sdague> gmann_: right, but we need to be careful how we explain all that to users so it's not hugely confusing
13:11:47 <sdague> I've got the rough structure of an introduction to urls and versions in my head that should go into the API ref document, but I want to wait until we've got the stub content over
13:11:51 <gmann_> yea that's key point
13:12:15 <gmann_> +1
13:12:24 <alex_xu> ok, sounds cool at here, if no more question, let's move on
13:12:49 <sdague> +1 move on
13:13:24 <alex_xu> for another priority item is policy discoverable, but i guess Andrew and claudio_ not here
13:13:33 <alex_xu> two specs i saw at here
13:13:38 <alex_xu> #link https://review.openstack.org/289405
13:13:47 <alex_xu> #link https://review.openstack.org/290155
13:14:19 <sdague> it's early for laski, claudio is in this TZ, maybe we can poke him to join next time
13:14:34 <johnthetubaguy> so there was discussion in channel
13:14:47 <johnthetubaguy> I really like us moving the policy defaults into the code
13:14:52 <alex_xu> From the discussion, the embed policy one sounds good, the policy api one have some discussion how to present it.
13:14:56 <alex_xu> sdague: +1
13:14:56 <johnthetubaguy> so folks run with an empty policy config file
13:15:13 <johnthetubaguy> and we can generate a sample file, with all the rules commented out
13:15:37 <johnthetubaguy> if we get that done this cycle, I think we are in a better position
13:15:51 <alex_xu> johnthetubaguy: +1
13:16:00 <sdague> yeh, agreed
13:16:01 <johnthetubaguy> I was talking to dolph and he supported the idea, seems like jaminelennox liked it too
13:16:13 <sdague> I believe that most of the keystone folks are on board with it
13:16:25 <johnthetubaguy> not sure they agree which release that happens in, but yeah, seems to be a good level of agreement on that
13:16:46 <alex_xu> sdague: that is good news
13:16:47 <sdague> so, honestly, I think from a feature perspective we should focus on that one this cycle
13:16:55 <johnthetubaguy> sdague: +1
13:17:04 <sdague> because I think if we got us all involved there, we could maybe get it done in a cycle
13:17:04 * mriedem joins super late
13:17:25 <sdague> and it would be nice to have everyone working on nova api stuff working on kind of the same feature and making progress
13:17:41 <sdague> alex_xu, gmann_ what do you think?
13:18:07 <alex_xu> sdague: yea, i'm in
13:18:13 <gmann_> sdague: totally agree. +1
13:18:15 <johnthetubaguy> I am hoping some of the OSIC folks might have time to help with the policy heavy lifting
13:18:30 <sdague> so next step please provide review on the specs. Lets try to get those close by summit
13:18:37 <sdague> we're going to have a dedicated session there on them
13:19:13 <johnthetubaguy> I am trying to write up my crazy thoughts around moving to oslo_config for the policy stuff, it seems less crazy when I think about it more
13:19:16 <gmann_> yea that will be nice, hope that session does not conflict with QA ones:)
13:19:27 <alex_xu> johnthetubaguy: +1, a lot of folks from OSIC :)
13:20:01 <sdague> gmann_: we can work with oomichi to make sure
13:20:06 <mriedem> hmm
13:20:16 <gmann_> yea, oomichi is here for that.
13:20:19 <mriedem> right now where i have it penciled in for nova it does, we'll have to adjust that
13:20:26 <mriedem> last 2 sessions on wed
13:20:36 <mriedem> we can move some though
13:21:01 <oomichi> nova sessions always run on the summit..
13:21:22 <oomichi> yeah, we can arrange it
13:21:41 <mriedem> thurs morning might work
13:21:57 <mriedem> there are no QA sessions on thurs until 11:50
13:22:15 <mriedem> oomichi: should we plan on thurs morning then?
13:22:20 <mriedem> 9 and 9:50
13:22:42 <alex_xu> the api was in Wed
13:22:57 <oomichi> mriedem: can we talk about it later, need to check my sched more
13:23:10 <mriedem> alex_xu: i just moved it
13:23:11 <mriedem> oomichi: sure
13:23:16 <oomichi> mriedem: thanks
13:23:18 <mriedem> fyi https://etherpad.openstack.org/p/newton-nova-summit-ideas
13:23:22 <alex_xu> mriedem: yea, i just saw that
13:23:24 <mriedem> bottom of there is the draft nova schedule
13:23:51 <gmann_> yea Thurs morning is nice
13:25:10 <sdague> ok, cool, next topic?
13:25:35 <alex_xu> cool, oomichi, if you get detail from you sched, you probably can reach to mriedem directly
13:25:50 <alex_xu> #topic Nova Microversion testing in Tempest
13:25:57 <alex_xu> gmann_: your turn
13:26:03 <gmann_> sure
13:26:07 <oomichi> alex_xu: yeah, will do that
13:26:19 <gmann_> so v2.10 tests are merged and running in gate
13:26:32 <gmann_> v2.20 one i need to update.
13:26:47 <alex_xu> oomichi: mriedem, thanks
13:26:50 <gmann_> So overall we have all framework in and doc also how to implement microversion tests
13:26:53 <gmann_> #link http://docs.openstack.org/developer/tempest/microversion_testing.html
13:26:57 <oomichi> gmann_: cool
13:27:12 <gmann_> hope i documented those well
13:27:23 <gmann_> so we can start implementing all now.
13:27:43 <mriedem> are those going in order?
13:27:44 <alex_xu> gmann_: where is > v2.20, it needs more people to help on?
13:27:53 <gmann_> I will keep implementing those and try to finish all M version in N cycle at least
13:27:58 <oomichi> gmann_: http://docs.openstack.org/developer/tempest/microversion_testing.html#microversion-tests-implemented-in-tempest doesn't contain v2.10
13:28:15 <gmann_> oomichi: ops, i will add :) nice catch
13:28:17 <oomichi> need to fix later ;)
13:28:46 <gmann_> alex_xu: yea that will be always nice, if we get more people on that
13:29:03 <alex_xu> gmann_: thanks
13:29:03 <alex_xu> but i think if people are interesting on that, free to help on that
13:29:05 <gmann_> only hard thing there can be response schema versioning
13:29:24 <cdent> Is there a spec or document somewhere that describes why microversion testing in tempest versus in functional or otherwise in-tree tests?
13:30:17 <sdague> cdent: can you rephrase the question?
13:30:25 <sdague> because I'm not sure I understand the alternative
13:30:56 <cdent> The essential question is: Why test multiple microversions in tempest?
13:31:02 <johnthetubaguy> so i intended to describe that here: http://docs.openstack.org/developer/nova/test_strategy.html
13:31:09 <gmann_> cdent: we have tempest spec which describe the testing need in tempest but not exactly the things vs in-tree
13:31:10 <gmann_> #link https://github.com/openstack/qa-specs/blob/master/specs/tempest/implemented/api-microversions-testing-support.rst
13:31:16 <cdent> With the secondary question of: How is it better than having those tests in functional tests?
13:31:17 <johnthetubaguy> oh, wait, thats a different question, we did write that somewhere
13:31:35 <cdent> thanks gmann_
13:31:37 <sdague> cdent: ok, so example. We add a parameter to a resource in 2.25
13:31:50 <sdague> are you suggesting that tempest now only tests that resource with the new parameter?
13:32:02 <sdague> which means now tempest fails on all clouds not on 2.25?
13:32:20 <cdent> I'm not really suggesting anything in particular, just trying to get a more clear picture.
13:32:42 <sdague> tempest is a test suite that tests all active branches of openstack
13:32:58 <cdent> If anything I would expect tempest to do microversion discovery on the cloud in question and "do the righ tthing"
13:33:10 <sdague> which is kind of what it does
13:33:11 <gmann_> cdent: along with single project version tests, tempest microversion testing can help in cross projects microversion testing too
13:33:33 <gmann_> i mean conbination of microversion supported by different services
13:34:04 <sdague> there are specific tests that are annotated for the microversions that they work on, and are only run if you specify your cloud is supposed to work with those
13:34:07 <cdent> What my brain is attempting to clarify is if tempest is attempting to validate artibtrary clouds against multiple microversions when it runs.
13:34:36 <cdent> Sounds like it is doing something akin to the right thing, so that's cool.
13:34:39 <sdague> right
13:34:57 <gmann_> yea
13:35:30 <sdague> I honestly feel like jay just sent a random email about it without looking into what was actually happening
13:35:37 <sdague> we did think this through a bit :)
13:35:55 <gmann_> :)
13:36:09 <cdent> The overlap or lack thereof with in-tree tests is still something I'd like to know more about but if that can come from something I can read that would be great.
13:36:16 <cdent> Oh, I wasn't even thinking about jay's email, I had forgotten about that.
13:36:38 <sdague> sorry, your question was very much like his email
13:36:56 <cdent> The genesis of this was latent "we should really let old stuff  break more often, people are resilient"
13:37:14 <sdague> cdent: our experience is that they aren't :)
13:37:29 <cdent> s/are/(should be|must be)/
13:37:34 * cdent is mean
13:37:43 <sdague> yeh, that just drives people to other platforms
13:38:06 * cdent holds his tongue so as not to distract the topic even more
13:38:14 <sdague> :)
13:38:16 <sdague> beer talk
13:38:25 <gmann_> heh
13:38:39 <oomichi> this is logged ;)
13:38:42 <alex_xu> ok, so that means let move on? :)
13:38:45 <cdent> :)
13:38:47 <sdague> alex_xu: yep
13:38:48 <gmann_> yea, we can move i think,
13:38:57 <alex_xu> #topic open
13:39:15 <alex_xu> Spec review for project_id validation with keystone: https://review.openstack.org/#/c/294337/
13:39:24 <alex_xu> mriedem: your turn
13:39:33 * edleafe is fighting my internet connection today
13:39:41 <cdent> so vaguely related to previous topic: sileht has figured out how to make gabbi work with tempest, which might be relevant for future api-in-tempest work if people are into that sort of thing
13:40:12 <mriedem> i don't have much to say about https://review.openstack.org/#/c/294337/ - i'm just advertising it here
13:40:12 <alex_xu> mriedem: or it is just a review request
13:40:26 <mriedem> when i went through it there were several times that i said, 'you should probably ask the api subteam about this'
13:40:41 <mriedem> it is a re-proposal from a previously approved spec and auggy is picking it up
13:40:44 <gmann_> cdent: oh, do you have any link,poc etc for that. anyways let's catch that later
13:40:46 <alex_xu> mriedem: ok, cool, i remember was approved long time ago, it's worth to revisit again
13:40:58 <mriedem> and there are a bunch of bugs that have been duplicated against this, so it is semi priority from a UX perspective
13:41:15 <sdague> mriedem: how do we have enough credentials to do this for quota changes, for instance?
13:41:15 <cdent> gmann_: https://review.openstack.org/#/c/301585/
13:41:35 <mriedem> sdague: not sure i understand your question
13:42:25 <sdague> how do we know if a project_id is valid without giving nova keystone admin credentials
13:42:35 <sdague> which... let nova delete arbitrary users as well
13:42:35 <mriedem> sdague: that's part of the proposed change
13:42:55 <mriedem> see the security impact section
13:43:30 <sdague> do we have keystone folks looking at that spec yet?
13:43:32 <mriedem> it's not proposing admin creds, from what i can tell, it's more of a service account
13:43:43 <mriedem> i added bknudson
13:44:00 <mriedem> but i haven't been loud about it in their channel yet
13:44:25 <sdague> it would be interesting to understand the implications of asking for an existance test for normal users
13:44:26 <mriedem> #action mriedem to get some keystone people to review spec https://review.openstack.org/#/c/294337/
13:44:47 <sdague> GET /project_id/444444444444444
13:44:53 <sdague> being always an allowed thing
13:45:03 <sdague> I wonder how bad the security would be on that
13:45:08 <mriedem> yes, and since we have default quotas we return them if the project doesn't exist
13:45:21 <mriedem> even worse, we create stuff in the nova db quotas tables for the invalid id
13:45:25 <alex_xu> if that return a 403 or something like that...
13:45:26 <sdague> I mean on the keystone side
13:45:29 <bknudson> nova has a service user so we can use that (although I think it's given admin by default)
13:45:52 <sdague> bknudson: ok, maybe this is the audit role
13:45:54 <sdague> or something
13:46:24 <sdague> because we are litterally asking for 2 things. Is this project_id valid, is this user_id valid
13:46:34 <sdague> we don't even need info about them
13:46:41 <sdague> we just need to know they exist
13:46:56 <bknudson> this is for cases where the user is referencing another user or a different project?
13:47:00 <sdague> yes
13:47:07 <bknudson> because if it's the same user or the scope of the project it's in the token
13:47:09 <sdague> where the admin is adjusting quota, for instance
13:47:17 <sdague> that's the canonical case
13:47:46 <bknudson> sounds easy enough
13:47:56 <sdague> but there are some other ones where the admin is doing a thing to a user's resource
13:48:08 <sdague> and it's really easy to typo the user_id
13:48:13 <sdague> and end up not doing the thing you expect
13:48:23 <sdague> and not knowing until way down the road
13:48:50 <mriedem> i don't think it's all admin
13:48:52 <sdague> bknudson: maybe just pile on https://review.openstack.org/#/c/294337
13:48:56 <mriedem> e.g. "compute_extension:quotas:show": "rule:admin_or_owner",
13:49:12 <sdague> mriedem: sure, but owner is easy
13:50:35 <mriedem> umm
13:50:40 <mriedem> "os_compute_api:os-quota-sets:defaults": "@",
13:50:45 <mriedem> that takes a project id
13:50:48 <mriedem> and anyone can run it
13:51:06 <sdague> but, you can't change another users' thing if you aren't the admin
13:51:10 <sdague> it's caught later
13:51:21 <mriedem> i think defaults probably doesn't matter anyway
13:51:32 <mriedem> since that's the global defaults i think
13:51:53 <sdague> the crux of the problem is really an admin or super prived user in nova doesn't mean it's keystone admin
13:52:18 <sdague> super prived nova user should be able to ask whether certain keystone things exist though
13:52:30 <sdague> conceptually
13:52:38 <sdague> then this becomes just code in nova
13:52:56 <sdague> but I think that's keystone changes to make it sane
13:53:14 * breton -1d the spec
13:54:31 <sdague> breton: I don't understand how your suggestion fixes anything
13:56:04 <mriedem> i don't either, i replied inline
13:56:17 * alex_xu reminders 3 mins left
13:56:26 <mriedem> let's move this to the spec
13:56:32 <mriedem> bknudson: can you take a look at that one at some point?
13:56:41 <bknudson> mriedem: yep, it's on my short list
13:56:45 <mriedem> thanks
13:56:59 <alex_xu> so any more opens, otherwise we will close the meeting in 3 mins
13:57:44 <alex_xu> 3...
13:57:46 <alex_xu> 2..
13:57:49 <alex_xu> 1.
13:58:07 <alex_xu> ok, thanks for all!
13:58:13 <gmann_> Thanks all
13:58:16 <alex_xu> #endmeeting