13:00:01 #startmeeting nova api 13:00:02 Meeting started Wed Apr 6 13:00:01 2016 UTC and is due to finish in 60 minutes. The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:00:03 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 13:00:06 The meeting name has been set to 'nova_api' 13:00:11 who is here today? 13:00:29 Hi 13:00:37 hi 13:01:08 o/ 13:01:12 o/ 13:01:13 o/ 13:01:31 o/ 13:01:33 hello everyone, and let's start the meeting 13:01:48 #topic API Priorities 13:02:11 For API ref, I think the framework already merged 13:02:28 sdague: any more update for api ref, I didn't see anything more yet 13:02:38 o/ 13:02:45 we're still working on the converter and the initial dump 13:03:28 I'm hoping we'll have a new dump of files this week 13:03:32 that will be close enough to run with 13:03:43 auggy and karen are doing the various converters there 13:03:56 converters should a hard work 13:03:59 and the sphinx extension will probably need some updates to match the new data format 13:04:06 thats the WADL to rst? 13:04:07 yeh, but they are quite close 13:04:11 yes 13:04:15 cool 13:04:16 it's actually 2 bits 13:04:27 there is a wadl2rst project that is making the rst stubs 13:04:41 cool 13:04:42 then a different bit of modified fairy slipper which builds the parameters.yaml 13:05:15 ah, OK, got it 13:05:30 after that, then we probably can try some microversion doc 13:05:38 sounds cool 13:05:53 #link https://review.openstack.org/#/c/300264/ 13:06:01 as an example of some of the parameter dumps 13:06:09 yea, that will be nice but have we decided the doc structure for microversion? 13:06:26 * bauzas lurks behind the door 13:06:44 sdague: that is autogenerated ? 13:06:52 gmann_: those files are 13:06:59 from the wadl sources 13:07:06 sdague: ohh, cool 13:07:35 so, on microversions, the current POC is that parameters introduced at certain versions are tagged as such 13:08:04 and that adds some css that makes it easy to hide them 13:08:10 I guess we can do a similar tag for removals? 13:08:16 right 13:08:31 we also have new api and removed api, but i guess that won't be hard 13:08:45 honestly, I have some vague ideas here but it's hard to really POC that until we get more of our content into a maleable format 13:09:05 sdague: as the doc, will we write URLs with project-id or without project-id? 13:09:11 * johnthetubaguy is excited he can ee glimmers of light at the end of the tunnel 13:09:32 oomichi: for right now, project-id is still in 13:09:52 sdague: oh, ok. 13:09:57 lets decide about when we stop having it there after we get all the content in place 13:10:14 sdague: yeah, +1 13:10:17 +1 13:10:51 sdague: oomichi but doc for v 2.18 onwards we can hide 13:11:10 yea, that we can decide later. 13:11:17 gmann_: right, but we need to be careful how we explain all that to users so it's not hugely confusing 13:11:47 I've got the rough structure of an introduction to urls and versions in my head that should go into the API ref document, but I want to wait until we've got the stub content over 13:11:51 yea that's key point 13:12:15 +1 13:12:24 ok, sounds cool at here, if no more question, let's move on 13:12:49 +1 move on 13:13:24 for another priority item is policy discoverable, but i guess Andrew and claudio_ not here 13:13:33 two specs i saw at here 13:13:38 #link https://review.openstack.org/289405 13:13:47 #link https://review.openstack.org/290155 13:14:19 it's early for laski, claudio is in this TZ, maybe we can poke him to join next time 13:14:34 so there was discussion in channel 13:14:47 I really like us moving the policy defaults into the code 13:14:52 From the discussion, the embed policy one sounds good, the policy api one have some discussion how to present it. 13:14:56 sdague: +1 13:14:56 so folks run with an empty policy config file 13:15:13 and we can generate a sample file, with all the rules commented out 13:15:37 if we get that done this cycle, I think we are in a better position 13:15:51 johnthetubaguy: +1 13:16:00 yeh, agreed 13:16:01 I was talking to dolph and he supported the idea, seems like jaminelennox liked it too 13:16:13 I believe that most of the keystone folks are on board with it 13:16:25 not sure they agree which release that happens in, but yeah, seems to be a good level of agreement on that 13:16:46 sdague: that is good news 13:16:47 so, honestly, I think from a feature perspective we should focus on that one this cycle 13:16:55 sdague: +1 13:17:04 because I think if we got us all involved there, we could maybe get it done in a cycle 13:17:04 * mriedem joins super late 13:17:25 and it would be nice to have everyone working on nova api stuff working on kind of the same feature and making progress 13:17:41 alex_xu, gmann_ what do you think? 13:18:07 sdague: yea, i'm in 13:18:13 sdague: totally agree. +1 13:18:15 I am hoping some of the OSIC folks might have time to help with the policy heavy lifting 13:18:30 so next step please provide review on the specs. Lets try to get those close by summit 13:18:37 we're going to have a dedicated session there on them 13:19:13 I am trying to write up my crazy thoughts around moving to oslo_config for the policy stuff, it seems less crazy when I think about it more 13:19:16 yea that will be nice, hope that session does not conflict with QA ones:) 13:19:27 johnthetubaguy: +1, a lot of folks from OSIC :) 13:20:01 gmann_: we can work with oomichi to make sure 13:20:06 hmm 13:20:16 yea, oomichi is here for that. 13:20:19 right now where i have it penciled in for nova it does, we'll have to adjust that 13:20:26 last 2 sessions on wed 13:20:36 we can move some though 13:21:01 nova sessions always run on the summit.. 13:21:22 yeah, we can arrange it 13:21:41 thurs morning might work 13:21:57 there are no QA sessions on thurs until 11:50 13:22:15 oomichi: should we plan on thurs morning then? 13:22:20 9 and 9:50 13:22:42 the api was in Wed 13:22:57 mriedem: can we talk about it later, need to check my sched more 13:23:10 alex_xu: i just moved it 13:23:11 oomichi: sure 13:23:16 mriedem: thanks 13:23:18 fyi https://etherpad.openstack.org/p/newton-nova-summit-ideas 13:23:22 mriedem: yea, i just saw that 13:23:24 bottom of there is the draft nova schedule 13:23:51 yea Thurs morning is nice 13:25:10 ok, cool, next topic? 13:25:35 cool, oomichi, if you get detail from you sched, you probably can reach to mriedem directly 13:25:50 #topic Nova Microversion testing in Tempest 13:25:57 gmann_: your turn 13:26:03 sure 13:26:07 alex_xu: yeah, will do that 13:26:19 so v2.10 tests are merged and running in gate 13:26:32 v2.20 one i need to update. 13:26:47 oomichi: mriedem, thanks 13:26:50 So overall we have all framework in and doc also how to implement microversion tests 13:26:53 #link http://docs.openstack.org/developer/tempest/microversion_testing.html 13:26:57 gmann_: cool 13:27:12 hope i documented those well 13:27:23 so we can start implementing all now. 13:27:43 are those going in order? 13:27:44 gmann_: where is > v2.20, it needs more people to help on? 13:27:53 I will keep implementing those and try to finish all M version in N cycle at least 13:27:58 gmann_: http://docs.openstack.org/developer/tempest/microversion_testing.html#microversion-tests-implemented-in-tempest doesn't contain v2.10 13:28:15 oomichi: ops, i will add :) nice catch 13:28:17 need to fix later ;) 13:28:46 alex_xu: yea that will be always nice, if we get more people on that 13:29:03 gmann_: thanks 13:29:03 but i think if people are interesting on that, free to help on that 13:29:05 only hard thing there can be response schema versioning 13:29:24 Is there a spec or document somewhere that describes why microversion testing in tempest versus in functional or otherwise in-tree tests? 13:30:17 cdent: can you rephrase the question? 13:30:25 because I'm not sure I understand the alternative 13:30:56 The essential question is: Why test multiple microversions in tempest? 13:31:02 so i intended to describe that here: http://docs.openstack.org/developer/nova/test_strategy.html 13:31:09 cdent: we have tempest spec which describe the testing need in tempest but not exactly the things vs in-tree 13:31:10 #link https://github.com/openstack/qa-specs/blob/master/specs/tempest/implemented/api-microversions-testing-support.rst 13:31:16 With the secondary question of: How is it better than having those tests in functional tests? 13:31:17 oh, wait, thats a different question, we did write that somewhere 13:31:35 thanks gmann_ 13:31:37 cdent: ok, so example. We add a parameter to a resource in 2.25 13:31:50 are you suggesting that tempest now only tests that resource with the new parameter? 13:32:02 which means now tempest fails on all clouds not on 2.25? 13:32:20 I'm not really suggesting anything in particular, just trying to get a more clear picture. 13:32:42 tempest is a test suite that tests all active branches of openstack 13:32:58 If anything I would expect tempest to do microversion discovery on the cloud in question and "do the righ tthing" 13:33:10 which is kind of what it does 13:33:11 cdent: along with single project version tests, tempest microversion testing can help in cross projects microversion testing too 13:33:33 i mean conbination of microversion supported by different services 13:34:04 there are specific tests that are annotated for the microversions that they work on, and are only run if you specify your cloud is supposed to work with those 13:34:07 What my brain is attempting to clarify is if tempest is attempting to validate artibtrary clouds against multiple microversions when it runs. 13:34:36 Sounds like it is doing something akin to the right thing, so that's cool. 13:34:39 right 13:34:57 yea 13:35:30 I honestly feel like jay just sent a random email about it without looking into what was actually happening 13:35:37 we did think this through a bit :) 13:35:55 :) 13:36:09 The overlap or lack thereof with in-tree tests is still something I'd like to know more about but if that can come from something I can read that would be great. 13:36:16 Oh, I wasn't even thinking about jay's email, I had forgotten about that. 13:36:38 sorry, your question was very much like his email 13:36:56 The genesis of this was latent "we should really let old stuff break more often, people are resilient" 13:37:14 cdent: our experience is that they aren't :) 13:37:29 s/are/(should be|must be)/ 13:37:34 * cdent is mean 13:37:43 yeh, that just drives people to other platforms 13:38:06 * cdent holds his tongue so as not to distract the topic even more 13:38:14 :) 13:38:16 beer talk 13:38:25 heh 13:38:39 this is logged ;) 13:38:42 ok, so that means let move on? :) 13:38:45 :) 13:38:47 alex_xu: yep 13:38:48 yea, we can move i think, 13:38:57 #topic open 13:39:15 Spec review for project_id validation with keystone: https://review.openstack.org/#/c/294337/ 13:39:24 mriedem: your turn 13:39:33 * edleafe is fighting my internet connection today 13:39:41 so vaguely related to previous topic: sileht has figured out how to make gabbi work with tempest, which might be relevant for future api-in-tempest work if people are into that sort of thing 13:40:12 i don't have much to say about https://review.openstack.org/#/c/294337/ - i'm just advertising it here 13:40:12 mriedem: or it is just a review request 13:40:26 when i went through it there were several times that i said, 'you should probably ask the api subteam about this' 13:40:41 it is a re-proposal from a previously approved spec and auggy is picking it up 13:40:44 cdent: oh, do you have any link,poc etc for that. anyways let's catch that later 13:40:46 mriedem: ok, cool, i remember was approved long time ago, it's worth to revisit again 13:40:58 and there are a bunch of bugs that have been duplicated against this, so it is semi priority from a UX perspective 13:41:15 mriedem: how do we have enough credentials to do this for quota changes, for instance? 13:41:15 gmann_: https://review.openstack.org/#/c/301585/ 13:41:35 sdague: not sure i understand your question 13:42:25 how do we know if a project_id is valid without giving nova keystone admin credentials 13:42:35 which... let nova delete arbitrary users as well 13:42:35 sdague: that's part of the proposed change 13:42:55 see the security impact section 13:43:30 do we have keystone folks looking at that spec yet? 13:43:32 it's not proposing admin creds, from what i can tell, it's more of a service account 13:43:43 i added bknudson 13:44:00 but i haven't been loud about it in their channel yet 13:44:25 it would be interesting to understand the implications of asking for an existance test for normal users 13:44:26 #action mriedem to get some keystone people to review spec https://review.openstack.org/#/c/294337/ 13:44:47 GET /project_id/444444444444444 13:44:53 being always an allowed thing 13:45:03 I wonder how bad the security would be on that 13:45:08 yes, and since we have default quotas we return them if the project doesn't exist 13:45:21 even worse, we create stuff in the nova db quotas tables for the invalid id 13:45:25 if that return a 403 or something like that... 13:45:26 I mean on the keystone side 13:45:29 nova has a service user so we can use that (although I think it's given admin by default) 13:45:52 bknudson: ok, maybe this is the audit role 13:45:54 or something 13:46:24 because we are litterally asking for 2 things. Is this project_id valid, is this user_id valid 13:46:34 we don't even need info about them 13:46:41 we just need to know they exist 13:46:56 this is for cases where the user is referencing another user or a different project? 13:47:00 yes 13:47:07 because if it's the same user or the scope of the project it's in the token 13:47:09 where the admin is adjusting quota, for instance 13:47:17 that's the canonical case 13:47:46 sounds easy enough 13:47:56 but there are some other ones where the admin is doing a thing to a user's resource 13:48:08 and it's really easy to typo the user_id 13:48:13 and end up not doing the thing you expect 13:48:23 and not knowing until way down the road 13:48:50 i don't think it's all admin 13:48:52 bknudson: maybe just pile on https://review.openstack.org/#/c/294337 13:48:56 e.g. "compute_extension:quotas:show": "rule:admin_or_owner", 13:49:12 mriedem: sure, but owner is easy 13:50:35 umm 13:50:40 "os_compute_api:os-quota-sets:defaults": "@", 13:50:45 that takes a project id 13:50:48 and anyone can run it 13:51:06 but, you can't change another users' thing if you aren't the admin 13:51:10 it's caught later 13:51:21 i think defaults probably doesn't matter anyway 13:51:32 since that's the global defaults i think 13:51:53 the crux of the problem is really an admin or super prived user in nova doesn't mean it's keystone admin 13:52:18 super prived nova user should be able to ask whether certain keystone things exist though 13:52:30 conceptually 13:52:38 then this becomes just code in nova 13:52:56 but I think that's keystone changes to make it sane 13:53:14 * breton -1d the spec 13:54:31 breton: I don't understand how your suggestion fixes anything 13:56:04 i don't either, i replied inline 13:56:17 * alex_xu reminders 3 mins left 13:56:26 let's move this to the spec 13:56:32 bknudson: can you take a look at that one at some point? 13:56:41 mriedem: yep, it's on my short list 13:56:45 thanks 13:56:59 so any more opens, otherwise we will close the meeting in 3 mins 13:57:44 3... 13:57:46 2.. 13:57:49 1. 13:58:07 ok, thanks for all! 13:58:13 Thanks all 13:58:16 #endmeeting