13:00:28 <alex_xu> #startmeeting nova api
13:00:29 <openstack> Meeting started Wed Oct 12 13:00:28 2016 UTC and is due to finish in 60 minutes.  The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:33 <openstack> The meeting name has been set to 'nova_api'
13:00:35 <alex_xu> who is here today
13:00:48 <edleafe> \o
13:00:48 <gmann_> hi
13:00:50 <cdent> hi
13:00:57 <Kevin_Zheng> o/
13:00:59 <mriedem> o/
13:01:31 <alex_xu> let us wait two more minutes
13:02:24 <sdague> o/
13:02:39 <johnthetubaguy> o/
13:02:44 <alex_xu> sdague: johnthetubaguy cool
13:02:50 <alex_xu> let us start the meeting
13:02:57 <claudiub> o/
13:02:59 <alex_xu> #topic Summit ideas
13:03:19 <alex_xu> so..let us talk about summit ideas first
13:03:35 <alex_xu> #link https://etherpad.openstack.org/p/ocata-nova-summit-api
13:03:45 <alex_xu> ^ this is the etherpad for summit api session
13:03:52 <alex_xu> #link https://etherpad.openstack.org/p/nova_api_ocata_ideas
13:04:13 <alex_xu> ^ and I wrote down few things in another etherpad
13:05:26 <mriedem> fixing the sort/filter keys for listing servers is going to be a priority for ocata b/c of multi cells
13:05:35 <mriedem> we had a bug for that
13:05:45 * mriedem looks it up
13:05:49 <sdague> that is mostly about being strict on allowed keys, right?
13:05:53 <mriedem> yeah
13:05:56 <mriedem> so not instance.info_cache
13:05:59 <sdague> right
13:06:06 <alex_xu> sdague: yes
13:06:14 <mriedem> https://bugs.launchpad.net/nova/+bug/1609039
13:06:14 <openstack> Launchpad bug 1609039 in OpenStack Compute (nova) "Should not be able to sort instances based on joined tables" [High,In progress] - Assigned to Zhenyu Zheng (zhengzhenyu)
13:06:14 <alex_xu> and there is bug fix from Kevin_Zheng https://review.openstack.org/359134
13:06:23 <alex_xu> mriedem: yea, also that one
13:06:26 <sdague> there is a piece of infrastructure that is needed with is a jsonschema equivalent for parameters
13:07:01 <johnthetubaguy> yeah, would be good to finish off parameter validation
13:07:27 <sdague> I would start with generic parameter validation module, and then use that to apply to sort keys
13:08:20 <alex_xu> sdague: so we should merge the bug fix from Kevin_Zheng first, then use micorversion to strict the allowed keys?
13:08:43 <johnthetubaguy> sdague: you are right, we should add layer of validation, like we have for the body
13:09:12 <sdague> alex_xu: maybe... honestly it would be nice to have the validation layer to handle it all
13:09:23 <alex_xu> sdague: ok
13:09:30 <mriedem> https://review.openstack.org/#/c/359134/15/nova/api/openstack/compute/servers.py looks hard to maintain to me
13:09:33 <sdague> jsonschema has been really useful to ensure we are only working with good known data
13:09:38 <mriedem> i.e. easy to miss something and regress
13:09:46 <sdague> mriedem: yes, agreed
13:09:52 <alex_xu> mriedem: agreed also
13:10:49 <mriedem> i always envisioned the fix would involve some sqlalchemy magic on the instances table to get the list of columns that excludes joined tables
13:10:53 <gmann_> sdague: we will be having some json schema for all allowed query param and validate that in generic infra?
13:10:58 <sdague> plus, sorting by ephemeral_gb is going to be a DOS attack on large environments. Really nothing should be in the sort list that doesn't have an index
13:11:06 <sdague> gmann_: something like that
13:11:13 <gmann_> k
13:11:30 <sdague> mriedem: so, actually, I would push against magic
13:11:41 <sdague> I do think we're talking about large defined listed
13:11:50 <johnthetubaguy> I like how jsonscheme dropped a whole load of brittle code
13:11:59 <johnthetubaguy> yeah, a defined list seems safest at this point
13:12:00 <sdague> we're going to say that sort_keys is a enum, and here are the values
13:12:16 <mriedem> yeah i'm fine with that
13:12:40 <sdague> and this validation layer should return consistent error messages back to the user as well
13:12:41 <gmann_> that sounds nice
13:12:44 <sdague> like jsonschema
13:13:11 <mriedem> if we restrict to only columns that are indexed, i'd want to get agreement on that from the ops people first
13:13:11 <sdague> ok... so who can champion this and draft up a spec?
13:13:16 <mriedem> since we don't know what they are using
13:13:28 <johnthetubaguy> there is the obvious follow task of validating all the parameters, feels like a good idea really.
13:13:59 <sdague> mriedem: sure... honestly, we could make the first pass, which applies retroactively, be the "not completely dangerous list"
13:14:10 <sdague> and then do as alex_xu said and mv in a stricter list
13:14:43 <alex_xu> i can help on if nobody raise hand
13:15:25 <mriedem> i don't know the jsonschema plumbing well enough
13:15:28 <mriedem> to add a layer for that
13:15:31 <johnthetubaguy> alex_xu: I was of a similar view, i could help. Its important hardening
13:16:08 <alex_xu> johnthetubaguy: cool
13:16:25 <sdague> alex_xu: great, can you have a spec by next week?
13:16:41 <alex_xu> sdague: yea, I can
13:16:42 <sdague> #action alex_xu to write first draft of spec on parameter validation for next week
13:17:32 <sdague> alex_xu: great, thanks
13:17:35 <gmann_> alex_xu: sdague may be validation can be done with some overridden jsonscham validator like we do for some specific elements like UUID format etc ?
13:17:39 <alex_xu> sdague: np
13:17:49 <gmann_> and pass whatever we get in query
13:18:09 <sdague> gmann_: maybe. It might be based on something else as well. We'll let alex_xu run with it and provide the options.
13:18:24 <gmann_> yea.
13:18:29 <alex_xu> gmann_: yea, I will try that idea
13:18:55 <gmann_> sdague: alex_xu: sure, thanks
13:19:06 <mriedem> i also added https://bugs.launchpad.net/nova/+bug/1567655 to the idea etherpad
13:19:06 <openstack> Launchpad bug 1567655 in OpenStack Compute (nova) "500 error when trying to list instances and neutron-server is down" [Medium,In progress] - Assigned to Prateek Arora (parora)
13:19:12 <mriedem> about caching instance.security_groups
13:19:23 <mriedem> so we don't roundtrip to neutron for every instance when listing instances
13:19:50 <mriedem> it's always been a performance problem, but with TLS in the gate now we're also seeing connection failures due to that
13:20:17 <mriedem> it's less of a REST API thing though, so not sure if it fits into this list
13:20:55 <johnthetubaguy> I have this bit up for review: https://review.openstack.org/#/c/382414/
13:21:06 <mriedem> i need to review that today
13:21:07 <johnthetubaguy> to stop us having to cache
13:21:21 <mriedem> you're just not going to show sec groups on a server GET?
13:21:51 <mriedem> "server response, no longer includes security_groups"
13:21:51 <mriedem> yeah
13:21:52 <mriedem> you could provide a link to the API to list them for a given server
13:21:55 <mriedem> that's what i was going to do
13:23:11 <johnthetubaguy> I don't like those links
13:23:12 <gmann_> mriedem: but it might be little bit dangerous to cashing sec grp
13:23:25 <johnthetubaguy> really, if you want port_uuid info, go list the port
13:23:39 <gmann_> or show something "Not Available" etc
13:23:46 <johnthetubaguy> I guess we could add a link to the port, but I duno if its worth all the bother
13:24:14 <mriedem> i need to read the spec in detail and tinker with the neutron apis
13:24:20 <johnthetubaguy> gmann_: we cache almost everything else like floating ip, and they can get out of sync
13:24:28 <johnthetubaguy> mriedem: sounds good
13:24:30 <mriedem> was just going to say that ^
13:25:06 <johnthetubaguy> I would like us to stop caching anything, because we don't really need it anymore
13:25:11 <johnthetubaguy> but that is an *extreme* view
13:25:17 <gmann_> but in case of neutron server down, it does not sounds perfect
13:25:22 <johnthetubaguy> and requires lots of re-work in the metadata server proxy
13:27:12 <mriedem> johnthetubaguy: what do you mean we do'nt need it anymore?
13:27:49 <johnthetubaguy> mriedem: wondering if we can stop caching all that neutron data, as its dropping out of the APIs we want folks to use, it becomes less and less important
13:28:17 <mriedem> johnthetubaguy: was the nw info_cache only added b/c of the REST API usage?
13:28:22 <mriedem> nova-compute relies on it
13:28:48 <johnthetubaguy> but normally we would have fetched it for the specific operation we are doing, like creating config drive
13:29:18 <johnthetubaguy> there is a lot of polling and chatter to keep it fresh, it might be more efficient to not cache, it
13:29:21 <johnthetubaguy> but thats a distraction
13:29:27 <johnthetubaguy> I need to do more research first
13:29:32 <sdague> yeh, I was going to say, that seems like a big rabbit hole
13:29:40 <sdague> and it's a short cycle
13:29:46 <johnthetubaguy> yeah, its a massive one, its not this cycle
13:30:04 <sdague> so focus on things like sane security groups, which will expose some of these other things
13:30:10 <johnthetubaguy> I was looking at optimizing fetching things from neutron quickly, and it made me wonder why we bothered with all that stuff
13:30:21 <johnthetubaguy> yeah, focus on secuirty group tidy up makes total sense to me
13:31:00 <johnthetubaguy> just before we add loads more caching for an edge case, we might want to look a bit deeper there
13:31:10 <sdague> yeh
13:31:17 <gmann_> true
13:31:36 <mriedem> i'm not sure what 'expose bdm' is in the etherpad
13:31:40 <mriedem> what's the use case for that?
13:32:03 <johnthetubaguy> I attempted to add detail, I don't remember why that came up now
13:32:15 <mriedem> i think it was because of the diagnostics API discussion right?
13:32:20 <mriedem> about disk id or something
13:32:23 <johnthetubaguy> it might have been tidying up the volume APIs, and tags, but I don't remember why
13:32:27 <alex_xu> mriedem: yes
13:32:48 <johnthetubaguy> seems like we need a spec drafting on that, a short one, by whoever remembers
13:32:59 <alex_xu> we probably said expose bdm is right thing
13:33:07 <johnthetubaguy> maybe I could trade you alex_xu for that parameter validation one?
13:33:22 <mriedem> ok i still don't get the use case
13:33:27 <mriedem> but i guess if someone wants to write a spec
13:33:51 <johnthetubaguy> I don't remember the use case, appart from remember it seemed like a good solution to something
13:34:06 <alex_xu> johnthetubaguy: you want to write the spec for parameter validation one? it is ok for me
13:34:13 <sdague> yeh, I'd say make the progress on the others first.
13:34:42 <alex_xu> yea, sounds like it is a thing people want to clearly
13:34:52 <alex_xu> s/it is/it isn't/...
13:34:55 <sdague> the bdm one probably needs a real champion that needs that info, and that doesn't seem to exist right now, so I'd back burner it
13:35:39 <johnthetubaguy> yeah, as you reminded us before, its a short cycle
13:35:56 <johnthetubaguy> alex_xu: I think its best you keep that parameter one if you still have the time
13:36:45 <alex_xu> johnthetubaguy: ok, looks like I have the time
13:37:00 <mriedem> there will be a xp session on capabilities discovery,
13:37:05 <mriedem> that's why we don't have one called out specifically in the nova track
13:37:14 <mriedem> but it might be part of the api session, to follow up from the xp session
13:37:23 <johnthetubaguy> yeah, we said alaski would get a slot to report back
13:37:23 <mriedem> and figure out nova-specific details and assigning people to do the work
13:37:34 <johnthetubaguy> I added a bit in that session etherpad for that
13:38:09 <mriedem> cool
13:38:25 <mriedem> i know there are some paging specs floating around that didn't make newton either
13:38:33 <mriedem> so really, i feel like we're already overcomitted on api
13:38:42 <mriedem> and would like to hunker down on what's already on our plate
13:39:02 <johnthetubaguy> the tidying up loose ends list looks quite a good thing to focus on
13:39:05 <cdent> mriedem: I think it is probably fair to say we're overcommitted everywhere
13:39:23 <mriedem> cdent: we don't need to do that though
13:41:17 <johnthetubaguy> did the paging ones get code up?
13:41:19 <mriedem> https://review.openstack.org/#/q/APIImpact+status:open+project:openstack/nova-specs
13:41:57 <mriedem> not sure,
13:42:05 <mriedem> the instance actions one got caught up
13:42:17 <mriedem> there was a ML thread on that, i think on the marker to use
13:42:47 <mriedem> another fairly obvious one https://review.openstack.org/#/c/366469/ tag a server on boot
13:43:22 <sdague> mriedem: +2, that should just move forward I think
13:43:24 <johnthetubaguy> yeah, true, feels a bit odd we don't have that one
13:43:27 <mriedem> takashi also brought up some things at the midcycle
13:43:34 <mriedem> like https://review.openstack.org/#/c/334725/
13:43:49 <mriedem> basically i think they were things we allow in the live migration api, but didn't in the cold migration api
13:43:52 <mriedem> like passing hosts
13:44:11 <mriedem> and we said at the midcycle we were ok with doing them
13:44:21 <mriedem> yeah https://review.openstack.org/#/c/334286/
13:44:47 <mriedem> anyway i'll throw that random stuff into the ideas etherpad
13:45:13 <alex_xu> mriedem: cool
13:45:58 <johnthetubaguy> should that not be one spec, in a single microversion?
13:46:12 <johnthetubaguy> like made cold migrate do what live-migrate takes for host hints
13:46:20 <johnthetubaguy> anyways, an talk about that on the review
13:46:23 <johnthetubaguy> can
13:46:27 <mriedem> shrug
13:46:39 <mriedem> if it makes the single thing overly complex to review so it doesn't get in, then no
13:46:53 <johnthetubaguy> well seems odd to add the broken version then the fixed version
13:46:57 <mriedem> frankly i've been spending a lot of my review time this cycle early on single patch blueprints
13:47:19 <mriedem> rather than the 20 patch never ending beasts
13:49:30 <alex_xu> anything more for summit ideas?
13:51:01 <alex_xu> johnthetubaguy: sdague hope you can help on to fill the offical api etherpad
13:51:08 <sdague> alex_xu: sure
13:51:18 <alex_xu> thanks
13:52:08 <alex_xu> emm...so let us move on?
13:52:16 <sdague> yeh
13:52:28 <alex_xu> #topic API deprecation in Ocata
13:52:31 <johnthetubaguy> alex_xu: happy to help with that, added some bits already I believe
13:52:42 <alex_xu> johnthetubaguy: cool, thanks a lot
13:52:55 <alex_xu> the news from me is image metadata one merged
13:52:58 <alex_xu> one more I added
13:53:01 <alex_xu> #link https://review.openstack.org/384261
13:53:32 <alex_xu> the security group one, johnthetubaguy submitted, we already talk about that
13:54:19 <alex_xu> anymore news for proxy api deprecation?
13:54:50 <alex_xu> johnthetubaguy: sdague mriedem need your reviwe for the multinic and floatingip actions one
13:55:03 <johnthetubaguy> yeah, I should look at that one today
13:55:14 <alex_xu> johnthetubaguy: thanks
13:55:42 <alex_xu> if no more question, let us jump to open directly
13:55:45 <alex_xu> #topic open
13:56:09 <alex_xu> anything more people want to bring up? 4 mins left...
13:56:57 <alex_xu> ok...I guess no
13:57:17 <alex_xu> so let us end the meeting, thanks all!
13:57:21 <alex_xu> #endmeeting