13:00:04 <alex_xu> #startmeeting nova api
13:00:05 <openstack> Meeting started Wed Sep 21 13:00:04 2016 UTC and is due to finish in 60 minutes.  The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:08 <openstack> The meeting name has been set to 'nova_api'
13:00:13 <alex_xu> who is here today?
13:00:15 <mriedem> o/
13:00:24 <cdent> o/
13:00:27 <edleafe> \o
13:00:47 <mriedem> i know sdague is around
13:00:53 <johnthetubaguy> o/
13:00:57 <alex_xu> yea, I guess so :)
13:01:08 <sdague> o/
13:01:08 <mriedem> is gmann here?
13:01:14 <jichen> o/
13:01:25 <sdague> gmann is in germany I think for the infra/qa sprint
13:01:30 <mriedem> ah
13:01:44 <mriedem> no wonder mtreinish has avoided me all week
13:01:59 <alex_xu> so I didn't have specfic thing in the list, but there are have some of items from mriedem, so just let us go through your items?
13:02:08 <mriedem> sure
13:02:29 <alex_xu> #link https://review.openstack.org/#/c/301372/
13:02:35 <alex_xu> mriedem: ^ the first one
13:02:48 <mriedem> yeah so i need help with that one
13:03:16 <mriedem> like, i need some direction from sdague and/or ken'ichi
13:03:20 <mriedem> on what they wanted to see there
13:03:36 <mriedem> but this is the thing where the security_groups in a server create response are possibly wrong when using neutron
13:03:49 <mriedem> so we want to return a link instead for security_groups
13:03:57 <johnthetubaguy> shouldn't we just drop that feature in a new API version
13:04:13 <johnthetubaguy> so don't add any security_groups be default? or does that break something else?
13:04:29 <sdague> mriedem: ok, this actually relates to a thing we were just talking about in #openstack-nova
13:04:50 <mriedem> johnthetubaguy: i thought about that too
13:04:52 <sdague> what would a real security_groups representation look like?
13:05:12 <johnthetubaguy> it would look like a full proxy of neutron's APIs?
13:05:46 <alex_xu> looks like just a wrap from per port to per instance
13:05:47 <johnthetubaguy> what I mean is, each port can have its own security group
13:05:53 <sdague> do we have some examples of neutron vs nova?
13:06:27 <johnthetubaguy> examples of the current API response?
13:06:35 <sdague> ok, personally I'd like to stare at the data comparisons a little bit before deciding what a best path forward is
13:06:56 <sdague> because we've got a related issue that a server get, with neutron service down, causes a 500, because of security group listing
13:07:14 <johnthetubaguy> oh, good point
13:07:28 <sdague> and I think the eventual approach should cover both cases
13:07:33 <johnthetubaguy> honestly, it feels like security groups and floating ips should be removed from the get server response
13:07:44 <mriedem> http://developer.openstack.org/api-ref/networking/v2/#security-groups-security-groups
13:07:47 <johnthetubaguy> now that doesn't fix the older versions
13:08:09 <mriedem> i feel like we're talking about different problems here,
13:08:16 <johnthetubaguy> ah, possibly
13:08:17 <sdague> mriedem: we are
13:08:26 <sdague> but I think they are related
13:08:35 <mriedem> the point of the spec, and the bug, is that when creating the server, we're returning the default secgroup b/c nova adds that by default, but neutron may not actually set that up
13:08:40 <mriedem> if the network doesn't have port security enabled
13:08:47 <sdague> mriedem: right
13:09:02 <mriedem> so the spec was suggesting returning a link to the secgroup api for nova
13:09:15 <johnthetubaguy> I guess I am questioning why nova sets the default security group any more
13:09:19 <mriedem> before 2.36 that was a proxy to neutron
13:09:23 <sdague> but the case in both of these is that we're building a server document with information where the source of truth is from neutron
13:09:35 <sdague> and that goes wrong a few different ways
13:09:49 <sdague> because of timing, defaults, and service outages
13:09:57 <alex_xu> I thought the security-group is really for nova-network
13:10:14 <sdague> alex_xu: neutron also has them, they act a little differently
13:10:26 <mriedem> neutron applies a default secgroup by default i think
13:10:31 <mriedem> if you have port security enabled on the network
13:10:55 <alex_xu> sdague: i'm curious what nova api show when we have two different security-groups for two ports which belong to one instance
13:10:58 <mriedem> iff you don't provide a non-default secgroup
13:11:41 <johnthetubaguy> ah, that used to be us setting that, but I think we stopped doing that at some point
13:11:55 <prateek> johnthetubaguy, regarding your comments on https://review.openstack.org/#/c/294513/18/nova/compute/api.py, what you wrote is definitely more elegant than what I did, but both excpetions had different attr when raising the exception, did that just slip by you or am I missing something
13:12:23 <mriedem> prateek: we're not talking about that right now
13:12:27 <johnthetubaguy> prateek: please ask questions in the nova channel, happily answer in there
13:12:35 <prateek> ohkk sorry
13:12:47 <prateek> i didnt look at that channel
13:13:12 <mriedem> so is the question if the server create response should even return security_groups at all?
13:13:28 <johnthetubaguy> thats where I am at, leave them out
13:13:39 <johnthetubaguy> and don't allow users to pass them in, either
13:13:41 <alex_xu> prefer to remove them
13:14:16 <mriedem> why wouldn't we allow users to pass them in?
13:14:19 <mriedem> that seems odd
13:14:24 <mriedem> we allow you to pass in specific nics
13:14:36 <johnthetubaguy> so it should be a per nic thing, if we wanted it
13:14:46 <johnthetubaguy> really I would want folks to set it on the port, and pass us the port_uuid
13:14:47 <sdague> johnthetubaguy: it seems like we need to redo the expected way to pass things around
13:15:15 <mriedem> can we decouple the two issues for now?
13:15:17 <sdague> I feel like what we have is a flow from nova-network
13:15:28 <johnthetubaguy> sdague: yeah, +1
13:15:34 <mriedem> b/c my spec is not about redoing the entire server create api for how secgroups are passed in
13:15:35 <sdague> and at least 2 (probably more) issues with that flow
13:15:55 <johnthetubaguy> I know, it just feels odd to allow people to pass things in, and not return them in any way
13:15:56 <sdague> and I think we should walk through what the flow should look like in the neutron world
13:16:26 <sdague> mriedem: honestly, I'm kind of -1 on doing that, because I think we're going to need to change a bunch of other stuff and there will end up being 4 microversions of fixing security groups
13:17:05 <sdague> johnthetubaguy: do you have a more ideal flow in your head you could sketch out in an etherpad?
13:17:30 <johnthetubaguy> honestly, it feels like folks should just set this on their port outside of nova
13:18:00 <johnthetubaguy> or we expand --nic to include a security group thingy, referencing the one in neutron
13:18:01 <sdague> johnthetubaguy: ok, that's fine, can you write that up, and the point to all the ways we would change the API to enforce that?
13:18:11 <johnthetubaguy> yeah, I should do that thing in some etherpad
13:18:30 <sdague> because this is going to unwind like the proxy deprecation did, I think there are second order impacts that will only be clear once we write the whole thing out
13:18:42 <johnthetubaguy> #link https://etherpad.openstack.org/p/ocata-nova-security-groups
13:19:44 <mriedem> can you use nova-network after 2.36? i can't remember
13:19:49 <sdague> ok, lets let that churn for a couple of days
13:19:50 <mriedem> how the proxy api breaks using nova-net
13:20:01 <sdague> mriedem: you can't use nova-net after that
13:20:13 <mriedem> even on server create?
13:20:23 <mriedem> because the server api isn't deprecated
13:20:37 <sdague> we didn't change the server representation
13:20:48 <sdague> but the server representation never said "use_neutron"
13:21:21 <mriedem> right so i can create a server with nova-network and 2.latest
13:21:24 <johnthetubaguy> I think you can boot an instance with default setting using nova-net, but the good stuff breaks, but I could be remembering that wrongly
13:21:39 <mriedem> but if i try to list ips from that server at 2.latest with nova-network it should fail
13:21:47 <sdague> right
13:22:29 <sdague> mriedem: but server show still has them
13:22:32 <mriedem> i would just like to be clear about what breaks if we change the server create api wrt secgroups
13:22:46 <sdague> mriedem: yes, agreed
13:23:38 <mriedem> ok let's move on
13:23:57 <alex_xu> #link https://review.openstack.org/#/c/350277/
13:24:02 <sdague> so... I think we should make security groups not be a lie with neutron, which they kind of are. I think a couple of days of johnthetubaguy and others trying to figure out where all the rough edges are will help get a handle on the problem.
13:24:23 <johnthetubaguy> yeah, thats a fair summary I feel.
13:24:29 <mriedem> We have to think through this a bit since we should get it done in Ocata: https://review.openstack.org/#/c/350277/ - I like the suggestion from johnthetubaguy to model this like the volume attachments API.
13:24:40 <mriedem> "Deprecate os-interface proxy API"
13:24:46 <mriedem> this is the thing we missed in 2.36
13:24:55 <mriedem> where we have os-interface and os-virtual-interfaces
13:25:13 <mriedem> os-virtual-interfaces is purely nova-net today
13:25:19 <mriedem> b/c of the vifs in the nova db
13:25:37 <mriedem> os-interface is the proxy to neutron for listing ports
13:25:41 <mtreinish> mriedem: I've been avoiding you?
13:25:46 <mriedem> and attach/detach ports
13:26:02 <johnthetubaguy> yeah, it feels like the cinder volume model is what we should model for VIFs, flow wise
13:26:03 <mriedem> mtreinish: yeah i've been driving by your house late at night and everything!
13:26:10 <mtreinish> hah
13:26:13 <sdague> johnthetubaguy: that seems reasonable
13:26:18 <mriedem> johnthetubaguy: yeah i like that too
13:26:31 <mriedem> so you'd have basically show, list, attach/detach
13:26:39 <sdague> we should realistically probably pile up somewhere the rest of the things we were going to deprecate but never got around to
13:27:11 <mriedem> it was at least this and the image metadata proxy api
13:27:25 <mriedem> oh but os-cloudpipe too and things like that
13:28:09 <alex_xu> also multi-nic which johnthetubaguy point out few days ago
13:28:38 <mriedem> so i think the path forward on https://review.openstack.org/#/c/350277/ is for me to rewrite it with this in mind:
13:28:46 <mriedem> 1. make os-interface more like os-volume_attachments
13:28:49 <johnthetubaguy> yeah, there a few I keep finding when looking at the api-ref patches
13:28:54 <mriedem> 2. deprecate os-virtual-interface
13:29:27 <mriedem> since os-virtual-interface only works for nova-net, deprecating that shouldn't be a big deal, it should have happened in 2.36
13:30:01 <sdague> yeh, seems reasonable
13:30:13 <mriedem> we might also change os-interface GET/list to build that from the nova db model instead of a proxy to neutron
13:30:23 <mriedem> i'd have to think that through
13:30:34 <mriedem> we might have to proxy for anything that was created before the 2.32 microversion
13:31:32 <mriedem> anyway, i'll move forward with that and see what pits i fall into
13:32:10 <mriedem> next on the agenda:
13:32:10 <mriedem> We need someone to write a spec for supporting the standardized  (formerly v3) instance diagnostics compute API from the REST API with a  microversion, as a dependency for hyper-v to support that API: https://blueprints.launchpad.net/nova/+spec/hyperv-vm-diagnostics
13:32:32 <mriedem> claudiub was going to see if the person that was going to work on ^ was going to work on the api change
13:32:46 <mriedem> i'm throwing it out here in case someone wants to get a head start on the spec
13:33:21 <mriedem> this is the thing that garyk worked on to create the standard instance diagnostics object in the virt layer
13:33:33 <mriedem> but with the v3 api dropping out, we never ended up exposing that code from the REST API
13:33:36 <claudiub> yeah, so that needs an api change as well. my coleague can take care of the hyper-v bits, but he's pretty loaded with tasks, and won't have enough time for the api changes, especially since ocata is so sohort
13:34:12 <mriedem> #help need someone to write the spec for exposing the standardized instance diagnostics out of the REST API
13:34:17 <mriedem> i guess i could get a spec started
13:34:23 <mriedem> it's pretty straight-forward i think
13:34:26 <alex_xu> mriedem: I can do that
13:34:31 <mriedem> ok
13:34:46 <sdague> #action alex_xu to work on generic diagnostics spec
13:35:01 <sdague> we should also probably mark actions from before as well
13:35:09 <mriedem> i think it's basically after microversion x you call nova.compute.api.API.get_instance_diagnostics
13:35:30 <alex_xu> yea, i guess so
13:35:35 <mriedem> sdague: i'm not sure what that means
13:35:39 <mriedem> mark actions from before
13:35:39 <sdague> #action johnthetubaguy to sketch out what an ideal security group workflow looks like in the nova api now with neutron as the presumed backend
13:35:57 <mriedem> oh
13:35:58 <mriedem> i see :)
13:36:07 <sdague> #action mriedem to write up spec for os-virtual-interface deprecation
13:36:20 <mriedem> next topic
13:36:20 <sdague> then they are recorded for revisit in next meeting
13:36:26 <mriedem> Does alex_xu have a spec for deprecating the proxy API for set/delete image metadata?
13:36:38 <mriedem> alex_xu: i know you reported a bug for that
13:37:02 <alex_xu> mriedem: yea, I can begin to do that. so that need a separated microversion?
13:37:49 <alex_xu> do we merge that with other thing we forget to deprecate? like multi-nic?
13:38:18 <mriedem> alex_xu: we could lump it in with others if that is easy
13:38:51 <alex_xu> mriedem: ok, let me check how image metadata deprecation look like first
13:38:59 <mriedem> the novaclient changes are probably the worst part, but at least we know how to do those now
13:39:43 <mriedem> #action alex_xu to write a spec for deprecating the proxy api to set/delete image metadata
13:40:03 <mriedem> ok, last item, i swear :)
13:40:05 <mriedem> How can we test swap volume from Tempest? gmann was trying that at one point.
13:40:11 <mriedem> ^ is why i was looking for gmann
13:40:28 <mriedem> so apparently we regressed swap volume, which was already broken
13:40:33 <johnthetubaguy> ah bauzas was looking a some nasty bits around there recently
13:40:34 <sdague> maybe mtreinish can take that back into the room there
13:40:36 <mriedem> it's just a terrible terrible api which isn't tested
13:41:03 <johnthetubaguy> does our default policy allow folks other than cinder to call it? I guess it does?
13:41:03 <mriedem> i know gmann was working on a test for swap volume at one point and ran into an issue but i can't remember what it was now
13:41:14 <mriedem> we changed the default policy to be admin-only back in i think mitaka
13:41:22 <mriedem> i thought gmann was running into policy issues
13:41:28 <johnthetubaguy> yeah, I didn't think calling it directly was meant to be a thing
13:41:32 <sdague> mriedem: given the TZ delay, perhaps we stick it out on the mailing list?
13:41:34 <mriedem> johnthetubaguy: but it is
13:41:40 <mtreinish> sdague: gmann isn't here, he couldn't make it
13:41:41 <mriedem> johnthetubaguy: nova volume-update :)
13:41:55 <mriedem> maybe i can just dig up his change in tempest
13:41:59 <johnthetubaguy> mriedem: yeah, that was a discovery for me
13:42:05 <mriedem> #action mriedem to follow up with gmann about testing swap-volume in tempest
13:42:27 <mriedem> the problem might have been he was using a non-admin to call nova, but then nova calls an admin api in cinder
13:42:29 <mriedem> and it blow up
13:42:30 <mriedem> *blew
13:42:39 <johnthetubaguy> ah
13:42:47 <mriedem> that should be fixed if you just use an admin throughout
13:43:13 <sdague> mriedem: is http://developer.openstack.org/api-ref/compute/?expanded=update-a-volume-attachment-detail the thing we're talking about?
13:43:14 <mriedem> it would also be totally conditional on tempest since only libvirt supports the swap-volume method
13:43:18 <johnthetubaguy> was there a reason disconnect one then attach another is no good?
13:43:24 <mriedem> sdague: yes
13:44:19 <sdague> johnthetubaguy: yeh, that's a question for me, this seems a bit too magical
13:44:40 <johnthetubaguy> I wonder if the answer is something about preserving device ordering, but I really have no idea
13:45:13 <sdague> ok, well the action is out there
13:45:18 <johnthetubaguy> well, a new day, a new crazy API found
13:45:19 <sdague> I have one thing
13:45:27 <alex_xu> :)
13:45:34 <bauzas> johnthetubaguy: yeah https://review.openstack.org/#/c/373390/
13:45:44 <mriedem> i think it has something to do with the blockRebase that's done https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1181
13:45:45 <sdague> I've been pondering API capabilities a bit, and am trying to figure out the best way to flesh out the approach
13:46:00 <mriedem> swap-volume is really about an api flow for libvirt and certain cinder backends for migration i think
13:46:30 <sdague> it's not clear to me that there is time in ocata to do it, but we probably need a pretty detailed spec (and discussions) before we move forward
13:46:37 <johnthetubaguy> mriedem: its live-migration for the volume backend, for non-clustered storage, as I understood it
13:46:39 <sdague> so my thinking is an ocata spec
13:46:41 <alex_xu> but I remember swap-volume works for normal-user, user can just pass a new volume, and that swap works.
13:46:43 <johnthetubaguy> sdague: +1 for moving forward on that
13:47:07 <johnthetubaguy> sdague: I think my problem is matching capabilities with the actual REST calls
13:47:14 <sdague> and we can figure out if there is ground work that falls out
13:47:40 <johnthetubaguy> I think there is a task to go back and add docs into our policy rules? maybe?
13:48:02 <mriedem> there is a task to build the default policy into the nova devref
13:48:05 <mriedem> like we have for the sample config
13:48:09 <johnthetubaguy> mriedem: +1
13:48:31 <mriedem> the keystone guys showed up yesterday asking about putting comments in the policy.json with instructions on how to generate it,
13:48:43 <mriedem> i'd prefer to just have a readme like we do for the nova.conf.sample that has those instructions
13:48:49 <mriedem> so we're consistent
13:48:55 <sdague> mriedem: yeh, given json doesn't support comments, I agree
13:48:58 <johnthetubaguy> mriedem: +1
13:49:20 <johnthetubaguy> maybe there is enough of "policy tidy ups" to be done
13:49:47 <johnthetubaguy> sdague: I don't remember spotting anything around that in the operator feedback, I guess its a release or two early for that?
13:50:07 <sdague> johnthetubaguy: on policy, they liked our new approach, they were just concerned about docs
13:50:09 <sdague> it's in there
13:50:14 <mriedem> sdague: might want to keep claudiub in the loop on any policy discoverability / capability stuff since i think he's also looking at that
13:50:21 <johnthetubaguy> sdague: right, thats fair
13:50:33 <claudiub> o/
13:50:38 <mriedem> i'll follow up on the readme for generating the default policy if ayoung didn't push that yet
13:50:39 <sdague> mriedem: sure, well, a spec will be a reasonable place to keep all that public and iterable
13:50:47 <johnthetubaguy> ah, so at the midcycle, we said we would try add two things for policy discovery
13:51:00 <johnthetubaguy> I guess, we need a spec up with that, so we can discuss something concrete
13:51:53 <sdague> #action sdague to start in on a capabilities spec
13:51:59 <claudiub> sorry, a bit out of the loop here. so, I have a WIP patch for matching policy rules and capabilities and different api endpoints.
13:52:16 <claudiub> also trying to take into account the given api microversion
13:53:17 <sdague> claudiub: where is that patch?
13:53:19 <claudiub> it's a bit slow, but i have something that outputs what the nova-api docs have in about 75-80%
13:54:21 <claudiub> https://review.openstack.org/#/c/280539/
13:54:40 <claudiub> still have to upload what i have in my work env
13:55:07 <claudiub> i've changed things since then
13:55:14 <sdague> ok, that's probably a bit different granularity than I was thinking
13:55:19 <alex_xu> 4 mins left
13:56:03 <claudiub> i was planning to include the post method, api endpoint and request body in the output
13:56:19 <claudiub> api method *
13:56:24 <diga_> alex_xu: Hi, https://blueprints.launchpad.net/nova/+spec/display-host-hypervisor-numa-topology can I submit the spec for this BP for ocata-1
13:56:44 <claudiub> or more like, the request body schema
13:56:52 <alex_xu> diga_: yea, sure, i think it is time to submit spec
13:57:02 <diga_> alex_xu: Sure, thx!
13:57:31 <alex_xu> diga_: np
13:58:06 <diga_> :)
13:58:18 <alex_xu> ok, 2 mins left
13:58:43 <alex_xu> so let us back to the nova channel?
13:59:11 <sdague> sounds good
13:59:20 <alex_xu> ok, thanks all!
13:59:21 <alex_xu> #endmeeting