13:00:03 <alex_xu> #startmeeting nova api
13:00:04 <openstack> Meeting started Wed Mar 15 13:00:03 2017 UTC and is due to finish in 60 minutes.  The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:08 <openstack> The meeting name has been set to 'nova_api'
13:00:10 <alex_xu> who is here today?
13:00:20 <gmann> o/
13:00:30 <sdague> o/
13:01:08 <alex_xu> I guess johnthetubaguy still in lunch
13:01:18 <johnthetubaguy> o/
13:01:25 <alex_xu> ok :)
13:01:32 <alex_xu> #topic priorities
13:01:37 <alex_xu> #link https://etherpad.openstack.org/p/pike-nova-priorities-tracking
13:02:27 <alex_xu> beside policy, just want to mention, this API spec looks like very close https://review.openstack.org/265282
13:03:07 <sdague> alex_xu: cool, will need to revisit that one
13:03:23 <johnthetubaguy> oops, i should hit that one
13:03:29 <alex_xu> sdague: yea, just for you and johnthetubaguy :)
13:04:07 <johnthetubaguy> there was another, just trying to find it
13:04:11 <alex_xu> for the policy, maybe we should adverties the doc one in the nova weekly meeting, that is simple and can be merge quick
13:04:34 <johnthetubaguy> #link https://review.openstack.org/#/c/440580/
13:04:49 <johnthetubaguy> alex_xu: a good idea, been trying to bug folks directly too
13:05:21 <gmann> this one also very close, #link https://review.openstack.org/#/c/415315/
13:05:34 <alex_xu> ah, yea, i forget to revisit that one
13:05:39 <gmann> johnthetubaguy: you had +2 then some trivial updates
13:06:17 <alex_xu> so we want to have a limit dict for the response of hints?
13:06:35 <alex_xu> I guess no, that isn't match the request
13:07:04 <alex_xu> gmann: thanks
13:07:09 <alex_xu> gmann: i put it in the therpad
13:07:27 <johnthetubaguy> alex_xu: what limit do you mean?
13:08:05 <johnthetubaguy> I think we have a limit on the size of the tag, and limit on the number of tags anyways, so that should just carry over here
13:08:08 <alex_xu> johnthetubaguy: the scheduler hints schema have 'addtionalProperties': True
13:08:22 <johnthetubaguy> oh right
13:08:31 <alex_xu> that means user can put anything into the hints, and that store in the db, and can be get from the response
13:08:39 <johnthetubaguy> we left that as an extension point, mostly until we have traits sorted I guess
13:08:42 <alex_xu> another open metadata api
13:08:57 <johnthetubaguy> yeah, it kinda is open
13:09:19 <johnthetubaguy> do we limit the number of scheduler hits you can set?
13:09:25 <alex_xu> good news, the hints can't update
13:09:26 * mriedem sneaks in late
13:09:34 <johnthetubaguy> alex_xu: yeah
13:09:49 <alex_xu> I think no, just limit by the length of request body
13:10:24 <johnthetubaguy> hmm, probably should fix that up
13:10:44 <johnthetubaguy> tags has a limit of 50
13:10:52 <alex_xu> then I will drop in the question whether we need microversion :) i hate that question
13:11:20 <johnthetubaguy> I think we can pretend its a security fix, but I am willing to have that shot down
13:11:53 <mriedem> what is the question/issue?
13:12:11 <johnthetubaguy> bigger question, do we want to fix up the scheduler hint limit before we allow the adding of the GET api?
13:12:25 <johnthetubaguy> mriedem: scheduler hit spec, link is above
13:12:32 <mriedem> right, but summarize?
13:13:08 <johnthetubaguy> you can set an unlimited number of hints today
13:13:10 <mriedem> this is the one that's proposing to return scheduler hints from the original instance build request right?
13:13:14 <johnthetubaguy> then we start returning them too
13:13:18 <johnthetubaguy> yeah
13:13:31 <mriedem> so the GET response could have a million hints in it
13:13:48 <alex_xu> mriedem: the hints in the request is open for any key-value, and then can get from the API response from that spec
13:14:01 <mriedem> what is the size on the key/value?
13:14:10 <sdague> johnthetubaguy: is there a reason why it's in the server document instead of dedicated sub resource?
13:14:20 <mriedem> sdague: they're following the flavors spec
13:14:23 <mriedem> flavors in server response
13:14:27 <alex_xu> no limit as i know
13:14:30 <sdague> I get the idea of why we'd roll flavor up here, because everyone has flavors
13:15:00 <sdague> but scheduler hints seem really specific, and only some percentage of applications is going to be using them
13:15:12 <mriedem> alex_xu: what table stores scheduler hints anyway? request_specs?
13:15:24 <johnthetubaguy> we kinda agreed to it at the PTG, on the basis of what goes in, should usually come out somehow
13:15:43 <mriedem> "somehow" could be a subresource,
13:15:46 <mriedem> is i think sean's point
13:15:49 <sdague> right
13:15:56 <mriedem> and doesn't clutter up the main, already large, response
13:16:01 <johnthetubaguy> oh, I missed what you meant there, that does make sense
13:16:16 <sdague> I think the important difference between flavor and scheduler hints
13:16:22 <sdague> *everyone* has a flavor
13:16:30 <johnthetubaguy> sdague: yeah, that makes good sense
13:16:36 <sdague> *only some instances* have hints set
13:16:43 <johnthetubaguy> I don't think its needed in /servers/detail either
13:16:49 <sdague> right
13:17:20 * johnthetubaguy wishes he didn't always forget about the sub resource as an option
13:17:27 <sdague> johnthetubaguy: the use case for this is?
13:17:27 <alex_xu> mriedem: yea, i guess in request spec
13:17:35 * alex_xu always forget those detail
13:17:55 <mriedem> sdague: rebuilding an instance using the same hints as before
13:17:59 <johnthetubaguy> sdague: I think it was launching another VM that is similar
13:17:59 <mriedem> was what i remember from the ptg
13:18:09 <mriedem> *not rebuilding, duplicating
13:18:21 <johnthetubaguy> or working out what state your instance groups are in, I guess
13:18:38 <johnthetubaguy> well, server groups
13:19:13 <mriedem> hmm,
13:19:19 <mriedem> the request_spec is not scoped to an instance,
13:19:30 <mriedem> oh wait yes it is
13:19:43 <sdague> where is this sitting the db models?
13:19:54 <mriedem> so we get the request_spec by instance_uuid from the api db,
13:20:03 <mriedem> deserialize the blob,
13:20:12 <mriedem> pull out the scheduler hints, and return those in the response
13:20:21 <mriedem> so yeah there is no key/value size limit on the hints in the db
13:20:30 <alex_xu> can we query the member of server-group by /server-groups api?
13:20:57 <johnthetubaguy> mriedem: yeah, I think thats all correct
13:20:59 <mriedem> we have some known in-tree hints in the json schema validation,
13:21:03 <mriedem> but additionalProperties=True
13:21:06 <mriedem> so it's totally open ended
13:21:11 <gmann> yea
13:21:14 <johnthetubaguy> +1
13:21:15 <mriedem> i could build a server with a million-char key/value
13:21:26 <johnthetubaguy> +1 for that being the current state, not +1 for keeping it like that
13:21:34 <mriedem> right, i'm just getting caught up,
13:21:44 <mriedem> seems we should limit the size of the key/value in the request schema,
13:21:49 <johnthetubaguy> mriedem: thats probably a security related bug, all bit not a major one
13:21:57 <mriedem> and as for the total number of hints per request, we could use quota like we do for metadata
13:22:03 <johnthetubaguy> mriedem: +1 I was suggesting we follow tags with a static 50 limit
13:22:16 <alex_xu> +1 for a hard-code number
13:22:27 <johnthetubaguy> metadata I think is a quota based limit, which feels like overkill as a starting point
13:22:31 <mriedem> i was thinking metadata, which is i think 255
13:22:39 <sdague> mriedem: well, if this is just a GET, what is the limit concern?
13:22:49 <johnthetubaguy> mriedem: Oh, you mean the string lenght?
13:22:58 <mriedem> johnthetubaguy: yeah
13:23:09 <mriedem> do we leave the string length on the key/values unbounded?
13:23:11 <johnthetubaguy> mriedem: I was thinking the number of strings, but both need fixing I gues
13:23:21 <sdague> mriedem: it already is right?
13:23:37 <johnthetubaguy> mriedem: I thought the DB limited that, but maybe I miss-remembered that
13:23:51 <mriedem> this is stored in the request_specs table in a json blob,
13:23:53 <sdague> so, I'm still trying to figure out where this is stored. Is this just jammed into user_data?
13:24:00 <mriedem> in a Text field, so there is a limit in the db, but it's rather large
13:24:01 <johnthetubaguy> oh right
13:24:07 <mriedem> sdague: request_specs,
13:24:08 <mriedem> nova_api db
13:24:18 <mriedem> https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api_models.py#L171
13:24:19 <alex_xu> ah
13:24:31 <sdague> oh, api_models.py
13:24:39 <mriedem> the RequestSpec object serializes/deserializes from the json blob
13:24:54 <sdague> when are request specs deleted?
13:25:08 <mriedem> i wouldn't be surprised if they aren't, but would need to check
13:25:41 <alex_xu> i guess there will be 500 when over the Text field limit
13:26:00 <mriedem> alex_xu: yup
13:26:09 <mriedem> bauzas: are request_specs ever deleted?
13:26:22 <mriedem> they are in the api db so when we delete an instance we're in the wrong db to clean those up
13:27:14 <sdague> I think my biggest concern to exposing this is that it locks us into lifecycle management constraints on that
13:27:26 <sdague> because it's in a different table
13:27:52 <sdague> in the flavor case, it's all super easy, because it's in the same tabl
13:28:00 <mriedem> well, not really,
13:28:05 <mriedem> flavors are in instance_extra
13:28:08 <mriedem> but same db :)
13:28:17 <mriedem> from what i can tell, we never delete request_specs records
13:28:31 <sdague> mriedem: ok, well in a table linked to the instance pretty strongly
13:28:52 <mriedem> so i can fill up your api db with just bogus server build requests with a million hints
13:28:55 <sdague> mriedem: right, I'm sure we're going to have to solve that
13:29:22 <mriedem> i suppose i should open a bug
13:29:36 <johnthetubaguy> yeah, bug time for sure
13:29:47 <gmann> but if limiting to 50 is not too much ? does anyone use 50 hints all together ?
13:30:09 <johnthetubaguy> mriedem: quota does usually limit these attacks though
13:30:26 <johnthetubaguy> gmann: does it matter? its about the next instance that gets created really
13:30:45 <johnthetubaguy> if we break that person, its a good fix
13:31:03 <sdague> honestly, I'm less concerned on the limit here
13:31:50 <alex_xu> sdague: what is your main concern?
13:31:55 <mriedem> johnthetubaguy: we don't have quotas on hints do we?
13:32:01 <sdague> honestly, I just wonder about exposing this
13:32:06 <johnthetubaguy> mriedem: I mean on build instance
13:32:13 <sdague> I do get that "it would be nice"
13:32:26 <alex_xu> sdague: if expose it just for server-group...then we have /server-groups API
13:32:32 <sdague> but, I also think that anyone that's actively using scheduler hints, isn't doing them in an add hoc way
13:32:41 <sdague> they are doing them with a system to coordinate them
13:33:03 <mriedem> johnthetubaguy: i'm saying, single server create request, with a scheduler hints dict with a million keys in it
13:33:09 <mriedem> we don't have quota limits on hints
13:33:11 <johnthetubaguy> mriedem: I couldn't find where the that DB entry is deleted either
13:33:12 <gmann> why people want that use case implement new sch hint as "same_scheduler_hint" like same_host
13:33:15 <mriedem> like we do for metadata or injected files
13:33:16 <sdague> and so using nova as redundant key retreival for this seems... odd
13:33:19 <johnthetubaguy> mriedem: sorry, yep +1 that
13:33:43 <sdague> gmann: right, but if they launched that server from their program, they know what hints those were
13:34:03 <sdague> the only real case is when Bob manually does an ``openstack server create .... hints``
13:34:13 <sdague> outside of a programatic environment
13:34:16 <gmann> humm
13:34:20 <sdague> then Jane wants to duplicate that
13:34:37 <sdague> which, seems like a nice thing to do, but doesn't seem urgent
13:35:06 <mriedem> i don't want to make assumptions on the usefulness or use case for this,
13:35:07 <sdague> especially if we want to spend brain power around quotas / policy
13:35:10 <gmann> if nova implement that sch hint so that Jane can use that sch hint and nova internally does magic at very starting
13:35:17 <johnthetubaguy> sdague: thats basically what I was thinking too, nice to have, acceptable, but not urgent
13:35:18 <mriedem> if we want to know if this is a silly idea, we should ask the ops list
13:35:40 <sdague> mriedem: well, it's a trade off. How many hours are we going to spend on this one.
13:35:52 <johnthetubaguy> we have that feedback at the PTG already, saying it was useful for the folks in the room
13:36:05 <johnthetubaguy> not sure it was urgent as such
13:36:15 <mriedem> it seems relatively straight-forward if we agree on where it lives in the REST API (subresource or not),
13:36:33 <mriedem> i think the quota limit and deleting request specs are separate issues that need to be dealt with anyway
13:36:34 <mriedem> as bugs
13:36:36 <sdague> johnthetubaguy: do we have ideas about who those folks were, and vs. policy or  other things
13:36:52 <mriedem> i think it was a team working with gibi
13:37:02 <mriedem> so probably related to ericsson and telcos
13:37:06 <johnthetubaguy> sdague: I thought it was chris f as well?
13:37:32 <mriedem> so do we want to take a couple of actions and move on?
13:37:41 <mriedem> 1. ask the ops list about usefulness of this?
13:37:48 <mriedem> 2. report bugs for the limits/delete issue?
13:38:08 <mriedem> and make the bugs a prerequisite for the api change?
13:38:13 <mriedem> if we do the api chngae
13:38:14 <mriedem> *change
13:38:29 <sdague> I want a more comprehensive use case
13:38:35 <sdague> I just left -1 feedback for that
13:38:37 <johnthetubaguy> yup, agreement on 2 seemed important
13:38:43 <sdague> the current use case is just "I want to get this out of nova"
13:38:50 <johnthetubaguy> sdague: better use case makes good sense
13:38:51 <sdague> That's not a use case
13:38:53 <alex_xu> +1 for #2 should be fixed
13:39:10 <johnthetubaguy> sdague: did you add about the sub resource thing as well?
13:39:34 <mriedem> #action mriedem to open bugs for lack of limits on scheduler hints and a bug about how we never delete request_specs
13:39:36 <gmann> yea, it really save to blowup the DB
13:39:43 <sdague> I did not, because I think that until there is a more detailed use case I'm -1 regardless
13:39:44 <edleafe> sdague: when migrating an instances, wouldn't the hints be necessary to ensure that it lands on a host that meets the original requirements?
13:39:59 <edleafe> s/instances/instance
13:40:03 <sdague> edleafe: I think that's a different issue
13:40:12 <johnthetubaguy> sdague: fair enough
13:40:19 <sdague> migrating an instance should keep the hints, the user shouldn't have to respecify them
13:40:22 <mriedem> edleafe: we already pull the hints when migrating
13:40:23 <gmann> and limit on sch hint as version bump right or we can discuss on bug anyways
13:40:24 <mriedem> from the request spec
13:40:32 <johnthetubaguy> edleafe: that already all fixed I believe, ask bauzas for details
13:40:34 <mriedem> sdague: that's how it already works i think, yeah
13:40:36 <edleafe> mriedem: ah, ok
13:40:38 <alex_xu> i guess we can move on, as we said, this isn't urgent one, otherwise, we will spend whole hour on that :)
13:40:44 <johnthetubaguy> ++
13:40:44 <edleafe> mriedem: I thought this was tied to that problem
13:41:01 <sdague> edleafe: it might be, and the proposers are a couple of releases back
13:41:11 <sdague> hence, please provide more detailed use case
13:41:38 <edleafe> sdague: got it
13:42:03 <alex_xu> johnthetubaguy: anything more for policy? otherwise we done for the priorites
13:42:15 <johnthetubaguy> other than be smiling at mriedem about the spec?
13:42:24 <mriedem> i've got it in a tab :)
13:42:47 <mriedem> https://review.openstack.org/#/c/433010 right?
13:42:49 <johnthetubaguy> mriedem: cool, I hope your screen looks better than mine, but I suspect not
13:42:50 <alex_xu> mriedem: it's worth to adverties on the nova weekly meeting for getting more feedback for the policy spec
13:42:59 <johnthetubaguy> mriedem: thats the one
13:43:11 <mriedem> alex_xu: ok can you update the weekly nova meeting agenda and post highlights for the api subteam?
13:43:30 <mriedem> when we have the weekly meeting, there is usually not many people around that were also at the api meeting, depending on time of day when we have the nova meeting
13:43:46 <alex_xu> mriedem: yea, I will update the agenda, but i can't attend the meeting this week, it is in the super early time for me
13:43:51 <mriedem> i'm only attending this one because i'm working from home now :)
13:43:55 <johnthetubaguy> alex_xu: I think the follow on specs need your feedback
13:44:02 <alex_xu> johnthetubaguy: got it
13:44:07 <mriedem> alex_xu: right - just need the agenda update with highlights, i'll repeat them
13:44:10 <johnthetubaguy> mriedem: welcome to the club, btw
13:44:13 <alex_xu> mriedem: thanks
13:44:25 <mriedem> johnthetubaguy: i can feel my health deteriorating already :)
13:44:31 <johnthetubaguy> mriedem: :)
13:44:33 * cdent expects mriedem's usually stellar hygiene to begin a slow gentle decay
13:44:50 <mriedem> leg atrophy and whatnot
13:44:58 <cdent> sigh, yet more crypto-jinxing
13:45:08 <alex_xu> we talk about extension removing, so I did some experiment on removing the stevedore https://review.openstack.org/445864
13:45:38 <alex_xu> #link https://review.openstack.org/445864
13:45:53 <alex_xu> another single one
13:45:56 <alex_xu> #link https://review.openstack.org/445361
13:45:57 <johnthetubaguy> alex_xu: any way to avoid api-paste.ini changes?
13:46:26 <johnthetubaguy> alex_xu: I should read your change better!
13:46:27 <sdague> johnthetubaguy: it should be easy to do by just changing the class inline
13:46:38 <johnthetubaguy> yeah, he fixed it already, I was just being dumb
13:47:02 <alex_xu> johnthetubaguy: i guess so, i just get circle import problem, but i want to make it works before the meeting, so i just hack the code quickly
13:47:18 <johnthetubaguy> ah, thats no fun
13:47:56 <alex_xu> it copies the placement api pattern to list the routes explicitly https://review.openstack.org/#/c/445864/2/nova/api/openstack/compute/router.py@110
13:48:44 <sdague> right, the explicit routes is very nice
13:48:52 <mriedem> when i saw that i was thinking router as in neutron router
13:48:54 <sdague> it makes it so much easier to understand what is going on
13:49:22 <gmann> +1
13:49:38 <johnthetubaguy> alex_xu: looking good, +1 all what they just said
13:49:55 <sdague> there is still a lot to unwind to really clean this up, but I think this a very good push
13:50:00 <alex_xu> and before that, we should think about removing the extension point in the server controller https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L155
13:50:27 <alex_xu> probably similar way as what i do in the patch
13:50:41 <johnthetubaguy> yeah, those ones would be good to kill
13:51:16 <alex_xu> is it something we want to do in Pike?
13:51:33 <sdague> alex_xu: ++
13:51:34 <gmann> alex_xu: that will go away by merging the extension class in server.py's create() right
13:51:46 <sdague> alex_xu: I think this would be a great debt reduction
13:51:51 <johnthetubaguy> as long as policy is still higher in priority, it beats and API features I have seen so far
13:51:59 <johnthetubaguy> so +1 for pike
13:52:15 <johnthetubaguy> closely followed by work on the API concept guide I guess
13:52:41 <alex_xu> gmann: not sure, based on if merge didn't generate too much works
13:53:20 <alex_xu> ok, i can do more futher work, and try to get more detail out, and I need a spec before freeze
13:53:49 <johnthetubaguy> maybe specless bp is fine for this
13:53:52 <gmann> spec ?
13:53:52 <sdague> alex_xu: one thing to make sure to keep an eye on is that this doesn't break project_id in the url by changing up the router
13:53:58 <johnthetubaguy> best to evolve the idea in gerrit I guess
13:54:15 <alex_xu> sdague: yea, i just break the project_id in the afternoon :)
13:54:21 <gmann> +1 for specless bp
13:54:47 <alex_xu> ok, i will work more poc code, until people feel it is good, then request the bp approve
13:55:34 <alex_xu> #topic open
13:55:39 <alex_xu> so 5 mins left...
13:55:57 <alex_xu> please bring up anything for the last 5 mins
13:55:59 <johnthetubaguy> lots of talk about the forum, are there any API things we want to raise at the forum?
13:56:18 <johnthetubaguy> #link https://etherpad.openstack.org/p/BOS-Nova-brainstorming
13:56:28 <sdague> so, fyi, I started writing up the microversion architecture document, which is taking the old blog post and readapting it for more permanent archival https://review.openstack.org/#/c/444892/
13:56:31 <johnthetubaguy> #link https://wiki.openstack.org/wiki/Forum/Boston2017
13:56:55 <sdague> including input from things heard at the PTG which are common questions that I think we have answers with
13:56:58 <sdague> answers for
13:57:05 <alex_xu> are most of developers going to the summit?
13:57:15 <johnthetubaguy> alex_xu: my gut tells me no
13:57:40 <mriedem> i doubt it,
13:57:50 <mriedem> i actually wanted to start getting a list of who is going, or trying to get approval to go
13:58:10 <mriedem> #action mriedem to start an etherpad to get an idea of who is going to the summit/forum
13:58:17 <johnthetubaguy> I am trying to be there, but not yet confirmed
13:58:17 <alex_xu> ok, i also didn't hear anything about summit travel strategy from my manager also
13:58:29 <johnthetubaguy> I was thinking what do we want feedback on from operators and users
13:58:46 <mriedem> johnthetubaguy: there is a thread about something like that in the ops ML,
13:58:47 <johnthetubaguy> the ironic raise min version discussion is the only thing that came to mind really
13:58:56 <mriedem> and basically there wasn't a lot of feedback, or requests for sessions,
13:59:11 <mriedem> so the forum organizers are a bit surprised and nervous from what i gather,
13:59:22 <johnthetubaguy> yeah, there is a dev thread on that too, I think
13:59:23 <mriedem> and a lot of operators won't be there because the summit is so close after the ops meetup
13:59:40 <mriedem> so i suspect a fair amount of thumb twiddling
13:59:43 <alex_xu> ...
13:59:59 <alex_xu> it is the time close the meeting
14:00:03 <mriedem> one last thing,
14:00:05 <alex_xu> #endmeeting