13:00:12 <alex_xu> #startmeeting nova api
13:00:13 <openstack> Meeting started Wed Nov 30 13:00:12 2016 UTC and is due to finish in 60 minutes.  The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:16 <openstack> The meeting name has been set to 'nova_api'
13:00:21 <alex_xu> who is here today?
13:00:27 <cdent> o/
13:01:04 <alex_xu> I guess johnthetubaguy is going to lunch
13:01:35 <alex_xu> wait few more minutes
13:01:37 <jichen> o/
13:02:07 <johnthetubaguy> sorry, yeah, back online
13:02:50 <alex_xu> cool
13:03:08 <alex_xu> daylight saving changed again, so I guess the meeting is right on your lunch time
13:03:38 <johnthetubaguy> yeah, its 1pm, which is right at the end of my usual lunch
13:03:59 <alex_xu> heh
13:04:21 <johnthetubaguy> I should remember to actually have lunch on time on a wednesday, my bad
13:04:21 <alex_xu> so the only topic I have for now is the spec https://review.openstack.org/393205
13:04:33 <alex_xu> johnthetubaguy: no worries
13:04:34 <johnthetubaguy> its way better time than for most folks, so I certainly can't complain!
13:04:48 <alex_xu> johnthetubaguy: but you should lunch on time for health
13:05:05 <johnthetubaguy> thats true :)
13:05:20 <johnthetubaguy> I skip some things for work, food is not one of those
13:05:35 <alex_xu> :)
13:05:56 <alex_xu> for the spec, currently found that the sort can't get any help from db index
13:06:02 * cdent has not yet had breakfast or lunch :(
13:06:15 <alex_xu> cdent: !
13:06:23 <cdent> I got distracted :)
13:06:37 * cdent blames johnthetubaguy
13:06:40 <cdent> (but not really)
13:06:44 <johnthetubaguy> heh
13:06:45 <alex_xu> heh
13:06:51 <jichen> :)
13:07:18 <alex_xu> I got feedback from jay, keep focus on the filters with index. just good luck for sort
13:08:41 <alex_xu> so two choices: 1. keep all the sort whatever they can't get help from index. 2. continue to remove those sort with rarely use-case
13:09:07 <alex_xu> or any better choice
13:10:50 <johnthetubaguy> I am still thinking it through
13:11:13 <johnthetubaguy> what do we do today, add lots of different sorts, whatever the user says?
13:12:32 <alex_xu> i don't know
13:13:33 <johnthetubaguy> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L1103
13:14:52 <alex_xu> johnthetubaguy: those are filters
13:15:01 <johnthetubaguy> yeah, sorry
13:15:31 <johnthetubaguy> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L330
13:16:18 <alex_xu> johnthetubaguy: you mean?
13:16:25 <johnthetubaguy> https://github.com/openstack/nova/blob/master/nova/api/openstack/common.py#L145
13:16:36 <johnthetubaguy> so I guess we allow *anything* for sort params right now
13:16:51 <johnthetubaguy> and we allow mulitple
13:17:10 <alex_xu> yes
13:17:59 <johnthetubaguy> would it be simpler to just have the same list for filtering and sorting?
13:18:05 <alex_xu> there is no different for allow one or multiple, they can't get benefit from index anyway
13:18:53 <johnthetubaguy> oh, OK, it does just the one sort run I guess
13:18:54 <alex_xu> johnthetubaguy: yea,it can be similar list. But filter with uuid is useful. sort with uuid is useless.
13:19:05 <johnthetubaguy> why?
13:19:55 <johnthetubaguy> I mean why is sorting with uuid useless? its a choice I guess
13:20:19 <alex_xu> sort on random generated uuid, sounds like useless
13:20:20 <johnthetubaguy> what I mean really is, someone could be doing that today, and we probably shouldn't break them
13:21:15 <alex_xu> if the rule is 'don't break them', that confuse me also :)
13:21:38 <johnthetubaguy> well I want to only break them, if we are protecting people from something bad
13:21:52 <alex_xu> johnthetubaguy: all the sort is bad
13:22:08 <johnthetubaguy> sorting on something thats not returned in the API with the same key, is silly
13:22:09 <alex_xu> none of them can get benefit from index
13:22:28 <alex_xu> ah, +1 for that point, yea
13:22:58 <johnthetubaguy> so we protect them form accidentally being tied to the DB model rather than the API, thats a good step forward here
13:23:06 <alex_xu> we don't have mapping for display_name=>name, that is another bad thing.
13:23:28 <johnthetubaguy> yeah, that should be fixed
13:23:37 <johnthetubaguy> well allowing both
13:23:38 <alex_xu> fix in the spec or later?
13:23:46 <alex_xu> ok
13:24:05 <johnthetubaguy> although that needs a microversion, in theory, so maybe leave that for later
13:24:15 <johnthetubaguy> yuck, I hate how complicated our past is making all this
13:24:34 <johnthetubaguy> anyways... baby steps forward here
13:25:07 <alex_xu> i hate also :)
13:25:07 <johnthetubaguy> longer term, we can add discoverable policy around sort, and encourage folks to use searchlight instead I guess
13:25:20 <alex_xu> yea
13:25:30 <johnthetubaguy> but for now, folks really seem to rely on sorting, so its here to stay
13:26:08 <johnthetubaguy> I think restricting sort and filtering to the same set seems like a minimal level of protection, thats better than the current free for all, but shouldn't break everyone?
13:26:12 <alex_xu> johnthetubaguy: just one question, so for small size deployment, to use searchlight looks like overhead, so just keep nova native sort for them in the future?
13:27:05 <alex_xu> really not sure 'break', we can't ensure what people use
13:27:31 <johnthetubaguy> scratch that last one
13:27:36 <johnthetubaguy> we should just call searchlight I guess
13:27:40 <johnthetubaguy> eventually
13:29:09 <alex_xu> ok, leave the burden to the deployment tools, hope deployment tools can ease the pain of people :)
13:30:04 <johnthetubaguy> well, I guess I mean the small deployments can just use the DB
13:30:17 <johnthetubaguy> elasticsearch is great, but you do need to feed it memory, from what I remember
13:31:09 <johnthetubaguy> anyways, we got a bit distracted
13:31:09 <alex_xu> johnthetubaguy: ok,then I'm little prefer to keep sort key which can mapping to the API
13:31:39 <johnthetubaguy> I think people rely on us doing sort right now, so we can't remove that
13:31:40 <alex_xu> we can control that with policy in the future
13:31:44 <johnthetubaguy> yeah
13:31:47 <johnthetubaguy> possibly
13:33:30 <alex_xu> johnthetubaguy: yea, that what i thought, we just remove the bad thing. the sort can't get benefit from index, so enable one sort key or enable ten sort keys is same
13:34:36 <johnthetubaguy> I was thinking that would mean sorting the list several times, but I assume mysql doesn't do it that way
13:35:43 <alex_xu> maybe I can do a test later, to see is there any different for one sort key and multiple sort key
13:36:08 <johnthetubaguy> its worth a quick check I guess
13:36:17 <johnthetubaguy> particularly on the longer string fields
13:37:55 <alex_xu> johnthetubaguy: just done, very close with one key and mutiple keys
13:38:15 <johnthetubaguy> cool, lets just not worry about that
13:38:58 <alex_xu> ok
13:39:38 <alex_xu> so the plan is the filter list probably is same with current proposal
13:39:54 <alex_xu> remove the sort which didn't mapping to the API.
13:41:11 <alex_xu> add mapping for display_name => name
13:41:22 <alex_xu> I need to check another mapping
13:41:33 <alex_xu> s/another/any other/
13:41:53 <alex_xu> does that sounds cool?
13:42:33 <johnthetubaguy> so filter and sort being the same list?
13:42:39 <johnthetubaguy> or is that list too small?
13:42:55 <alex_xu> I guess filter list is small
13:43:07 <alex_xu> special for non-admin filter list
13:44:40 <johnthetubaguy> well non-admins should be able to sort on project_id, but I guess it doesn't matter as its a no op sort
13:44:49 <johnthetubaguy> I mean should not be able
13:45:39 <alex_xu> yea
13:46:14 <alex_xu> look at the API response, looks like they are close
13:46:51 <johnthetubaguy> anyways, sounds like we are closer to agreement
13:47:08 <alex_xu> yea
13:48:54 <alex_xu> at least agreement on the direction. i will work out the final list. I guess that won't be too much different than your expect
13:50:40 <alex_xu> emm...I need to check the client also
13:50:50 <alex_xu> ensure there isn't anything we break
13:52:00 <johnthetubaguy> true, it would be bad to break what we advertise in the client
13:52:07 <alex_xu> johnthetubaguy: emm...we wil remove some filters which support in the client. so I think we can't remove them
13:52:17 <johnthetubaguy> ah, what are the examples?
13:52:40 <alex_xu> like --ip, --ip6, --image, --flavor
13:53:00 <johnthetubaguy> image and flavor seem worth keeping
13:53:07 <alex_xu> ok
13:53:30 <johnthetubaguy> I guess we should keep ip/ip6 if thats in the client, but I can't say I like that :(
13:53:38 <alex_xu> yea
13:54:01 <alex_xu> and --name
13:55:48 <alex_xu> probably the client just adverties the filters for non-admin and few admin-only
13:55:53 <johnthetubaguy> --name has to stay
13:56:18 <johnthetubaguy> I know those don't have an index, but I think thats intentional, becuase the index is more expensive than the gain we get by having it
13:56:20 <alex_xu> https://github.com/openstack/nova/blob/f8a81807e016c17e6c45d318d5c92ba0cc758b01/nova/api/openstack/compute/servers.py#L1103
13:56:32 <alex_xu> heh, ^ probably we just keep this list
13:57:37 <alex_xu> yea, name is pattern matching, index won't help
13:57:58 <alex_xu> 3 mins left
13:58:21 <alex_xu> johnthetubaguy: so i guess that is the plan, anything more you hope me to notice?
13:58:32 <johnthetubaguy> alex_xu: so IP filter happens in code btw: https://github.com/openstack/nova/blob/3cd2005523dc83b1cadc956c4f86385766aaf679/nova/compute/api.py#L2433
13:58:50 <alex_xu> johnthetubaguy: yea
13:58:54 <johnthetubaguy> alex_xu: thats in some ways worse, as it means we have to turn off paging
13:59:10 <johnthetubaguy> alex_xu: anyways, don't need to worry about an index for that one
13:59:12 <alex_xu> johnthetubaguy: how that works in paging for now?
13:59:16 <alex_xu> I need to check
13:59:41 <alex_xu> it's time to close
13:59:45 <johnthetubaguy> well, I think the expectation is there will only be a very small number of matches
14:00:00 <alex_xu> johnthetubaguy: ok, got it
14:00:05 <alex_xu> #endmeeting