15:01:12 <TravT> #startmeeting openstack search
15:01:12 <openstack> Meeting started Thu Jan 14 15:01:12 2016 UTC and is due to finish in 60 minutes.  The chair is TravT. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:16 <openstack> The meeting name has been set to 'openstack_search'
15:02:09 <sjmc7> mornin
15:02:25 <RickA-HP> Hello
15:02:32 <TravT> okay, had some connection flakiness there!
15:02:37 <TravT> o/ RickA-HP
15:02:57 <TravT> did i miss any other greetings?
15:03:14 <RickA-HP> Just Steve.
15:03:57 <TravT> ok. well, we'll give a minute.
15:05:03 <TravT> maybe we scared 'em all away. ;D
15:05:24 <sjmc7> :)
15:05:38 <TravT> if you aren't already there, agenda is here: https://etherpad.openstack.org/p/search-team-meeting-agenda
15:05:46 <TravT> #topic mitaka 3
15:06:00 <TravT> #undo
15:06:01 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x95b9b50>
15:06:05 <TravT> #topic mitaka 2
15:06:12 <TravT> okay, so mitaka 2 is next week
15:06:42 <TravT> which means we'll have to run the release for the milestone at some point next week.
15:06:46 <sjmc7> the big outstanding spec i can think of is the notification publishing one
15:07:11 <TravT> yes, i've given it a positive vote
15:07:14 <TravT> based on concepts
15:07:30 <TravT> our specs don't have to be perfect waterfall design documents
15:08:00 <TravT> but i don't know if lei-zh is still planning to work on it.
15:08:02 <yingjun> seems late for meeting
15:08:04 <TravT> this release
15:08:07 <TravT> o/ yingjun
15:08:16 <yingjun> o/
15:08:33 <TravT> i'll try to reach out to lei-zh and see if they have plans to implement
15:08:50 <TravT> if it doesn't sound like it, then we'll need it to be re-proposed for n-release
15:09:03 <sjmc7> ok. i’m fine approving the spec, though i have some concern about getting it implemented in time
15:09:42 <TravT> ok, i'll mention that. will also reach out to malini
15:10:21 <TravT> the other spec is zero downtime.  it has a few comments to address.
15:10:22 <TravT> https://review.openstack.org/#/c/245222/
15:10:38 <TravT> RickA-HP: do you want me to address the current comments and then you add anything else to it?
15:11:16 <RickA-HP> Yes, I'm also adrerssing the comments.
15:11:30 <TravT> sjmc7: i also have a few response / questions inline to your comments
15:11:40 <sjmc7> ok, will take a look this morning
15:11:41 <TravT> that have followup question for you
15:12:04 <TravT> RickA-HP: um, so does that mean you'll just do the next rev of it and I shouldn't? either way is okay with me.
15:12:09 <RickA-HP> I'm also breaking it down into separate tasks like we discussed.
15:12:21 <RickA-HP> I'l handle the next rev.
15:12:54 <TravT> okay, that sounds good.
15:13:33 <TravT> #topic bug review / bp review day
15:13:59 <TravT> So, each milestone since starting this up, we've done a bug review / bp review day
15:14:05 <TravT> well, not day
15:14:15 <TravT> just an hour or so in the #openstack-searchlight room
15:14:22 <sjmc7> yeah, they’ve been quite useful
15:14:33 <TravT> i think we should do that again
15:14:59 <TravT> unfortunately, it seems nikhil, rosmaita, and david-lyle aren't around today to see what works for them
15:15:16 <sjmc7> maybe try asking in the sl room later today
15:15:32 <TravT> yeah... or we could schedule in next week's meeting.
15:15:34 <sjmc7> it’d be good to get some other eyes on the pile of reviews
15:15:57 <TravT> definitely
15:16:44 <TravT> ok, so i will follow up on this, but i'm thinking probably the week after next we should go through them all.
15:16:50 <TravT> see what is left for mitaka 3
15:17:26 <TravT> so next...
15:17:30 <TravT> #topic python-searchlightclient CLI Discussion
15:17:58 <TravT> yingjun has three reviews up on the client
15:18:16 <TravT> the command structure was based on some initial input primarily from me
15:18:41 <TravT> and after going through them yesterday i saw some opportunity for improvement
15:18:46 <yingjun> #link https://etherpad.openstack.org/p/searchlight-cli-brainstorming
15:18:55 <TravT> thanks yingjun
15:19:07 <TravT> so, let's just walk through it and get it settled in this meeting
15:19:51 <sjmc7> ok
15:20:16 <sjmc7> we can always make changes later, too - it doesn’t have to be exactly right first time
15:21:27 <yingjun> so Base plugin name?
15:21:39 <TravT> okay, my wifi is being really flaky
15:21:50 <TravT> did i miss anything since 08:19?
15:22:02 <sjmc7> nothing useful
15:22:05 <TravT> i suppose that's my time zone...
15:22:21 <TravT> so, yesterday i organized and put some questions on there.
15:22:50 <sjmc7> so for the ‘first’ patch (260899)
15:23:29 <sjmc7> your suggestion is to use ‘openstack search query ‘{json blob}’ or ‘openstack search query-string “search string”’
15:24:14 <sjmc7> i think i’m ok with that; it does differ fairly radically from how most of the other OSC commands are structured but i think that’s unavoidable
15:24:46 <RickA-HP> Steve, for your first example, did you mean "query --query-string '{json blob}'" ?
15:24:52 <TravT> okay, that's the 3rd patch
15:25:16 <sjmc7> :)  ok, sorry - which one should we look at first
15:25:18 <TravT> line 24 in etherpad
15:25:40 <sjmc7> openstack search i’m definitely ok with
15:25:50 <TravT> ok yes, question 1
15:26:07 <TravT> openstack search vs openstack searchlight
15:26:20 <TravT> looks like we have 3 +1's for openstack search
15:26:21 <sjmc7> it should definitely be the thing it does, not the project name
15:26:38 <TravT> especially in case we ever get hit with some name trademark issue
15:27:06 <RickA-HP> I vote for "openstack search" also.
15:27:24 <TravT> ok, rick, if you want to add +1 to the etherpad, that'd be great!
15:27:34 <TravT> question 2
15:27:36 <TravT> For commands like facet-list and query-string, what option should be used to limit the resource types being searched?
15:27:54 <sjmc7> —type is what we’ve used in the query structure already
15:28:35 <yingjun> +1 for --type
15:28:46 <TravT> it would be a lot easier to type!
15:29:08 <RickA-HP> Will there ever be other types besides resources?
15:29:29 <sjmc7> all types will always be uniquely named
15:29:48 <sjmc7> depending on what we call a ‘resource’, maybe - but not in the near future
15:30:35 <TravT> whatever we do, we should make it consistent with searchlight manage.
15:30:42 <TravT> (which isn't too late to change)
15:30:44 <TravT> currently it is
15:30:46 <TravT> searchlight-manage index sync --type OS::Glance::Image
15:31:09 <sjmc7> well, we should make it consistent with the query language
15:31:17 <TravT> that is true as well
15:31:25 <sjmc7> we’ve used “type” everywhere
15:31:28 <TravT> we should type to match up with the ES "type"
15:31:46 <TravT> s/should/used
15:31:56 <TravT> for better or worse
15:32:40 <sjmc7> let’s go with type
15:32:56 <TravT> RickA-HP: point taken about the namespacing of type, which is why i was kind of on the fence, but from a pure ease of use, --type really is a lot nicer to use repeatedly
15:33:19 <sjmc7> that does roll into a question i had though - if we add ‘type’ as a separate param (whatever we call it)
15:33:27 <RickA-HP> In that case, we should be more unix-like and use "typ" :)
15:33:29 <sjmc7> do we also add things like ‘sort’, ‘size’ ?
15:33:37 <TravT> -t or --type  :)
15:34:03 <sjmc7> making the argument to ‘query’ just be the “query” part of the JSON payload that’s sent to the API?
15:34:14 <TravT> hmm, that's a good question.
15:34:46 <sjmc7> {“query”: {“query-string”: “something”}, “order”: “id”, “size”: 1}   == openstack search query-string “something” —order id —size 1 ?
15:35:06 <TravT> there are some "standard" openstack CLI options for things like sort
15:35:26 <TravT> which are simply not as powerful as what you can do with native ES
15:36:04 <TravT> i was thinking that we could support a simplified CLI structure for search as well that wouldn't require all the nasty json query
15:36:15 <sjmc7> well, our “—order” could be —order ‘{“field”: “id”, “order”: “desc”}'
15:36:27 <sjmc7> E-S supports “order”: “-id”
15:36:32 <sjmc7> for descending
15:36:48 <sjmc7> we could support multiple arguments too, that’d cover all the useful cases
15:38:15 <sjmc7> i’ll add something to the etherpad
15:38:57 <TravT> so, i think we should have an ability for somebody not versed in ES to still do cross resource searching with simple options
15:39:02 <sjmc7> though i think it’s “sort” in e-s, keep misremembering
15:39:49 <TravT> something like
15:39:56 <TravT> openstack search faceted-query --type OS::Glance::Image --container-format raw --and-visibility shared --or-visibility public
15:40:01 <RickA-HP> It's "sort"
15:40:05 <TravT> where the above are <tab> completed
15:40:20 <TravT> see lines 37 - 45
15:40:25 <TravT> on etherpad
15:40:38 <sjmc7> ah, i scrolled down and saw that bit :)
15:40:59 <TravT> that would be complementary to being able to just send in native syntax
15:41:04 <sjmc7> so anything that’s not recognized as one of the top level things gets munged into a query_string i guess?
15:41:42 <sjmc7> oh, the leading —and —or . yeah, i can see that
15:42:18 <RickA-HP> If the user can specify both, do we need to be concerned with the case where the query JSON conflicts with the options?
15:42:22 <sjmc7> handling ‘and’ and ‘or’ might get a bit weird
15:42:35 <TravT> well, that where i was thinking
15:43:24 <TravT> openstack search query --query-string    OR   openstack search query
15:43:28 <TravT> for the native queries
15:43:42 <TravT> and a different command for the tab assist one
15:43:58 <TravT> openstack search faceted-query
15:44:09 <TravT> just as a brainstorming idea
15:46:08 <yingjun> is —query-string optional?
15:46:39 <sjmc7> i’d be fine providing openstach search query <json blob>  and openstack search query-string “string” for now
15:46:44 <sjmc7> along with the optional params
15:47:30 <TravT> ahh... so query is just native syntax
15:47:36 <RickA-HP> Or do we think there will be other options for "openstack search query"?
15:47:37 <sjmc7> yeah
15:47:40 <TravT> even if we someday support something other than ES
15:47:45 <TravT> then it is very portable
15:48:02 <TravT> there could be a --syntax option later if needed for syntax negotiation
15:49:07 <TravT> and openstack search query-string internally wraps it
15:49:09 <TravT> so:
15:50:11 <TravT> openstack search query '{ "query": { {"term": {"name": "something"}}}'
15:50:17 <TravT> or some nastiness like that
15:50:33 <TravT> and then
15:51:05 <TravT> openstack search query-string "name: foo AND tag: database"
15:51:14 <TravT> in ES query-string accepted format
15:51:21 <sjmc7> yes. although i think the leading “query” could be left off in the first instance
15:51:43 <sjmc7> maybe the CLI adds it if it’s not there
15:51:53 <TravT> the other way would be
15:52:03 <RickA-HP> I would reverse the commands. I think "query" would be the generic type and query-string would be the specific JSON object.
15:52:14 <TravT> openstack search query --native <json nasty>
15:52:28 <TravT> openstack search query --query-string <nicer string>
15:53:20 <TravT> rick, i'm using query string to match to the ES specific type of search
15:53:21 <RickA-HP> I like having a single query command with options associated with it. Insted of having multiple types of query commands.
15:54:01 <TravT> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
15:54:02 <sjmc7> option 3 - let the client figure it out :)
15:54:03 <RickA-HP> I know, I always tr yto decouple an interface from a specific back-end technology :)
15:54:12 <sjmc7> if it’s JSON, it’s a query
15:54:48 <sjmc7> or even simpler, if it ends and/or starts with brackets, it’s an attempt at a query
15:55:31 <TravT> RickA-HP: the decouple part is where the tab complete comes in i think
15:55:44 <TravT> with the openstack search faceted-query idea i mentioned earlier
15:56:25 <TravT> otherwise, we can just have the available options (--query-string, --simple-query-string) come from the supported search provider
15:56:34 <TravT> and --native is no parsing
15:57:47 <TravT> okay, almost out of time. I think we should jump to the room to finish this part of the discussion.
15:57:51 <TravT> but from a summary
15:58:25 <TravT> it seems that this patch is ready except for some confusion i have on making it work with devstack
15:58:26 <TravT> https://review.openstack.org/#/c/249076
15:58:55 <TravT> and this patch https://review.openstack.org/#/c/255027/
15:59:05 <TravT> just needs --resource changed to --type
15:59:20 <TravT> and we need to continue discussion in #openstack-searchlight
15:59:24 <TravT> ok?
15:59:33 <RickA-HP> Sounds good.
15:59:36 <yingjun> ok
15:59:57 <TravT> thanks. i'll end meeting here and see you there. :)
16:00:07 <TravT> #endmeeting