Thursday, 2016-01-14

*** yingjun_ has joined #openstack-searchlight00:36
*** TravT has quit IRC00:49
*** bpokorny has quit IRC01:05
*** yingjun_ has quit IRC01:13
*** yingjun_ has joined #openstack-searchlight01:13
*** lakshmiS_ has quit IRC01:59
*** TravT has joined #openstack-searchlight02:36
*** ekarlso has quit IRC02:52
*** bpokorny has joined #openstack-searchlight02:55
*** yingjun_ has quit IRC02:57
*** yingjun_ has joined #openstack-searchlight02:57
*** yingjun_ has quit IRC03:01
*** TravT has quit IRC03:03
*** ekarlso has joined #openstack-searchlight03:06
*** bpokorny has quit IRC03:07
*** yingjun_ has joined #openstack-searchlight04:26
*** yingjun_ has quit IRC04:31
*** TravT has joined #openstack-searchlight04:41
*** bpokorny has joined #openstack-searchlight04:41
*** bpokorny has quit IRC05:00
*** yingjun_ has joined #openstack-searchlight05:29
*** yingjun_ has quit IRC05:38
*** TravT has quit IRC05:42
*** TravT has joined #openstack-searchlight05:45
*** TravT has quit IRC05:50
*** yingjun_ has joined #openstack-searchlight05:56
*** TravT has joined #openstack-searchlight06:05
*** TravT has quit IRC07:07
*** TravT has joined #openstack-searchlight07:12
*** TravT has quit IRC07:17
*** TravT has joined #openstack-searchlight07:17
*** TravT has quit IRC07:36
*** TravT has joined #openstack-searchlight07:41
*** TravT has quit IRC07:50
*** TravT has joined #openstack-searchlight07:51
*** TravT has quit IRC08:00
*** TravT has joined #openstack-searchlight08:04
*** TravT has quit IRC08:13
*** TravT has joined #openstack-searchlight08:15
*** TravT has quit IRC08:19
*** TravT has joined #openstack-searchlight08:24
*** yingjun_ has quit IRC08:28
*** yingjun_ has joined #openstack-searchlight08:29
*** TravT has quit IRC08:32
*** TravT has joined #openstack-searchlight08:38
*** TravT has quit IRC08:47
*** TravT has joined #openstack-searchlight08:52
*** TravT has quit IRC09:01
*** TravT has joined #openstack-searchlight09:02
*** TravT has quit IRC09:11
openstackgerritLi Yingjun proposed openstack/python-searchlightclient: Add resource type list CLI  https://review.openstack.org/24907609:11
*** TravT has joined #openstack-searchlight09:12
*** ig0r_ has joined #openstack-searchlight09:17
*** TravT has quit IRC09:25
*** yingjun_ has quit IRC09:32
*** ig0r_ has quit IRC09:41
*** TravT has joined #openstack-searchlight10:22
*** TravT has quit IRC10:31
*** TravT has joined #openstack-searchlight10:36
*** TravT has quit IRC10:51
*** GB21 has joined #openstack-searchlight11:05
*** GB21 has quit IRC11:15
*** TravT has joined #openstack-searchlight11:51
*** TravT has quit IRC12:00
*** TravT has joined #openstack-searchlight12:06
*** TravT has quit IRC12:14
*** TravT has joined #openstack-searchlight12:15
*** TravT has quit IRC12:20
*** TravT has joined #openstack-searchlight12:25
*** yingjun_ has joined #openstack-searchlight12:25
*** yingjun_ has quit IRC12:30
*** TravT has quit IRC12:34
*** yingjun_ has joined #openstack-searchlight12:35
*** TravT has joined #openstack-searchlight12:39
*** yingjun_ has quit IRC12:42
*** yingjun_ has joined #openstack-searchlight12:42
*** TravT has quit IRC12:43
*** yingjun_ has quit IRC12:47
*** TravT has joined #openstack-searchlight13:07
*** TravT has quit IRC13:17
*** TravT has joined #openstack-searchlight13:17
openstackgerritHarshada Mangesh Kakad proposed openstack/searchlight: Replace deprecated library function os.popen() with subprocess  https://review.openstack.org/26755513:24
*** TravT has quit IRC13:30
*** TravT has joined #openstack-searchlight13:36
*** yingjun_ has joined #openstack-searchlight13:40
*** TravT has quit IRC13:45
*** TravT has joined #openstack-searchlight13:46
*** TravT has quit IRC13:55
*** TravT has joined #openstack-searchlight14:00
*** TravT has quit IRC14:18
*** TravT has joined #openstack-searchlight14:19
*** TravT has quit IRC14:24
*** TravT has joined #openstack-searchlight14:29
*** yingjun has quit IRC14:33
*** TravT has quit IRC14:38
*** GB21 has joined #openstack-searchlight14:46
*** yingjun_ has quit IRC14:54
*** yingjun has joined #openstack-searchlight14:54
*** TravT has joined #openstack-searchlight15:00
TravTCourtesy Searchlight meeting reminder now in #openstack-meeting-4: lakshmiS, nikhil_k, rosmaita, sigmavirus24, TravT, david-lyle, kragniz, sjmc7, ekarlso, abhijeetm, itisha, GB21, briancline, lei-zh-, yingjun15:00
*** TravT has quit IRC15:01
*** TravT has joined #openstack-searchlight15:02
*** sigmavirus24_awa is now known as sigmavirus2415:07
*** TravT has quit IRC15:20
*** TravT has joined #openstack-searchlight15:21
*** RickA-HP has joined #openstack-searchlight15:59
TravTyingjun: what are your thoughts on everything we just discussed in the meeting?16:00
*** GB21 has quit IRC16:00
TravTi guess i'll summarize current options again.16:01
yingjuni’d prefer openstack search query16:01
yingjunand adding optional arguments if needed16:02
TravTso then16:02
TravThttps://review.openstack.org/#/c/26089916:02
TravTreally just chances the command from resources to query16:03
TravTopenstack search query [—type [<resource-type> [<resource-type>…]]] [--query-string <query-string>] [--all-projects] [--details]16:03
TravTand we add follow patch with --native for a full native query16:04
TravTopenstack search query [—type [<resource-type> [<resource-type>…]]] [--query-string <query-string>] [--native <json>] [--all-projects] [--details]16:04
sjmc7what does it mean if you don’t provide --query-string?16:05
TravTand possibly a --simple-query-string16:05
yingjundefaults to match everything16:05
RickA-HPAnd the user can specify only one of the options [ --query-string | --native | --simple-query-string ]16:06
TravThmm, [ [--query-string <query-string>]  |  [--native <json>]  |  [--simple-query-string] ]16:06
rosmaitaTravT: sorry i missed the mtg16:06
TravTrosmaita: yeah, we missed you!16:07
TravTstill time to catch up!16:07
TravTi think perhaps more interesting than usual today16:07
TravThttp://eavesdrop.openstack.org/meetings/openstack_search/2016/openstack_search.2016-01-14-15.01.log.html16:07
TravTFYI16:07
TravTso yingjun, RickA-HP, sjmc7, if it is just a [ --query-string | --native | --simple-query-string ]16:08
TravTthen will it become really complex figuring out what additional parameter options are possible with each one?16:09
TravTgoing back to some of sjmc7s questions about sorting, etc.16:09
sjmc7if you think there’d be different options then yes, it will become very complex16:09
RickA-HPThe --native option is easy!16:10
TravTthat's why perhaps the16:11
RickA-HPGoing back to Steve's question about a default, I agree with yingjun in that it would match everything. If no option is specified it would default to "--native '{ "query" : { "math_all" : {} } }'16:11
sjmc7i’m not really a big fan of mandatory optional arguments16:11
TravTwith some limit...16:11
TravTsjmc7: please expand?16:12
openstackgerritLi Yingjun proposed openstack/python-searchlightclient: Add facet list support  https://review.openstack.org/25502716:12
sjmc7with a default i think it’s better16:13
sjmc7i’d still prefer to make it smart if we can16:13
sjmc7detecting whether you’re trying a query string versus a json payload isn’t hard16:13
TravTok, couple examples again? :)16:14
sjmc7search query “name: something”16:14
sjmc7search query ‘{“term”: {“field”: “name”, “query”: “something”}}'16:14
TravTthat is nice.16:15
sjmc7that’s a bit harder if you want to support16:15
sjmc7search query ‘“match_all”: {}'16:15
sjmc7though even then, if the CLI can parse it as JSON with surrounding brackets16:15
sjmc7then it probably is16:16
TravTand for simple string query support16:16
TravThow do we figure it out?16:16
sjmc7you could also use —query-stirng as a flag16:16
sjmc7what’s ‘simple query string’? i think i missed that16:16
ekarlsoanything I can help with ? :p16:17
TravT    simple-query-string --> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html16:17
sjmc7ekarlso - there’re a bunch of reviews need doing, and a few bugs16:17
TravTekarlso, we're having debates on CLI16:17
sjmc7goodness gracious16:17
sjmc7that doesn’t seem simple :)16:18
TravTdefinitely if you want to reach the meeting logs from today and catch up16:18
TravThttp://eavesdrop.openstack.org/meetings/openstack_search/2016/openstack_search.2016-01-14-15.01.log.html16:18
sjmc7i’d use a flag16:18
TravT--syntax16:18
sjmc7openstack search query “magical simple string” —simple-query-string16:18
sjmc7i prefer the structure of a mandatory positional argument, and optional flags to alter behavior16:18
sjmc7the default would then be equivalent to —native16:19
sjmc7and if possible make the CLI try to figure it out16:19
TravTthat seems pretty reasonable...16:19
rosmaitajust want to interject that the complexity of this issue has exceeded my available brain cells ATM, so i'm not going to express an opinion16:19
TravTi still would like the faceted query idea though16:19
sjmc7mine too rosmaita :)16:20
sjmc7honestly, i’d like to get the simpelst useful thing merged16:20
TravTrosmaita: LOL16:20
sjmc7and then go from there16:20
rosmaitabummer, because i was going to say, i will go with what sjmc7 says!16:20
sjmc7lesson learned!16:20
RickA-HPDo we think that the "--native" vs "query-string" vs "simple-query-string" are different enough that we can easily detect which is which in the CLI?16:20
sjmc7simple vs query-string, no16:21
RickA-HPI would hate for the CLI being too smart and returning the wrong query that the user is not expecting!16:21
TravTi think --query-type16:21
sjmc7however, i’ve never used simple query string16:21
TravTsjmc7 i played with it as part of the ui dev16:21
sjmc7the docs make it sound like a query_string that silently eats errors16:22
TravTit was a little more forgiving16:22
TravTbut if i remember right, not quite as powerful16:22
sjmc7RickA-HP: i think reliably we can determine query vs query-string16:23
TravTin either case, they all have these extra params16:23
TravTthat differ based on query type16:23
RickA-HPWould we pass in the simple-query-string as is? For example will we allow the user to specify an analyzer?16:23
sjmc7i’m struggling to understand why we’d try to provide options for all this16:24
sjmc7if you’re doing a really crazy query, do it with json16:24
sjmc7otherwise we’re trying to reinterpret the DSL16:24
TravTso, i generally like simplest approach16:25
RickA-HPSteve, are you proposing not to have simple-query-string and to limit what we allow for query-string?16:25
TravTbut i don't like backing into a corner16:25
TravTand on the UI, definitely found some of the extra params very nice16:25
TravTlike phrase slop16:25
sjmc7the UI isn’t USING the cli16:25
sjmc7here’s the problem16:26
sjmc7if we have, say 100 options16:26
TravTthe UI defnitely isnt' using the CLI16:26
sjmc7we have to document all of them, and make sure we target a specific e-s API version16:26
TravTbut it is really nice to put in a simple query without a bunch of json and be able to say "please don't use exact phrase matching" --> eg phrase_slop != 016:27
TravTi'm thinking we could do16:27
TravTbasically what you are saying16:27
TravTbut have ability to pass through additional query type params.16:28
RickA-HPTravis, I think your Wifi is dropping again!16:28
TravThmm16:28
TravTwhy?16:28
sjmc7ok. what are you wanting yingjun to do for these patches?16:29
sjmc7my vote is still to support openstack search query <json blob>16:29
sjmc7and go from there16:29
TravTas mentioned at end of the meeting16:29
RickA-HPSteve, do you mean the first patch would do 'JSON blob' only?16:29
sjmc7yes.16:29
TravT08:58 TravT: it seems that this patch is ready except for some confusion i have on making it work with devstack16:30
TravT08:58 TravT: https://review.openstack.org/#/c/24907616:30
TravT08:58 TravT: and this patch https://review.openstack.org/#/c/255027/16:30
TravT08:59 TravT: just needs --resource changed to --type16:30
sjmc7ok, good16:30
TravTyingjun already fixed the above16:30
TravT25502716:30
TravTthe last one https://review.openstack.org/#/c/260899/16:30
TravTsounds like we are leaning towards a sytnax like this16:31
TravTopenstack search query <json or string which client figures out> [—type [<resource-type> [<resource-type>…]]]  [--all-projects] [--details]16:32
yingjunok16:32
sjmc7ok, i’m good with that16:33
TravTwith possibility of sending additional params through16:33
TravTe.g. phrase_slop....16:33
RickA-HPWe will need to define what "string" supports. Starting with a few options as needed.16:33
TravTi'm kind of thinking that we could treat any unrecognized param e.g. --phrase_slop as something that we just pass straight through.16:34
RickA-HPIf this is too large, we can go with Steve's idea of support JSON blob only for the first patch and then follow it up with the simple search string in a subsequent patch.16:34
TravTto be clear, yingjun's patch already support query string, i thinkg16:34
TravTnot json blob16:34
sjmc7that’s fine too, either works as a start point16:35
TravTline 41 here: https://review.openstack.org/#/c/260899/1/searchlightclient/osc/v1/search.py16:35
RickA-HPOk16:35
TravTand line 7816:35
TravTwell, then i think we're in agreement. ?16:36
yingjunsorry guys, i have to take care of my crying baby, please comment in the patch if any conclusion come out.. and i’ll do the change.16:36
ekarlsoso baically a discussion on how to implement query stuff from a cli perspective ?16:37
TravTyingjun: no, our apolozies to you!16:37
ekarlsoyingjun: heh, I know that problem ;)16:37
TravTekarlso: yeah16:37
RickA-HPTravis, I believe we are in agreement.16:37
TravTok, i'll add comment on patch16:37
TravTopenstack search query <json or string which client figures out> [—type [<resource-type> [<resource-type>…]]]  [--all-projects] [--details]16:37
TravTthanks for pounding through this.16:39
TravTjust didn't want to see these patches quagmired for ever. :)16:39
RickA-HPJust in time, the way the wind is blowing off the mountains I may be joining you in Wifi limbo.16:39
TravTyeah, it's crazy!16:39
TravTi keep expecting a tree to come flying through the window16:40
ekarlsolol, you should've seen the weather here in december :P16:41
ekarlsorecord rains and winds : P16:41
RickA-HPWhere is that, in the South?16:43
ekarlsoRickA-HP: Norway :P16:43
RickA-HPekarlso Yikes, stay dry!16:43
*** yingjun has quit IRC16:46
*** yingjun has joined #openstack-searchlight16:54
*** nikhil_k has joined #openstack-searchlight17:27
*** nikhil has quit IRC17:29
sjmc7i’d really like to get https://review.openstack.org/#/c/266587/ merged; it makes it a lot easier to add tests for other plugins we work on17:29
krotscheckTravT: In response to https://review.openstack.org/#/c/265396/ - I can just remove the http methods.17:33
krotscheckTL/DR my comment: If we miss something, there's easy workarounds available17:34
openstackgerritMichael Krotscheck proposed openstack/searchlight: Added Keystone and RequestID headers to CORS middleware  https://review.openstack.org/26539617:35
sjmc7TravT / lakshmiS / RickA-HP - any thoughts on https://bugs.launchpad.net/searchlight/+bug/153297217:39
openstackLaunchpad bug 1532972 in OpenStack Search (Searchlight) "Restructure parent-child plugins" [Medium,New]17:39
sjmc7i think neutron’s going to be parent/child and suffer from the same problem17:40
*** GB21 has joined #openstack-searchlight17:59
*** nikhil_k is now known as nikhil18:11
TravTsjmc7 i do agree it is problematic and i recall us pondering that at the time.  are you just thinking of combining to single plugin18:11
TravTbut still have the two types?18:12
sjmc7i think keep the separate plugins, but make the relationship known at index time18:12
sjmc7so either zone knows ‘recordset’ is a child or the other way round18:12
sjmc7and don’t index ‘children’ independently18:12
sjmc7everything else, i think, can remain the same18:13
TravTis there value to keeping separate plugin?18:14
sjmc7being able to search them independently18:14
*** sigmavirus24 is now known as sigmavirus24_awa18:15
TravTok, i'm confused18:15
TravTreading the bug it says: It doesn't make sense for Zone to be a first-class plugin.18:16
sjmc7it doesn’t make sense to INDEX it as a first class plugin18:16
sjmc7yes, i wasn’t very clear18:17
sjmc7there’s no problem searching RecordSet separately18:17
sjmc7it doesn’t make sense to *index* RecordSet18:17
TravTokay, now that seems a little more clear, especially since zone is the parent18:18
sjmc7i’ll edit the description, it’s not very clear18:19
sjmc7in my brain it made sense18:19
*** sigmavirus24_awa is now known as sigmavirus2418:19
TravTno worries. i often find that by the time my thoughts reach my fingers that they must taken a detour along the way.18:19
sjmc7yeah, i should’ve written ‘RecordSet’ too :)18:20
sjmc7yeah, i think i’m right; reindexing RecordSets makes no sense on their own18:21
TravTno, not really18:21
sjmc7since you have to also ensure Zones exist18:22
sjmc7so i think it’s fine to allow searches to treat them separately18:22
sjmc7but for indexing i think we’d present it as18:22
sjmc7 - OS::Nova::Server18:22
sjmc7 - OS::Designate::Zone (OS::Designate::Recordset)18:22
TravTThat could work18:24
TravTother than maybe something to do with performance i'm not sure of a value to not doing it that way18:25
sjmc7right now i think it’s broken18:31
sjmc7with the notifications we can somewhat rely on a zone existing before we try to index recordsets for it, but if you’re reindexing that’s not necessarily the case18:32
*** bpokorny has joined #openstack-searchlight18:32
TravTwell, that's interesting.18:38
TravTbecause recordset notifications happen independent of zone notificatoins18:38
sjmc7right, so that’s fine18:38
sjmc7because if we presume we’re up to date with zones we should be ok updating recordsets18:39
TravTjust read your updated description18:40
sjmc7i’m bringing this up because it will affect neutron, presuming we treat subnets and ports as children of networks18:41
sjmc7simplest option, of course, is to not do parent-child at all18:41
TravTi can say that at a concept level, what you are saying seems right, but i'm not visualizing the concrete code change being suggested yet18:41
sjmc7initial_indexing won’t do anything for recordsets - recordsets will mark itself as a child of ‘zones'18:42
sjmc7and the mapping and indexing will be driven from there18:42
TravTvery small lightbulb finally started light up18:42
TravTthat makes sense i believe.18:43
TravThumor me and let me restate in my own words.18:43
TravTsearchlight-manage index sync  will not let you specify OS::Designate::Recordset on its own and if you don't specify anything it just lists OS::Designate::Zone (OS::Designate::Recordset)18:44
sjmc7i think for the sake of efficiency, it’s ok to index all Zones, and then all Recordsets18:44
sjmc7right18:44
sjmc7yeah - relaly, that’s the only difference18:44
TravTyeah, some algorithm would be fine18:45
sjmc7the issue is, for something like neutron, it makes reindexing quite expensive18:45
TravTnot sure if there'd be a race condition, but that must already exist18:45
sjmc7although since there are at least as many children as parents, maybe that’s ok18:45
sjmc7reindexing Networks independently of Ports would make no sense18:46
TravTit is kind of weird to conceptually think about "let me reindex all my records because i now my zones are all correct"18:46
TravTnow -> know18:46
TravTagree it could get expensive, but you'd have to be pretty brilliant to realize that only one type of child entity is out of whack18:47
TravTit is likely more error prone to not index from parent on down.18:48
*** bpokorny_ has joined #openstack-searchlight18:49
sjmc7grr, having wifi issues of my own18:50
*** bpokorny_ has quit IRC18:50
sjmc7i missed anything since 12:4718:50
*** bpokorny_ has joined #openstack-searchlight18:51
*** bpokorny has quit IRC18:53
TravTjust me writing a note on the notification forwarding spec18:54
*** bpokorny_ has quit IRC18:55
*** bpokorny has joined #openstack-searchlight18:56
sjmc7you want me to +2 it?18:58
sjmc7i will do so, but i’m not intending to spend any time implementing it18:59
*** bpokorny has quit IRC19:01
TravTlet's wait to see what reply happens from what i posted19:10
*** RickA-HP has quit IRC19:15
openstackgerritTravis Tripp proposed openstack/searchlight: Add python 3.4 to programming languages in setup.cfg  https://review.openstack.org/26778319:20
*** lakshmiS has joined #openstack-searchlight19:32
*** TravT_ has joined #openstack-searchlight19:44
*** TravT has quit IRC19:48
*** TravT_ has quit IRC19:51
*** TravT has joined #openstack-searchlight19:52
*** yingjun has quit IRC19:53
*** yingjun has joined #openstack-searchlight19:53
*** yingjun has quit IRC19:57
*** TravT has quit IRC19:58
*** TravT has joined #openstack-searchlight20:03
*** TravT has quit IRC20:23
*** TravT has joined #openstack-searchlight20:24
*** GB21 has quit IRC20:28
*** TravT has quit IRC20:31
*** TravT has joined #openstack-searchlight20:32
*** TravT_ has joined #openstack-searchlight20:43
*** TravT has quit IRC20:44
*** TravT has joined #openstack-searchlight20:50
*** TravT_ has quit IRC20:50
*** TravT_ has joined #openstack-searchlight20:55
*** TravT has quit IRC20:55
*** bpokorny has joined #openstack-searchlight21:05
*** TravT_ has quit IRC21:06
*** TravT has joined #openstack-searchlight21:07
*** TravT_ has joined #openstack-searchlight21:13
*** TravT has quit IRC21:13
*** TravT_ has quit IRC21:21
*** TravT has joined #openstack-searchlight21:21
*** TravT has quit IRC21:28
*** TravT has joined #openstack-searchlight21:28
*** TravT has quit IRC21:36
*** TravT has joined #openstack-searchlight21:37
*** TravT_ has joined #openstack-searchlight21:43
*** TravT has quit IRC21:43
*** TravT has joined #openstack-searchlight21:50
*** TravT_ has quit IRC21:50
*** TravT_ has joined #openstack-searchlight21:53
*** TravT has quit IRC21:54
*** TravT has joined #openstack-searchlight22:10
*** TravT_ has quit IRC22:12
*** TravT_ has joined #openstack-searchlight22:15
*** TravT has quit IRC22:16
*** TravT has joined #openstack-searchlight22:21
*** TravT_ has quit IRC22:23
*** TravT_ has joined #openstack-searchlight22:26
*** TravT has quit IRC22:28
*** sigmavirus24 is now known as sigmavirus24_awa22:59
TravT_sjmc7: u there?23:02
*** TravT_ is now known as TravT23:02
*** bpokorny_ has joined #openstack-searchlight23:17
*** bpokorny has quit IRC23:20
openstackgerritSteve McLellan proposed openstack/searchlight: WIP Index child plugins as part of parent  https://review.openstack.org/26786423:47
*** TravT has quit IRC23:54
*** TravT has joined #openstack-searchlight23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!