15:01:12 <TravT> #startmeeting openstack search
15:01:13 <openstack> Meeting started Thu Sep 24 15:01:12 2015 UTC and is due to finish in 60 minutes.  The chair is TravT. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:18 <openstack> The meeting name has been set to 'openstack_search'
15:01:38 <lakshmiS> o/
15:01:45 <sjmc7> hey
15:01:46 <TravT> hey-o
15:02:04 <rosmaita> o/
15:02:25 <TravT> nikhil: still here?
15:02:55 <TravT> #link https://etherpad.openstack.org/p/search-team-meeting-agenda
15:03:13 <TravT> we'll wait just a moment for people to stream in...
15:03:22 <TravT> rosmaita: i have a patch that perhaps you could look at
15:03:38 <rosmaita> TravT: sure
15:03:45 <TravT> https://review.openstack.org/#/c/224411/
15:04:23 <TravT> ok, we'll get going.
15:04:35 <TravT> basically today, we just need to run through the rc1 status
15:04:46 <TravT> #topic rc status
15:04:58 <TravT> #link https://launchpad.net/searchlight/+milestone/liberty-rc1
15:05:30 <TravT> looking like we are in good shape.
15:06:05 <TravT> lakshmi: sjmc7: what's going on with the function tests? https://blueprints.launchpad.net/searchlight/+spec/set-up-functional-tests
15:06:17 <nikhil> TravT here now
15:06:22 <sjmc7> i need to figure out why they don't work for me
15:06:28 <TravT> hi nikhil!
15:06:29 <sjmc7> zuul's not running them, best i can make out
15:06:29 <lakshmiS> TravT:  it will be nice if someone else also tests them
15:06:32 <nikhil> hi
15:06:34 <nikhil> TravT
15:06:57 <lakshmiS> It works for me but not for sjmc7
15:07:11 <TravT> #link https://review.openstack.org/#/c/217580/
15:07:37 <TravT> so sjmc7, what do you mean zuul isn't running them?
15:08:02 <lakshmiS> es is not available
15:08:03 <lakshmiS> on zuul tests
15:08:12 <sjmc7> http://logs.openstack.org/80/217580/10/check/gate-searchlight-python27/de23b19/testr_results.html.gz
15:08:21 <sjmc7> shows it skipping 28 of them
15:08:29 <TravT> ok...
15:08:59 <TravT> and they also don't work for you locally?
15:09:08 <sjmc7> i'm ok getting that enabled later, but yeah, the metadef ones fail for me
15:09:16 <sjmc7> looks like all the scripted ones
15:09:23 <sjmc7> i didn't get a chance to dig in
15:09:50 <TravT> which ES do you guys have locally?
15:09:54 <lakshmiS> how about atleast some of them on this meeting quickly test that in background
15:10:05 <sjmc7> 1.7. i will run it in the debugger later
15:10:27 <TravT> ok, i have what devstack installs, which I think is 1.4.?
15:10:38 <TravT> what do you have lakshmiS?
15:10:49 <lakshmiS> 1.7.2
15:11:58 <TravT> ok, i will try a run of them.
15:12:06 <lakshmiS> thx
15:12:14 <TravT> should they be running under py34?
15:12:47 <sjmc7> both
15:12:58 <lakshmiS> only skips if es is not available
15:13:06 <TravT> do they work on one, but not the other for you sjmc7?
15:13:36 <sjmc7> only tested py27. i can try 34, though i would be flabbergasted if there was a difference
15:14:16 <TravT> :)
15:14:40 <lakshmiS> yeah i would doubt
15:15:20 <TravT> ok, i've got to py34 started up in the background
15:15:23 <TravT> will see what happens
15:15:32 <sjmc7> running it now. i'll take a closer look after this
15:15:49 <TravT> skips=28
15:16:21 <sjmc7> export PATH=$PATH:<path to elasticsearch>
15:17:17 <TravT> include bin
15:17:18 <TravT> ?
15:17:24 <TravT> or just base path?
15:17:28 <sjmc7> bin
15:19:00 <TravT> ran 122 tests
15:19:04 <TravT> don't see any skips
15:19:20 <TravT> py34: commands succeeded
15:19:51 <lakshmiS> how many tests were there when it skipped?
15:20:05 <TravT> i knew you'd ask that
15:20:10 <sjmc7> 94
15:20:40 <TravT> Here's run with exporting path: Ran 122 tests in 24.367s (+24.048s) PASSED (id=5)
15:20:40 <sjmc7> ok. then it's something mysterious in my setup
15:21:18 <sjmc7> i'll try and figure it out then; if i can't by EOD i'm ok approving this in the absence of any other failures
15:21:50 <TravT> Ran 122 (+31) tests in 0.321s (-0.002s)
15:21:50 <TravT> PASSED (id=4, skips=28)
15:21:59 <TravT> Without exported path^
15:22:29 <TravT> ok, so, to conclude, it appears sjmc7 has an environmental issue
15:22:50 <sjmc7> small sample size! :)
15:23:01 <lakshmiS> i will also try in a new setup and comment on patch if it doesnt work
15:23:06 <TravT> ok
15:23:10 <TravT> lakshmiS:
15:23:28 <TravT> are any of the other patches we have up for review going to break these tests?
15:23:40 <lakshmiS> need to figure out how to install es on zuul environment
15:23:50 <TravT> is there a reason to hold off on more merges until functional tests?
15:23:57 <TravT> just wondering if we need to do any rebasing here
15:24:07 <TravT> setting up dependencies
15:24:46 <TravT> although i guess zuul isn't really testing them right now
15:25:00 <sjmc7> don't think so. i would like to see why it won't work for me (on two systems) though. i'll try to do so this morning
15:25:09 <TravT> #action lakshmiS figure out how to make them run in zuul
15:25:19 <lakshmiS> :)
15:25:43 <TravT> ok, so then on to the rest of these
15:25:44 <TravT> https://launchpad.net/searchlight/+milestone/liberty-rc1
15:26:05 <TravT> i bumped Use keystone.session.Session for clients to Essential
15:26:28 <TravT> because with leaving my system running for long periods I see that listener loses its connection to nova
15:26:36 <TravT> and i have to restart the listener
15:26:39 <sjmc7> yeah, it's helpful. i'll retest that, too. i think it needs a documentation update
15:26:50 <TravT> apparently the patch fixes that
15:26:59 <sjmc7> it should do, yes
15:27:05 <TravT> https://review.openstack.org/#/c/219271/
15:27:12 <lakshmiS> will take a look
15:27:46 <TravT> ekarlso usually isn't available at this time of day, so we'll try to catch him later.
15:28:10 <TravT> next Add a pattern for specifying pr plugin configuration
15:28:16 <TravT> this is medium.
15:28:23 <TravT> do we believe this is right priority?
15:28:48 <TravT> if so, then we probably need to set a deadline for it merging or for bumping from RC
15:29:15 <sjmc7> it's useful but not necessarily essential for this release
15:29:28 <lakshmiS> i think medium sounds right since this is good to have feature
15:29:42 <sjmc7> it's more to assist deployers than anything
15:30:35 <TravT> i'm going to propose that EOD wednesday is our deadline for merges.
15:30:40 <TravT> next wednesday
15:30:42 <sjmc7> ok
15:30:59 <TravT> then thursday / friday we work on getting the release done
15:31:11 <TravT> nikhil: rosmaita: lakshmiS: what do you think?
15:31:24 <lakshmiS> will that be RC1 and more later if required?
15:31:38 <nikhil> the notification patch is up for review
15:31:45 <nikhil> I am waitinf for one more +2
15:32:02 <sjmc7> that'd be good
15:32:06 <TravT> nikhil: what are you doing for glance deadlines?
15:32:11 <nikhil> bt it may be this or next week
15:32:18 <nikhil> if it gets in by today then in RC1
15:32:30 <nikhil> else RC2 is possibly planned for next week
15:32:49 <nikhil> please ping hemanth for the other review
15:33:05 <nikhil> jokke was going to help but not sure how much time he has atm
15:33:10 <lakshmiS> yeah i did. will ping again
15:33:37 <TravT> lakshmiS: yes, I think next wednesday is RC1 deadline.  After that we should only look at high / critical bugs.
15:34:02 <TravT> nikhil: thanks for helping to push that through
15:34:20 <TravT> lakshmiS: but ideally, would like to not have to do a RC2
15:34:25 <nikhil> np!
15:34:40 <TravT> let's vote on that in next Thursday's meeting.
15:34:46 <nikhil> yes please
15:34:51 <lakshmiS> TravT: ok
15:35:16 <TravT> so back to: https://launchpad.net/searchlight/+milestone/liberty-rc1
15:35:37 <TravT> next up is faceting support (https://review.openstack.org/222388)
15:35:43 <TravT> I see you +2'd it lakshmi
15:35:52 <TravT> I have one final query that I'm trying out
15:36:05 <TravT> run through designate, images, metadefs, and most of nova
15:36:25 <TravT> but i'm not getting results from a nova filter query... which I think is my problem
15:36:31 <TravT> just ran out of time before this meeting started
15:36:47 <lakshmiS> ok
15:36:59 <TravT> ok, on to bugs
15:37:15 <TravT> 5 in progress
15:37:26 <TravT> 1 new that we can talk through momentarily
15:37:35 <TravT> but before we do that... are there any bugs missing?
15:37:47 <TravT> that we should target for rc1
15:37:53 <TravT> https://bugs.launchpad.net/searchlight
15:38:05 <sjmc7> there was another config one that i filed i think
15:38:13 <sjmc7> to allow separate sections for the api and listener
15:38:26 <sjmc7> mainly because otherwise, deploying with a log file is painful
15:38:57 <TravT> ok, so do you think this is high / medium?
15:39:05 <sjmc7> https://bugs.launchpad.net/searchlight/+bug/1493975
15:39:06 <openstack> Launchpad bug 1493975 in OpenStack Search (Searchlight) "Need separate config sections for api and listener" [Undecided,New] - Assigned to Steve McLellan (sjmc7)
15:39:18 <sjmc7> mm... probably medium, though it will be a pain for deployers
15:39:23 <sjmc7> it's also probably not a big change
15:39:50 <TravT> well, we can target for rc1, but reviews should be done in accordance with priority
15:40:30 <sjmc7> yep
15:40:33 <TravT> we'll re-evaluate it for rc1 next thursday
15:40:41 <TravT> there is one other bug i filed
15:40:50 <TravT> https://bugs.launchpad.net/searchlight/+bug/1497122
15:40:52 <openstack> Launchpad bug 1497122 in OpenStack Search (Searchlight) "Search results incomplete during full indexing." [Undecided,New]
15:41:16 <TravT> i don't think on its own it is critical
15:42:14 <sjmc7> i'd agree - i think we can roll that into deployment hardening in mitaka
15:42:53 <TravT> ok.  let's not pull it into liberty then.
15:43:19 <TravT> lakshmiS: https://bugs.launchpad.net/searchlight/+bug/1492579
15:43:20 <openstack> Launchpad bug 1492579 in OpenStack Search (Searchlight) "Property mapping for Metadef properties regressed to "name" from "property"" [Undecided,New]
15:44:03 <TravT> is that one still a problem or did your recent metadef patch address it?
15:44:22 <lakshmiS> its still a bug
15:44:36 <lakshmiS> i will put a patch up
15:44:59 <TravT> ok, i think it'll take me several minutes to actually understand it
15:45:04 <TravT> so will look later.
15:45:10 <TravT> but, what priority should we put on this?
15:45:40 <lakshmiS> it should be high since it fails the current code during listener updates
15:45:54 <TravT> ok, i updated that bug
15:46:43 <TravT> there is one other bug i filed yesterday as a medium, which makes it unlikely for this release.
15:46:49 <TravT> however the change is simple
15:46:55 <TravT> https://bugs.launchpad.net/bugs/1498995
15:46:56 <openstack> Launchpad bug 1498995 in OpenStack Search (Searchlight) "Use doc_values in mappings" [Medium,New]
15:47:04 <sjmc7> i'm not so sure about this one
15:47:17 <TravT> i went to an elasticsearch webinar yesterday on scaling.
15:47:31 <TravT> and they spent nearly 20 minutes of it extolling the virtues of making this change
15:47:49 <TravT> saying it is so important that they just do it by default starting in 2.0 elasticsearch
15:47:52 <sjmc7> yes.. but we're not scaling :)
15:48:50 <sjmc7> i guess there's probably no harm doing it
15:48:52 <TravT> sjmc7: other than it probably not being very impactful at this point, do you have other concerns with it?
15:48:55 <sjmc7> i doubt it will benefit us
15:49:43 <TravT> probably, but the guy on the call said it is the very first thing he checks on any deployment.
15:50:08 <TravT> i agree it isn't critical to us right now, which is why i set as medium.
15:50:14 <sjmc7> we're not a typical deployment :)  the others i've used/seen typically are storing many tens of millions of text documents
15:50:27 <sjmc7> what i think maybe is more an issue is what we analyze/don't analyze
15:50:36 <TravT> i think swift will be when this would be more likely to be needed
15:50:55 <TravT> yes i agree the way we analyze fields is definitely a pain point right now
15:51:26 <TravT> particularly the name field...
15:51:30 <sjmc7> ok. i've no real problem doing it - but i'd rather not do it in the sorting patch
15:51:36 <sjmc7> yes, the name one's the difficult one
15:51:44 <sjmc7> for nova i stored it not_analyzed to allow exact matching
15:51:46 <TravT> ok, i can remove -1 on sorting over that.
15:52:10 <TravT> i have found the searching on name field from my horizon panel to be frustrating.
15:52:17 <sjmc7> trouble then is that a search for "first" won't return "first instance"
15:52:26 <sjmc7> yeah.. for the prototype we were doing a wildcard search
15:52:54 <sjmc7> we will now be storing both
15:53:02 <sjmc7> but that won't help with name:something
15:53:55 <TravT> well, with the sorting patch, it should help
15:54:17 <TravT> the base field will be anyalyzed and raw field not analyzed, right?
15:54:26 <sjmc7> depends how you set it up
15:54:32 <TravT> https://www.elastic.co/guide/en/elasticsearch/guide/current/multi-fields.html
15:54:34 <sjmc7> but still the problem remains for a field search
15:55:31 <TravT> FYI nikhil: rosmaita: sigmavirus24_awa: david-lyle: I put up a horizon search panel patch
15:55:32 <TravT> https://review.openstack.org/#/c/227036/
15:55:44 <TravT> so you can actually search against searchlight from horizon
15:56:09 <TravT> it is driving some of the items we are discussing right now.
15:56:38 <nikhil> \o/
15:56:52 <sjmc7> yeay
15:57:17 <TravT> well sjmc7: i'll go back through sorting.
15:57:58 <TravT> nikhil: i know you are super busy, but if you have a chance to look through the function tests patch as well.
15:58:16 <TravT> https://review.openstack.org/217580
15:58:18 <nikhil> surely! kept the tab open!
15:58:34 <nikhil> thanks!!
15:58:35 <TravT> Well, great work everybody.
15:58:40 <TravT> We are really close
15:59:25 <TravT> I'll spend all day on reviews, but also there is still a horizon master bug affecting the search panel that I'm working on as well.
15:59:35 <TravT> #link https://review.openstack.org/#/q/status:open+project:openstack/searchlight,n,z
15:59:48 <TravT> We're out of time for today!
16:00:01 <TravT> see you in the room
16:00:10 <TravT> #endmeeting