15:00:31 <sjmc7> #startmeeting openstack search
15:00:32 <openstack> Meeting started Thu Oct 13 15:00:31 2016 UTC and is due to finish in 60 minutes.  The chair is sjmc7. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:36 <openstack> The meeting name has been set to 'openstack_search'
15:00:44 <RickA-HP> o/
15:00:47 <TravT> o/
15:00:55 <sjmc7> morning all
15:01:00 <yuriyz|2> o/
15:01:02 <yingjun> o/
15:01:03 <lei-zh> o/
15:01:05 <sjmc7> https://etherpad.openstack.org/p/search-team-meeting-agenda
15:01:12 <matt-borland> o/
15:01:15 <rosmaita> o/
15:01:39 <sjmc7> don’t have too many things on the agenda so if anyone has anything they want to bring up, there will be plenty of time
15:02:06 <sjmc7> give folks one more minute to get coffee / out of bed
15:02:24 * matt-borland is out of coffee
15:02:30 <yuriyz|2> hi, good news form ironic notification spec merged https://review.openstack.org/#/c/347242/
15:02:31 <sjmc7> but not bed?
15:02:38 <tyr> o/
15:02:41 <matt-borland> lol
15:02:47 <sjmc7> ah, splendid yuriyz|2
15:02:52 <Kevin_Zheng> o/
15:02:58 <sjmc7> i’ll add that to the end of the agenda, i meant to bring that up
15:02:58 <TravT> yuriyz|2: great!
15:03:22 <sjmc7> ok
15:03:24 <sjmc7> #topic Project Teams Gathering
15:03:46 <sjmc7> as most people know, the powers that be have changed the format of the summit after barcelona
15:04:16 <sjmc7> there’ll now be the summit at the regular time but the design summit has been replaced by a project teams gathering held before the start of hte next cycle
15:04:30 <sjmc7> the intent is it will be similar to the midcycles many projects hold, but all together
15:04:51 <sjmc7> the first one will be in atlanta end of february 2017
15:05:11 <sjmc7> i’ve been asked if we a) need a room or b) will be sending anyone to it
15:06:20 <matt-borland> I doubt that I will be going
15:06:24 <RickA-HP> Do we know of any big changes for the 'Q' release, or will it be trying to get more plugins?
15:06:30 <sjmc7> i’m tempted to say we don’t need space devoted to us, since it’s a small team and we can roam around looking for space
15:06:39 <sjmc7> don’t know yet
15:06:53 <sjmc7> if loads of people are planning to go i’ll ask for a room
15:07:14 <RickA-HP> So far the silence is implying no room :)
15:07:17 <sjmc7> so maybe just indicate on the agenda etherpad if you are definitely/maybe going by EOD tomorrow, that’d be helpful
15:07:19 <sjmc7> yeah :)
15:07:23 <TravT> sjmc7: we haven't had an actual mid-cycle yet for searchlight...
15:07:34 <TravT> and at the summit's we've only had 2 - 3 rooms.
15:07:38 <rosmaita> i think we should ask for a room
15:07:44 <rosmaita> just to keep a buzz going
15:07:58 <rosmaita> problem is, it's hard to get a commitment to travel this far out
15:08:02 <sjmc7> right
15:08:06 <rosmaita> at least at rackspace
15:08:10 <sjmc7> and i have a feeling i’m in costa rica that week
15:08:13 <TravT> its pretty much impossible
15:08:23 <rosmaita> PTG at steve's place!
15:08:35 <sjmc7> gatecrash my friend’s wedding in CR!
15:08:38 <rosmaita> well, at least atlanta != hoboken
15:08:47 <sjmc7> hoboken’s next summer
15:08:55 <lei-zh> I always thought people in us can drive to any place inside the country :）
15:08:58 <sjmc7> rosmaita: do you know if room space is limited?
15:09:06 <rosmaita> sjmc7: don't know
15:09:17 <sjmc7> lei-zh: i’m not originally from here so i’ve never got used to it :)
15:09:23 <rosmaita> i filled out the survey and said i couldn't guarantee actual attendance
15:09:30 <sjmc7> ok. i’ll do the same
15:09:34 <sjmc7> let them figure it out
15:09:40 <rosmaita> last week i asked glance community to start talking to their managers
15:10:05 <sjmc7> i imagine it’s a series of largeish meeting rooms versus the full-on convention center treatment
15:10:06 <rosmaita> because one problem is, you sometimes need to present to go somewhere
15:10:21 <rosmaita> so if so, i am going to formalize the glance sessions to ahve actual presentations
15:10:28 <sjmc7> yeah, that’s been our company’s stance this year
15:10:36 <TravT> lei-zh, you can, but most people don't.  It'd be about a two day drive to get to Atlanta from where I live
15:10:38 <rosmaita> which is against the spirit of loose discussions, but what can you do?
15:10:49 <rosmaita> i think we will have 4 total glance people in barcelona
15:10:55 <sjmc7> ouch :(
15:11:02 <sjmc7> the times are a changin'
15:11:04 <rosmaita> and we are supposed to be one of hte 6 core projects
15:11:08 <rosmaita> yeah
15:11:17 <sjmc7> ok. i’ll tentatively say yes but no guarantees
15:11:54 <sjmc7> this was always a danger with splitting the conference i guess
15:12:04 <rosmaita> i hope the foundation can put pressure on members to send contributors
15:12:17 <rosmaita> the info about the PTG says "any active contributor should go"
15:12:23 <rosmaita> but it doesn't say who will pay for it
15:12:27 <sjmc7> right :)
15:12:35 <rosmaita> so, it's not supposed to be cores only
15:12:54 <rosmaita> but the way things are going, it may just be the PTL in a room by him/herself
15:13:06 <sjmc7> or not even in my case if i’m out of the country :)
15:13:15 <rosmaita> sorry for being so pessimistic, i will shut up now
15:13:23 <sjmc7> i’ll bring you a very small violin
15:13:32 <sjmc7> ok, moving on
15:13:37 <sjmc7> #topic Reviews
15:13:47 <yuriyz|2> I have a question, we (ironic team) limited levels set to INFO an ERROR. Add new one static handler or use config option? the patch is https://review.openstack.org/#/c/364461/
15:14:11 <sjmc7> yuriyz|2: i’d go with a static handler
15:14:37 <RickA-HP> If it's just one additional handler, I agree with adding a static handler.
15:14:47 <yuriyz|2> ok it will be a simple solution
15:15:23 <sjmc7> yeah, that should simplify things
15:15:35 <sjmc7> i will try to get ironic working on one of our systems in the next few days
15:15:38 <sjmc7> so i can test it out properly
15:15:46 <sjmc7> apologies for not getting to it earlier
15:16:25 <sjmc7> thanks for doing it; it’s great you were able to submit the review without needing much help from us
15:17:36 <sjmc7> the other one was the pipeling architecture (https://review.openstack.org/#/c/359972)
15:18:24 <lei-zh> yeah, thanks to rick and steve, I've read your comments
15:18:44 <sjmc7> i finally got round to taking another look at this yesterday and left a comment on the review, but i think it would be worth considering simplifying such that notifications are always piushed to elasticsearch (or potentially in future some other database) but also additional publishers
15:19:07 <sjmc7> reason being that the notification handlers don’t treat ES as a write-only store, they read from it
15:19:14 <sjmc7> so without it a number of things won’t work
15:20:21 <RickA-HP> Yes, I think it makes sense to serialize the data and send it to ES. Then take the raw payload and send it to the pipeline of publishers.
15:20:48 <sjmc7> the alternative is to refactor so that ES is only used as a write-only store
15:21:06 <sjmc7> the advantage doing it that way is the code will be vastly simplified
15:21:14 <sjmc7> (the first way)
15:21:28 <lei-zh> so the structure looks like previous zaqar publisher patch, keep es part unchanged and put pipeline after documents are indexed, is that right?
15:22:02 <sjmc7> yeah - return the document from the handler. there might be some places that requires a bit of extra work but not many
15:22:30 <sjmc7> and again, i’m really sorry it took so long to get around to looking at it again
15:22:47 <sjmc7> had trouble finding a few hours where i could concentrate without interruptions
15:22:55 <lei-zh> yeah, to turn es document into a generic one maybe
15:23:20 <TravT> one reason i like the "goes to ES first" is that this should make the error handling easier as well.
15:23:23 <sjmc7> yeah, the document before we add the admin/user role field
15:23:28 <lei-zh> it's ok, cause we cannot figure out the whole thing until some code is show
15:23:32 <sjmc7> yeah - it means that ‘success’ is easier to define
15:23:59 <TravT> basically, this IS searchlight. it MUST be searchable.
15:24:14 <RickA-HP> But do other publishers care about the ES document? Or do they only care about the raw payload. Is the mappings and normalized fields useful for publishers?
15:24:26 <sjmc7> that i don’t know, and could be configurable
15:24:36 <sjmc7> the handler could return the raw and processed payload
15:24:44 <sjmc7> and be up to a publisher
15:24:51 <sjmc7> you might only care about the deltas
15:25:36 <lei-zh> maybe can return raw payload and es document together, and let publishers decide what they wanna do?
15:25:44 <sjmc7> yeah, that might make sense
15:26:03 <TravT> i still also think that if searchlight is effectively a caching layer that anybody chaining onto it's pipeline could guarantee that the cache is current with anything it forwards along...
15:26:34 <sjmc7> TravT: you mean beyond just the one event being processed?
15:26:51 <sjmc7> we can’t absolutely guarantee anything
15:26:56 <TravT> well if it is just fanning out notifications
15:27:05 <TravT> and one of them happens to be a UI
15:27:17 <TravT> and that UI uses searchlight
15:27:18 <sjmc7> yeah, eventually you’d get everything that’s going on
15:27:22 <sjmc7> ah, i see
15:27:26 <sjmc7> yes, there’d be internal consistency
15:27:53 <TravT> but, there's never any guarantees of perfection.
15:28:24 <TravT> distributed systems are pretty much always eventual consistency
15:28:38 <TravT> anyway, please feel free to ignore me.
15:28:55 <sjmc7> :)
15:28:58 <TravT> i think the base idea has merit for the reasons sjmc7 and RickA-HP said
15:29:07 <sjmc7> does that make sense lei-zh ?
15:29:13 <sjmc7> i’m not trying to be a dictator :)
15:29:39 <RickA-HP> I think I'll write a MongoDB publisher and have a competitor to Searchlight :)
15:29:40 <sjmc7> and i think the original patch was closer to this before i suggested doing it the other way
15:29:51 <lei-zh> yes, it will definitely simplify things a lot
15:30:15 <sjmc7> ok. i do have more time to be looking at reviews now so i’ll pay more attention to it. we can definitely get it in this cycle
15:31:07 <lei-zh> so ack could be done after successfully es indexing, like we do now
15:31:32 <RickA-HP> Yes.
15:31:35 <sjmc7> yep
15:32:55 <TravT> +1 to that
15:33:24 <TravT> i think that helps keeps searchlight's primary mission simpler as well.
15:33:25 <TravT> also...
15:33:28 <TravT> now that we are 1.0
15:33:43 <TravT> this seems like it would align better
15:34:47 <lei-zh> yeah, if we do it progessively, no need for me to intro writing new plugins : )
15:34:56 <lei-zh> aggressively
15:35:12 <sjmc7> both!
15:35:16 <sjmc7> ok, let’s do that
15:35:41 <sjmc7> ok, last thing i had on the agenda
15:35:43 <sjmc7> #topic Elasticsearch 5.0 awareness
15:36:24 <sjmc7> ES 5.0 will be released before the end of the year. i’m going to test our compatibility and file bugs for stuff that has been removed, but it’d be nice to be both 2.0 and 5.0 compatible before it comes out
15:36:36 <sjmc7> it shouldn’t be as big a change as 1.x to 2.x
15:37:27 <sjmc7> but a lot of things were deprecated in 2.x and will be removed in 5
15:38:19 <TravT> thanks for looking at that steve
15:38:51 <sjmc7> one really nice thing in 5.0 is proper ipv6 support
15:39:48 <sjmc7> can do range queries for toasters and such
15:39:53 <yingjun> sounds like it's a big jump from 2.x to 5.x
15:40:05 <sjmc7> 2.0 indexes are backwards compatible, allegedly
15:40:13 <sjmc7> there is a big list of breaking changes though
15:40:29 <sjmc7> they have a relaly good migration tool which i used before
15:40:42 <sjmc7> scans mappings and lets you know what problems you’ll have
15:41:12 <sjmc7> the issues we had with 2 were really easy to fix
15:41:34 <sjmc7> that was pretty much it for the agenda - anyone have any other stuff they wanted to discuss?
15:42:30 <RickA-HP> Nope.
15:43:01 <lei-zh> nope
15:43:09 <sjmc7> going… going
15:43:31 <TravT> nope
15:43:37 <sjmc7> ok. thanks all!
15:43:43 <sjmc7> #endmeeting