15:00:19 <TravT> #startmeeting openstack search
15:00:20 <openstack> Meeting started Thu Jul 30 15:00:19 2015 UTC and is due to finish in 60 minutes.  The chair is TravT. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:23 <openstack> The meeting name has been set to 'openstack_search'
15:00:27 <david-lyle> o/
15:00:30 <lakshmiS> o/
15:00:41 <TravT> good morning, afternoon, evening...
15:00:44 <sigmavirus24> o/
15:00:52 <sigmavirus24> good tomorrow
15:00:53 <TravT> will give a couple mins for people to join.
15:00:56 <sigmavirus24> I mean... yesterday
15:00:57 <sigmavirus24> er
15:01:00 <sigmavirus24> whatever
15:01:07 <TravT> im actually looking forward to good Saturday
15:01:15 <sigmavirus24> Is Saturday Yesterday?
15:01:18 <sigmavirus24> When can then be now?
15:01:24 <TravT> not soon enough
15:01:48 <TravT> hey sjmc7_
15:01:53 <sjmc7_> hi
15:01:57 <sjmc7_> sorry, having network issues
15:02:03 <TravT> isp?
15:02:11 <sjmc7_> no, software. morning!
15:02:16 <sigmavirus24> So, neutron?
15:02:20 <sjmc7_> :)
15:02:31 <sigmavirus24> neutron problems are unheard of =P
15:03:10 <TravT> Ok, agenda is here: https://etherpad.openstack.org/p/search-team-meeting-agenda
15:03:20 <TravT> please add any items you want to discuss to it
15:03:49 <TravT> to start with, sigmavirus24, how did the glance meetup go?  anything we should be aware of?
15:04:07 <sigmavirus24> I was only able to make it on Tuesday for part of the day
15:04:18 * sigmavirus24 is not in Blacksburg
15:04:25 <TravT> i wanted to join and listen in on a few things, but this week has been hectic
15:04:36 <sigmavirus24> my week has been hectic as well
15:04:45 <sigmavirus24> many a head of hair on fire
15:05:04 <TravT> well, i appreciate the reviews on that critical bug this week
15:05:11 <TravT> good to have glance images fully working
15:05:35 <TravT> #topic General Status Updates
15:05:59 <TravT> just wanted to mention that the API WG has put out more guidelines for discussion
15:05:59 <TravT> Cross Project API Guidelines: http://lists.openstack.org/pipermail/openstack-dev/2015-July/069765.html
15:06:08 <TravT> i am guilty of not reviewing them yet
15:06:26 <TravT> but if you can take a look and see if there is an impact we need to consider
15:06:59 <TravT> Also, I created a tinyurl for our review board. just wanted to share it
15:07:02 <TravT> #link http://tinyurl.com/searchlight-reviews
15:07:42 <TravT> sjmc7, want to say something about our session submssions for the summit?
15:07:54 <sjmc7_> no, just that they're there :)
15:07:57 * david-lyle wishes the horizon dashboard looked like that
15:08:07 <TravT> lol david-lyle
15:08:11 <TravT> i do too
15:08:35 <TravT> so, if you can, consider voting on them as well.
15:08:42 <sigmavirus24> or glance dashboard
15:08:56 <sigmavirus24> then again, I use gertty so it wouldn't help me anyway =P
15:09:01 <TravT> if anybody would like to co-present, we can do that as well.
15:09:04 <TravT> https://www.openstack.org/summit/tokyo-2015/vote-for-speakers/presentation/4507
15:09:13 <TravT> https://www.openstack.org/summit/tokyo-2015/vote-for-speakers/presentation/5289
15:09:48 <TravT> #topic Notification Payload vs API for Glance (TravT, sjmc7)
15:09:57 <sigmavirus24> Didn't voting close yesterday or this morning?
15:10:08 <TravT> hmm, guess I should have voted...
15:10:27 <david-lyle> then you might have to give the presentation
15:10:33 <david-lyle> there are risks
15:10:39 <TravT> i have a feeling lakshmiS didn't mean to drop out
15:10:39 <sigmavirus24> lol
15:10:49 <sigmavirus24> "Ping timeout" says no he didn't
15:11:06 <TravT> kind of need him in this next topic
15:11:28 * TravT wonders where nikhil_k is, but sigmavirus24 will have to represent glance
15:11:45 <sigmavirus24> yeah, I'm apparently the new go to for questions about glance
15:11:54 <sigmavirus24> I'll have to do
15:12:16 <TravT> so, the notification payload looks like v1 API.
15:12:34 <TravT> and we have to do mapping.
15:12:39 <TravT> and it doesn't include tags
15:13:01 <TravT> it is great to not have to make an API callback for efficiency reasons
15:13:07 <sigmavirus24> sure
15:13:20 <TravT> let me copy past repeat for lakshmiS
15:13:29 <TravT> 09:12 TravT: so, the notification payload looks like v1 API.
15:13:29 <TravT> 09:12 TravT: and we have to do mapping.
15:13:29 <TravT> 09:12 TravT: and it doesn't include tags
15:13:31 <TravT> 09:13 TravT: it is great to not have to make an API callback for efficiency reasons
15:13:40 <TravT> But, I'm wondering if there is more data we're missing
15:13:41 <lakshmiS> had to reconnect<network issues...>
15:13:49 <TravT> and I'm particularly concerned about the lack of tags
15:14:05 <TravT> searching based on tags seems like something we'd particularly want to support
15:14:19 <sjmc7_> longer term it'd be best if everything was in the notifications, for efficiency. right now, i don't think we have much choice but to go to the API unfortunately
15:14:35 <sigmavirus24> That's something I don't immediately know the answer to TravT
15:14:41 <TravT> but here's the problem i see also with the notifications.
15:14:50 <TravT> they are an API contract of their own.
15:15:03 <TravT> the publisher doesn't know the consumer
15:15:20 <TravT> so, changing a notification is like breaking a contract.
15:15:38 <TravT> with API, you can specify API version format at least.
15:16:03 <TravT> so, a couple of options (at least for Glance), are
15:16:15 <TravT> 1) switch to go straight to API and not use payload
15:16:30 <TravT> 2) Get tags into payload
15:16:43 <sigmavirus24> TravT: so additions to the payload would be a microversion bump
15:16:48 <sigmavirus24> Right?
15:16:57 <lakshmiS> no microversions in glance i think
15:16:59 <sigmavirus24> As long as we don't change existing members it should be "backwards compatible"
15:17:08 <sigmavirus24> lakshmiS: more of an analogy here
15:17:34 <TravT> 3) perhaps support a new notification for the searchlight_indexer topic that always has latest API results (i'm not sure this is very realistic)
15:17:35 <sigmavirus24> TravT: the question I have is "Is ceilometer okay with us sending extra information in our notifications or will it have a sad?"
15:17:46 <sigmavirus24> TravT: that was an idea I had
15:18:01 <sigmavirus24> Our notifications handlers could be intelligent about what it sends to what listener
15:18:15 <sigmavirus24> This is kind of the benefit of the onion approach, right?
15:18:31 <TravT> i don't know about ceilometer's happiness with changes to existing
15:18:54 <sigmavirus24> Right so there are things to be researched before Glance can update its notifications
15:18:55 <david-lyle> ping gordc and ask?
15:19:10 <sigmavirus24> Also, one thing I'm pretty sure Glance won't consent to do is extra DB look ups just to add data to a notification
15:19:28 <david-lyle> in #openstack-ceilometer
15:19:43 <sigmavirus24> i.e., if what we currently have doesn't include tags, I doubt we'd do an extra database call to get that information for notification purposes
15:20:01 <TravT> sigmavirus24: oddly, that means a whole API callback will happen (if searchlight is deployed)
15:20:08 <TravT> maybe it could be configurable in glance?
15:20:11 <sigmavirus24> Right
15:20:23 <TravT> david-lyle: yeah, we should follow up there.
15:20:24 <sjmc7_> i think at least for the next while (this cycle, maybe the next) we're stuck with what we have in notifications
15:20:31 <sjmc7_> ultimately it would be best to have everything in there
15:20:38 <sigmavirus24> The API callback will probably trigger a worse DB call but we'd have to make that obvious and the best way to argue for this would be a spec I think
15:20:39 <sjmc7_> because throughput's really limited if we have to go to the API
15:20:53 <sigmavirus24> sjmc7_: yeah, I get that
15:21:06 <lakshmiS> sigmavirus24: there's already db calls for notification but yeah it would be one more extra call
15:21:12 <sigmavirus24> Glance's DB is currently not well optimized and has more joins than nikhil_k likes iirc
15:21:34 * TravT are any of the openstack DB's well optimized?
15:21:50 * sigmavirus24 shrugs
15:22:13 <TravT> well, i'm not sure we even have a BP or bug open on this topic
15:22:32 <TravT> but we should address the fact that we won't be indexing tags without some sort of change here
15:23:00 <sigmavirus24> Probably a bug in each of searchlight and glance (perhaps the same bug affecting both?) would be a helpful start
15:23:05 <lakshmiS> in interim, we can update the document for no tags
15:23:23 <TravT> sigmavirus24, that makes sense. Easy way to track across
15:23:48 <sigmavirus24> Yep. Also we should bring this up next week at the Glance Team Meeting
15:24:13 <TravT> Ok, I can open the initial bug
15:24:14 <sigmavirus24> mclaren, jokke_ and flaper87 will likely have some good insights into this too
15:24:57 <sigmavirus24> lakshmiS: yeah, I think people's reactions will be "it's easy to say, oh it's just one more" but we really don't want to slow glance down. We already have enough people complain about Glance's performance
15:25:17 <TravT> sigmavirus24, i really wonder if it could be a flag
15:25:22 <TravT> config item
15:25:58 <TravT> so that way you only take the extra hit if it is needed, knowing it saves the follow-up call
15:26:12 <sjmc7_> there's a similar problem in nova, fwiw
15:26:38 <TravT> yes, there will be similar across the board.
15:26:45 <TravT> but we'll deal with each individually
15:27:25 <sigmavirus24> right
15:27:27 <TravT> #action TravT open tag bug in Glance and Searchlight
15:27:33 <sigmavirus24> So the flag would be whether or not searchlight is enabled basically
15:27:43 <TravT> yeah, i think so.
15:28:12 <sigmavirus24> I think we all agree that a better db call on the notification side is better performance for the whole cloud than searchlight making yet another request against glance/every other service
15:28:35 <TravT> kinda hard to argue with that... :)
15:28:36 <lakshmiS> i agree
15:28:53 <sigmavirus24> note "better" not "additional" ;P
15:29:00 <lakshmiS> :)
15:29:04 <TravT> although, i think we also know some services can take a long time to get any kind of changes in
15:29:07 * TravT looks at nova
15:29:30 <sigmavirus24> TravT: better start planning now for the N cycle
15:29:31 <sigmavirus24> =P
15:29:33 <lakshmiS> direct db maynot be that bad in that case ;)
15:29:36 <TravT> lol
15:29:36 <sigmavirus24> L and M are all but decided :D
15:29:49 <TravT> let's see, another API call or 2 years....
15:30:09 <sigmavirus24> lakshmiS: yeah, searchlight should just directly access every service's DB directly =P
15:30:18 <sigmavirus24> ok, I'm sorry for pulling us off track
15:30:20 <sigmavirus24> let's continue
15:30:24 <TravT> no worries
15:30:27 <TravT> ok, moving on
15:30:44 <TravT> #topic     Ceilometer / Notifications: https://review.openstack.org/#/c/202392
15:30:59 <TravT> i think Lakshmi's fix looks good on our side
15:31:13 <lakshmiS> yeah start the workflow :)
15:31:28 <sjmc7_> yep
15:31:57 <sigmavirus24> so zuul is under heavy load
15:32:07 <sigmavirus24> they've asked projects to hold off on approving things for a while
15:32:14 <TravT> oh, interesting
15:32:22 <TravT> what's been going on with zuul the last few days?
15:32:46 <sigmavirus24> http://status.openstack.org/zuul/ is really really really really backed up
15:32:50 <sigmavirus24> TravT: disk space issues
15:33:01 <TravT> ok...
15:33:19 <TravT> i thought this was cloud... maybe zuul should burst out to AWS.
15:33:23 * TravT ducks and hides
15:34:18 <TravT> lakshmiS: have you had any further discussion on this with ceilometer. or is it final that we just use a different topic and no other changes needed?
15:35:06 <lakshmiS> yeah that was final from their PTL and other core members from their weekly IRC meeting
15:35:13 <lakshmiS> i have the transcript saved with the decision
15:35:31 <TravT> ok, i guess you could post that into the bug notes (if you didn't already)
15:35:51 <lakshmiS> its in the comments on patchset. will update bug notes too
15:36:02 <TravT> #topic other bugs
15:36:12 <TravT> We've got a few other open looking for owners
15:36:18 <TravT> https://bugs.launchpad.net/searchlight
15:36:41 <TravT> I saw a new one show up about the dev env docs. I'll look into that today
15:36:57 <TravT> but let's move on
15:37:17 <TravT> #topic     BP: https://blueprints.launchpad.net/searchlight/+spec/dont-expose-index-doctype
15:37:24 <sjmc7_> is this my badly named BP?
15:37:26 <TravT> sjmc7:
15:37:29 <sjmc7_> it is!
15:37:53 <sjmc7_> the reasoning is.. our API now exposes index_name and doc_type
15:38:08 <sjmc7_> which means that they effectively cannot be configurable
15:38:32 <sjmc7_> because then nobody will know what they are. so the proposal is to refer to a plugin by an unchangeable name (OS::Glance::Image)
15:38:37 <sjmc7_> and map that internally
15:38:57 <lakshmiS> layer of indirection. sounds good to me
15:38:57 <sjmc7_> index name and doc type won't necessarily be hidden, but not relied upon
15:39:11 <TravT> lakshmiS: and use the heat resource name
15:39:30 <lakshmiS> perfect
15:39:31 <sjmc7_> i started on it yesterday - it's relatively simple to do the mapping on the two api endpoints (search and index)
15:39:49 <sjmc7_> so unless there are objections i'll get a patch up over the next day or so
15:40:33 <TravT> ok cool.
15:40:41 <sigmavirus24> no objection here
15:41:17 <TravT> #topic designate plugin question
15:41:21 <TravT> whose is that?
15:41:43 <lakshmiS> me
15:41:44 <sjmc7_> looks unrelated to the designate plugin per se - failure was indexing images
15:41:57 <TravT> there was a failure yesterday which merged
15:41:59 <TravT> but different error
15:42:09 <TravT> when is your devstack from?
15:42:18 <lakshmiS> failed numerous times setting up designate and one of the issue was devstack for searchlight
15:42:31 <TravT> I have not seen that error: publicURL endpoint for image service not found
15:42:36 <TravT> my devstack is from Monday morning
15:42:41 <lakshmiS> was using the localrc from ekarlso
15:42:55 <lakshmiS> today morning(-8 hrs from now)
15:43:39 <TravT> well, i used my own local.conf and just added ekarlso's stuff to it
15:43:57 <lakshmiS> i saw sjmc7_ install designate.
15:44:23 <sjmc7_> i did what now? yeah, i ran it a few days ago
15:44:35 <TravT> i could send that to you, but i didn't actually get to testing designate
15:44:38 <lakshmiS> TravT: so it installed searchlight with devstack successfully?
15:44:46 <TravT> yes
15:44:56 <TravT> and it worked for contacting glance
15:45:25 <lakshmiS> TravT: ok maybe its some intermittent error then. will retry again
15:45:35 <TravT> i can kick off another devstack this morning
15:46:06 <sjmc7_> i do want to file a bug about running indexing within the stevedore setup (which you mentioned yesterday TravT)
15:46:17 <lakshmiS> Need to check if there is a test for devstack install
15:46:30 <sjmc7_> it's hard to track down errors and it also means i think indexing can't be limited to a single plugin which would be useful for testing
15:46:51 <TravT> sjmc7_ yeah
15:47:08 <lakshmiS> ekarlso: ?
15:47:17 <sjmc7_> he's on vacation still
15:47:36 <lakshmiS> will check with hime during his timezone
15:47:38 <sjmc7_> european vacation time :)
15:47:43 <lakshmiS> s/hime/him
15:47:57 <TravT> lakshmiS: we'd also like to enable us to be able to call indexing for a particular resource type
15:48:08 <TravT> right now it tries to do all of them
15:48:34 <TravT> this is a growing pain of moving to being bigger than just glance.
15:48:52 <TravT> or put another way, indexing more than just glance
15:49:24 <sjmc7_> i'll file a bug for it
15:49:27 <sjmc7_> or a BP or something
15:49:34 <TravT> ok.
15:49:42 <TravT> #topic L2 tagging.
15:49:46 <TravT> david-lyle ^
15:49:50 <sjmc7_> this crept up on us :)
15:49:59 <TravT> yeah, went too fast.
15:50:15 <david-lyle> that was just a time check for everyone :)
15:50:52 <lakshmiS> forgot about that
15:51:17 <TravT> so, next week, we should perhaps do a serious review of what we are targeting for l3.
15:51:18 <sigmavirus24> heh
15:51:23 <sigmavirus24> yep
15:51:30 <sigmavirus24> see what's achievable etc.
15:51:31 <david-lyle> There is a self-service page for tagging releases, but we should coordinate with the Release Management team for L-3
15:52:07 <TravT> nova, images, designate from functionality.
15:52:21 <TravT> some of these other micro improvements probably less important.
15:52:32 <TravT> getting API and indexing right is more important
15:52:49 <TravT> such as that resource mapping.
15:53:34 <sjmc7_> nudge nudge
15:53:38 <TravT> speaking of which sjmc7_: I spend time with the nova instance plugin yesterday.  become friends with it, but ultimately had quite a few comments
15:53:44 <sjmc7_> :)
15:53:52 <sjmc7_> ok. be brutal, i can take it
15:54:11 * TravT has zero grammar skills in irc
15:54:32 <TravT> lakshmiS: 09:53 TravT: speaking of which sjmc7_: I spend time with the nova instance plugin yesterday.  become friends with it, but ultimately had quite a few comments
15:54:45 <TravT> lakshmiS: if you could also look over it, would be great.
15:55:03 <lakshmiS> yeah its on my review list:) will do
15:55:03 <david-lyle> wow check queue 671 deep
15:55:08 <TravT> ouch
15:55:36 <david-lyle> 7.5 hours out
15:55:41 <TravT> david-lyle: sjmc7_ and I have been talking about starting that cross resource page on horizon.
15:55:48 <TravT> for searching via searchlight.
15:56:00 <sigmavirus24> david-lyle: and I guarantee people will be ignoring Infra's requests
15:56:20 <david-lyle> sigmavirus24: I actually hadn't seen the request
15:56:37 * david-lyle proactively hadn't done any reviews this morning yet
15:56:38 <sigmavirus24> david-lyle: they used openstackstatus to send messages to every channel that it is registered to notify
15:56:51 <david-lyle> right, might need a refresh for US morning
15:57:06 <TravT> sigmavirus24: i don't think searchlight gets those
15:57:17 <TravT> how do we sign up?
15:57:22 <TravT> something you know how to do?
15:57:25 <david-lyle> TravT: I can add it
15:57:25 <sigmavirus24> yep
15:57:29 <sigmavirus24> It's in project-config iirc
15:57:29 <david-lyle> or sigmavirus24
15:57:31 <sigmavirus24> or david-lyle can do it
15:57:32 <sigmavirus24> lol
15:57:35 <TravT> david-lyle: +1
15:57:37 <sigmavirus24> hot potatoe
15:57:40 <sigmavirus24> ;)
15:57:45 <TravT> #topic open discussion
15:57:46 <david-lyle> alright, I'll add
15:57:50 <TravT> might as well change the topic
15:58:12 * sigmavirus24 misspells potato to rile people up who notice that
15:58:21 <TravT> FYI, i'm out most of tomorrow. will check in briefly in the morning, but have to go out of town for friend's wedding
15:58:23 <david-lyle> you could be president
15:58:24 <lakshmiS> For glance midcyle meeting topic, do anyone know what time is Glance + Searchlight topic is today(time?)
15:58:25 <sjmc7_> consider me duly riled
15:58:47 <david-lyle> we have low standards
15:59:19 <lakshmiS> never mind: its at 2.45-3.30: Session 5 - Glance + SearchLight
15:59:32 <david-lyle> err VP
16:00:06 <TravT> can you send that to me lakshmiS?
16:00:13 <lakshmiS> https://etherpad.openstack.org/p/liberty-glance-mid-cycle-meetup
16:00:17 <TravT> guess our time is up.
16:00:26 <TravT> let's chat re: meetup in room
16:00:29 <TravT> #endmeeting