15:01:56 <krotscheck> #startmeeting StoryBoard
15:01:57 <openstack> Meeting started Mon Jun 23 15:01:56 2014 UTC and is due to finish in 60 minutes.  The chair is krotscheck. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:01 <openstack> The meeting name has been set to 'storyboard'
15:02:03 <krotscheck> Allo allo!
15:02:03 <ttx> o/
15:02:10 <jeblair> o/
15:02:12 <NikitaKonovalov> hi
15:02:15 <mordred> o/
15:02:30 <krotscheck> Agenda is here:
15:02:30 <krotscheck> https://wiki.openstack.org/wiki/StoryBoard#Agenda
15:02:47 <krotscheck> Anyone have last minute additions they’d like to put on there?
15:03:10 * fungi lurks
15:03:17 <krotscheck> #topic Reviews: Validate project & group names
15:03:19 <NikitaKonovalov> I'd like to extend Search UI to Search UI and API
15:03:23 <Ish__> Hi..
15:03:24 <krotscheck> NikitaKonovalov: Got it
15:03:42 <Ish__> I am Aishwarya.
15:03:52 <krotscheck> Hi there!
15:03:53 <SergeyLukjanov> o/
15:03:59 <Ish__> , a CS grad student from Stony Brook University, the new intern to your team.
15:04:19 <krotscheck> To everyone, Ish__ is joining StoryBoard for the summer :)
15:04:25 <jeblair> Ish__: welcome!
15:04:32 <mordred> woot. welcome Ish__
15:04:37 <ttx> Ish__: welcome!
15:04:40 <NikitaKonovalov> Ish__: o/
15:04:48 <ttx> Where is Stony Brook?
15:04:51 <Ish__> Thank you, all..!
15:05:00 * ttx should google that
15:05:01 <Ish__> in NY
15:05:03 <krotscheck> NikitaKonovalov: Anything on Project & Group names?
15:05:08 <ttx> Ish__: cool.
15:05:15 <NikitaKonovalov> I've added a migration
15:05:28 <NikitaKonovalov> It will rename a Project if it does not pass validation
15:05:37 <krotscheck> NikitaKonovalov: Awesome, thanks for that.
15:05:43 <krotscheck> Anything else?
15:05:45 <NikitaKonovalov> a previous name will be appended to the description
15:05:54 <NikitaKonovalov> so supersuers can see it and restore
15:05:55 <krotscheck> Oh, smart move.
15:05:58 <jeblair> NikitaKonovalov: i like that solution
15:06:07 <krotscheck> ++
15:06:14 <jeblair> NikitaKonovalov: it has the right balance of "keep things working" and "let the admin know something is wrong so they can fix it"
15:06:35 * ttx likes too.
15:07:05 <krotscheck> Awesome. Review is here: https://review.openstack.org/#/c/98728/
15:07:27 <krotscheck> #topic Search UI and API
15:08:01 <krotscheck> Ok, so I put up a mostly functional multidimensional search UI here: https://review.openstack.org/#/c/99975/
15:08:21 <krotscheck> It still needs some love with no-result edge cases, however the basic feel is there.
15:08:39 <krotscheck> Note that right now it only pretends to be a search, it’s really a browse/filter UI.
15:08:54 <ttx> ok
15:08:58 <NikitaKonovalov> Seen that, looks great
15:09:25 <krotscheck> Also, it’s not the fastest thing in the world, as it waits for multiple search requests to come back before rendering.
15:09:52 <krotscheck> I would like some feedback on the UX on that before going much farther, simply as a sanity check.
15:10:04 <mordred> krotscheck: is taht speed thing a thing that gets helped with search index? or is there a different thing at play?
15:10:28 <mordred> (I mean, I know you said it's actually just doing browse/filter right now - so that might be part of it?)
15:10:38 <krotscheck> mordred: Right now it seems to be mainly network latency that’s the issue.
15:11:17 <krotscheck> mordred: See, it’s actually trying to resolve search dimensions via the API, meaning that every time you type a key, it goes out and queries the User/Project/Story API’s all at once.
15:11:19 <jeblair> krotscheck: i think i left comments to this effect in some review (i think it may be the api side), but yeah, having to hit multiple endpoints seems awkward ...
15:11:33 <krotscheck> It is.
15:11:54 <jeblair> krotscheck: what kind of flexibility do we have to say violate the rest standards and make a /search?q=... endpoint?
15:11:56 <krotscheck> Having said that though, I was able to make the UI work in the way that I wanted it to with the current API features, so that’s good.
15:12:10 <krotscheck> jeblair: We have all the flexibility we want? In fact we probably should.
15:12:27 <krotscheck> jeblair: That begs the question on browse vs. search though
15:12:39 <krotscheck> NikitaKonovalov should probably weigh in on that.
15:13:21 <NikitaKonovalov> well, we may have browse and search on the same endpoint
15:13:26 <krotscheck> Oh?
15:13:36 <NikitaKonovalov> depending on a q parameter
15:14:08 <NikitaKonovalov> but I'd preferr to have /search?q=
15:14:18 <krotscheck> How would that work if I, say, asked for /stories?q=foo&project_id=22
15:14:33 <krotscheck> (since the latter implies a filter and the former implies a query
15:15:32 <mordred> it sounds like the API kinda wants to ahve both, yeah? like a /search?q= _and_ a /stories?q=foo&project_id=22 for different purposes?
15:15:40 <mordred> or maybe I'm crazypantshead?
15:15:49 <NikitaKonovalov> but we want to have q=<something for a full text search>&k=v to filter
15:16:09 <krotscheck> mordred: Well, how do full text searches interact with indexes?
15:16:24 <krotscheck> NikitaKonovalov: We don’t necessarily want that.
15:16:35 <NikitaKonovalov> http://sqlalchemy-searchable.readthedocs.org/en/latest/
15:16:55 <NikitaKonovalov> ^^ can wratp filter queries with add full-text stuff
15:16:58 <mordred> krotscheck: oh, we'd need the indexingy things to implement /search?q= in a truly performant way - but we could have a v1 of what the API wants to look like without the goodness probably
15:16:58 <krotscheck> …postgres only? Srsly?
15:17:15 <mordred> krotscheck: the python community has an unnatural obession with postgres
15:17:32 <krotscheck> mordred: The python community has an unnatural obsession with python :-P
15:17:38 <mordred> krotscheck: :)
15:17:41 <mordred> rewrite it all in go!
15:17:47 <ttx> in javascript!:
15:17:56 <NikitaKonovalov> Postgres is for an example there
15:17:58 <krotscheck> I recommend we rewrite it all in chicken
15:17:59 <NikitaKonovalov> i hope
15:18:38 <mordred> NikitaKonovalov: it seems worthwhile to explore whether or not that will function with mysql
15:18:47 <NikitaKonovalov> mordred: agree
15:18:59 <jeblair> oh, i also left some nodes in a review about the full-text indexing... i think we may want to consider using the messaging framework to keep solr up to date near-real-time
15:19:08 <mordred> oh. I guess "Currently it only supports PostgreSQL." is what krotscheck was reading
15:19:13 * krotscheck is honestly not that familiar with how search indexes work, as long as the API behaves sanely he’ll happily stay on the client side and let everyone else argue it out.
15:19:44 <mordred> krotscheck: yah - I mean, honestly, I think we should get a search api written even if the impl is terrible so that it doesn't have to be faked ui side
15:19:44 <NikitaKonovalov> I've updated the Search spec with Spihnx section
15:19:59 <mordred> and then we can try different api server impl choices.
15:20:00 <NikitaKonovalov> it can keep it's indexes pretty up to date
15:20:17 * mordred also agrees with jeblair on messaging framework for keeping search indexes up to date
15:20:26 <krotscheck> jeblair: You mean the messaging async processing spec I put up there?
15:20:33 <jeblair> krotscheck: yep
15:20:34 <NikitaKonovalov> this is what their indexes up to date http://sphinxsearch.com/docs/current.html#delta-updates
15:20:59 <NikitaKonovalov> looks like we will need messaging for both indexing and notifications
15:21:13 <jeblair> krotscheck: since the search spec seemed to be reaching for a way to get async notifications that it needs to reindex stuff
15:21:19 <krotscheck> Before we go off the rails here, can we focus on what decisions actually need to be made right now?
15:21:51 <krotscheck> I think I agree with mordred that “implementing the search API no matter what the impl behind the scenes is” is good.
15:21:57 <jeblair> ++
15:22:20 <krotscheck> However, I also feel that the UI I put together buys us more time on the need for this implementation, because it actually provides a pretty sophisticated way of discovering things.
15:22:25 <mordred> ++
15:22:32 <NikitaKonovalov> then the LIKE statement is the solution
15:22:39 <krotscheck> Right
15:23:21 <krotscheck> It doesn’t remove the need for a fulltext search, it only patches the pain
15:23:38 <krotscheck> So it sounds like we want more comments and discussion on Nikita’s review
15:23:57 <krotscheck> The real decision to make is: Separate search API endpoint, or munge it in with browse?
15:24:35 <NikitaKonovalov> pecan allows to do that
15:24:53 <NikitaKonovalov> I've done a similar thing for GET projects
15:24:57 <mordred> krotscheck: what's the benefit of munging it with browse?
15:25:34 <krotscheck> mordred: Can’t think of one. I feel that it’s mostly a stylistic choice.
15:25:44 <krotscheck> Well
15:25:47 <krotscheck> Philosophical
15:25:56 <jeblair> krotscheck: fundamentally, i think we want one endpoint to handle the one operation -- what i'm not sure about is whether it needs to be completely separate, or if it's enough to hang it off of stories because that's ultimately what we're going to display to the user...
15:26:15 <krotscheck> jeblair: point
15:26:45 <krotscheck> jeblair is suggesting search as a resource-wide action, not just specific to individual resources.
15:26:49 <krotscheck> Sorry
15:26:52 <krotscheck> API-wide action
15:27:47 <krotscheck> mordred: Philosophical difference is as follows- search is inclusive: “Give me things that are close to this value”. Browse is exclusive: “Only give me things where this == value“
15:28:23 <krotscheck> NikitaKonovalov: Thoughts about /search?q= vs /stories/search?q=
15:28:40 <NikitaKonovalov> my vote is for the second
15:28:58 <NikitaKonovalov> that one clearly says what are we looking for
15:29:22 <krotscheck> Doesn’t searching imply that we don’t actually know what we’re looking for?
15:30:11 <NikitaKonovalov> so then we need an ability to limit the search if the user actually knows what he is looking for
15:30:36 <NikitaKonovalov> search?q=<whatever>&target=stories
15:30:37 <krotscheck> NikitaKonovalov: Right - at that point it becomes a munged search/browse
15:31:08 <jeblair> krotscheck: maybe we need to lay out some use cases and say "if you want to find a specific comment, do this" "if you want to find a story that has a comment with text, do this"....
15:31:22 <krotscheck> jeblair: That’s a good idea.
15:31:25 <mordred> ++
15:31:37 <jeblair> krotscheck: i think doing straw-man versions of cases (especially edge cases) like that may be instructive
15:32:06 <krotscheck> jeblair: You know, i can go back into the Atlanta UX sessions and extract some of the search cases that were highlighted there.
15:32:19 <jeblair> krotscheck: sounds like a good source
15:32:22 <krotscheck> Indeed
15:33:05 <krotscheck> Do we need to pause the spec while we figure that out?
15:33:16 <krotscheck> Or can the spec be nonspecific as to where the endpoint lives?
15:33:44 <jeblair> i think this would be good stuff to get into the spec
15:33:50 <krotscheck> Alright, so high priority.
15:34:06 <krotscheck> NikitaKonovalov: Do you mind if I add the use cases to your spec directly?
15:34:18 <NikitaKonovalov> krotscheck: I'd appreciate that
15:34:25 <krotscheck> Works for me.
15:34:41 <krotscheck> #action krotscheck Extract search use cases from UX sessions and add to search spec.
15:35:09 <krotscheck> SPeaking of specs....
15:35:13 <krotscheck> #topic Specifications
15:35:29 <krotscheck> We just talked about Search: https://review.openstack.org/#/c/97254/
15:35:48 <krotscheck> NikitaKonovalov added a section on solr.
15:35:55 <krotscheck> We’ve got some use cases to add
15:36:08 <krotscheck> And an outstanding decision on where the endpoint should live.
15:36:19 <krotscheck> endpoint(s)
15:36:24 <krotscheck> Did I miss anything?
15:36:30 <NikitaKonovalov> I'll remove that sqlalchemy-searchable section
15:36:38 <NikitaKonovalov> we do not have postgress
15:36:47 <krotscheck> Right
15:37:04 <krotscheck> Anything else? Else I’ll move on.
15:37:14 <jeblair> NikitaKonovalov: btw, in the commit msg, you can have "Story:" and "Change-Id" in the same paragraph (like in https://review.openstack.org/#/c/95307/ ); they don't need a blank line between them
15:37:41 <NikitaKonovalov> jeblair: not a problem
15:37:48 <krotscheck> jeblair: I dunno, I feel that two lines is more legible for humanz
15:38:04 <krotscheck> My meat brain is bad at regex :)
15:38:08 <jeblair> krotscheck: definitely 2 lines, just saying not 3 lines
15:38:12 <krotscheck> Oh, got ti
15:38:21 * mordred votes for 5 lines
15:38:35 <krotscheck> There’s also the tagging spec: https://review.openstack.org/#/c/97211/
15:38:50 <krotscheck> Which I know I promised I was going to write a sibling for, but I haven’t gotten around to it because of the search UI
15:38:57 <jeblair> krotscheck: we should also make sure to get sdague and clarkb to look over the search spec and provide feedback on the fulltext engines
15:39:30 <krotscheck> jeblair: Last time I asked clarkb on that his response was “Oh god not elasticsearch please god not that"
15:39:37 * krotscheck may be paraphrasing
15:39:39 <jeblair> yeah, that's good feedback to have :)
15:40:00 <mordred> that sounds like something he'd say
15:40:24 <krotscheck> #action NikitaKonovalov Get sdague and clarkb’s feedback on Search Spec.
15:40:39 <krotscheck> Looks like we have a new update on tagging
15:40:48 <krotscheck> ttx: Care to summarize?
15:42:07 <ttx> Mostly clarifications
15:42:17 <krotscheck> Alright, we can read those on our own time.
15:42:31 <krotscheck> The messaging spec is here: https://review.openstack.org/#/c/95307/
15:42:32 <ttx> on what's accepted in a tag name, and how protected tags would work
15:42:54 <krotscheck> jeblair: I left a comment for you regarding the actual implementation that would make a streaming API easy
15:43:06 <ttx> krotscheck: it still targets a story-specific case, so if you want to introduce a more general reousrce tag, you can file a separate one
15:43:12 <jeblair> krotscheck: cool, thx
15:43:18 <krotscheck> ttx I will.
15:43:40 <krotscheck> jeblair: That comment will also address the question of updating any indexes.
15:44:24 <krotscheck> (Well, it addresses the question of “Hey we have a stream of changes, what useful things can we do with that?”)
15:44:35 <krotscheck> #topic Ongoing work
15:44:47 <krotscheck> In alphabetical order this week!
15:45:02 <krotscheck> I was mostly working on the search UI, as well as tag completion.
15:45:24 <krotscheck> This week I’m going to write all the specs and try to advance the storyboard puppet module changes
15:45:40 <krotscheck> So that we can get rabbit onto the SB server and we can start broadcasting changes.
15:46:03 <krotscheck> NikitaKonovalov?
15:46:42 <NikitaKonovalov> I've been working on 4xx codes support for UI and API
15:47:06 <NikitaKonovalov> now 401 and 403 should work properly
15:47:17 <krotscheck> Awesome
15:47:19 <mordred> woot
15:47:24 * krotscheck is surprised that’s not baked into pecan
15:47:37 <mordred> wasn't there discussion at some point of pecan growing that?
15:47:42 <mordred> dhellmann: ^^ ?
15:48:06 <NikitaKonovalov> krotscheck: I'd say I had to fight agains pecan mechanisms
15:48:13 <krotscheck> dhellmann: Context: https://review.openstack.org/#/c/101146/
15:48:29 <NikitaKonovalov> on rendering the correct code
15:48:34 <krotscheck> NikitaKonovalov: Yeah, I looked at that review and was somewhat surprised that there wasn’t a more elegant way
15:48:48 <NikitaKonovalov> but still that works
15:49:25 <NikitaKonovalov> and there is a new modal for UI to warn a user if cant perform some action
15:49:35 <NikitaKonovalov> https://review.openstack.org/#/c/101867/
15:49:41 <krotscheck> Oh neat!
15:50:21 <NikitaKonovalov> I've done some one-line fixed for timeline and docs
15:50:23 <krotscheck> NICE. Love the implementation
15:51:12 <NikitaKonovalov> and there is a WIP change for search endpoints, but it'll wait until we decide how the final interface should look like
15:51:21 <krotscheck> ttx? Thanks for the updated spec and for your reviews, anything in addition to that?
15:51:28 <ttx> no
15:51:53 <krotscheck> Ok, moving to open discussion
15:51:55 <ttx> slowly working to the bottom of my TODO pile though so I still hope to be able to propose some code to keep current
15:52:01 <krotscheck> I stand corrected.
15:52:30 <krotscheck> ttx: I could use some help with oslo.messaging :)
15:52:50 <krotscheck> Anything else?
15:53:00 <ttx> nope
15:53:03 <krotscheck> #topic Open Discussion
15:53:39 <NikitaKonovalov> There is going to be an Infra-qa meetup soon
15:53:47 <NikitaKonovalov> any StoryBoard stuff there?
15:54:14 <krotscheck> You mean the midcycle in Dortmund? I won’t be able to make it.
15:54:26 <jeblair> darmstadt
15:54:33 <ttx> neither will I
15:54:38 <krotscheck> Well, i had one letter right!
15:54:53 <jeblair> so we probably won't do much storyboard stuff then, except perhaps to talk about bulk import
15:55:06 <jeblair> since that's the main blocker of increased infra adoption right now
15:55:08 <mordred> bulk import ftw
15:55:37 <krotscheck> Indeed. That discussion was last left with “maybe doing a direct-to-db dump would be best to ensure associations are maintained"
15:57:00 <jeblair> good to know
15:57:12 <krotscheck> That’s from the summit
15:57:30 <jeblair> we may end up pinging people in irc if we get to talking/hacking about it
15:57:55 <krotscheck> jeblair: Works for me.
15:58:02 <krotscheck> Anyone have anything else they want to bring up?
15:58:12 <NikitaKonovalov> nothing from my side
15:59:59 <krotscheck> Ok, thanks everyone!
16:00:01 <krotscheck> #endmeeting