15:00:42 #startmeeting Storyboard 15:00:42 Meeting started Mon Apr 14 15:00:42 2014 UTC and is due to finish in 60 minutes. The chair is krotscheck. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:43 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:44 hi everyone 15:00:45 hi 15:00:46 The meeting name has been set to 'storyboard' 15:00:50 Hey hey 15:01:12 #topic Work on Soft/Hard delete. 15:01:14 * corvus is aka jeblair 15:01:22 Quick update: We’ve got a patch in for tasks 15:01:26 thanks to NikitaKonovalov 15:01:28 o/ 15:01:40 I’ve got a broken tasks for stories. 15:01:42 Sorry 15:01:45 Broken patch 15:01:57 Anyone been working on projects? 15:02:23 Guess not. 15:02:45 Well, mordred went through and deleted all the not-using-storyboard tasks on thursday, so that might not come up for a while yet. 15:03:00 #topic Last meeting agenda 15:03:07 Sorry, that’s a more appropriate topic. 15:03:44 Re: UX/UI resources, I’m in contact with HP’s internal resources, but they’’re just now hiring up that team, so unlikely to get much help re testing labs, etc from there. 15:03:52 Re: ElasticSearch/Sphinx. 15:04:11 NikitaKonovalov did a really good analysis of the advantages of ES 15:04:38 #link https://etherpad.openstack.org/p/Storyboard_ES 15:05:10 I guess we are considering only ES now, no Sphinx anymore 15:05:13 krotscheck: did you get input from clarkb and mordred? 15:05:17 I took the time to talk to mordred about that as well, and between NikitaKonovalov and mordred’s comments it quickly came out that everyone (in OS) is using ES, and nobody seems to be using sphinx. 15:05:41 krotscheck: who is 'everyone'? 15:05:44 I didn’t have the time to talk to clarkb - mostly because clarkb was in HeartBleed hell. 15:06:09 corvus: Well, ruhe did a search across the openstack codebase and couldn’t find any references to sphinx 15:06:33 So “everyone” to me means “the skill set in openstack seems to be heavily biased towards elastic search" 15:07:11 In short: We have a bunch of information on the ES side, but little on sphinx. 15:07:22 krotscheck: right, but we can't expect those people to just show up and start working on this; there's typically very little overlap. moreover, they're extremely unlikely to show up and run it... 15:07:42 So I’m not quite comfortable making a decision on that yet without additional ideas of what sphinx would buy us. 15:08:10 we should also consider things like ease of deployment, extra dependencies for people running it, resources needed, administration, etc 15:08:15 corvus: how hard is it to start a minimal ES installation? 15:08:23 * ruhe is here 15:08:31 corvus: I’m less concerned with them “running it”, and more of “Who can we annoy with questions when we try to build up our own skills in that area” 15:08:53 to start local ES for development you need three steps: 1. download archive 2. unpack it 3. run shell command 15:09:08 NikitaKonovalov: i don't know actually. the only one i've seen in practice falls over all the time and consumes about 0.5fte just to keep it running. 15:09:26 corvus: Sounds like you’re pretty invested in making sure that we make the right decision here, would you like to come up with an etherpad of pro/con arguments and make a recommendation? 15:09:48 As I said, so far we’ve only got half of the picture. 15:10:11 (or one third, or one fourth, given that there may be other solutions we’re not considering) 15:10:41 krotscheck: i don't have time for that in the next several weeks. i think we should consult clarkb on es... 15:10:48 krotscheck: what did you learn from mordred about sphinx? 15:10:59 Given that the person I was going to ask about sphinx basically said : Ehn, everyone else is using elasticsearch... 15:11:09 corvus: ^ 15:11:38 I can dig through those logs. 15:12:07 krotscheck: so we'll learn a lot just by asking clarkb. i expect he'll say one of "no way man i'm not running another one of those" or "sure, a small one will probably be fine". and then we'll have the ops story. :) 15:12:09 Extract more information, but the only meaningful statement about sphinx was that it was the de-facto standard back when mordred was an sql consultant. 15:12:48 if anyone else knows someone who has run a small-scale es, that would be great to know. 15:12:56 corvus: Well, I want to make sure that storyboard is as self-contained as possible, right? 15:13:17 Requiring a large dedicated ops team doesn’t lead to easy adoption inside of orgs, etc etc... 15:13:31 But yeah, I’ll check with clarkb 15:13:36 krotscheck: exactly; i think we should target being able to run this all on a single modest server 15:14:01 Ok, so ES/Sphinx investigation to continue. 15:14:03 (for a small set of projects; and scale up after that) 15:14:07 Anyway, we may say that ES is optional and servers for a performance 15:14:29 If some one does not use it then fall back to sql 15:14:31 NikitaKonovalov: oh that makes sense; would you fall back on a full table scan? 15:14:34 ok cool 15:14:55 * corvus reads etherpad quickly 15:14:59 Ok, so that’s an appropach should we decide to go with ES 15:15:08 * krotscheck speak good engrish 15:16:05 i wonder if we could trigger a background es update from the api? 15:16:08 Honestly, search isn’t a super pressing issue _yet_, since at best we’re a week out of actually implementing it, but let’s all keep it in mind as we continue. 15:16:22 Anything else re: es/sphinx? 15:16:34 so we're not blocking like the explicit flow 15:16:39 and we don't have to wait for esriver 15:17:13 corvus: we can use evenetlet or even another process to do that 15:17:54 NikitaKonovalov: cool. maybe we could start with explicit but then make that a future enhancement for performance... 15:18:15 oh, i just noticed the 'updated_at' part of es river 15:18:21 that might not be so bad then. 15:19:11 With any luck, we’ll be able to reduce the number of search queries anyway by being smart about determining what’s relevant to a particular user. 15:19:16 i think i still lean toward explicit or something like it because it's simpler (fewer dependencies) 15:20:23 heh, i can't really decide between those two; i'll keep thinking about it :) 15:20:24 Feels like y’all want to do some design discussion, let’s put search into the design discussion list and move on to the MVP 1.01 topics ttx didn’t get around to last time. 15:20:37 krotscheck: ++ 15:20:49 #topic Stories with all alnded tasks should not be in primary UI filter 15:21:02 So, I’m working on that right now. 15:21:09 Because it annoys the living daylights out of me. 15:21:16 My current approach is…. 15:21:21 Well, I’d like some feedback on it. 15:21:41 FTR the MVP 1.01 stuff are all the things I *think* would make a better experience at our early dogfooding stage 15:21:42 ttx: before I continue too much down that road, could you look at the patch and see if it does what you want it to? 15:21:56 feel free to add more stuff to it 15:22:15 #link https://review.openstack.org/#/c/86452/ 15:22:18 krotscheck: sure 15:22:38 might not manage to get to it today though 15:22:42 this week is a bit busy 15:22:47 ttx: That’s ok, I don’t know why it’s breaking anyway :) 15:23:01 of the things in ttx's list, the ones i most feel the need for are: ui filter, assignments, story activity 15:23:18 kk 15:23:23 one that i would like to add is: some kind of priority. 15:23:34 “Tasks can’t be edited” is done. 15:23:39 the projectgroup thing makes a better experience for storyboard specifically, since that allows to see "all of it" 15:23:44 i'm not sure where the priority discussions ended up at; i wasn't keeping up with that... 15:23:49 but it's useless without the ui filter 15:24:04 krotscheck: it should be easy to filter completed tasks on server side, and have a fetch_all_flag to get all the rest 15:24:07 priority handling is still a bit up in the air 15:24:25 You have to click on the actual task to bring up the edit form, so the UI needs to change, but you can do it. 15:24:35 Ok, guys. One topic at a time please. 15:24:37 I mean, we know how to do it the dumb way (high/medium/low) 15:24:45 * ttx freezes 15:25:10 corvus: ttx I agree that priority should probably be mvp 1.01 15:25:11 krotscheck: i'll mark it done 15:25:15 Anyone else? 15:26:05 krotscheck: the ideas we had for "priority" were a bit ambitious, with crazy kanbans and stuff :) 15:26:05 Sorry, edited agenda 15:26:06 Ok 15:26:10 Support for Project Groups 15:26:18 #topic Support for Project Groups 15:26:23 project group is a key feature as we add more projects 15:26:41 at this point it's only useful to see storyboard (two projects) in one shot 15:26:42 ttx: How many levels deep were you thinking on that? 15:27:10 krotscheck: you mean, shall we have groups of groups ? 15:27:25 ttx: shoud those groups match OS Programs 15:27:28 ttx: Yeah, for instance: We could have “Openstack-Infra” as a group, “Storyboard” as a subgroup, etc. 15:27:40 if so, let's call them programs also? 15:28:06 NikitaKonovalov: that's a neat idea 15:28:13 krotscheck: in my view we can model everything with a single level 15:28:43 Works for me 15:28:44 i/e/ storyboard-webclient would be part of several groups ("storyboard", infra, official openstack stuff, etc) 15:28:51 Gotit 15:28:58 that said, groups of groups would make specifying those groups easier 15:29:18 i.e. if you add a project to "infra" it could automatically appear in "official stuff" 15:29:36 so I can see the benefit of that 15:29:41 Yeah, but if one project can be a part of many groups, then we could have a parent group composed of subgroups and individual projects, and quite frankly that’s a set of edge cases that may be a little too complex for how young the project is. 15:29:54 the only "needed" part is the ability for one project to belong to multiple groups 15:30:06 * krotscheck is just trying to keep our features and commits digestible :) 15:30:10 because some of them just won't be a subgroup of another 15:30:42 for example you could have a "UI project group" that goes Horizon, Storyboard-webclient etc 15:30:52 Oh man, that would make me so happy 15:31:05 I could create a “Javascript only” project group and ignore all the silly python stuff. 15:31:07 :D 15:31:07 i agree, groups of groups would be nice; we can probably start with just direct membership and expand it later 15:31:16 krotscheck: ideally we would also support "personal project groups" which would be like your personal list 15:31:24 should just need an extra mapping table 15:31:45 Alright, so any disagreements with keeping project groups in MVP 1.01? 15:31:45 that can be baked into the same system, or be done more like a subscription thing 15:31:54 (gerrit lets you compose group membership, and we use it quite often) 15:32:15 corvus: You suggested that it might be less important than UI filter, activity, and assignments? 15:32:30 ttx: ++; i lean toward subscription there for ease of end-user use 15:32:43 krotscheck: only because we only have 4 projects now, but yes 15:32:45 Subscription is definitely a better metaphor. 15:32:47 krotscheck: I think projectgroup is not very useful until we have ui filter 15:32:53 kk 15:33:02 But still MVP 1.01? 15:33:04 Or not? 15:33:06 the idea being to get to the relevant information 15:33:13 for me yes 15:33:18 corvus? 15:33:22 NikitaKonovalov? 15:33:27 SergeyLukjanov? ^ 15:33:28 agree 15:33:34 krotscheck: yes, we'll be able to use it in mvp1.01 when it exists 15:33:35 juggling between storyboard and storyboard-webclient is my #1 pain with SB at the moment 15:33:42 ok 15:33:43 so projectgroup would fix that for me 15:33:57 ttx: Quick side question: Is that for creating tasks, or for listing things? 15:34:16 for listing things. Tasks should always point to a specific project 15:34:21 kk 15:34:25 Good to know. 15:34:34 Ok, Project groups stays in MVP 1.01 15:34:42 #topic Story Activity 15:34:52 probably we should start from simple filtering on ui 15:34:59 SergeyLukjanov: I agree. 15:35:12 I added this one because it's still difficult to follow what happens on a story 15:35:21 * SergeyLukjanov batch processing meeting logs like Hadoop 15:35:42 status changes, who did what is as valuable as the discussion 15:35:53 we can emulate it by leaving comments, but the time info is still missing 15:36:19 ttx: yes, but should it be mixed with comments in one flow? 15:36:43 ttx: Part of me thinks that this is a part we really should give a lot of design attention to, because it’s going to be the primary way in which we engage our users. 15:36:48 NikitaKonovalov: that would be my preference, I think. Could be convinced otherwise I guess 15:36:58 Deltas on tasks and stories are going to drive things like emails, notifications, etc. 15:37:09 usually a comment can only be parsed as part of the story history 15:37:37 reading comments out of context of what happened to that story would imho be difficult 15:38:05 but maybe I'm spoiled with Launchpad habits here 15:38:06 so we are replicating LP behavior as is 15:38:17 so I'm willing to hear alternate suggestions 15:38:36 I didn't say it's bad 15:38:42 I haven’t thought enough about the problem to have alternate suggestions. 15:38:58 But I really want to have them. 15:39:08 ttx: i like the updates in comments flow; if we put enough metadata in the db, we can change it later 15:39:18 I think they should appear in the same timeline -- we could have timeline filters if people really don't want to see certain things in that 15:39:43 but maybe planning for that basic functionality now is a reasonable approach 15:39:44 personally, I like the LP-way with status change == comment but with ability to filter only service comments 15:39:45 ttx: +1 for filtering comments 15:39:52 but I never heard the complaint that LP discussion contained extraneaous info 15:40:23 krotscheck: would you keep both comments and activity in the same table, or try to mix two tables ? 15:40:42 ttx: Honestly, I agree with you on UI completely, I just have strong opinions about the implementation. 15:41:03 krotscheck: in what way? 15:41:13 I feel that it should be modeled as a stream of events of different types, which then carry with them references to the appropriate records. 15:41:25 I don’t feel overloading the comments table is the right way to go about it. 15:41:48 And that a story event could be, say “status changed”, or “comment, status changed”, etc etc. 15:41:52 krotscheck: fair enough. i just fear we would duplicate the comment in the activity table saying "dude left a comment" 15:41:58 krotscheck: so a timeline with ids and timestamps, and then 'comment' and 'event' tables link to that? 15:42:14 jira has a nice way to separate different type of events/comments. it has a tab for full-history changes, tab for user comments, tab for status changes 15:42:16 corvus: ….maaaybe. 15:42:36 corvus: As I said, haven’t had much time to think it through. 15:42:53 krotscheck: possibly a TimelineEvent table for status changes, and comment_id field to link a comment to it 15:42:55 corvus: Immediate concerns are holy crap big table of ID’s is that going to be performant 15:43:01 krotscheck: don't have strong opinions on how to do it under the hood. I think by default it should show up in the same timeline, but that's about it 15:43:34 NikitaKonovalov: Yeah, something like that. 15:43:37 krotscheck: oh, one thing 15:43:41 Yes 15:43:42 ? 15:43:59 krotscheck: in LP there was the ability to leave a comment when you changed status, like to justify your change 15:44:05 krotscheck: ok. i don't have terribly strong feelings about the implementation at this point; i do think we all agree that what's stored in the db should be expressive enough to support the kind of filtering,etc we might want to do later 15:44:13 and then it would appear on the same event in the timeline 15:44:25 ttx: Yeah, already flagged that as a consideration. 15:44:31 It was a good thing because it forced people to explain why they had changed something 15:44:44 I fear if we sperarate them too much we would encourage the wrong behavior 15:44:45 ttx: That’ll be interesting to figure out since status happens on tasks and comments happen on stories :) 15:45:15 ttx: And it’s not very RESTful to modifiy two resources in one call :) 15:45:21 yes, that'w why I bring it up. I hope we can preserve it, but it may not fit that well 15:45:38 ttx: I feel we SHOULD preserve that, even if it doesn’t fit well. 15:45:45 But given those constraints I think we can figure something out. 15:45:46 for example, someone removing a task should almost always explain why 15:46:02 because otherwise you just don't understand what happened 15:46:29 he could leave a comment after the fact... but good UI would let him do it in one go. Might be a REST nightmare with two calls :) 15:46:33 Ok, so summary sounds like: “Yes we all want this, possibly in one filterable timeline, but design needs some offline gray metter.” Sound about right? 15:46:41 ayup 15:46:44 ayup 15:46:53 Alright. 15:46:58 lgtm 15:47:00 +1 15:47:09 Honestly, I don’t think we have enough people right now to assign someone to it. 15:47:14 At least not before next monday 15:47:51 #agreed We all want a story timeline/activity history, possibly in one filterable timeline, but design and implementation needs oflfine gray matter 15:48:06 arguably less urgent than the ui filter 15:48:19 ttx: Yeah, taht’s why I’m focused on that :) 15:48:28 Actually, NikitaKonovalov, what are you working on right now? 15:48:36 Want to take this on? 15:48:42 I'll take it 15:48:45 Awesome. 15:48:56 #action NikitaKonovalov Implement story actions/timelines 15:49:06 Ok, next topic 15:49:14 #topic Story priority 15:49:19 * krotscheck unfreezes ttx 15:49:41 story priority? 15:49:52 ok... 15:50:11 that's one of the areas where we actually want to get smarter than Launchpad 15:50:33 and let different groups of stakeholders have different priorities for stuff 15:50:52 trick is it's easily said, not so easy to design 15:51:05 My last try at it was at https://wiki.openstack.org/wiki/StoryBoard/Task_Lists 15:51:18 #link https://wiki.openstack.org/wiki/StoryBoard/Task_Lists 15:51:26 i.e. use several ordered task lists instead of a "priority" 15:51:32 so let's take a practical example 15:51:36 ok 15:51:41 for that MVP 1.01 15:52:09 (quick aside: I can probably knock out Task Assignments pretty quickly, I think it’s already supported by the API) 15:52:20 we could have had a task list where we would have listed all the things we want there, and then play with their ordering 15:52:28 (since we’re running low on time) 15:52:29 (relative priotity by ranking) 15:53:06 It's a bit ambitious and we are not exactly following on clear footsteps here 15:53:12 krotscheck: the assignee_id is pretty availbel to use 15:53:19 so it might just be a bit crazy / unusable 15:53:46 It's also quite a bit of work to do the crazy trello-like UI 15:54:32 ttx: So, to me this sounds like priority is an integer rather than a specific status. 15:54:49 And that the basic UI decision is “This is more important/less important” than it currently is. 15:55:24 The question is whether we care about explicit priority, or relative priority. 15:55:29 krotscheck: right, but also priority is only relevant in the context of a specific task list 15:55:39 i.e. “This task is more important than that one” vs. “This is in the same group as these” 15:55:53 i.e. the priority for release management may not be the priority for the core devs etc. 15:56:04 (fwiw, i'm very open to experimentation here; i don't mind at all of we go through several completely different iterations. i'd rather have something sooner, even if we throw it out and have to reprioritize everything) 15:56:16 hence the idea of having multiple task lists, and infdicate priority by ordering that subset of tasks 15:56:25 ttx: How many different task lists do you think there will be? 15:57:12 krotscheck: i expect people to have their own task list, then at least one project-level one for milestone planning 15:57:35 but i also expect people to run with them and start listing their little group priority 15:57:53 sorry for offtopic: any thoughts when we could add subscriptions? 15:57:59 (see types of task lists in the wiki doc) 15:58:05 We have two minutes 15:58:21 SergeyLukjanov: As soon as someone builds it :). Which, given our current backlog, looks about… 3 weeks? 15:58:32 krotscheck: to simplify first I would keep task lists at the project level 15:58:43 ttx: Ok, priority by project first. 15:58:44 Check 15:59:05 i.e. a project could have multiple task lists in a table 15:59:06 FTR: I love having personal vs. project priority 15:59:47 krotscheck: might be worth baking in a personal task list (same as the personal project group) 16:00:03 Yeah, but not MVP 1.01 :) 16:00:22 krotscheck: do you mean the tasks that you have created have a bigger priority than other in the same project? 16:00:27 Oh sure, I'm not sure we can do something with priority in 1.01 :) 16:00:32 Honestly, this is one place where not having FK’s is going to be magical, because we can do fun overloady thingies in the ORM. 16:00:33 task lists could also live at projectgroup level, fwiw 16:01:00 i.e. "storyboard priorities" rather than "storyboard-webclient priorities"' 16:01:05 Anyway, we’re out of time. Any last comments before we get kicked out? 16:01:16 ttx: :( we need some kind of priority soon 16:01:16 nope thx! 16:01:41 corvus: 1.02? 16:02:15 corvus: We can do a really stupid task-level priority to start with if you need something RTFN 16:02:16 i find it _very_ hard to work without some kind of sorting 16:02:19 corvus: maybe it can be done incrementally, starting with a unique per-project list 16:02:28 krotscheck: yeah, anything would help :) 16:02:47 Ok, so let’s start with that for 1.01 and then give it some love next release 16:02:50 #endmeeting