17:00:25 <alaski> #startmeeting nova-cells
17:00:31 <openstack> Meeting started Wed Nov 19 17:00:25 2014 UTC and is due to finish in 60 minutes.  The chair is alaski. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:32 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:34 <openstack> The meeting name has been set to 'nova_cells'
17:00:42 <dansmith> o/
17:00:42 <leifz> :-)
17:00:43 <bauzas> \o
17:00:44 <mriedem> hi
17:00:47 <alaski> woo
17:01:10 <alaski> welcome everyone to the first meeting on this new effort
17:01:11 <edleafe> o/
17:01:23 <dheeraj-gupta-4> hi
17:01:24 <alaski> I have a simple agenda setup at https://wiki.openstack.org/wiki/Meetings/NovaCellsv2#Agenda
17:01:25 <gilliard> \o
17:01:37 <VineetMenon_> \m/
17:01:47 <belmoreira> hi
17:01:49 <alaski> and feel free to add to that for any particular week
17:02:09 <alaski> #topic Cells manifesto
17:02:32 <alaski> So dansmith had the idea that some sort of manifesto would be useful
17:02:36 <bauzas> +1
17:02:49 <dansmith> yeah, I was just thinking a summary of where we see this going
17:03:02 <dansmith> an overview of the plan, what things will look like when it's "done" and maybe a few first steps
17:03:24 <bauzas> I like the idea here https://review.openstack.org/#/c/133691/
17:03:27 <alaski> I think that's a very useful idea
17:03:29 <dansmith> and of course, a little text relating this to current cells to make it clear that we're not really changing the high-level organization very much
17:03:36 <bauzas> for upgrades
17:03:48 <dansmith> I'm happy to help with that, or start it, etc
17:04:09 <leifz> It may be worth a paragraph on different issues we run into when add features that could affect cells?
17:04:21 <alaski> dansmith: cool.  if you want to start it that would be great
17:04:36 <dansmith> leifz: maybe, I think the alternate organization makes that much less of a problem
17:04:36 <VineetMenon_> dansmith: functioanlly much will not change.. but implementation wise, lots gonna change, right
17:04:40 <alaski> I come from a place of familiarity with cells, so I may make a lot of assumptions as I go
17:04:42 <dansmith> leifz: that's all in my mine of course, but..
17:04:55 <dansmith> alaski: sure I can start it
17:05:35 <dansmith> VineetMenon_: sure, a lot needs changing, but not as much as if we were doing something totally wildly different, I think
17:05:44 <alaski> where should this live?
17:05:47 <bauzas> dansmith: what kind of deliverable do you plan ?
17:05:57 <VineetMenon_> dansmith: and the DAG organization of cell won't be any more.. it will be a two level tree
17:05:58 <mriedem> alaski: devref?
17:05:59 <alaski> we could put it into a spec for review, but it's not really a spec in the traditional sense
17:06:01 <bauzas> dansmith: I like the idea of a rst file
17:06:06 <dansmith> alaski: I was thinking a blog post would be a good first thing
17:06:07 <bauzas> see https://review.openstack.org/#/c/133691/
17:06:31 <dansmith> VineetMenon_: yep
17:06:36 <alaski> dansmith: would you want to put it out for feedback first?
17:06:38 <mriedem> http://www.openstack.org/blog/ ?
17:06:55 <belmoreira> i think is also important to take on board all the deployments that at the moment don't care about cells
17:07:05 <belmoreira> because thay will be affected as well
17:07:06 <dansmith> alaski: yeah, so maybe in a etherpad to start, then a post that can show up on the planet for wider audience
17:07:25 <dansmith> an rst in tree is good, but I think we might want to make some progress before we do that
17:07:32 <mriedem> +1
17:07:34 <bauzas> ack
17:07:36 <VineetMenon_> alaski: then why not a typical BP?
17:07:44 <alaski> dansmith: +1
17:07:45 <mriedem> this isn't a bp
17:07:45 <dansmith> belmoreira: the goal here is that those deployments shouldn't really notice this change if they don't want to
17:08:00 <bauzas> VineetMenon_: that's wider than just a specification
17:08:04 <alaski> VineetMenon_: this is more high level than a blueprint, as it's not describing specific work to be done
17:08:11 <mriedem> it's like an SLD right?
17:08:15 <dansmith> right, which is why I called it a manifesto
17:08:28 <dansmith> mriedem: don't bring your TLAs here
17:08:29 <dansmith> :)
17:08:31 <VineetMenon_> alaski: ack
17:08:33 <mriedem> muwahaha
17:08:35 <bauzas> mriedem: I call PRD on my side :)
17:08:49 <mriedem> gonna be a fine line to not cram every little detail into this thing
17:08:56 <dansmith> right
17:09:11 <dansmith> manifestos are usually heavy on the goals, light on the actual details, right? :)
17:09:11 <mriedem> like deployment impacts, a history of cells and deep dive, etc
17:09:17 <alaski> #action dansmith to start an etherpad with a cellsv2 manifesto
17:09:19 <mriedem> i guess
17:09:27 <mriedem> when i think of manifesto i only think of hitler, is that wrong?
17:09:34 <bauzas> woah, first action
17:09:50 <leifz> manifesto == tell me why I should care...
17:09:51 <dansmith> I think manifestos are usually "we want free icecream!" with no plan for how to pay for and deliver said icecream :)
17:09:52 <edleafe> mriedem: more like unabomber
17:10:12 <bauzas> isn't what the scrum world would call an epic ? :)
17:10:13 <mriedem> dansmith: so like a patent application :)
17:10:16 <VineetMenon_> mriedem: huehue
17:10:19 <dansmith> I'm suddenly uncomfortable with being tasked with the manifesto, and what that apparently connotes to people :)
17:10:23 <dansmith> mriedem: heh
17:10:37 <leifz> https://www.gnu.org/gnu/manifesto.html
17:10:48 <alaski> leifz: nice save
17:10:55 <leifz> it's old, but good.
17:11:02 <dansmith> yeah, and similar to what I'm talking about
17:11:05 <dansmith> light on the details
17:11:15 <mriedem> yeah, that's good
17:11:19 <alaski> right, details should go in specs
17:11:23 <mriedem> yup
17:11:56 <alaski> cool, moving on
17:12:02 <alaski> #topic Checkpoints
17:12:25 <alaski> I want to provide or get updates on various things we agreed to at the summit
17:12:48 <alaski> I now have two specs up for some initial cells work
17:13:04 <alaski> https://review.openstack.org/135424 and https://review.openstack.org/135644
17:13:34 <alaski> there will be more, but I could use as much feedback as possible
17:13:44 <dansmith> jogo and I were going to try to do a first pass at table analysis..
17:13:48 <dansmith> I don't think that needs to be a spec, right?
17:13:58 <alaski> dansmith: yeah, I don't think so
17:14:02 <dansmith> cool
17:14:22 <leifz> dansmith: can you put it somewhere to view? just want to get my hands around this aspect if possible.
17:14:23 <alaski> but the results should live somewhere discoverable, perhaps in devref?
17:14:32 <dansmith> leifz: totally
17:14:48 <mriedem> devref would be good, it's under utilized and probably pretty stale
17:14:49 <dansmith> alaski: devref might be a good place eventually, so that it's easy to refer to
17:14:55 <mriedem> like does devref say anything about objects?
17:15:06 <bauzas> mriedem: s/probably/obviously
17:15:10 <dansmith> my guess is that there will be a few that belong on one side or another, but need some work to make that true
17:15:22 <mriedem> the 2nd spec looks like a copy of the first when i reviewed the first yesterday
17:15:25 <dansmith> so either devref will have to wait, or lie, or call those out
17:15:31 <dheeraj-gupta-4> alaski: IIRC per the new specs each cell gets its own API and the cell table tells them how to contact each other?
17:15:51 <dheeraj-gupta-4> alaski: for the first spec
17:15:59 <bauzas> dansmith: can the analysis findings be provided in an etherpad ?
17:15:59 <alaski> dansmith: I would expect it to call them out
17:16:11 <bauzas> dansmith: so we can discuss on it
17:16:21 <alaski> but that is making me think an etherpad might be better for now
17:16:28 <dansmith> bauzas: yeah, certainly we'll do the work in an etherpad, but if we want to argue, that's better suited for gerrit I think
17:16:53 <alaski> dheeraj-gupta-4: each cell has it's own db, but there's one api
17:17:07 <alaski> dheeraj-gupta-4: and the table tells the api how to contact each cell
17:17:19 <dheeraj-gupta-4> alaski: Ahh OK
17:17:19 <bauzas> dansmith: agreed
17:17:52 <bauzas> dansmith: let's consider the etherpad as a first pass and then go to a devref file or whatever deliverable
17:18:12 <dansmith> yeah, I think that's what I said :)
17:18:31 <alaski> another action for dansmith!
17:18:35 <dansmith> bah
17:18:55 <alaski> #action dansmith to begin analysis of tables in an etherpad, to be moved to gerrit later
17:18:57 <alaski> :)
17:19:07 <bauzas> dansmith: I like paraphrasing :)
17:19:13 <dansmith> jogo is suppsoed to help, so that's fine
17:19:17 <dansmith> supposed even
17:19:25 <alaski> there was quite a list of people to help on that one
17:19:32 <dansmith> oh, okay cool
17:19:36 <alaski> (Dan, Joe, Nikola, CERN, bauzas, NeCTAR)
17:19:38 <VineetMenon_> alaski:... won't it be pertinent to discuss what all data need to reside where, at this point?
17:19:48 <dansmith> all of cern is helping with that? awesome :)
17:19:57 <alaski> dansmith: good luck with that
17:20:01 <dansmith> VineetMenon_: that's what we're talking about
17:20:04 <belmoreira> yes :)
17:20:13 <bauzas> alaski: don't know who this bauzas is
17:20:17 <bauzas> :)
17:20:39 <alaski> :)
17:20:42 <leifz> alaski: we should cross-check on our side as well.
17:20:55 <alaski> VineetMenon_: the etherpad will start that discussion.  it's a bit early to do it in a meeting
17:21:04 <VineetMenon_> alaski: sure
17:21:19 <belmoreira> alaski: for the initial work what are the other specs that you are thinking ?
17:21:34 <alaski> leifz: yeah, I didn't volunteer because I have a lot of other things I committed to, but Rackspace definitely has valuable input
17:22:05 <dansmith> leifz: the assumption here is that existing cells people have to validate that this is reasonable
17:22:08 <alaski> belmoreira: the specs only go as far as creating new tables and the ability to use them.  We need something for a migration of data into those tables and to start using them
17:22:25 <leifz> alaski: if you can't, we'll find someone to help.
17:22:39 * leifz thinking I'm still volunteering alaski for now :-)
17:22:51 <alaski> I'll definitely be involved, just maybe more on verification than writing it
17:23:09 <mriedem> there would need to be some tool to read data from the old nova db and write into the new nova_api db right?
17:23:10 <dansmith> I think that the manifesto will help us brainstorm what else we need to do along the way
17:23:13 <mriedem> assuming that's a later spec
17:23:31 <alaski> mriedem: right.  that's the next spec I'm looking to get together
17:23:32 <dansmith> like objects knowing whether they should talk to the api conductor or the local one, the eventlet python problem, etc
17:24:01 <belmoreira> alaski: the other thing that we discussed in the summit was testing
17:24:05 <alaski> dansmith: agreed
17:24:22 <alaski> belmoreira: yep, thanks
17:24:22 <belmoreira> alaski: should that be a spec?
17:24:39 <alaski> belmoreira: I don't think so at this point
17:24:43 <mriedem> belmoreira: i think it depends on changes needed to qa
17:24:47 <dansmith> yeah
17:24:51 <alaski> I've been doing some work in infra for that
17:24:53 <dansmith> anything we need to change in nova would be a bug
17:24:55 <mriedem> if we need features in tempest, a small qa-spec is easy
17:25:19 <mriedem> apparently i'm qa liasion so hopefully that goes both ways
17:25:22 <alaski> I did get a change in to exclude some tests from tempest on the experimental cells run
17:25:53 <mriedem> alaski: cool, i'd like to see that change at some point later if you can dig it up
17:26:05 <alaski> there's currently the issue that a flavor needs to exist in the cell db, which I'm hacking aroudn to get some data
17:26:28 <alaski> I would actually like to rely on dansmiths flavors in instance_extra patch series to ultimately resolve this though
17:26:41 <mriedem> alaski: link to that for reviews?
17:26:53 <alaski> but in the meantime it might be worth considering alternatives
17:26:55 <dansmith> https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/flavor-from-sysmeta-to-blob,n,z
17:27:04 <mriedem> #action review https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/flavor-from-sysmeta-to-blob,n,z
17:27:18 <dansmith> mriedem: everything up to the last one with the WIP is good, I'm currently splitting the last one for sanity
17:27:36 <mriedem> ok, i'll try to focus on those
17:27:43 <alaski> I reviewed the early ones but haven't gotten through the end yet
17:28:01 <VineetMenon_> alaski: what about this? https://review.openstack.org/#/c/108238/
17:28:28 <alaski> VineetMenon_: I didn't realize that had merged
17:28:39 <alaski> that may provide a short term workaround then
17:29:31 <alaski> #action alaski to see if https://review.openstack.org/#/c/108238/ can be used to fix cells testing
17:30:05 <alaski> there was also an item from the summit to get scheduling requirements from cells users
17:30:19 <bauzas> agreed
17:30:42 <alaski> johnthetubaguy agreed to help on that, but isn't here atm... so I don't have an update there
17:30:53 <dansmith> yeah, and I think he has a good handle on the issue
17:31:00 <dansmith> so I think it'd be good to have him own that
17:31:11 <alaski> yeah
17:31:29 <alaski> I'll ping him specifically next time to get an update on that
17:31:44 <dansmith> yeah, next meeting is late for him
17:32:18 <alaski> that leads nicely into something I wanted to ask about
17:32:24 <alaski> #topic open discussion
17:32:50 <alaski> I didn't get a lot of feedback on who might attend the 2200 meeting
17:33:03 <mriedem> that's 4pm for me i think so i should be able to attend
17:33:16 <belmoreira> for me ok
17:33:22 <bauzas> I guess most of the EU people can't decently attend
17:33:27 <alaski> so what I'm curious about is if it actually helps anyone?
17:33:38 <bauzas> oh, seems like CERN is now relocating to the US ? :)
17:33:40 <mriedem> tz wise?
17:33:42 <vineet> alaski: None from Asia, at least.
17:33:54 <alaski> I'm going to hold it for a bit to see who shows up, but if it's always the same group it may be easier to consolidate
17:34:08 <alaski> mriedem: right
17:34:16 <mriedem> sounds good
17:34:17 <dansmith> I prefer consolidation of course, but yeah, not sure who else would show up in that slow
17:34:18 <dansmith> er, slot
17:34:23 <belmoreira> bauzas: well is 2300 CET so is not to bad...
17:34:28 <dansmith> mikal maybe, but I doubt he'd need to be here really
17:34:39 <bauzas> belmoreira: :)
17:34:53 <alaski> dansmith: yeah, I think if we bring updates to the nova meeting he may be good
17:35:01 <dansmith> yeah, so not sure who else the later meeting helps
17:35:02 <vineet> was that specifically for people in US?
17:35:05 <mriedem> i think that was the plan at the summit
17:35:13 <alaski> anyways, just wanted to bring it up and see if anyone here was helped by that timeslot
17:35:20 <mriedem> the subgroups bring updates to the nova meeting, at least monthly or per milestone
17:35:30 <alaski> vineet: the other slot was for asia/pacific region
17:36:12 <vineet> alaski: aah.
17:36:28 <alaski> well, we'll see how it goes
17:36:39 <alaski> anyone have anything else to discuss?
17:36:45 <mriedem> i guess my only open topic thing would be,
17:36:51 <mriedem> work items,
17:37:02 <mriedem> like if you think of things people can start working on now,
17:37:06 <vineet> alaski: testing of present cell functionality?
17:37:06 <mriedem> make a list
17:37:32 <mriedem> since i don't know the existing cells design/code that well, if there are low hanging fruit things you want help with but don't have time, we should have a list for people to sign up,
17:37:41 <alaski> mriedem: good point
17:37:51 <mriedem> e.g. 'these unit tests suck, re-write them' or something, idk, like what jay did with the scheduling stuff
17:38:02 <mriedem> consider me an intern and code review monkey otherwise
17:38:07 <alaski> I'd like to get a list of what's left for testing, once I get more info about what happens when flavors is fixed
17:38:23 <alaski> and I will share that for everyone who wants to help
17:38:25 <dansmith> mriedem: my guess is that you can help with the testing effort without needing to know much
17:38:40 <dansmith> mriedem: I think plenty of those issues are just mechanical "didn't expect a ! in the instance name" thing or something
17:38:47 <mriedem> sure
17:39:01 <mriedem> if someone can just give me a task and say go, i can ask questions as needed
17:39:10 <vineet> can someone confirm this bug for me? https://bugs.launchpad.net/tempest/+bug/1394227
17:39:19 <dansmith> alaski: presumably you can cultivate such a list as you go along, right/
17:39:24 <leifz> alaski: a good example is the GET calls may be easier to move over to the new functionality while other things are being addressed.
17:39:27 <alaski> vineet: testing is mostly blocked by the flavors issue atm so that needs to be addressed first
17:39:30 <gilliard> I'm in the same situation. Where would the list live?
17:39:35 <bauzas> mriedem: it took some iterations to get the list we discussed at the summit for the scheduling stuff :)
17:39:39 <mriedem> gilliard: thinking etherpad
17:39:40 <vineet> alaski: okay.
17:39:54 <gilliard> makes sense
17:39:57 <bauzas> mriedem: that's just coming from our past weekly discussions :)
17:39:59 <jogo> vineet: I was just looking at that bug,
17:40:11 <alaski> dansmith: yes, I can keep a list going
17:40:18 <jogo> vineet alaski: my understanding is we aren't fixing existing cells issues
17:40:22 <vineet> jogo: sure.. please let me know if you are able to reproduce it
17:40:22 <alaski> I'll get an etherpad going for that
17:40:36 <mriedem> vineet: check the experimental queue cells tempest job to see if that test fails there,
17:40:37 <jogo> just making sure we don't code rot
17:40:45 <mriedem> if the test is using security groups or something that's probably why it's failing
17:40:53 <dansmith> yeah
17:41:09 <mriedem> which i think it does, sets up a security group and then ssh's into the vm
17:41:12 <alaski> #action alaski to cultivate a list of tempest tests that could use eyes or fixes
17:41:40 <mriedem> i'll find a link to the cells tempest job on experimental so we can refer to that for what's failing today
17:41:42 <vineet> mridem: okay.. I'll try that ASAP
17:41:56 <alaski> jogo: it may be necessary to fix some issues, but mainly we're trying to adapt testing to test what currently works
17:42:10 <jogo> alaski: ahh
17:42:17 <mriedem> yeah
17:42:19 <mriedem> baseline
17:42:23 <mriedem> so we don't regress cells v1
17:42:24 <dansmith> yeah, we probably just need to take each one
17:42:27 <alaski> mriedem: https://review.openstack.org/#/c/135064/ btw
17:42:32 <mriedem> alaski: thanks
17:42:34 <dansmith> "is this worth fixing because it increases coverage"
17:43:07 <alaski> exactly
17:43:56 <leifz> There weren't many, but when I checked last there were a small set that dealt with the cellname mucking up our response (looked to be translation issue of some sort), but that's only a few tests.
17:44:06 <alaski> also https://review.openstack.org/#/c/135621/ hacks around the flavors issue atm so results there are worth looking at
17:44:22 <mriedem> leifz: we could disable ceilometer in the cells exp queue run if needed,
17:44:29 <mriedem> but why would ceilometer muck up nova's responses?
17:44:40 <dansmith> cellname
17:44:58 <leifz> no these were http related (I don't remember the details) just went "hmmm... that's crazy we don't have the right code there".
17:45:08 <mriedem> color me lost
17:45:14 <dansmith> mriedem: he didn't say ceilometer
17:45:19 <mriedem> ah
17:45:20 <mriedem> ha
17:45:51 <leifz> mriedem: CELLNAME not to be confused with a monitoring project
17:45:52 <mriedem> i thought alaski was talking about that at the summit too, something about a hostname issue or something
17:45:54 <mriedem> maybe unrelated
17:46:04 <alaski> mriedem: nope, same thing
17:46:13 <vineet> mriedem: I'll post the nova log as well... tomorrow maybe
17:46:19 <alaski> hostname is cellname!hostname in current cells
17:46:26 <mriedem> i'd like to propose bubbles as the codename for the next openstack monitoring project, thanks
17:46:52 <vineet> mriedem: southpark?
17:47:18 <mriedem> vineet: bubbles was the joke codename for cellsv2
17:47:22 <mriedem> or not joke, i couldn't tell
17:47:22 <dansmith> I vote for Sparkly Twinkly Free Unicorn
17:47:25 <dansmith> or STFU for short
17:47:28 <mriedem> ha
17:47:36 <vineet> :D
17:47:47 <alaski> on that note, anything else? :)
17:47:51 <dansmith> please no.
17:48:04 <mriedem> nope
17:48:15 <alaski> excellent
17:48:19 <alaski> #endmeeting