22:04:13 <alaski> #startmeeting nova_cells
22:04:14 <openstack> Meeting started Wed Feb  4 22:04:13 2015 UTC and is due to finish in 60 minutes.  The chair is alaski. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:04:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:04:18 <openstack> The meeting name has been set to 'nova_cells'
22:04:34 <alaski> #topic Test failurs
22:04:38 <alaski> argh
22:04:53 <dansmith> use #undo
22:04:57 <alaski> #undo
22:04:58 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x33399d0>
22:04:59 <dansmith> (if you want)
22:05:02 <alaski> sweet
22:05:08 <beagles> I thought it was pretty appropriate
22:05:11 <alaski> #topic Test failures
22:05:13 <beagles> "failurs"
22:05:16 <melwitt> heh
22:05:30 <belmoreira> hehe
22:05:36 <alaski> :)
22:06:02 <alaski> last week bauzas exluded the last test we expect to fail on the check-tempest-dsvm-cells job
22:06:16 <alaski> so in theory it should be passing, but isn't
22:06:37 <alaski> the most common failure I've seen is an ec2 run idempotent job
22:06:45 <alaski> which I can't get to fail locally
22:07:15 <alaski> http://logs.openstack.org/04/153004/3/check/check-tempest-dsvm-cells/dc1ee1f/console.html has an example
22:07:54 <alaski> any eyes on that are appreciated so we can get the job voting at some point
22:08:16 <melwitt> I'm also (still) running tests locally. since the instance object flavor field merged failures have changed a bit and I'm investigating
22:08:49 <alaski> melwitt: great
22:09:06 <alaski> any help on testing is much appreciated
22:09:17 <alaski> #topic WIP
22:09:33 <alaski> https://review.openstack.org/#/c/150381/
22:10:03 <alaski> There's a patch up for a quick PoC of multiple db support
22:10:09 <belmoreira> unfortunately Dheeraj is not here to comment
22:10:52 <alaski> that's alright
22:11:01 <belmoreira> he continues to look how to allow DB api to connect to different DBs
22:11:02 <alaski> dansmith added some good comments to the review
22:11:18 <belmoreira> yes, just saw it. Thanks
22:11:54 <alaski> he also mentioned passing info through the context for picking a db, which I think is a good idea
22:12:44 <dansmith> yeah, think that's how it's going to have to work,
22:12:49 <dansmith> else we'll need some major refactoring of db api
22:13:12 <alaski> agreed
22:13:23 <belmoreira> ok
22:13:57 <alaski> we can get into more specifics as work progresses, but that seems like the right mechanism to try
22:14:26 <alaski> on my end I've been busy with other things that are wrapping up now, so I'm going to be coding furiously on this now
22:14:43 <alaski> so there should be some more code to look at soon
22:15:11 <alaski> (it's also why this agenda is light)
22:15:21 <alaski> any other work to be called out?
22:15:34 <dansmith> alaski: the flavor stuff merged, so I should try fixing the libvirt driver now, right?
22:15:57 <alaski> dansmith: yes, that would be excellent
22:16:10 <dansmith> alaski: and, does that mean that the flavor stuff didn't break cells?
22:16:13 <alaski> I think that ties into what melwitt has been investigating
22:16:14 <dansmith> because that would be ... shocking
22:16:20 <dansmith> ah, melwitt you wanna do that?
22:16:23 <alaski> dansmith: it appears that it didn't
22:16:28 <dansmith> alaski: shocking
22:16:42 <alaski> dansmith: heh, we had faith in you
22:16:47 <dansmith> glad someone did :)
22:17:31 <alaski> but yes, the libvirt driver should be looked at in light of the flavors work
22:17:32 <belmoreira> dansmith: can you point me to the flavor stuff?
22:17:54 <alaski> it's likely that a previous workaround can now be removed
22:18:02 <melwitt> alaski, dansmith: yes, it didn't make it worse from what I see so far, there's something different wrong now which I'm working out
22:18:03 <dansmith> belmoreira: https://review.openstack.org/#/c/135700/
22:18:23 <belmoreira> dansmith: thanks
22:18:24 <dansmith> melwitt: boy, you really know how to compliment someone eh?
22:18:30 <dansmith> :P
22:19:14 <alaski> does someone want an action item to look at things?
22:19:34 <alaski> melwitt: I'm guessing you'll continue digging in?
22:20:06 <melwitt> dansmith: lol. not what I meant, sorry. from my understanding, the flavor stuff should have made 60 failures disappear but it didn't, the failures are now different. I've *almost* got a handle on what it is that I hope to confirm later today, then I will be asking you for help :P
22:20:32 <dansmith> melwitt: oh, really? I didn't expect it to fix any of those
22:21:13 <melwitt> dansmith: it should have because the problem was that cell couldn't access the flavor that existed only at the top. with flavor staying with the instance, it should have had all the info it needed to succeed
22:21:16 <alaski> it should mean that flavors don't need to exist in the cells db now
22:21:39 <dansmith> melwitt: ah, but only if the code that is looking up the flavor is using the per-instance interface
22:21:46 <dansmith> melwitt: so I think probably some more work is needed to convert those
22:21:48 <melwitt> I also figured out why the flavor not found problem got fixed and then returned. it got fixed the first time by garyk passing flavor obj down to driver
22:22:20 <dansmith> melwitt: so maybe tomorrow we can sync up on some of those failures and take a look at what changes might be easy?
22:22:27 <melwitt> but then it broke again when I added a get flavor call to fill in extra_specs that he asked for
22:22:54 <alaski> melwitt: ahh
22:22:57 <melwitt> flavor get fails if you're in the non-top cell.
22:23:01 <dansmith> this should fix that
22:23:15 <dansmith> but we have to change the code to use the flavor on the instance instead of looking it up
22:23:17 <alaski> yeah, the work from garyk shouldn't be necessary any longer
22:23:27 <dansmith> nor the extra_specs thing
22:23:37 <alaski> right
22:23:46 <alaski> there are some lookups in ironic that can be removed as well
22:23:52 <dansmith> cool
22:24:39 <alaski> it sounds like melwitt has this in hand, with some possible assistance from dansmith
22:25:02 <melwitt> s/possible/probable :)
22:25:03 <dansmith> cool
22:25:06 <dansmith> hah
22:25:09 <alaski> so we'll move on
22:25:17 <dansmith> I can offer probable assistance
22:25:20 <alaski> melwitt: keep him busy
22:25:26 <alaski> #topic open discussion
22:25:51 <melwitt> :)
22:26:01 <alaski> There's still an open question of networking
22:26:22 <alaski> I spoke with anteaya at the nova midcycle and she should be getting me in touch with someone from neutron
22:26:37 <dansmith> meaning whether we support n-net, or how we organize cells and neutron?
22:26:41 <alaski> the latter
22:26:43 <dansmith> so,
22:26:52 <dansmith> if we associate a neutron endpoint per cell,
22:26:53 <belmoreira> alaski: that is great
22:27:04 <dansmith> then they can be the same or one per cell, or one per few cells right?
22:27:29 <dansmith> if we're switching db/mq per cell, no reason we can't also switch the neutron being used
22:27:46 <alaski> dansmith: right.  but that assumes no longer using a global neutron
22:27:49 <dansmith> and then in the migration code, we can initially just fail if the target node isn't the same neutron endpoint as us
22:27:57 <alaski> or that it partitions similarly
22:28:07 <dansmith> alaski: ah, I guess that might break some of our API stuff that expect a single pool of network resources
22:28:53 <alaski> right, so we need some discussion on where best to address that
22:29:16 <alaski> we work around it at rackspace, but I need to reach out and see what limitations that causes
22:29:25 <dansmith> yeah
22:30:01 <belmoreira> this is the inital spec for nova net to neutron migration: https://review.openstack.org/#/c/147723/1
22:30:16 * tonyb is massively late.  Did I get volenteered for anything?
22:30:28 <dansmith> tonyb: yeah, you're figuring out networking
22:30:35 <alaski> ideally it would be nice if neutron partitioned along cell lines so we could do the multiple endpoint thing that you suggested dansmith
22:30:45 <tonyb> dansmith: awesome.  I can do that.
22:30:54 <alaski> tonyb: yep, please report back in a week with the solution
22:30:56 <dansmith> alaski: yeah, even if multiple partitions share a neutron
22:31:23 <belmoreira> alaski: in fact I was thinking in the other way arround... having a central neutron
22:31:47 <dansmith> that is likely going to require cells-like support in neutron I think,
22:31:53 <dansmith> which is something we could wait a long time for
22:32:25 <tonyb> I
22:33:02 <tonyb> I'll talk to gus on the neutron side.  It's something we should consider as part of the migration planning that's going on.
22:33:03 <belmoreira> alaski: seems interesting... but I think that should be a long term goal...
22:33:58 <alaski> long term it might make sense.  I'd like to get a sense of how the neutron group sees things
22:34:42 <alaski> I have the feeling that we're going to influence how neutron is deployed to work with cells
22:35:28 <alaski> but primarily I think we need to get everyone talking
22:35:37 <tonyb> alaski, dansmith (and others) do you have time after the nova meeting tomorrow to talk to some neutron folk? (if I can arrange it)
22:36:14 <alaski> which meeting time is it tomorrow?
22:36:17 <dansmith> that's the morning one
22:36:22 <dansmith> and yeah, I prolly could
22:36:27 <alaski> yeah, I'll be around then
22:36:49 <tonyb> okay I'll see what I can do.
22:36:56 <alaski> tonyb: thanks
22:37:19 <alaski> any other topics for discussion?
22:37:36 <dansmith> move to get back to work :)
22:37:57 <alaski> sounds good to me
22:38:09 <alaski> #endmeeting