21:00:30 <alaski> #startmeeting nova_cells
21:00:31 <openstack> Meeting started Wed Jan 13 21:00:30 2016 UTC and is due to finish in 60 minutes.  The chair is alaski. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:32 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:35 <openstack> The meeting name has been set to 'nova_cells'
21:00:46 <alaski> anyone around for cells today?
21:00:50 <melwitt> o/
21:00:51 <belmoreira> o/
21:01:03 <doffm> o/
21:01:08 <alaski> awesome
21:01:14 <alaski> #topic Cells testing
21:01:31 <alaski> things seem quiet
21:01:35 <alaski> melwitt: seen anything?
21:01:45 <melwitt> alaski: nope. all seems well
21:01:53 <alaski> excellent
21:02:14 <alaski> #topic Open reviews
21:02:21 <alaski> #link https://etherpad.openstack.org/p/mitaka-nova-priorities-tracking
21:02:29 <alaski> lots of patches to review
21:03:05 <doffm> I see some new stuff added. I'll get on it. :)
21:03:19 <alaski> we can touch on specific ones in open discussion if people would like
21:03:29 <alaski> doffm: awesome, thanks for that
21:03:31 <melwitt> it looks like nothing new yet on the tempest config option for disabling secgroups
21:03:41 <belmoreira> yes
21:03:43 <alaski> #topic Open Discussion
21:04:02 <alaski> melwitt: yeah, I think ccarmack was going to talk with the qa group about that
21:04:37 <melwitt> okay
21:04:43 <belmoreira> I missed last week discussion for https://review.openstack.org/#/c/201606
21:04:49 <ccarmack> Do we still need that tempest config change?
21:05:00 <ccarmack> Since v2 supports sec groups
21:05:18 <alaski> ccarmack: it's mostly about cleaning up testing configs for v1
21:05:38 <ccarmack> alaski: ok, I'll keep on it
21:05:40 <alaski> belmoreira: one minute
21:05:44 <melwitt> ccarmack: well, my concern is currently we don't have any testing of instance boot/network connectivity for cells v1 because we have all tests that do that blacklisted bc secgroups
21:06:01 <alaski> oh right, and that
21:06:27 <ccarmack> melwitt:  there was a comment about cells and neutron
21:06:52 <melwitt> so just... don't touch anything in cells v1 ;) so there's no way we could break the functionality
21:07:34 <ccarmack> melwitt:  I think the question was about cells v1 supporting neutron provided sec groups
21:07:48 <melwitt> ccarmack: ah, right
21:07:51 <alaski> ccarmack: cells isn't tested against neutron, only nova-network
21:08:06 <alaski> and we nixed plans to test against neutron
21:08:07 <ccarmack> I think it was a "what if" question
21:08:12 <ccarmack> ok
21:08:47 <alaski> it would be nice to do, but it was deemed too much effort for not much gain
21:08:56 <alaski> so it's not going to happen
21:09:19 <melwitt> I do wonder if qa would be more amenable to the idea of a "special" tempest test just for our situation that will boot and instance and ssh with default secgroup
21:09:37 <melwitt> but I dunno. it would be a useless test for everyone else
21:10:33 <alaski> it could be a reasonable middle ground though
21:10:54 <alaski> ccarmack: is that something you could try, or ask about?
21:11:05 <ccarmack> melwitt: does tempest open all the ports, because I was wondering how the ssh with cells v1 works without sec groups
21:11:39 <melwitt> ccarmack: it does. mriedem found the code sometime back when we were all talking about it
21:11:54 <melwitt> I'd have to dig it up again to show you
21:12:17 <ccarmack> melwitt: do we need the special test then?
21:12:43 <melwitt> ccarmack: well, none of the tests use the default group is what I meant. we'd have to make a new test that uses the default group
21:12:57 <melwitt> that we wouldn't blacklist
21:14:24 <ccarmack> melwitt: I guess I'm confused, why would we need to use the default group if the tests run ok now?
21:14:58 <alaski> none of the tests that aren't blacklisted use the default sec group
21:15:09 <melwitt> ccarmack: the tests don't run okay for us because they create a new secgroup on the fly to assign to the booted instance. so we blacklist them
21:15:26 <melwitt> they would run okay if they didn't create a new group and instead used the default group
21:16:01 <ccarmack> hmm.. I wonder how the experimental cells gate tests ran for me
21:16:07 <ccarmack> no errors
21:16:41 <alaski> a lot of cells tests are running and passing.  but none that ssh to an instance
21:17:15 <alaski> all current tests that would check ssh are blacklisted for not using the default sec group
21:17:36 <melwitt> there are only about 5 tests that ssh to an instance out of all the tempest tests, fwiw
21:17:58 <ccarmack> alaski:  I think I un-blacklisted them in my patch
21:18:17 <alaski> ccarmack: do you have a link?
21:19:00 <ccarmack> https://review.openstack.org/#/c/226043
21:19:52 <melwitt> that's after you disabled the new secgroup creation
21:20:10 <melwitt> and made it use the default group. right?
21:20:47 <ccarmack> melwitt:  Yes, I assume it use the default group now
21:20:59 <melwitt> it does
21:21:00 <alaski> that change passes because it was run on top of https://review.openstack.org/#/c/225199/
21:23:14 <alaski> ccarmack: can you try https://review.openstack.org/#/c/225204 and https://review.openstack.org/#/c/226043 without depending on https://review.openstack.org/#/c/225199/ ?
21:23:34 <alaski> if that works we can avoid the controversy on the last one
21:24:15 <alaski> nvm, you need that last patch
21:25:50 <alaski> it sounds like we still need more discussion with qa folks to get buyin on the current approach, or try adding a test for the default sec group situation
21:26:05 <melwitt> yeah
21:26:23 <alaski> ok, we just need to keep pushing on that then
21:26:39 <alaski> belmoreira: https://review.openstack.org/#/c/201606 ?
21:26:44 <ccarmack> alaski: I think if I had ran experimental https://review.openstack.org/#/c/225204 it would have failed
21:27:28 <ccarmack> alaski: I'll keep pushing, don't mean to monopolize the meeting
21:27:49 <alaski> ccarmack: no worries, it's a good discussion
21:28:25 <belmoreira> alaski: there wasn't many replies to the ML but seems clear that the soft delete should be removed
21:28:52 <alaski> ccarmack: if you find that we're testing ssh to an instance with the current tests somehow that would be great, as that's the goal more than disabling sec groups just to disable them
21:29:25 <alaski> ccarmack: or if it could be done without disabling them in current tests
21:29:28 <alaski> belmoreira: agreed
21:29:43 <belmoreira> do you think we can start? or wait a little more... I'm starting to be concerned about the timelines
21:29:45 <ccarmack> alaski: ok, I check if we are testing ssh
21:30:49 <alaski> belmoreira: me too...  we're not going to have the new flavor api changes in place in M I don't think
21:31:30 <alaski> belmoreira: we could still add the new flavor tables to the api db, but maybe we need to hold off on the migration for now
21:31:31 <belmoreira> but should that be a blocker for cellsV2?
21:32:18 <alaski> no, it shouldn't block it.  it may change how flavors are accessed a little
21:32:40 <alaski> if a flavor doesn't exist in the api db, but is in the nova db and deleted, don't migrate it but show it in the api
21:33:38 <alaski> let me back up.  what I'm thinking is that we can migrate flavors to the api db, but would still need to access them from the nova db
21:33:58 <alaski> until we get the flavor api changes, then we can remove them from the nova db
21:34:50 <belmoreira> so what happens when a new flavor is created?
21:35:05 <alaski> it would need to be created in both
21:35:37 <melwitt> alaski: based on the flavor api patch, I didn't see that we were removing them from nova db during migration
21:35:43 <belmoreira> and if it's deleted, is hard deleted in api db and soft deleted in nova db?
21:36:15 <alaski> belmoreira: yeah, or we could just wait on the migration and do it all later
21:36:55 <doffm> alaski: +1 Sounds complicated to go through this process and then go to the 'proper' way later.
21:37:14 <doffm> Is there any chance we can get the spec and code in for https://review.openstack.org/#/c/265282/?
21:37:17 <doffm> Before M?
21:37:32 <alaski> melwitt: hmm, good point.  I had assumed they were being removed but they don't appear to be
21:37:55 <alaski> doffm: I don't see it happening
21:38:12 <belmoreira> doffm: I agree
21:38:14 <alaski> primarily because we're way past spec freeze
21:38:26 <alaski> do we have consensus on waiting on the migration then?
21:39:26 <belmoreira> alaski: It would be nice to start merge at least the table in the api db to start having some progress
21:39:35 <alaski> so we would be looking at getting the flavor tables in the api db now, flavor api changes in N with migration to follow
21:40:28 <alaski> belmoreira: agreed.  mriedem had some comments that needed to be addressed, and removed the soft delete stuff, but then I think it should be good to go
21:41:41 <melwitt> yeah
21:42:39 <belmoreira> alaski: missing flavors in M means that we can't have a cellV2 proof of concept
21:43:30 <alaski> not entirely.  we can't do multiple cells until we get flavors migrated, and some other tables
21:43:30 <doffm> belmoreira: That might be the case anyway, cell0 still missing, scheduling interaction somewhat early in review process.
21:44:10 <alaski> yeah, there's a lot still needed
21:44:31 <belmoreira> as big cell user what concerns me is that we are extending the end of life of cellv1
21:44:32 <alaski> and it's not clear at what point we can say cells is somewhat real
21:44:54 <belmoreira> doffm: yeah
21:45:06 <doffm> alaski: Thats a concern here also, I have been telling people it will 'probably' be real in N.
21:45:12 <doffm> But thats just what I've been telling people.
21:46:12 <alaski> gotcha
21:46:38 <alaski> I will say that I'm now at a point where I have more time to devote to cells work
21:47:03 <alaski> and interest from more places, like your involvement doffm, will help
21:47:11 <doffm> Not to immidiately change topic, ccarmack and I would like to start work on cell0 spec.
21:47:25 <doffm> Do you think there is enough in the sheduler interaction POC for us to do that.
21:47:41 <alaski> yes
21:47:45 <doffm> Ok.
21:47:54 <doffm> I didn't see anything really missing.
21:48:33 <melwitt> I find I've worked on things that are not so immediately useful, like the db connection switching. if there's something more "now", I'm interested in that
21:49:46 <alaski> melwitt: sorry, I thought that would be more pressing than it was
21:50:15 <alaski> the scheduler interaction work is at a place where actual cells in the cell_mapping table are needed
21:50:29 <melwitt> alaski: no worries, it's good to do but I feel I haven't helped much being that we don't really need it for awhile
21:50:56 <alaski> melwitt: see the comment in https://review.openstack.org/#/c/263927/1/nova/compute/api.py
21:51:38 <melwitt> ah, I see
21:51:53 <alaski> my first thought was a nova-manage command, but we need some way to populate the cell_mapping table with info on the current db/mq
21:52:46 <melwitt> oh, I think I understand. I think nova-manage would be my thought too
21:52:47 <alaski> and then after a few more changes cell0 will be needed for when scheduling fails to find a cell
21:53:54 <melwitt> okay, as I review the changes you've got I'll see if I can figure out where I could pitch in. thanks
21:55:23 <alaski> melwitt: I'm going to keep pushing towards moving scheduling out of compute/api, so the work on creating cells is open
21:55:34 <alaski> it overlaps with the work on cell0 a bit though
21:56:03 <alaski> but cell0 doesn't need a mq I don't think, and needs some way to indicate that it's special
21:56:42 <melwitt> okay
21:57:02 <alaski> I hope we can get a group of us together at the midcycle to talk this through with higher bandwidth
21:57:36 <doffm> Sounds great.
21:58:08 <melwitt> +1
21:58:49 <alaski> in the meantime I'm going to keep trying to get things written down in hopes that it helps people jump in
21:59:41 <alaski> just about at time
21:59:45 <alaski> thanks everyone
21:59:50 <melwitt> thanks all
21:59:53 <doffm> Thanks.
21:59:55 <belmoreira> thanks
21:59:56 <alaski> #endmeeting