17:00:00 <dansmith> #startmeeting nova_cells
17:00:01 <openstack> Meeting started Wed Aug 30 17:00:00 2017 UTC and is due to finish in 60 minutes.  The chair is dansmith. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:04 <openstack> The meeting name has been set to 'nova_cells'
17:00:16 <melwitt> o/
17:00:54 <mriedem> o/
17:00:56 <dansmith> ohai
17:01:03 <melwitt> welcome mriedem
17:01:06 <dansmith> #topic bugs
17:01:21 <dansmith> mriedem: you had a couple in the last week or so right?
17:01:23 <mriedem> there is a backport proposed for the guest assisted snapshot one
17:01:33 <mriedem> https://review.openstack.org/#/c/498979/
17:01:37 <dansmith> cool
17:01:58 <mriedem> i think that's the only cells v2 bug recently, everything else is placement
17:02:03 <dansmith> okay cool
17:02:11 <dansmith> anything else on bugs?
17:02:15 <mriedem> do'nt think so
17:02:23 <dansmith> #topic open reviews
17:02:56 <dansmith> I don't have anything for this at the moment either, since I'm working on some placement things we need to get done ASAP
17:03:16 <mriedem> i don't think there are any open reviews for cells v2 that i know of
17:03:16 <dansmith> anyone else?
17:03:22 <dansmith> ack
17:03:28 <dansmith> #topic open discussion
17:03:35 <dansmith> mriedem: you wanted to scheme about ptg things right?
17:03:38 <mriedem> yes
17:03:42 <mriedem> https://etherpad.openstack.org/p/nova-ptg-queens
17:03:56 <mriedem> there are some random things
17:03:58 <dansmith> I've put a few things on there so far for cells
17:04:22 <mriedem> the main things i think are
17:04:25 <mriedem> 1. Use GET /usages from the Placement API for counting quotas rather than iterating cells.
17:04:33 <mriedem> 2. Scheduling alternative hosts in the cell for retries
17:04:43 <mriedem> edleafe has a spec related to ^
17:04:57 <mriedem> https://review.openstack.org/#/c/498830/
17:04:59 <dansmith> yep, I took a first pass on that spec,
17:05:14 <dansmith> although it's just a piece of the puzzle
17:05:24 <mriedem> right it's not the full picture
17:05:33 <mriedem> just the data format i think between conductor and scheduler right?
17:05:39 <dansmith> yeah
17:05:59 <mriedem> 3. all of the upcall stuff https://docs.openstack.org/nova/latest/user/cellsv2_layout.html#operations-requiring-upcalls
17:06:20 <mriedem> *all of the "other" upcall stuff
17:06:35 <dansmith> yeah I need to revisit those things and see what is actionable
17:06:40 <melwitt> on the GET /usages thing the missing piece will be counting instances independent of cells. and the need to keep instance mappings around forever because deleted instances. I was wondering if it would be kosher to add a deleted column to instance mappings to know whether to count
17:06:43 <dansmith> the affinity stuff just has to wait for the distance placement stuff
17:07:14 <dansmith> melwitt: if it's in placement we count it, and if it's not we don't right?
17:07:14 <mriedem> melwitt: knee jerk reaction is we don't want a deleted column on instance_mappings
17:07:19 <dansmith> yeah, do not want
17:07:32 <melwitt> does placement have instances count in it?
17:07:47 <mriedem> no,
17:08:01 <mriedem> i think that means, GET /allocations/{instance_uuid} returns 404 right?
17:08:11 <dansmith> right
17:08:25 <mriedem> and that would also tied into how GET /usages works when filtering on instance.project_id and instance.user_id
17:08:42 <mriedem> if the instance is deleted, it shouldn't be in the placement allocations table and it's usages won't be affected by the instance consumer
17:08:47 <melwitt> ohhh, I see what you're saying
17:09:06 <dansmith> we should be able to list allocations by project too, so you can get a list of all the instances for a given project/user and that will have consumer ids, which are instances
17:09:27 <melwitt> sweet.
17:09:45 <melwitt> that works for me
17:09:50 <mriedem> the counting via placement stuff is all a bit busted right now too with how we're accounting for allocations during a move
17:09:58 <mriedem> which is going to be fixed with dan's migration uuid stuff
17:10:06 <dansmith> yah
17:10:16 <melwitt> oh, right. so we won't get a double count for a move-in-progress
17:10:21 <melwitt> ?
17:10:35 <dansmith> well,
17:10:43 <dansmith> you won't if you consider only instances
17:10:58 <dansmith> but I think you'll want to consider migrations too so your quota covers things you have in resize_confirm state right?
17:11:48 <melwitt> maybe, now that it could potentially be consuming twice the resources, I wouldn't want that counting against quota
17:11:58 <dansmith> no?
17:12:09 <mriedem> we don't double account quota during a move today
17:12:12 <dansmith> I thought there was a request to prevent people from having 20 instances, all in resize_confirm state for a month
17:12:17 <dansmith> right, but that's a bug right?
17:12:19 <mriedem> we pike the bigger parts of the flavor and count that for quota
17:12:24 <melwitt> I didn't think so. well, it would be a regression or rather it will change how quota has been counted
17:12:31 <mriedem> *pick
17:12:33 <melwitt> because this double allocating thing is new right?
17:12:44 <mriedem> the double accounting is new
17:12:44 <dansmith> well, regardless, right now you can't tell the difference between the two halves of a migration,
17:12:59 <dansmith> and after my change, counting just the instance allocation will give you the target of the migration, which is what you want
17:13:14 <dansmith> whether or not we work in the source side is something we can discuss
17:13:27 <melwitt> okay
17:13:35 <dansmith> but because of the way I have it set up, it'll just do what you're expecting by default
17:13:51 <dansmith> so we can discuss at the ptg maybe and see what others think
17:14:02 <melwitt> k
17:14:07 <dansmith> pretty sure I was talking to johnthetubaguy about the migration quota limit thing, so if he's there we can get his input
17:14:21 <mriedem> i think we'd have to account for the source side
17:14:29 <mriedem> for an old flavor that has bigger cpu/ram than the new flavor
17:14:45 <mriedem> otherwise if you resize, then do something else to raise quota, and revert, you could go over quota on the revert
17:15:16 <mriedem> but yeah i suppose details can be hashed out at the ptg
17:15:49 <dansmith> I think that's confusing and not what _I_ would want, but yeah, we can argue later
17:15:54 <melwitt> yeah. I suspect you're right though. currently we're considering old flavor or new flavor depending on whether it's an upsize or downsize
17:16:38 <mriedem> for multi-cell ci testing,
17:17:14 <mriedem> i think in boston sdague said he wanted to write some new deployment tooling using ansible or something since devstack/devstack-gate/grenade gets too complicated with all the bash and multiple nodes and such
17:17:36 <mriedem> but, that seems like a herculian task
17:17:38 <dansmith> yeah I'm not so sure how likely that is to happen at this point
17:17:39 <dansmith> yeah
17:18:06 <mriedem> but i think we should definitely try a 2-cell ci job in queens
17:18:25 <dansmith> well, I'd like to, but there are a lot of things piling up quickly so I don't want to put it too high on the priority list
17:18:26 <dansmith> so far,
17:18:44 <dansmith> I haven't found anything that works with our gate but doesn't with two real cells
17:19:01 <dansmith> I'm sure there are some, but seems like not too major
17:19:16 <dansmith> anyway, worth discussion at the ptg, which is what we're doing here
17:19:19 <mriedem> yeah it doesn't need to be top priority, but if we can at least figure out some of the high level what needs to happen, i know enough about d-g to do some damage
17:19:42 <dansmith> woot
17:20:02 <mriedem> the efficient listing stuff...
17:20:03 <mriedem> idk
17:20:35 <dansmith> I need to sync back up with mdbooth on that, but I'm hoping he's still going to do it.. last I heard from him it sounded like he was,
17:20:43 <dansmith> but if he's not we have to get a plan for that I think
17:21:02 <melwitt> I'm willing to pick it up if he can't get to it
17:22:04 <dansmith> mriedem: what else?
17:22:45 <mriedem> i see melwitt added consoles
17:22:48 <melwitt> I'll be continuing the consoles stuff. there's some review on it that I need to address
17:23:51 <mriedem> re-propose the spec if you haven't already
17:24:18 <melwitt> yeah, will do that
17:24:40 <melwitt> I probably need a new spec for swapping the quota count with placement calls too
17:24:47 <melwitt> do you think?
17:24:58 <dansmith> yeah
17:25:18 <mriedem> yes
17:25:46 <melwitt> okay. what about for the DB magic instance listing?
17:26:04 <mriedem> that's purely poc at this point i think
17:26:28 <melwitt> so finish the patch and then write a spec?
17:26:35 <melwitt> if it works
17:26:37 <mriedem> not sure it requires a spec
17:26:46 <mriedem> it should be transparent to the user
17:26:52 <melwitt> oksy
17:26:55 <melwitt> *okay
17:26:56 <dansmith> what magic instance thing?
17:26:57 <mriedem> it's really fixing a bug
17:27:03 <melwitt> mdbooth's thing
17:27:14 <dansmith> oh listing
17:27:18 * dansmith read that wrong
17:27:52 <mriedem> well i think that's probably certainly enough crap to worry about
17:28:02 <melwitt> agreed
17:28:14 <dansmith> cool, so anything else?
17:28:18 <mriedem> no
17:28:23 <melwitt> nay
17:28:28 <dansmith> awesome
17:28:32 <dansmith> #endmeeting