17:00:03 <dansmith> #startmeeting nova_cells
17:00:04 <openstack> Meeting started Wed Nov  9 17:00:03 2016 UTC and is due to finish in 60 minutes.  The chair is dansmith. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:08 <openstack> The meeting name has been set to 'nova_cells'
17:00:10 <mriedem> o/
17:00:12 <dansmith> cells peeps say wha?
17:00:17 <bauzas> \o
17:00:17 <mriedem> https://www.youtube.com/watch?v=JHYIGy1dyd8
17:00:19 <melwitt> o/
17:00:19 <mriedem> is what i say
17:01:01 <dansmith> okay, I guess it's just us
17:01:06 <dansmith> #topic testing
17:01:12 <dansmith> anything on this?
17:01:20 <dansmith> we have some things pending for cellsv2 in grenade and devstack
17:01:22 <mriedem> well
17:01:24 <mriedem> yeah that
17:01:31 <mriedem> https://review.openstack.org/#/c/393441/
17:01:34 <mriedem> is grenade ^
17:01:43 <mriedem> https://review.openstack.org/#/q/topic:ocata-requires-cellsv2
17:01:53 <mriedem> i'm good with the nova change on top https://review.openstack.org/#/c/392227/
17:02:00 <mriedem> so bauzas melwitt ^
17:02:11 * bauzas looking
17:02:17 <mriedem> we kind of need sdague for the grenade change
17:02:22 <bauzas> heh, it was my question
17:02:23 <dansmith> yeah
17:02:24 <mriedem> but i'm not sure when he comes back
17:02:27 <bauzas> but https://review.openstack.org/#/c/392227/ resolved it
17:02:34 <dansmith> at least another week I think
17:02:40 <mriedem> i can ping him to see for sure
17:03:00 <dansmith> we have some things that can go in before the big ones in nova to get stuff done while we wait for that
17:03:04 <dansmith> so I'm not too concerned
17:03:28 <mriedem> the big issues is going to be,
17:03:33 <bauzas> okay, I'll dig into those changes
17:03:37 <mriedem> upgrading/starting api last in grenade/devstack for multicell
17:04:02 <mriedem> but before tackling that, we can probably deal with non-grenade multinode cells v2
17:04:08 <dansmith> mriedem: yeah, so I had some hacky patches up to try to do that, and have talked to sdague about it.. it's going to be a little tough because of when/where we setup the api endpoints in keystone, etc
17:04:17 <dansmith> yeah
17:04:22 <mriedem> also,
17:04:30 <mriedem> with https://review.openstack.org/#/c/392289/ we might just automatically get it...
17:04:37 <mriedem> because the neutron multinode job will now be running cells v2
17:05:28 <mriedem> so we'll see
17:05:40 <mriedem> sdague's orientation is the 29th
17:05:42 <dansmith> mriedem: that won't end up with old api I don't think
17:05:54 <mriedem> dansmith: no it would be full install multinode all ocata
17:06:14 <mriedem> wanted to make sure the scheduler changes are working in that case at least
17:06:24 <dansmith> oh I see what you mean
17:06:25 <dansmith> yep
17:06:50 <dansmith> we're kinda bleeding over already so let me just...
17:06:51 <mriedem> fudge....so,
17:06:55 <dansmith> #topic open reviews
17:07:05 <mriedem> sdague is basically out for the rest of the year...
17:07:14 <mriedem> maybe a week or two in december
17:07:16 <mriedem> just keep that in mind
17:07:20 <dansmith> I thought we'd get him for beginning of dec yeah
17:07:25 <mriedem> for like a week :)
17:07:31 <mriedem> and he'll be playing catchup
17:07:34 <dansmith> I'll probably drop off at some point too, haven't planned yet
17:07:50 <mriedem> alright
17:07:57 <dansmith> yeah, I know, but if we can at least get the grenade thing figured out,
17:07:59 <dansmith> there are a couple other grenade cores
17:08:04 <dansmith> if we know what he's okay with and not
17:08:13 <mriedem> there are other grenade cores?! :)
17:08:18 <dansmith> yeah man :)
17:08:21 <mriedem> treinish also being out forever
17:08:30 <mriedem> https://review.openstack.org/#/admin/groups/188,members
17:08:33 <mriedem> oh not even
17:08:35 <mriedem> so dean
17:08:54 <mriedem> we can move on
17:08:57 <dansmith> yeah
17:09:03 <dansmith> so, for open reviews,
17:09:06 <bauzas> interesting
17:09:13 <dansmith> I have the big set to work on moving scheduling to the conductor,
17:09:19 <dansmith> which is actually a lot of the work we have planned for this cycle
17:09:21 <dansmith> https://review.openstack.org/#/c/319379/
17:09:38 <dansmith> it was pretty far off a week ago, but it's actually starting to work a bit now
17:09:58 <dansmith> we've discussed some of the sticking points like secgroups this morning
17:10:02 <dansmith> so still some work to do there, but it's going pretty well
17:10:05 <dansmith> glad to have melwitt back so she can find all the bugs
17:10:10 <mriedem> so after this morning we have at least 2 changes right?
17:10:14 <dansmith> yeah
17:10:14 <mriedem> 1. store uuid in secgroup
17:10:16 <melwitt> heh
17:10:27 <mriedem> 2. get the uuid from the name in the api and use that for #1
17:10:35 <mriedem> i'll take a crack at #2
17:10:38 <bauzas> I'm cool with that plan
17:10:44 <dansmith> mriedem: yeah, so trying to get #2 into its own patch is going to be a little hard I think
17:10:50 <dansmith> mriedem: because of the changeover that the last patch makes
17:11:02 <dansmith> mriedem: oh okay I was going to do both but that's fine
17:11:22 <mriedem> i need you to review specs :)
17:11:27 <mriedem> i have to tap out of specs for awhile
17:11:33 <dansmith> heh
17:11:37 <dansmith> okay
17:12:10 <dansmith> okay so aside from that set,
17:12:16 <dansmith> what else is on the blocks right now?
17:12:31 <mriedem> so,
17:12:40 <mriedem> instances going to cell0 is not a thing yet right?
17:12:45 <dansmith> it is
17:12:47 <mriedem> and we talked about working that in with the scheudler stuff
17:12:48 <dansmith> as part fo that set
17:12:49 <mriedem> orly
17:12:52 <dansmith> yup
17:13:03 <dansmith> there is a lot wrapped up in there
17:13:10 <mriedem> https://review.openstack.org/#/c/367557/ ?
17:13:17 <bauzas> that's in the series
17:13:27 <dansmith> mriedem: well, yes, but not until the last patch does it actually happen of course
17:13:27 <mriedem> "- in the case of failure method creates instance in cell0"
17:13:28 <mriedem> yup
17:13:47 <mriedem> ok,
17:13:50 <mriedem> hmm,
17:14:02 <mriedem> i might just mark that cell0 bp as complete then, or superseded by the scheduler interaction one
17:14:04 <dansmith> and after this set, we'll be scheduling across cells
17:14:07 <dansmith> so we kinda have the beginnings of multicell
17:14:08 <mriedem> since we're tracking them together
17:14:40 <mriedem> ok so those were the big things on my radar
17:14:55 <mriedem> melwitt is back and can start digging in on this or the quotas stuff if needed,
17:15:00 <dansmith> what I meant above was.. are there other big sets of changes up for review?
17:15:02 <mriedem> as noted i looked into the quotas counting thing a bit yesterday
17:15:06 <mriedem> no
17:15:11 <dansmith> okay
17:15:14 <dansmith> #topic open discussion
17:15:19 <mriedem> there are some tricky bits with counting rams/cores in the api
17:15:33 <mriedem> placement api and resource provider usage might help with that?
17:15:38 <dansmith> yeah
17:15:40 <mriedem> but that's not per-tenant is it...
17:15:42 <dansmith> although they don't know about tenants
17:15:44 <mriedem> right
17:15:52 <dansmith> so you'd have to do a pretty big query of them
17:16:02 <mriedem> i know a guy that likes to write big sql queries
17:16:08 <dansmith> well,
17:16:15 <dansmith> it can't be a sql query
17:16:34 <dansmith> we have to basically get a list of instance uuids and then query all the ram/cores taken by $uuids
17:16:38 <mriedem> you'd have to map instances to computes and computes to resource providers right?
17:16:41 <dansmith> which could potentially be yuuge
17:16:46 <bauzas> dansmith: talking about statistics ?
17:16:57 <dansmith> mriedem: no we have allocations by instance uuid (consumer)
17:16:58 <dansmith> bauzas: no?
17:17:24 <dansmith> mriedem: unfortunately, the data is still in our own db so we could do something super efficient,
17:17:41 <dansmith> mriedem: but the virtual wall between us an placement prevents it by decree
17:17:45 <bauzas> dansmith: http://developer.openstack.org/api-ref/compute/?expanded=show-hypervisor-statistics-detail ?
17:17:56 <dansmith> which is.. kinda the suck
17:18:03 <mriedem> bauzas: that's not tenant specific though
17:18:04 <dansmith> bauzas: I'm not sure what that has to do with what we're talking about
17:18:11 <mriedem> we need tenant specific ram/core usage
17:18:21 <bauzas> dansmith: okay, me not understanding what you're talking about :)
17:18:28 <dansmith> bauzas: quotas by counting
17:18:32 <dansmith> bauzas: as we discussed at summit
17:18:32 <bauzas> ack
17:18:39 <mriedem> there are some details in the ML
17:18:48 <mriedem> i had to talk to dan yesterday afternoon to understand more about that
17:18:51 <mriedem> i was a bit lost in the session
17:18:59 <bauzas> as well, I have to admit
17:19:24 <mriedem> so maybe we don't worry about that today
17:19:31 <dansmith> yeah,
17:19:39 <dansmith> let's let people (i.e. melwitt) stew on options a bit
17:19:42 <mriedem> we do have some other things that need to happen for cells v2, like the instance sort/filter spec alex_xu has up
17:19:54 <mriedem> https://review.openstack.org/#/c/388518/
17:20:12 <dansmith> I will look at that when we're done here
17:20:18 <mriedem> ok
17:20:19 <mriedem> cool
17:20:23 <mriedem> that's all i have
17:20:28 <melwitt> there isn't anything in the allocations table yet, right? I don't know how that all works
17:20:28 <dansmith> mriedem: have you looked at it?
17:20:36 <dansmith> melwitt: there is
17:20:37 <dansmith> melwitt: and it's what we need
17:20:50 <dansmith> melwitt: but we're not supposed to query it directly from the api, even though we can
17:20:56 <mriedem> dansmith: not yet
17:20:57 <melwitt> okay. I set up a devstack yesterday and didn't find anything in there so there must be something else I have to do
17:20:58 <mriedem> it's on my growing list
17:21:05 <dansmith> mriedem: ack
17:21:12 <dansmith> melwitt: you didn't have placement enabled
17:21:14 <mriedem> melwitt: right
17:21:22 <dansmith> melwitt: which is required for ocata, but probably not defaulted in newton yet
17:21:22 <mriedem> placement-api needs to be in enabled services
17:21:26 <dansmith> er, defaulted in devstck
17:21:27 <melwitt> ah, okay. I'll redo that then
17:21:40 <melwitt> *enable it then
17:21:49 <bauzas> yeah, you need to enable placement-api service
17:21:51 <mriedem> melwitt: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/nova.yaml#n51
17:21:54 <mriedem> is the placement job config if that helps
17:21:57 <bauzas> oh snap
17:22:07 <melwitt> thanks
17:22:30 <mriedem> which reminds me i need to get on the scheduler boyz
17:22:32 <dansmith> okay so sounds like that's most of the status.. good progress and inertia on the important things
17:22:35 <mriedem> about plans to make that required in ocata
17:22:42 <dansmith> anything else we need to discuss here
17:22:42 <dansmith> ?
17:22:45 <mriedem> nope
17:23:03 <dansmith> going once
17:23:15 <melwitt> there is the consoleauth open question still I guess (sorry)
17:23:27 <dansmith> melwitt: say more things
17:23:42 <melwitt> I don't remember seeing PaulMurray at the summit
17:23:52 <dansmith> I don't think he was there, at least that I saw
17:24:08 <melwitt> but basically for the console token auth there's an upcall that happens to the consoleauth service that's assumed to be running at the API cell
17:24:21 <melwitt> over rpc
17:24:48 <melwitt> and PaulMurray in the past had some plans to deprecate that service and replace it with some auth that happens in the DB
17:24:53 <melwitt> directly, I think
17:25:14 <dansmith> hmm, an upcall really?
17:25:17 <melwitt> so if that were to be deprecated we could avoid dealing with an rpc call up to the API cell
17:25:18 <dansmith> I don't think I know about that
17:25:38 <dansmith> I liked his auth-in-the-db thing anyway,
17:25:52 <dansmith> so that seems good
17:26:28 <melwitt> I'll re-dig up some details to put on one of our etherpads so you can see in detail what I'm talking about. I looked into it several weeks ago and don't remember everything now
17:27:03 <dansmith> okay cool
17:27:15 <dansmith> anything else?
17:28:09 <dansmith> okay then
17:28:12 <dansmith> #endmeeting