14:00:13 <mriedem> #startmeeting nova
14:00:15 <openstack> Meeting started Thu Jul 13 14:00:13 2017 UTC and is due to finish in 60 minutes.  The chair is mriedem. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:19 <openstack> The meeting name has been set to 'nova'
14:00:24 <edleafe> \o
14:00:25 <alex_xu> o/
14:00:25 <efried> \o
14:00:28 <takashin> o/
14:00:29 * ildikov is lurking
14:00:36 <bauzas> \o
14:01:00 <mriedem> alright let's get started
14:01:03 <mriedem> #link agenda https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
14:01:09 <mriedem> #topic release news
14:01:14 <mriedem> #link Pike release schedule: https://wiki.openstack.org/wiki/Nova/Pike_Release_Schedule
14:01:21 <mriedem> #info July 27 is feature freeze (2 weeks away)
14:01:33 <mriedem> #info July 17 (next week) is the release freeze for non-client libraries (oslo, os-vif, os-traits, os-brick): https://releases.openstack.org/pike/schedule.html#p-final-lib
14:01:50 * gibi sits in a bit late
14:02:07 <mriedem> i haven't gone digging yet, but if we have any changes to os-vif or os-traits that are going to be dependencies for other work within nova, please let me know asap so i can monitor for a final release
14:02:22 <bauzas> none I saw for os-traits
14:03:03 <mriedem> ok
14:03:10 <efried> I have a question
14:03:13 <mriedem> shoot
14:03:17 <efried> "feature"
14:03:27 <efried> meaning things that affect functionality or change APIs or whatnot?
14:03:34 <efried> Not just "you can't drop code for a blueprint after this date"
14:03:43 <edleafe> efried: as in "bug fixes still ok"
14:03:51 <efried> Cause the service catalog endponit stuff may be tight.
14:04:05 <mriedem> efried: doesn't that rely on a ksa release?
14:04:16 <efried> Yes, we're still waiting on a big pile of stuff from ksa, sta, and the newly-minted os-service-types
14:04:21 <mriedem> ksa is a non-client lib so that would have to get released next week
14:04:33 <mriedem> mordred: ^
14:04:41 <mriedem> efried: which blueprint is that holding up?
14:04:54 <efried> service catalog for endpoints - lemme get the name...
14:05:02 <mriedem> https://blueprints.launchpad.net/nova/+spec/use-service-catalog-for-endpoints
14:05:10 <efried> yuh
14:05:58 <mriedem> so how about we talk with mordred about the state of things after the meeting?
14:06:18 <efried> Sure.  Been working closely with mordred, and I know we're close.  But yes.
14:06:21 <mriedem> #action mriedem and efried to talk to mordred about the final ksa release for next week
14:06:36 <mriedem> remind me if i don't follow up :)
14:06:41 <efried> rgr
14:06:48 <mriedem> #info Blueprints: 65 targeted, 63 approved, 29 completed (+3 from last week)
14:06:55 <mriedem> so we're making some progress,
14:07:20 <mriedem> and there are at least about 5 other bps i'm watching that are very close to done
14:07:41 <mriedem> maybe i'll send those to the ML after the meeting to focus our attention on pushing those over the line
14:07:48 <bauzas> yeah good idea
14:07:59 <bauzas> if some are close to be merged, gtk
14:07:59 <gmann_> mriedem: this one can be closed or after pythonclient change - https://blueprints.launchpad.net/nova/+spec/fix-quota-classes-api
14:08:01 <mriedem> #action send list of close to done bps to ML after the meeting
14:08:14 <mriedem> gmann_: after the client change
14:08:18 <gmann_> ok
14:08:22 <mriedem> i don't consider the api microversion ones done until the novaclient patch is also merged
14:08:46 <mriedem> ok any other questions about the release?
14:08:56 <mriedem> i know it's tight and we have a lot of work yet to do
14:09:02 <mriedem> so just put a pot of coffee on
14:09:19 <mriedem> #topic bugs
14:09:32 <mriedem> there are no critical bugs
14:09:37 <mriedem> #help Need help with bug triage; there are 115 (+6 from last week) new untriaged bugs as of today (July 13).
14:10:17 <mriedem> gate status is mostly ok
14:10:25 <mriedem> there were some things yesterday with zuul but infra sorted it out
14:10:35 <mriedem> no news for 3rd party ci
14:10:51 <mriedem> #topic reminders
14:10:58 <mriedem> #link Pike Review Priorities etherpad: https://etherpad.openstack.org/p/pike-nova-priorities-tracking
14:11:10 <mriedem> i don't know how realistic that is at this point
14:11:22 <mriedem> #link Consider signing up for the Queens PTG which is the week of September 11 in Denver, CO, USA: https://www.openstack.org/ptg
14:11:44 <mriedem> #topic stable branch status
14:11:51 <mriedem> #link stable/ocata: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/ocata,n,z
14:11:54 <cdent> is there ptg planning etherpad yet?
14:11:56 <mriedem> #link stable/newton: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/newton,n,z
14:11:58 <mriedem> cdent: nope
14:12:04 <cdent> should we intentionally wait?
14:12:12 <cdent> (so as not to be distracted)
14:12:14 <bauzas> that's very early
14:12:21 <mriedem> cdent: that's why i haven't created one yet
14:12:24 <edleafe> +1 on too early
14:12:27 <cdent> ✔
14:12:30 <bauzas> for stable branches, I did a bit of review for ocata
14:12:31 <mriedem> my focus is about hour to hour right now :)
14:12:33 <gmann_> yea lot of time
14:12:46 <mriedem> bauzas: yeah i saw that, thanks
14:12:49 <bauzas> do we need some point release soon ?
14:12:57 <mriedem> i wasn't planning one
14:13:01 <mriedem> we've done several this release already
14:13:13 <mriedem> we'll do another one around the rc1 i suppose
14:13:13 <bauzas> okay, so maybe after pike-3 I guess
14:13:18 <bauzas> yeah ok
14:13:21 <bauzas> so nothing urgent
14:13:29 <mriedem> i don't really have other news except they are finally killing off mitaka
14:13:35 <mriedem> infra is removing jobs, etc
14:13:45 <bauzas> yeah there was a thread
14:13:47 <mriedem> and the stable/mitaka branch is gone
14:14:08 <mriedem> ok moving on
14:14:14 <mriedem> #topic subteam highlights
14:14:28 <mriedem> i'll cover cells v2 as dan is not around this morning
14:14:36 <mriedem> #help Need reviews on quotas https://review.openstack.org/#/c/416521/ and fixing server external events https://review.openstack.org/#/c/445142/
14:14:52 <mriedem> i've spent a good chunk of 2 days reviewing the bottom quotas change
14:14:56 <mriedem> and talking through things with melwitt
14:15:01 <mriedem> so she has some stuff to update in there
14:15:13 <bauzas> the quota thing is a bit frightening :(
14:15:19 <mriedem> that bottom one is the change to start counting instances, which is the last reservable resource we have left
14:15:25 <mriedem> yes so it's complicated
14:15:35 <mriedem> and melwitt has done an awesome job working through this and being patient
14:16:03 <mriedem> the other patch there is another dependency for multi-cell,
14:16:05 <bauzas> sure, my main question is how we could raise our confidence in that patch
14:16:05 <mriedem> and a bug fix,
14:16:09 <mriedem> looks like that needs to be rebased
14:16:16 <mriedem> bauzas: you could review it for one :)
14:16:22 <bauzas> mriedem: I began
14:16:35 <bauzas> I have some comments but nothing uploaded yet
14:16:35 <mriedem> and get it merged so we have time to bake it in and flush out issues
14:17:03 <mriedem> i would love to see someone do a rally comparison before and after that patch to see if performance is impacted much
14:17:23 <mriedem> dtp was looking for things to help with, maybe he could do that
14:17:30 <mordred> mriedem, efried (yes to ksa discussion)
14:17:40 <mriedem> #action mriedem to talk to dtp about running rally against the bottom quotas change
14:17:50 <bauzas> mriedem: could we see the quota patch being tested live ?
14:18:00 <mriedem> bauzas: what does 'live' mean?
14:18:10 <mriedem> it's tested in our ci like everything else
14:18:11 <bauzas> some end-to-end testing
14:18:14 <bauzas> anyway
14:18:24 <bauzas> let's not derail the meeting
14:18:32 <mriedem> the thing that worries me the most about quotas test coverage,
14:18:47 <mriedem> is that tempest uses a separate tenant for every test
14:19:02 <mriedem> so if we start leaking quota, we won't find that in a normal tempest dsvm run
14:19:10 <mriedem> especially for complicated things like resize
14:19:14 <bauzas> :/
14:19:16 <mriedem> so,
14:19:34 <mriedem> something someone could do is pull that change down and do like 20 resizes in a row or someting
14:19:37 <mriedem> and find out if we blow up quota
14:19:59 <mriedem> or, push a DNM patch that does that with tempest
14:20:08 <bauzas> that's the end-to-end testing I had in mine
14:20:10 <bauzas> mind*
14:20:10 <mriedem> just run the resize tests 10 times in a row with the same tenant
14:20:12 <gmann_> mriedem: separate tests you mean separate quota tests?
14:20:23 <bauzas> but I could try to devstack it
14:20:39 <mriedem> gmann_: no the tenant isolation in tempest
14:20:46 <mriedem> quotas are counted against the project
14:20:52 <mriedem> so each test has it's own project and quota
14:21:14 <mriedem> so if we're leaking quota, we won't notice because we aren't using a single tenant through the entire tempest run
14:21:36 <mriedem> gmann_: are you aware of any ci jobs that run tempest without tenant isolation? i know there used to be a periodic job that did that
14:21:54 <mriedem> the novaclient functional tests also do that, fwiw
14:22:08 <gmann_> it used to be a serial job which use single tenant
14:22:09 <mriedem> so we could point a dnm novaclient change at that series and see what happens
14:22:22 <mriedem> the novaclient functional test is a serial job with a single tenant
14:22:25 <mriedem> so we can start with that
14:22:48 <mriedem> #action run a DNM novaclient patch against dan's fleetify devstack change which depends on the quotas series to see if we're leaking quota at all
14:23:07 <mriedem> i still think we need to do something like resize 20 times in a row
14:23:10 <gmann_> mriedem: but we have deprecated that things and we can do via account file which has pre defined tenants to run tempest against
14:23:36 <mriedem> but let's move on - if we're worried about this stuff, we have things we can do, like reviews, rally testing and endurance testing with a single tenant
14:23:37 <bauzas> mriedem: writing a novaclient change that would do 20 resizes ?
14:23:53 <mriedem> bauzas: yes maybe
14:23:57 <bauzas> mmm
14:23:57 <mriedem> moving on
14:24:10 <mriedem> i will rebase and address comments on the services/hypervisors API uuid changes
14:24:24 <mriedem> thanks to alex_xu for the thorough reviews there
14:24:30 <alex_xu> np
14:24:32 <mriedem> #info Started an etherpad to track remaining TODOs for Pike (like docs gaps): https://etherpad.openstack.org/p/nova-pike-cells-v2-todos
14:24:45 <mriedem> i should throw the test ideas up in ^
14:25:10 <mriedem> ok moving on to scheduler
14:25:11 <mriedem> edleafe:
14:25:15 <edleafe> We briefly discussed the spec amendment regarding picking up the Ironic flavor migration work that jroll was going to do. In that discussion, dtantsur brought up his work to make devstack use custom resource classes in Ironic by default. Patches to use Traits with AllocationCandidates have also been pushed by alex_xu.
14:25:20 <edleafe> We then spent the next half hour discussing jaypipes's patch for claiming in the scheduler, and the implications of those changes. We managed to achieve harmony, then held hands and sang folk songs afterwards.
14:25:24 <edleafe> Finally there was a discussion about only implementing half of the proposed change to the scheduler for claiming, and that until that is made, complex resource providers cannot be supported. Since that's not happening until Queens, it was agreed that it was fine to change the method signatures twice to reflect that support for complex RPs would not happen in Pike.
14:25:29 <edleafe> EOM
14:26:26 <mriedem> ok
14:26:58 <mriedem> for those wondering,
14:27:07 <mriedem> the current focus for that series is starting here https://review.openstack.org/#/c/482381/
14:27:14 <mriedem> #link claims in scheduler series is here now https://review.openstack.org/#/c/482381/
14:27:39 <mriedem> ok moving on to api
14:27:41 <bauzas> not really nope ?
14:27:46 <mriedem> alex_xu: anything to bring up here?
14:27:49 <alex_xu> Talk about 'PUT /os-services' API, agree with the API should be idempotent
14:27:57 <alex_xu> #link https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/api-no-more-extensions-pike
14:27:58 <bauzas> mriedem: https://review.openstack.org/#/c/482382/ is the bottom one now
14:28:02 <bauzas> AFAIK
14:28:12 <bauzas> anyway
14:28:42 <alex_xu> need review help for the api-no-more-extension-pike, they are easy, just need one more core reviewer
14:28:54 <alex_xu> that's all
14:28:54 <edleafe> bauzas: the dependencies are mixed up for that series
14:29:04 <jaypipes> and then jaypipes almost committed suicide last night attempting to deal with the awfulness of the scheduler.
14:29:34 <mriedem> #action mriedem to create a scheduler hotline for people working on it
14:30:06 <mriedem> ok moving on to notifications
14:30:08 <mriedem> gibi: you're up
14:30:14 * edleafe has an idea for a new market for Valium
14:30:39 <gibi> We discussed the last pieces of the additional-notification-fields-for-searchlight bp and as a result mand
14:30:42 <gibi> atory part of the BDM piece has already been merged. The tag support at instance create is still ongoing and in focus.
14:30:45 <gibi> Another patch in focus is a bugfix for https://bugs.launchpad.net/nova/+bug/1684860 https://review.openstack.org/#/c/475276/
14:30:46 <openstack> Launchpad bug 1684860 in OpenStack Compute (nova) "Versioned server notifications don't include updated_at" [Undecided,In progress] - Assigned to Takashi NATSUME (natsume-takashi)
14:30:48 <gibi> Most of the notification transformation patches need a rebase due to the BDM addition mentioned above.
14:30:53 <gibi> that is all
14:31:11 <mriedem> takashin: fyi on https://review.openstack.org/#/c/475276/
14:31:35 <takashin> mriedem: Thank you.
14:31:50 <mriedem> yeah and https://review.openstack.org/#/c/469800/ is on my short list of bps
14:31:57 <mriedem> that's the tags at server create one
14:32:14 <mriedem> i know alex_xu was +2 on it at one point
14:32:26 <mriedem> ok moving on
14:32:29 <mriedem> cinder stuff
14:32:36 <mriedem> needs reviews: https://review.openstack.org/#/q/status:open+project:openstack/nova+topic:bp/cinder-new-attach-apis+branch:master
14:32:51 <mriedem> specifically the swap volume change has a +2 https://review.openstack.org/#/c/456971/
14:33:00 <mriedem> so it would be cool to see another core get to that one
14:33:11 <ildikov> Cinder dependencies are all merged for the patches in the chain
14:33:18 <mriedem> stvnoyes is working on some grenade testing
14:33:27 <mriedem> hyper-v live migration is passing with the new flow
14:33:39 <mriedem> need testing from xenapi team at Citrix
14:33:40 <ildikov> XenAPI is confirmed to work as well
14:33:46 <mriedem> ok, yeah i saw they were reviewing
14:33:54 <mriedem> the live migration patch is https://review.openstack.org/#/c/463987/
14:34:03 <mriedem> i just started back on that before the meeting
14:34:31 <mriedem> but, i posted a change last week that runs the new flow through with the live migration changes and with the volume-backed live migration tests in tempest and things were all passing
14:34:53 <mriedem> so i'm fairly confident in the functionality, just need to dig into edge cases
14:35:10 <mriedem> moving on
14:35:13 <ildikov> we will have the Cinder-Nova meeting soon, so can sort those out hopefully
14:35:13 <mriedem> #topic stuck reviews
14:35:26 <mriedem> nothing on the agenda - is there anything to bring up here?
14:35:41 <mriedem> #topic open discussion
14:35:49 <mriedem> anything anyone wants to bring up?
14:35:50 <stephenfin> I've got one
14:35:57 <stephenfin> https://review.openstack.org/#/q/topic:doc-migration+project:openstack/nova+status:open
14:36:14 <stephenfin> ^ they need +2s if we ever want to have documentation again
14:36:15 <mriedem> #link nova docs migration starts here https://review.openstack.org/#/c/478472/
14:36:52 <mriedem> "It'll be a great day in the parish"
14:36:54 <mriedem> oh come on
14:37:02 <mriedem> your irish catholicism is bleeding through
14:37:09 <stephenfin> personally, I think if we ignore stylistic things and merge anything that's not actually broken, we can come back and revisit after p-3
14:37:24 <stephenfin> mriedem: Never forget where you came from ;)
14:37:37 <edleafe> Can we discuss the ironic migration? https://review.openstack.org/#/c/481748/
14:37:54 <mriedem> edleafe: sure if jaypipes is around for it
14:37:58 <mriedem> i know dan isn't
14:38:06 <edleafe> I'm still not clear on the order that things will be changing for this
14:38:11 <jaypipes> I'm here.
14:38:31 <mriedem> step 1 is getting the custom resource class reported into allocations
14:38:39 <mriedem> we report the inventory, but not the actual allocation
14:39:08 <mriedem> and i believe the vehicle for doing that is putting the resource class info in the flavor extra spec on startup of the driver
14:39:20 <jaypipes> mriedem: step 1 is getting the custom resource class added to the flavor so we can tell that it's being requested.
14:39:27 <jaypipes> mriedem: yes, bingo.
14:39:28 <mriedem> because then the scheduler report client can pull the resource class data off the extra spec in the update_available_resource periodic
14:39:39 <jaypipes> ++
14:39:49 <mriedem> that's the order i have in the work items section anyway
14:40:06 <mriedem> https://review.openstack.org/#/c/481748/1/specs/pike/approved/custom-resource-classes-in-flavors.rst@165
14:40:23 <edleafe> So what about existing ironic nodes? Will the extra_spec change show up in their instance.flavoe?
14:40:35 <edleafe> s/flavoe/flavor
14:40:41 <mriedem> yes
14:40:45 <mriedem> that's the whole poin
14:40:47 <mriedem> *point
14:40:56 <mriedem> on startup of nova-compute, the driver gets it's existing instances,
14:41:04 <jaypipes> right
14:41:07 <mriedem> and for their corresponding ironic node,
14:41:07 <edleafe> ok, just checking whether this would be a new flavor or an update
14:41:10 <bauzas> that's in the spec, nope ?N
14:41:19 <mriedem> get the node.resource_class and shove that into the embedded instance.flavor.extra_specs
14:41:29 <mriedem> bauzas: it was never clear in the original spec
14:41:33 <mriedem> which is why i was amending it
14:41:39 <bauzas> ok, I'm unclear about the problem
14:41:42 <bauzas> ok
14:41:52 <mriedem> edleafe: to be clear, this isn't updating existing global flavors,
14:41:57 <mriedem> it's updating the embedded flavor in the existing instances
14:42:11 <jaypipes> right
14:42:16 <jaypipes> re: edleafe
14:42:24 <mriedem> because that's what the scheduler report client looks at when putting allocations
14:42:26 <edleafe> so the first step is adding the resource class to existing nodes?
14:42:31 <mriedem> no
14:42:33 <jaypipes> 's question about should be zero out cpu/ram/disk, etc... I think the aNSWER TO THAT IS YES.
14:42:38 <jaypipes> oh ffs, typing sucks.
14:42:57 <mriedem> if the ironic node.resource_class is not None, then do data migration steps
14:42:59 <mriedem> else skip
14:43:03 <edleafe> ok, compute starts up and looks at node.resource_class. Where/how does that get populated?
14:43:12 <mriedem> it's populated in ironic
14:43:17 <mriedem> when the node is created in ironic
14:43:18 <jaypipes> edleafe: in the ironic get_inventory()
14:43:24 <jaypipes> here:
14:43:39 <edleafe> so all existing ironic nodes will be updated to return that class?
14:43:47 <mriedem> if we're going to step through this, can we do it in -nova rather than hold up the meeting?
14:43:50 <jaypipes> https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L615
14:43:57 <edleafe> mriedem: sure.
14:43:59 <jaypipes> edleafe: that's already been done.
14:44:01 <mriedem> ok anything else?
14:44:12 <edleafe> jaypipes: ok, that wasn't clear
14:44:29 <mriedem> alright i'm going to end it, thanks everyone
14:44:31 <mriedem> #endmeeting