16:00:34 <gibi> #startmeeting nova
16:00:35 <openstack> Meeting started Thu Sep 10 16:00:34 2020 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:38 <openstack> The meeting name has been set to 'nova'
16:01:13 <bauzas> \o
16:01:14 <gibi> o.
16:01:20 <elod> o/
16:02:12 <stephenfin> o/
16:02:12 <gibi> lets get started
16:02:18 <gibi> #topic Bugs (stuck/critical)
16:02:33 <gibi> the https://bugs.launchpad.net/nova/+bug/1882521 has been marked Critical
16:02:35 <openstack> Launchpad bug 1882521 in OpenStack Compute (nova) "Failing device detachments on Focal" [Critical,Confirmed] - Assigned to Lee Yarwood (lyarwood)
16:02:58 <gibi> lyarwood opened a qemu bug https://bugs.launchpad.net/qemu/+bug/1894804
16:02:59 <openstack> Launchpad bug 1894804 in QEMU "Second DEVICE_DELETED event missing during virtio-blk disk device detach" [Undecided,New]
16:04:08 <bauzas> well, technically, we haven't regressed and our gate isn't broken
16:04:21 <lyarwood> \o
16:04:21 <bauzas> so the Critical status is remarkable
16:04:27 <gibi> do we consider this as release critical? shall I mark it with  victoria-rc-potential tag?
16:04:42 <lyarwood> which bug is this sorry?
16:04:51 <gibi> https://bugs.launchpad.net/nova/+bug/1882521
16:04:52 <openstack> Launchpad bug 1882521 in OpenStack Compute (nova) "Failing device detachments on Focal" [Critical,Confirmed] - Assigned to Lee Yarwood (lyarwood)
16:05:06 <lyarwood> so how is this is not critical?
16:05:12 <bauzas> my point was, I totally understand this being a priority
16:05:18 <lyarwood> focal is the supported os from V
16:05:21 <lyarwood> for*
16:05:29 <bauzas> is this used in the gate ?
16:05:33 <lyarwood> yes
16:05:58 <bauzas> ok, then tagging the bug
16:06:16 <bauzas> if so, nothing to complain
16:06:26 <bauzas> do we have a logstash query ?
16:06:42 <lyarwood> I can write one up if it's required
16:06:45 <bauzas> trying to see how much this impacts
16:06:58 <gibi> lyarwood: it is actually not breaking the gate right now, is it?
16:07:01 <lyarwood> at the moment we've added the failing tests to a blacklist on a number of jobs
16:07:04 <bauzas> gibi: to answer your question, looks like it's a good candidate indeed
16:07:13 <bauzas> gibi: that's my whole question
16:07:24 <gibi> lyarwood: and we have the intention to remove them from the backlist before V?
16:07:33 <bauzas> Criticals are for either regression or gate failures
16:07:34 <gibi> if so then it is release critical
16:07:49 <bauzas> agreed on this one
16:07:49 <lyarwood> gibi: right that's my understanding
16:07:53 <gibi> OK
16:08:17 <gibi> then I added victoria-rc-potential to the bug too
16:08:19 <bauzas> again, I think we need to identify the impact
16:08:43 <bauzas> I haven't seen personnally the gate borked by this one but I could be wrong
16:08:52 <sean-k-mooney> by which?
16:08:54 <gibi> bauzas: it is not borked as the test is turned off
16:08:56 <sean-k-mooney> missed the start
16:08:56 <lyarwood> because it hasn't moved to focal yet
16:09:03 <sean-k-mooney> oh focal
16:09:03 <lyarwood> sean-k-mooney: the focal detach bug
16:09:09 <lyarwood> sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1882521
16:09:11 <openstack> Launchpad bug 1882521 in OpenStack Compute (nova) "Failing device detachments on Focal" [Critical,Confirmed] - Assigned to Lee Yarwood (lyarwood)
16:09:20 <sean-k-mooney> ya so we think that likel a cirros/qemu bug not nova right
16:09:27 <bauzas> ok, now I have a better understanding, thanks
16:09:33 <sean-k-mooney> i know we may be able to adress it in nova better
16:09:55 <sean-k-mooney> but the difference in behavior on focal is likely cause by somethign external to nova
16:10:19 <sean-k-mooney> i dont think we should be holding swapting the gate jobs to focal because of this
16:10:41 <lyarwood> well the knock on impact is that we lose some coverage in the gate
16:10:48 <bauzas> sean-k-mooney: is that 100% reproducible when you detach a device on Focal ?
16:10:50 <sean-k-mooney> if we really need too we could run a centos-8 job for the test coverage
16:10:52 <lyarwood> but you're right there's not a lot we can do about it in Nova
16:11:02 <sean-k-mooney> bauzas: not 100%
16:11:04 <lyarwood> sean-k-mooney: what about Fedora? /s
16:11:08 <sean-k-mooney> bauzas: you need suffenct load
16:11:10 <sean-k-mooney> lyarwood: no
16:11:21 <sean-k-mooney> i dont think we shoudl have gating jobs on fedora
16:11:24 <gibi> so we move to focal, keep the test disabled, and if qemu or us figure out a fix then we build an RC+1 out of it
16:11:27 <lyarwood> note the /s
16:11:39 <lyarwood> gibi: ack
16:11:39 <sean-k-mooney> centos-8 has the same verion of qemu and libvirt as focal
16:11:42 <bauzas> sean-k-mooney: lyarwood: so the logstash query is like super important
16:12:07 <gibi> #action lyarwood to provide a logstash query
16:12:09 <bauzas> gibi: is there something we need to do from a nova perspective ?
16:12:09 <gibi> can we move on?
16:12:32 <gibi> bauzas: handle the communication in the qemu bugreport?!
16:12:33 <bauzas> because from my understanding, I don't see anything actionable from nova during RC1
16:12:37 <sean-k-mooney> bauzas: we may be able to handel the detach more gracefully
16:12:53 <lyarwood> sean-k-mooney: that still wouldn't fix it btw
16:12:53 <bauzas> or requiring a patch to be shipped in RC1, if you prefer
16:13:01 <bauzas> my whole point
16:13:16 <bauzas> I don't see a need for us to leave a Critical bug unresolvable on our side
16:13:19 <gibi> bauzas: if qemu devs could point out a workaround that is implementable on nova side then we can release that
16:13:21 <lyarwood> there's nothing in Nova at the moment but I think we still need to track this to remove tests from the blacklist before GA if QEMU is fixed
16:14:02 <sean-k-mooney> ok so maybe just drop critical and make it high
16:14:16 <lyarwood> sure if it's going to stop us going around in circles ;D
16:14:23 <sean-k-mooney> :)
16:14:38 <gibi> lets back to this next week
16:14:41 <gibi> moving on
16:14:44 <bauzas> the point is, we're going to track this bugreport as something due for RC1
16:14:59 <bauzas> but if that doesn't bother people here...
16:15:06 <bauzas> :)
16:15:18 <gibi> I think we would like to track it
16:15:22 <gibi> anyhow
16:15:33 <gibi> beside this single bug there are  36 new untriaged bugs (+7 since the last meeting): https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New
16:15:51 <gibi> so there are inflow we have to look at
16:15:54 <gibi> and triage
16:16:07 <gibi> if you see regressions then please tag them with victoria-rc-potential
16:16:10 <gmann> yeah that is one the blocker for Focal migration - http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017104.html
16:16:30 <bauzas> gibi: I'm on bug triaging next week
16:16:47 <gibi> bauzas: thanks. I will also try. I will let you know when taking the baton
16:16:53 <bauzas> hopefully the number will decrease
16:16:59 <bauzas> gibi: sure
16:17:19 <gibi> any other bug we need to talk about here?
16:18:17 <gibi> #topic Runways
16:18:23 <gibi> etherpad #link https://etherpad.opendev.org/p/nova-runways-victoria
16:18:31 <gibi> runways is being closed now for Victoria due to feature freeze
16:18:57 <gibi> thanks for participating in it
16:19:08 <gibi> #topic Release Planning
16:19:17 <gibi> we have Feature Freeze today
16:19:22 <gibi> as of Milestone 3
16:19:30 <gibi> Feature Freeze Exception can be requested on the ML. See my mail #link http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017111.html
16:19:52 <gibi> I don't expect a long list of requests
16:20:07 <gibi> we have the release tracking etherpad #link https://etherpad.opendev.org/p/nova-victoria-rc-potential
16:20:25 <gibi> please add release TODOs there
16:20:31 <gibi> as well as critical patches
16:20:48 <gibi> we have the last client library release for victoria today, so python-novaclient has been released #link https://review.opendev.org/#/c/750429/
16:21:06 <gibi> We need to produce release notes prelude before RC1 (in two weeks), bauzas started to write it up
16:21:29 <gibi> We need to set RPC version aliases before RC1
16:21:33 <bauzas> gibi: I'll provide the change number before the meeting end
16:21:38 <gibi> bauzas: thanks!
16:22:13 <bauzas> gibi: remember we had a discussion last cycle on RPC major versioning
16:22:25 <gibi> bauzas: yeah I started on it but lost focuse
16:22:27 <bauzas> and we said it was quite a large effort for a little gain
16:22:36 <bauzas> should we try to make it real ?
16:22:46 <gibi> I will re-try it in W
16:22:51 <bauzas> I can take a look if that helps
16:23:22 <gibi> let's get back to this early W, I will add a line to the PTG etherpad about it
16:23:23 <bauzas> gibi: we're two weeks from RC1, maybe we can try it
16:23:29 <bauzas> kk
16:23:43 <bauzas> I used to write some of such changes in the past
16:23:50 <gibi> last cycle we discussed that this should be started lot earlier than M3
16:23:51 <bauzas> I'm maybe rusty but I feel brave enough
16:23:55 <bauzas> right
16:24:02 <bauzas> but I forgot
16:24:17 <bauzas> well, the point is, you need at least two changes
16:24:30 <gibi> yepp
16:24:56 <gibi> anything else about the coming release?
16:25:53 <gibi> #topic Stable Branches
16:25:59 <gibi> any news?
16:27:27 <elod> nothing much from my side. patches landed slowly but steady since last week.
16:27:35 <gibi> thanks elod
16:27:39 <gibi> #topic Sub/related team Highlights
16:27:45 <gibi> API (gmann)
16:27:52 <gmann> policy file json to yaml migration patches are merged.
16:28:02 <gmann> we have officially deprecated the JSON format of the policy file (but having a fallback to old default of json file).
16:28:21 <gmann> also upgrade check is added for detect the JSON formatted file. sometime in future, we will be able to remove the JSON support itself and can have erorr from oslo side.
16:28:23 <gmann> error
16:28:36 <gibi> cool
16:28:43 <gmann> but from nova side, all work is done.
16:28:48 <stephenfin> \o/
16:28:55 <gibi> thanks
16:29:14 <gibi> Libvirt (bauzas)
16:29:27 <bauzas> nothing to report but lyarwood had a thing last week IIRC
16:29:38 <bauzas> which was... versioning !
16:29:40 <lyarwood> just the QEMU and libvirt version bumps
16:29:48 <lyarwood> but that's on hold until we move to focal
16:29:59 <lyarwood> reviews still welcome
16:30:00 <bauzas> which makes sense
16:30:06 <bauzas> lyarwood: link ?
16:30:09 <lyarwood> https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bump-libvirt-qemu-victoria
16:30:11 <bauzas> ++
16:30:32 <gibi> thanks
16:30:53 <gibi> #topic PTG and Forum planning
16:30:57 <stephenfin> lyarwood: I thought I'd all those done?
16:31:03 <stephenfin> wait, nvm
16:31:30 <gibi> I've booked PTG slots for us see mail #link http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017063.html
16:31:37 <gibi> please collect PTG topics in #link https://etherpad.opendev.org/p/nova-wallaby-ptg
16:32:02 <gibi> if you need cross project sessions please let me know
16:32:12 <gibi> cyborg already did
16:32:29 <gibi> any other PTG related topic?
16:32:31 <sean-k-mooney> im not sure if we will need with with cinder or neutron
16:32:44 <gibi> at least no topics yet
16:33:19 <gibi> #topic Open discussion
16:33:22 <gibi> I have one
16:33:32 <gibi> I will approve the vmware undeprecation patch after this meeting if no objection. #link https://review.opendev.org/#/c/742407/
16:33:56 <gibi> so any objections?
16:33:59 <sean-k-mooney> we have one potential downstream request realted to ceph that would require cinder/nova coperateion but at present i think its premature to add to the ptg for what its worth
16:34:22 <lyarwood> sean-k-mooney: the users thing?
16:34:23 <gibi> sean-k-mooney: ack, just plug it to the ethepad if it become serious
16:34:27 <sean-k-mooney> lyarwood: yes
16:35:12 <gibi> so vmware going once
16:35:25 <gibi> going twice
16:35:33 <gibi> approved
16:35:39 <sean-k-mooney> cool
16:35:43 <bauzas> you're the PTL, dude
16:35:45 <bauzas> stand up
16:35:54 <bauzas> people had time to argue
16:36:15 <bauzas> :)
16:36:17 <gibi> :)
16:36:23 <gibi> any other open discussion topic?
16:37:20 <gibi> if not then thanks for joining today
16:37:29 <gibi> it is beer time in the EU anyhow
16:37:42 <gibi> #endmeeting