Thursday, 2020-11-26

*** jlibosva has joined #openstack-meeting-300:21
*** jlibosva has quit IRC00:29
*** DinaBelova has quit IRC00:55
*** freerunner has quit IRC00:55
*** freerunner has joined #openstack-meeting-300:55
*** DinaBelova has joined #openstack-meeting-300:55
*** freerunner has quit IRC01:16
*** freerunner has joined #openstack-meeting-301:16
*** baojg has joined #openstack-meeting-301:50
*** jlibosva has joined #openstack-meeting-302:15
*** jlibosva has quit IRC02:20
*** baojg has quit IRC02:39
*** hemanth_n has joined #openstack-meeting-302:51
*** hemanth_n has quit IRC02:58
*** baojg has joined #openstack-meeting-303:02
*** hemanth_n has joined #openstack-meeting-303:20
*** hemanth_n has quit IRC03:25
*** psachin has joined #openstack-meeting-303:31
*** psachin has quit IRC03:32
*** psachin has joined #openstack-meeting-303:33
*** lifeless has quit IRC03:41
*** lifeless has joined #openstack-meeting-303:42
*** jlibosva has joined #openstack-meeting-304:05
*** jlibosva has quit IRC04:10
*** baojg has quit IRC04:18
*** jlibosva has joined #openstack-meeting-304:39
*** jlibosva has quit IRC04:44
*** baojg has joined #openstack-meeting-305:04
*** jlibosva has joined #openstack-meeting-305:12
*** hemanth_n has joined #openstack-meeting-305:13
*** jlibosva has quit IRC05:16
*** jlibosva has joined #openstack-meeting-306:06
*** jlibosva has quit IRC06:10
*** baojg has quit IRC06:14
*** jlibosva has joined #openstack-meeting-306:15
*** baojg has joined #openstack-meeting-306:17
*** jlibosva has quit IRC06:20
*** jlibosva has joined #openstack-meeting-306:56
*** e0ne has joined #openstack-meeting-306:58
*** belmoreira has joined #openstack-meeting-306:59
*** ralonsoh has joined #openstack-meeting-307:07
*** e0ne has quit IRC07:09
*** slaweq has joined #openstack-meeting-307:11
*** hemanth_n has quit IRC07:29
*** hemanth_n has joined #openstack-meeting-307:29
*** e0ne has joined #openstack-meeting-307:34
*** e0ne has quit IRC08:05
*** e0ne has joined #openstack-meeting-308:10
*** tosky has joined #openstack-meeting-308:37
*** e0ne has quit IRC09:15
*** e0ne has joined #openstack-meeting-309:41
*** slaweq has quit IRC09:45
*** slaweq has joined #openstack-meeting-309:46
*** lpetrut has joined #openstack-meeting-310:39
*** ricolin has quit IRC11:34
*** jlibosva has quit IRC11:39
*** hemanth_n has quit IRC11:39
*** hemanth_n has joined #openstack-meeting-311:40
*** raildo has joined #openstack-meeting-311:45
*** e0ne has quit IRC12:08
*** e0ne has joined #openstack-meeting-312:23
*** ianychoi_ has joined #openstack-meeting-312:35
*** ianychoi has quit IRC12:36
*** lpetrut has quit IRC13:08
*** lpetrut has joined #openstack-meeting-313:10
*** slaweq has quit IRC13:17
*** slaweq has joined #openstack-meeting-313:20
*** hemanth_n has quit IRC13:45
*** e0ne has quit IRC13:52
*** baojg has quit IRC14:11
*** e0ne has joined #openstack-meeting-314:52
*** frickler is now known as frickler_pto15:02
*** lpetrut has quit IRC15:33
*** lpetrut has joined #openstack-meeting-315:48
*** elod has joined #openstack-meeting-315:58
gibi#startmeeting nova16:00
openstackMeeting started Thu Nov 26 16:00:10 2020 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: nova)"16:00
openstackThe meeting name has been set to 'nova'16:00
gibio/16:00
lpetruto/16:00
stephenfino/16:01
bauzas\o16:01
gibi#topic Bugs (stuck/critical)16:02
*** openstack changes topic to "Bugs (stuck/critical) (Meeting topic: nova)"16:02
gibino critical bug16:02
gibi#link 14 new untriaged bugs (+1 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New16:02
gibiit is a steady increase +1/week16:02
gibiso we would only need a bit more effor to keep it stable16:02
gibi#link 75 bugs are in INPROGRESS state without any tag (+0 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?field.tag=-*&field.status%3Alist=INPROGRESS16:03
*** macz_ has joined #openstack-meeting-316:03
gibithese are potentially un-triaged bugs. Check if they are still valid16:03
lyarwood\o16:03
gibiI'm traking these to try not to forget they exists16:03
*** sean-k-mooney has joined #openstack-meeting-316:03
gibiany specific bug we need to talk about?16:04
lyarwoodnot a Nova bug16:04
lyarwoodbut https://bugs.launchpad.net/tempest/+bug/1905725 is blocking the ceph job at the moment on master16:05
openstackLaunchpad bug 1905725 in tempest "test_attach_scsi_disk_with_config_drive fails during cleanup when using RBD for Glance and Nova" [Undecided,New] - Assigned to Lee Yarwood (lyarwood)16:05
lyarwoodhttps://review.opendev.org/c/openstack/tempest/+/764337 posted and I've asked for reviews in #openstack-qa16:05
gibiI hope gmann can look at it16:05
gibifor me the fix looks sensible16:06
gibiany other bugs?16:07
gibi#topic Gate status16:08
*** openstack changes topic to "Gate status (Meeting topic: nova)"16:08
gibiClassification rate 24% (-4 since the last meeting) #link http://status.openstack.org/elastic-recheck/data/integrated_gate.html16:08
lyarwoodI'll add an er for that last issue after the meeting16:08
gibilyarwood: thanks16:08
gibithis is pretty low, and got even lower this week, meaning we have a lot of failures that has no e-r signature16:09
gibiPlease look at the gate failures, file a bug, and add an elastic-recheck signature in the opendev/elastic-recheck repo (example: #link https://review.opendev.org/#/c/759967)16:09
gibiany gate bug we need to talk about?16:09
gibipersonally I got hit by https://bugs.launchpad.net/nova/+bug/1823251 this week so I spend days running func tests locally in random order but I still cannot reproduce it locally16:10
openstackLaunchpad bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed]16:10
*** slaweq has quit IRC16:10
gibiwhat I see is that the test cases in question takes a long time to run on the gate but also the runtime is varying a lot16:11
stephenfinI've seen a couple of the unit test failures due to eventlet, but I don't think there's much to be done about that that isn't serious work16:11
gibilike 50sec - 800sec16:11
lyarwoodgibi: yeah I've also hit that one16:11
lyarwoodand the negative evacuation bug elod has listed in the stable section of the meeting16:11
gibilyarwood: for the evac one there is a comment in the bug with a suggestion16:12
*** slaweq has joined #openstack-meeting-316:13
gibior at least with an observation16:13
lyarwoodgibi: ack yeah I'm looking at it now, it's weird how this is being seen more on the older bionic nodes16:13
lyarwoodgibi: I'll push an ER and update the change shortly16:14
gibicool16:14
gibithanks16:14
gibiany other gate failure we need to discuss?16:14
gmanno/16:15
gibi#topic Release Planning16:15
*** openstack changes topic to "Release Planning (Meeting topic: nova)"16:15
gmann+2 on 764337.  asked other  core to +A it16:15
gibigmann: thanks!16:15
gibiWallaby Milestone 1 is on 3rd of December, which is 1 weeks from now16:15
gibiA second spec review day has been proposed for 1st of Dec: #link http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018932.html16:15
gibiwhich is next Tuesday16:16
gibiis there any other release specific info?16:16
gibi#topic Stable Branches16:17
*** openstack changes topic to "Stable Branches (Meeting topic: nova)"16:17
gibielod (I assume) left a comment16:17
gibistable/victoria seems to be blocked by bug in live-migration job: https://bugs.launchpad.net/nova/+bug/190397916:17
openstackLaunchpad bug 1903979 in OpenStack Compute (nova) "nova-live-migration job fails during evacuate negative test" [High,In progress] - Assigned to Lee Yarwood (lyarwood)16:17
gibithis bug we dicussed above16:17
gibianything else about stable?16:17
gibi#topic Sub/related team Highlights16:18
*** openstack changes topic to "Sub/related team Highlights (Meeting topic: nova)"16:18
gibiLibvirt (bauzas)16:19
bauzasnothing to report, sir.16:19
*** baojg has joined #openstack-meeting-316:19
gibithen moving on16:19
gibi#topic Open discussion16:19
*** openstack changes topic to "Open discussion (Meeting topic: nova)"16:19
gibiwe have two items16:19
gibi(lpetrut): Hyper-V RBD support16:19
gibiThe upcoming Ceph release will include RBD Windows support, so we'd like to add Hyper-V driver support16:19
gibispecless bp: https://blueprints.launchpad.net/nova/+spec/hyperv-rbd16:19
gibilpetrut: as far as I understand it only impact the hyperv driver16:20
lpetrutyep16:20
lpetrutthe nova changes are minimal16:20
lpetrutbasically just letting the driver know that RBD is supported while os-brick handles the plumbing16:20
sean-k-mooneymy main concern was testing16:20
sean-k-mooneyi think it makes sense to proceed as a specless blueprint16:21
sean-k-mooneybut it would be nice to see it tested in third party ci16:21
sean-k-mooneyyou expalined you plan to test it in os-brick16:21
sean-k-mooneyso my main question is does the nova team feel that is enough16:21
sean-k-mooneyor do we want the same job to run on changes to the hyperv driver16:21
lpetrutwe already have a Hyper-V CI testing all the OpenStack projects that support Windows (e.g. nova, neutron, cinder, os-brick)16:22
sean-k-mooneylpetrut: yep but you are only doing lvm testing16:22
lyarwoodsean-k-mooney: nope, they use os-brick on Windows now16:22
lyarwoodsean-k-mooney: so I assume this is using their full blown third party setup16:22
lpetrutwhat Sean's saying is that the Nova job only covers iSCSI while we have os-brick test jobs for other backends16:23
sean-k-mooneyyep16:23
sean-k-mooneythere are 3 os-brick jobs for hyperv16:23
sean-k-mooneyonly the isusi one triggers on nova16:23
sean-k-mooneyif we are fine with that then cool16:23
lpetrutwe're trying to limit the number of jobs because of the available infrastructure. any nova change that breaks other storage backends will be caught by os-brick16:24
sean-k-mooneyonly after the change merges16:24
lpetrutthat's ok, 3rd party CIs aren't voting anyway16:24
sean-k-mooneyright but we do look at them16:24
sean-k-mooneyas i siad im +1 on this porposal16:25
sean-k-mooneyjust wanted to bring up that point16:25
lyarwoodah right sorry misunderstood the issue, I'd assume we would want a simple third party job still to catch any virt driver breakage early16:25
lyarwoodwithin Nova16:25
lpetrutsean-k-mooney: definitely, thanks for raising those concerns16:25
sean-k-mooneylyarwood:that was basically what i was asking16:26
lyarwoodright, I'd vote to have it assuming rbd will be as popular with hyperv as it is with libvirt16:26
lyarwoodbut it shouldn't block it either way16:26
lyarwoodas long as it's covered somewhere16:26
sean-k-mooneyyep if capstity is a concern then i think its fine to rely on os-brick but even a nightly or weekly nova run of all 3 jobs might be nice16:27
gibiI'm OK to go forward with the bp. lpetrut if it is possible try to provide CI coverage in nova for the RBD track too16:27
gibia periodic run like sean-k-mooney suggest could be a minimal extra load on your infra16:28
lpetrutgibi: you mean testing RBD for one in every X patches ?16:28
lpetrutthat would work16:28
sean-k-mooneylpetrut: yep so trigger a run once a week or once a night16:28
gibilpetrut: running the RBD job once a day16:29
gibior so16:29
*** e0ne has quit IRC16:29
lpetrutok, we can do that16:29
sean-k-mooneylpetrut: its mainly to narrow down the set of possible problem patches16:29
gibithen I don't see any objects. So I will approve the bp without spec after the meeting16:30
gibilpetrut: thanks for pushing tihs16:30
lpetrutthanks!16:30
sean-k-mooneyhttps://review.opendev.org/c/openstack/nova/+/763550 is the change for those that are intersted in reviewing16:30
gibisean-k-mooney: you have a topic too16:31
lpetrutone difference compared to libvirt is that we're mounting the disk, hyper-v can't use librbd as qemu does. we're exposing a virtual SCSI disk16:31
sean-k-mooneyyes basiclly for the libvirt driver we currentl use the metadat element in the xml to store some useful info for debugging16:31
lpetrutgibi: I've just set the topic to "hyperv-rbd"16:32
gibilpetrut: ack thanks, the implementation details can be discussed in the review16:32
gibisean-k-mooney: yes16:32
lpetrutdefinitely, just thought that it might be worth mentioning16:32
sean-k-mooneywas waiting for previous topic to finish16:33
gibilpetrut: thanks16:33
gibisean-k-mooney: I think we can move to your topic16:33
sean-k-mooneylpetrut: we do somethign simiar in livbirt in that the gust sees a virtio device or scie device depneing on the bus selected anyway i dont think that is a problem16:33
sean-k-mooneycool16:33
sean-k-mooneyam so ya today we store info like the flavor name and how many cpus we have in the medata in the libvit xml16:34
sean-k-mooneybut we dont store the flavor extra spec or image properties16:34
sean-k-mooneywhen we review reports form cosutoem we often have the logs16:35
sean-k-mooneywith the xmls but no access to the api to get the flavors16:35
sean-k-mooneyso i want to start storign the falvor extra specs and image prperties in the xml16:35
sean-k-mooneyand normalise soem ot the other element16:35
sean-k-mooneyfor example we have the flavor name but not uuid but image uuid but not name16:35
sean-k-mooneyso i want to put both16:35
gibimake sense to me16:36
sean-k-mooneyi would like to do this as a specless blueprint and discuss in the code patch but i can do a small spec if required16:36
lyarwoodI'm okay with that if it doesn't bloat the XML16:36
lyarwoodotherwise we can just log it at an INFO level within the driver16:36
lyarwoodduring the build16:36
sean-k-mooneyit should not extend it much16:36
lyarwoodbut I guess that doesn't persist as well16:36
lyarwoodas logs rotate etc16:37
sean-k-mooneywe dont typically have more then ~10 extra specs16:37
lyarwoodkk16:37
sean-k-mooneyand the xml is only logged at debug level16:37
lyarwoodright but if we ever need it again we can just dump it from the domain16:37
gibisean-k-mooney: I'm OK to have this as a specless bp as this only change a freeform part of the domain xml16:37
sean-k-mooneyyep its technically not part of any public api we provide its just debug metadata16:38
sean-k-mooneyok ill file the blueprint after the meeting and submit a patch16:38
sean-k-mooneywe can discuss futher in gerrit and if there are concerns we can create a spec if need16:39
gibialso as we discussed earlier when the flavor and image data changes then also the domain is regenerated16:39
gibiso it is easy to keep it up tod ate16:39
gibito date16:39
sean-k-mooneyyep in rebuild and resize we already regerneate teh xml and metadata16:39
gibiOk16:40
gibianyhing else for today?16:40
sean-k-mooneythat looks like a no16:41
sean-k-mooneynothing else from me in anycase16:41
gibithen thanks for joining today16:41
sean-k-mooneyo/16:41
gibihave a nice rest of the day!16:41
*** sean-k-mooney has left #openstack-meeting-316:41
gibi#endmeeting16:41
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:41
openstackMeeting ended Thu Nov 26 16:41:48 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:41
openstackMinutes:        http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-11-26-16.00.html16:41
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-11-26-16.00.txt16:41
openstackLog:            http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-11-26-16.00.log.html16:41
*** psachin has quit IRC16:52
*** ricolin has joined #openstack-meeting-317:02
*** tosky has quit IRC17:17
*** belmoreira has quit IRC18:46
*** e0ne has joined #openstack-meeting-319:32
*** e0ne has quit IRC19:32
*** e0ne has joined #openstack-meeting-319:48
*** e0ne has quit IRC19:52
*** e0ne has joined #openstack-meeting-320:02
*** lpetrut has quit IRC20:02
*** e0ne has quit IRC20:03
*** e0ne has joined #openstack-meeting-320:03
*** e0ne has quit IRC21:04
*** tosky has joined #openstack-meeting-321:09
*** raildo has quit IRC21:31
*** slaweq has quit IRC21:45
*** slaweq has joined #openstack-meeting-321:47
*** e0ne has joined #openstack-meeting-322:25
*** slaweq has quit IRC22:26
*** e0ne has quit IRC22:42
*** e0ne has joined #openstack-meeting-322:56
*** e0ne has quit IRC22:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!