*** gyee has quit IRC | 00:27 | |
*** brinzhang has joined #openstack-nova | 00:35 | |
*** ociuhandu has joined #openstack-nova | 00:46 | |
*** sapd1 has joined #openstack-nova | 00:48 | |
*** mlavalle has quit IRC | 00:50 | |
*** ociuhandu has quit IRC | 00:50 | |
*** mlavalle has joined #openstack-nova | 00:50 | |
*** Liang__ has joined #openstack-nova | 01:24 | |
*** yingjisun has joined #openstack-nova | 01:37 | |
*** songwenping__ has quit IRC | 01:52 | |
*** songwenping_ has quit IRC | 01:52 | |
*** Liang__ has quit IRC | 02:26 | |
*** Liang__ has joined #openstack-nova | 02:27 | |
*** rcernin has quit IRC | 02:38 | |
*** zzzeek has quit IRC | 02:40 | |
*** zzzeek has joined #openstack-nova | 02:41 | |
*** rcernin has joined #openstack-nova | 02:41 | |
*** sapd1 has quit IRC | 02:47 | |
*** sapd1 has joined #openstack-nova | 02:47 | |
*** zzzeek has quit IRC | 02:50 | |
*** zzzeek has joined #openstack-nova | 02:52 | |
*** efried has quit IRC | 02:55 | |
openstackgerrit | Rui Zang proposed openstack/nova master: DO NOT REVIEW https://review.opendev.org/753919 | 03:00 |
---|---|---|
*** Yumeng has joined #openstack-nova | 03:00 | |
*** xinranwang__ has joined #openstack-nova | 03:06 | |
*** psachin has joined #openstack-nova | 03:27 | |
*** yingjisun has quit IRC | 03:46 | |
*** dave-mccowan has quit IRC | 03:47 | |
*** aj_mailing has quit IRC | 03:53 | |
*** ratailor has joined #openstack-nova | 03:56 | |
*** evrardjp has quit IRC | 04:33 | |
*** evrardjp has joined #openstack-nova | 04:33 | |
openstackgerrit | Hemanth Nakkina proposed openstack/nova master: Update pci stat pools based on PCI device changes https://review.opendev.org/749175 | 04:34 |
*** brinzhang_ has joined #openstack-nova | 04:37 | |
*** brinzhang has quit IRC | 04:41 | |
*** psachin has quit IRC | 04:45 | |
*** yingjisun has joined #openstack-nova | 04:58 | |
*** aj_mailing has joined #openstack-nova | 05:03 | |
*** HagunKim has joined #openstack-nova | 05:54 | |
*** maciejjozefczyk has joined #openstack-nova | 06:07 | |
*** lpetrut has joined #openstack-nova | 06:10 | |
*** slaweq has joined #openstack-nova | 06:11 | |
*** maciejjozefczyk_ has joined #openstack-nova | 06:15 | |
*** maciejjozefczyk has quit IRC | 06:17 | |
*** slaweq has quit IRC | 06:29 | |
*** vishalmanchanda has joined #openstack-nova | 06:30 | |
*** slaweq has joined #openstack-nova | 06:32 | |
*** aj_mailing has quit IRC | 06:35 | |
*** ralonsoh has joined #openstack-nova | 06:36 | |
*** dklyle has quit IRC | 06:48 | |
*** belmoreira has joined #openstack-nova | 06:54 | |
*** aj_mailing has joined #openstack-nova | 07:08 | |
*** nightmare_unreal has joined #openstack-nova | 07:09 | |
*** links has joined #openstack-nova | 07:16 | |
gibi | I have to be offline for the next 2-3 hours but I will be back after lunch (UTC 11) | 07:33 |
*** zzzeek has quit IRC | 07:33 | |
*** zzzeek has joined #openstack-nova | 07:36 | |
*** martinkennelly has joined #openstack-nova | 07:41 | |
*** xek has joined #openstack-nova | 07:43 | |
*** dtantsur|afk is now known as dtantsur | 07:47 | |
*** ratailor has quit IRC | 07:50 | |
*** zzzeek has quit IRC | 08:01 | |
*** zzzeek has joined #openstack-nova | 08:01 | |
*** tosky has joined #openstack-nova | 08:04 | |
*** k_mouza has joined #openstack-nova | 08:09 | |
*** ralonsoh has quit IRC | 08:23 | |
*** ralonsoh has joined #openstack-nova | 08:24 | |
openstackgerrit | Lucas Alvares Gomes proposed openstack/nova master: DO NOT REVIEW: Test OVN devstack module https://review.opendev.org/748226 | 08:26 |
*** rcernin has quit IRC | 08:35 | |
*** derekh has joined #openstack-nova | 08:38 | |
*** rcernin has joined #openstack-nova | 08:41 | |
*** k_mouza has quit IRC | 08:49 | |
*** k_mouza has joined #openstack-nova | 08:50 | |
*** rcernin has quit IRC | 08:57 | |
*** martinkennelly has quit IRC | 09:03 | |
*** maciejjozefczyk has joined #openstack-nova | 09:06 | |
*** maciejjozefczyk_ has quit IRC | 09:08 | |
*** spatel has joined #openstack-nova | 09:14 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Replace nova-live-migration with zuulv3 jobs https://review.opendev.org/752557 | 09:14 |
lyarwood | stephenfin: https://review.opendev.org/#/c/744883 - would you mind hitting this back into the gate | 09:15 |
stephenfin | sure, looking | 09:16 |
lyarwood | had to be rebased to pickup the test-evacuate.sh change as the original live-migration job is still being used until https://review.opendev.org/752557 lands | 09:16 |
stephenfin | yup, makes sense | 09:17 |
stephenfin | +W | 09:17 |
*** spatel has quit IRC | 09:18 | |
*** LinPeiWen has quit IRC | 09:26 | |
*** LinPeiWen has joined #openstack-nova | 09:26 | |
*** ociuhandu has joined #openstack-nova | 09:29 | |
*** xinranwang__ has quit IRC | 09:34 | |
lyarwood | bauzas: https://review.opendev.org/#/c/753281/ - Morning would you mind landing this in stable/ussuri please? | 09:35 |
*** jangutter has joined #openstack-nova | 09:42 | |
*** dtantsur is now known as dtantsur|brb | 09:43 | |
CeeMac | lyarwood: morning :) I'm still trying to get my head around the process involved in the cinder retype initiated nova swap_volume. Is nova calling libvirt drive-mirror (or blockdev-mirror) to perform the sync, or is it relying on cinder copying/syncing the data to the new volume before doing an attachment swap? Trying to work out at which point the swap_volume job failure could be affecting the instance | 09:45 |
CeeMac | i've been reading through the driver.py and qemu block job docs | 09:46 |
*** jangutter_ has quit IRC | 09:46 | |
CeeMac | but I lack a lot of context unfortunately | 09:46 |
CeeMac | i'm also trying to work out if any of the operations rely on / prefer the qemu quest agent and connection being present | 09:47 |
*** adriant has quit IRC | 09:52 | |
*** adriant has joined #openstack-nova | 09:53 | |
*** zzzeek has quit IRC | 09:57 | |
*** zzzeek has joined #openstack-nova | 10:00 | |
*** xek has quit IRC | 10:01 | |
*** Liang__ has quit IRC | 10:04 | |
lyarwood | CeeMac: Morning sorry was afk | 10:06 |
*** aj_mailing has quit IRC | 10:07 | |
lyarwood | CeeMac: so we call the https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockRebase API at the moment (if libvirt is < 6.0.0) | 10:07 |
lyarwood | CeeMac: AFAIK we don't use the guest agent at all during the process | 10:08 |
lyarwood | CeeMac: in terms of https://bugs.launchpad.net/nova/+bug/1896621 the issue is with parallel requests to retype multiple attachments to the same instance AFAICT | 10:09 |
openstack | Launchpad bug 1896621 in OpenStack Compute (nova) "instance corrupted after volume retype" [High,Confirmed] - Assigned to Lee Yarwood (lyarwood) | 10:09 |
lyarwood | CeeMac: virDomainBlockRebase is disk specific so that isn't an issue | 10:09 |
lyarwood | CeeMac: but the final thing we do when swapping disks is to update the persistent config for the overall domain | 10:10 |
lyarwood | CeeMac: AFAICT this fails when other calls to virDomainBlockRebase and thus QEMU block jobs are still active | 10:10 |
lyarwood | CeeMac: I was going to throw an instance wide lock on the compute _swap_volume call to avoid this as I can reproduce the failure with libvirt > 6.0.0 where we now use the https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockCopy libvirt API | 10:12 |
*** martinkennelly has joined #openstack-nova | 10:16 | |
lyarwood | CeeMac: I'll try to find some time this afternoon to post a WIP | 10:19 |
lyarwood | CeeMac: I'll also try to work out if we can recover from this in your env, I've got a feeling it might be possible if we can work out the details of the new volume and attach it somehow to the instance | 10:21 |
*** rcernin has joined #openstack-nova | 10:35 | |
*** salmankhan has joined #openstack-nova | 10:42 | |
*** maciejjozefczyk_ has joined #openstack-nova | 10:45 | |
brinzhang_ | stephenfin: thanks, yes, I missed sth, updated, and replied in the patch :D | 10:47 |
*** maciejjozefczyk has quit IRC | 10:48 | |
*** rcernin has quit IRC | 10:56 | |
*** dtantsur|brb is now known as dtantsur | 10:57 | |
CeeMac | lyarwood: sorry, was just in a long meeting. thanks for the notes, I'll do some more reading! :) | 10:58 |
CeeMac | lyarwood: so, i wonder if it is the domain persistent config update. Or if it is just a direct result of the multiple block jobs, I need to stand up a disposable environment to validate if this affects all instances or specific ones | 10:59 |
CeeMac | lyarwood: for reference, i seem to have the most issue with Microsoft Active Directory Controllers. They really don't seem to like a lot of the normal operations. removing/recreating the instance from the same boot volume causes the same level of corruption. As does deleting the instance and recreating it from a nova snapshot. | 11:01 |
CeeMac | lyarwood: I had to delete the affected instance in the production environment, but I think I still have a corrupt instance in my dev area I could hold off on killing and rebuilding | 11:02 |
*** sapd1 has quit IRC | 11:03 | |
*** JamesBenson has joined #openstack-nova | 11:06 | |
*** JamesBenson has quit IRC | 11:06 | |
*** JamesBenson has joined #openstack-nova | 11:06 | |
CeeMac | lyarwood: so if I'm understanding the docs correctly, you're using VIR_DOMAIN_BLOCK_REBASE_REUSE_EXT with VIR_DOMAIN_BLOCK_REBASE_COPY_DEV to point the rebase job at a volume device that has already been created and syning the content across to it? | 11:15 |
lyarwood | CeeMac: yes, I believe some cinder-volume backends clone things in the background so we don't end up copying everything across | 11:22 |
lyarwood | CeeMac: just the delta but backends like the LVM/iSCSI backend don't do this | 11:22 |
lyarwood | CeeMac: we then switch the volumes around within Cinder | 11:23 |
lyarwood | CeeMac: I think that's likely led to the corruption as we are pointing at the old likely unmapped volume | 11:23 |
lyarwood | CeeMac: but I'll need to reproduce to confirm | 11:23 |
CeeMac | that would make sense for the case where I've been able to log on to the instance fine but it broke after a reboot | 11:24 |
CeeMac | i'll have a poke around the configurations of one of the corrupt instances see if I can compare a dumpxml of attached volumes against expected attached volumes | 11:25 |
gibi | cores, I see the libvirt cleanup https://review.opendev.org/#/q/topic:bump-libvirt-qemu-victoria and the live migration / evacuation part of the zuul migration https://review.opendev.org/#/q/topic:native-zuulv3-migration+project:openstack/nova still actively pushed before RC1 | 11:54 |
gibi | is there anything else that I need to keep track of before / when I update the RC1 release patch? | 11:55 |
gibi | are we OK if they are not made it to RC1? | 11:55 |
gibi | I think the libvirt bump is merged just the cleanup is open so that is OK if we are not able to merge it before RC1 | 11:55 |
gibi | but what about the zuul migration? | 11:55 |
gibi | can we not merge them before RC1? | 11:56 |
gibi | gmann, lyarwood: ^^ | 11:56 |
lyarwood | gibi: I'm fine with the cleanups not landing | 11:58 |
lyarwood | gibi: for zuulv3 I'll defer to gmann | 11:58 |
gibi | thanks, noted | 11:58 |
gibi | I would like to make the RC1 right after the weekly meeting if possible | 11:58 |
gibi | but if things are on the gate then I think we can wait for them | 11:59 |
*** priteau has joined #openstack-nova | 12:00 | |
lyarwood | gibi: yeah I'd only wait for the zuulv3 stuff and only if gmann confirms that we need to tbh | 12:00 |
gibi | ack | 12:01 |
gibi | thanks | 12:01 |
kashyap | lyarwood: Where can I update this DevStack job using F31 (which is EOL) to F32? - https://zuul.opendev.org/t/vexxhost/job/devstack-platform-fedora-latest-virt-preview# | 12:06 |
kashyap | I'm wading through different CI / DevStack repos, thought I'd ask here first | 12:06 |
*** adrianc has quit IRC | 12:06 | |
lyarwood | kashyap: I'm trying to do that https://review.opendev.org/#/c/750292/ already | 12:06 |
kashyap | Oh, good that I asked | 12:06 |
lyarwood | kashyap: https://review.opendev.org/#/q/topic:f32 for the full set of changes required FWIW | 12:07 |
kashyap | lyarwood: I was just fixing the conflicts here and rebasing the CI job (https://review.opendev.org/#/c/708701/) | 12:07 |
kashyap | lyarwood: Sweet | 12:07 |
* kashyap clicks | 12:07 | |
kashyap | Ah, phew; I already was thinking about 'd-i-b', I see that it's already merged | 12:08 |
*** zzzeek has quit IRC | 12:09 | |
sean-k-mooney | speaking of dib i need to talk to infra or in openstack-dib about alpine support | 12:09 |
sean-k-mooney | i have started on it but im hitting some issues | 12:10 |
kashyap | One would think using a image would be a simple "drag-n-drop" of a pre-built image cache (a.k.a. virt-builder) | 12:10 |
kashyap | ... instead of this kind of a 'diff' across eight files: https://review.opendev.org/#/c/737217/ | 12:10 |
kashyap | But, that ship has sailed eight years ago ;-) | 12:11 |
*** adrianc has joined #openstack-nova | 12:11 | |
sean-k-mooney | yep i really dislike virt-builder so im happy we use dib | 12:11 |
kashyap | sean-k-mooney: *Alpine*? I don't think it is one of the distros that matter | 12:11 |
kashyap | :D | 12:11 |
sean-k-mooney | kashyap: im trying to replace cirros | 12:11 |
kashyap | sean-k-mooney: What is your reason for dislike of 'virt-builder'? I think it does a solid job for its purpose | 12:11 |
sean-k-mooney | with a maintianed disto | 12:11 |
kashyap | And has saved my times countless times | 12:11 |
kashyap | It's well-documented, deadly simple to use, and it does many smart-and-useful things. | 12:12 |
*** zzzeek has joined #openstack-nova | 12:12 | |
kashyap | I don't see a single reason to dislike it :D | 12:12 |
sean-k-mooney | kashyap: i really dont like its workflow and it had rather poor supprot for non rpm distros orginailly | 12:12 |
sean-k-mooney | i want a declaritive interface which si what the element provide | 12:12 |
kashyap | sean-k-mooney: Even for non-RPM distros, it even lets you create Ubuntu and Debian images | 12:12 |
sean-k-mooney | kashyap: right but i hae always had issue trying to use it | 12:13 |
kashyap | Its main purpose is to allow you creating / modifying images _quickly_ | 12:13 |
kashyap | What exact issue? I'm surprised such a simple tool fails for you | 12:13 |
sean-k-mooney | for me its never been quick and i dont find its docs that easy to folow | 12:13 |
sean-k-mooney | the first road block whas always been user managment | 12:14 |
sean-k-mooney | injection of ssh keys ectra | 12:14 |
sean-k-mooney | the package customisation is not that bad but it used to not work well behind proxies | 12:14 |
kashyap | - First, on quickness: it uses minimal kickstart for RPM-based, and similar for Debian-based | 12:14 |
kashyap | It can't get quicker than what the mirrors allow | 12:14 |
sean-k-mooney | so it was a pain to get working behind the intel firewall | 12:14 |
kashyap | - Second, the "docs" are quite readable too (very few open source tools can claim that): https://libguestfs.org/virt-builder.1.html | 12:15 |
kashyap | For Fedora users, I wrote this dead-simple guide some years ago - https://developer.fedoraproject.org/tools/virt-builder/about.html | 12:15 |
sean-k-mooney | i can take a look but i have read the docs and i dont find them that useful | 12:16 |
sean-k-mooney | i tried virt builder before dib | 12:16 |
sean-k-mooney | i only tried dib after i could not get it to do what i wanted | 12:17 |
kashyap | sean-k-mooney: User management is also easy: | 12:17 |
kashyap | virt-builder [...] --firstboot-command ' | 12:17 |
kashyap | # Create the user account. | 12:17 |
kashyap | useradd -m -p "" -G wheel stack | 12:17 |
kashyap | chage -d 0 stack | 12:17 |
kashyap | chmod 0755 /home/stack | 12:17 |
kashyap | mkdir -m 0755 /home/stack/.ssh' | 12:17 |
sean-k-mooney | how do you do that declaritvly with out writing a bash script | 12:17 |
kashyap | sean-k-mooney: I know we're going deep into a non-Nova topic, but 'd-i-b' had the terrible practise of using root overall | 12:17 |
kashyap | sean-k-mooney: We discussed it when the tool was being written at that time; and pointed to better alternatives ... but they went NIH | 12:18 |
sean-k-mooney | it use a chroot | 12:18 |
sean-k-mooney | so root is not system root | 12:19 |
sean-k-mooney | but ya not a nova topic lets drop it | 12:19 |
kashyap | sean-k-mooney: It used to use system root, and we discussed the security implications at length. HP at that time wasn't convinced. | 12:20 |
kashyap | (Yeah, non-Nova, but indirectly very useful topic for Nova Infra :D) | 12:20 |
kashyap | sean-k-mooney: Not to belabour, but the "diskimage-builder" was so terribly unsafe, at that time the libguestfs folks went and created a "safe wrapper" for it, 'virt-dib': https://manpages.debian.org/testing/libguestfs-tools/virt-dib.1.en.html | 12:27 |
*** salmankhan has quit IRC | 12:28 | |
sean-k-mooney | i see well i still dont think we should replace it with virt-install | 12:33 |
*** nweinber has joined #openstack-nova | 12:33 | |
sean-k-mooney | *virt-builder | 12:33 |
sean-k-mooney | regardless of the implementaion the design is more approtiate for a git based declaritve workflow IMO | 12:34 |
CeeMac | lyarwood: quick question, when the domain persistence config is recreated following a successful swap_volume operation, would this generate any new SSIDs against or within the config? | 12:34 |
sean-k-mooney | which is why i wrote https://github.com/intel-orchestration-software/dib-elements a few year ago using it instead of virt builder | 12:34 |
*** zzzeek has quit IRC | 12:40 | |
kashyap | sean-k-mooney: Yeah, yeah ... I wasn't arguing we shouldn't replace it now (too tedious, and can't justify the effort). The purpose of both tools are different | 12:40 |
*** maciejjozefczyk has joined #openstack-nova | 12:41 | |
*** zzzeek has joined #openstack-nova | 12:42 | |
*** maciejjozefczyk_ has quit IRC | 12:44 | |
*** salmankhan has joined #openstack-nova | 12:44 | |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 12:56 |
kashyap | Cc: lyarwood --^ In my above modif, I've nuked the "check" queue altogether, as I see that the 'virt-preview' job is now added to the experimental queue upstream | 12:57 |
kashyap | (Also see that I've added your F32 support as a Depends-On. Let me know if I messed up anything in the above job config :-)) | 13:00 |
*** priteau has quit IRC | 13:02 | |
lyarwood | kashyap: yeah you can't do that | 13:02 |
lyarwood | kashyap: you need to leave a single job in the check queue | 13:02 |
lyarwood | kashyap: just move the virt-preview job there | 13:02 |
kashyap | lyarwood: Aah, okay. Noted. Lemme edit :) | 13:02 |
kashyap | lyarwood: Aside: Sorry, but your "You can't do that" reminded me of a certain president and his conversation with an Australian journalist ;-) | 13:03 |
*** eharney has joined #openstack-nova | 13:06 | |
kashyap | lyarwood: BTW, also should at least one job be present in the 'gate' queue? | 13:09 |
lyarwood | kashyap: yes | 13:10 |
lyarwood | brb | 13:10 |
kashyap | Thx | 13:11 |
kashyap | lyarwood: When you're back, that good? — | 13:12 |
kashyap | check: | 13:12 |
kashyap | jobs: | 13:12 |
kashyap | - devstack-platform-fedora-latest-virt-preview | 13:12 |
kashyap | gate: | 13:12 |
kashyap | jobs: | 13:12 |
kashyap | - nova-next | 13:12 |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 13:14 |
* kashyap bbiab | 13:14 | |
gmann | lyarwood: gibi : zuulv3 grenade is not must for RC. I can try to fix that today but if it goes in W branch then also no issue in backport to V | 13:15 |
*** sapd1 has joined #openstack-nova | 13:15 | |
gibi | gmann: thanks | 13:19 |
gibi | then I will check the gate after the weekly meeting and decide if anything is close, if not we deferr them | 13:20 |
gibi | gmann: does the same true for the other two zuul migration patch (live migration + evac) | 13:20 |
gmann | gibi: let me check | 13:20 |
*** priteau has joined #openstack-nova | 13:21 | |
*** priteau has quit IRC | 13:33 | |
*** rcernin has joined #openstack-nova | 13:35 | |
*** nweinber has quit IRC | 13:36 | |
gmann | lyarwood: gibi nova-grenade-multinode job disappear from run in -https://review.opendev.org/#/c/752557/6 | 13:36 |
gmann | commented on review | 13:36 |
*** nweinber has joined #openstack-nova | 13:36 | |
gibi | gmann: good point | 13:38 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1896463 in func env https://review.opendev.org/754100 | 13:41 |
openstack | bug 1896463 in OpenStack Compute (nova) "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,Confirmed] https://launchpad.net/bugs/1896463 - Assigned to Balazs Gibizer (balazs-gibizer) | 13:41 |
gibi | this is a nasty race ^^ | 13:41 |
sean-k-mooney | that a knonw issue | 13:43 |
sean-k-mooney | let me see if i can find the downstream bug | 13:44 |
lyarwood | gmann: urgh sorry, weird how zuul don't complain about that | 13:44 |
gmann | lyarwood: yeah, that was strange, it think it skipped running it as base job it found from some stable branch and made this job valid only for stable branch. but yes this is confusing | 13:45 |
lyarwood | loverly | 13:45 |
gmann | it should be error instead of skip | 13:46 |
sean-k-mooney | gibi https://bugzilla.redhat.com/show_bug.cgi?id=1852110 | 13:46 |
openstack | bugzilla.redhat.com bug 1852110 in openstack-nova "nova host-evacuation returns erroneous pci addresses and an error: Unable to correlate PCI slot" [High,New] - Assigned to smooney | 13:46 |
lyarwood | yeah agreed | 13:46 |
sean-k-mooney | gibi ill add https://bugs.launchpad.net/nova/+bug/1896463 ad the upstream track of the downstream bug | 13:47 |
openstack | Launchpad bug 1896463 in OpenStack Compute (nova) "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,Confirmed] - Assigned to Balazs Gibizer (balazs-gibizer) | 13:47 |
gibi | sean-k-mooney: thanks | 13:47 |
sean-k-mooney | it exist in qeens for what it worth | 13:48 |
sean-k-mooney | gibi: there are other similar races for retries to other host and a much of issues with shevle and other move operations | 13:49 |
sean-k-mooney | also all move operation involving PFs dont work as we never update the neutorn port mac | 13:49 |
*** xek has joined #openstack-nova | 13:50 | |
*** yingjisun has quit IRC | 13:50 | |
gibi | sean-k-mooney: yeah, the problem seems generic for every move that uses migration context and claims | 13:50 |
sean-k-mooney | yep | 13:51 |
*** rcernin has quit IRC | 13:51 | |
gibi | now I have a reproducer for evacuate | 13:51 |
sean-k-mooney | it should not affect live migration for what its worth | 13:51 |
sean-k-mooney | ya i did not have time to look into this yet since it was only reported downstream in august | 13:51 |
sean-k-mooney | having a repoducer will certenly help figure out the fix | 13:52 |
sean-k-mooney | this is the unshevle case i think https://bugzilla.redhat.com/show_bug.cgi?id=1767797 | 13:53 |
openstack | bugzilla.redhat.com bug 1767797 in openstack-nova "When unshelving an SR-IOV instance, the binding profile isn't reclaimed or rescheduled, and this might cause PCI-PT conflicts" [High,New] - Assigned to nova-maint | 13:53 |
sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1851545 | 13:53 |
openstack | Launchpad bug 1851545 in OpenStack Compute (nova) "Port update exception on nova unshelve for instance with PCI devices (part 2)" [Medium,Triaged] | 13:53 |
sean-k-mooney | it might be the same root cause. perhapse we coudl copy your repoducer for shelve and see it it causes it | 13:54 |
sean-k-mooney | gibi: ill try and review it later | 13:54 |
gibi | could be, I haven't checked the live migration relevance here | 13:55 |
sean-k-mooney | live migration does not use claims | 13:55 |
sean-k-mooney | for sriov devies | 13:55 |
sean-k-mooney | which si why i think it would be fine | 13:56 |
gibi | ahh, true, | 13:56 |
sean-k-mooney | pfs still wont work becaue of the mac issue but vfs should be fine | 13:57 |
gibi | yeah, I keep the PF-mac issue separate | 13:57 |
gibi | do we have a bug and a reproducer there? | 13:57 |
sean-k-mooney | while you are in a sriov context you might want to take a look at https://review.opendev.org/#/c/749175/ | 13:57 |
sean-k-mooney | i need to review the latest version but i think its more or less ready to merge after we reopen master and rc1 is done | 13:58 |
*** mriedem has joined #openstack-nova | 13:58 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056 | 13:59 |
gibi | sean-k-mooney: ack, I have to get back to that | 13:59 |
gibi | at some point I started reading it but got pulled away | 13:59 |
sean-k-mooney | i tested it shortly after your attach stuff merged and it did not break anything but i can repoduce the issue so that is all i could say | 14:04 |
*** efried has joined #openstack-nova | 14:05 | |
gibi | sean-k-mooney: do you mean you could not able to reproduce the bug before the fix? | 14:05 |
sean-k-mooney | yes it required the pci config space capablities to change for the device | 14:11 |
sean-k-mooney | that normally need a bios or firmware update | 14:11 |
sean-k-mooney | or change to the kernel/kernel_module paramters | 14:12 |
sean-k-mooney | gibi: for exampl if you put some intel nics into datacenter brinding mode it disable sriov capablity and repalces it with vmdq | 14:12 |
sean-k-mooney | that remove the virtual_funciotns capablity form the pci config space | 14:13 |
sean-k-mooney | meaning it would show up as type-PCI | 14:13 |
sean-k-mooney | if you put it back in sriov mode it woucl change to type-PF | 14:13 |
sean-k-mooney | but i do not have any hardwar that can do that to test with | 14:13 |
*** hemna has quit IRC | 14:13 | |
*** nweinber has quit IRC | 14:14 | |
*** nweinber has joined #openstack-nova | 14:14 | |
gibi | I see. When I get back to this bug I will check if I can test this or not in my SRIOV hardware | 14:16 |
*** nweinber has quit IRC | 14:20 | |
*** nweinber has joined #openstack-nova | 14:21 | |
*** hemna has joined #openstack-nova | 14:23 | |
*** maciejjozefczyk_ has joined #openstack-nova | 14:25 | |
sean-k-mooney | gibi: i asked for a functional test so we atleast have that | 14:25 |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056 | 14:26 |
*** maciejjozefczyk has quit IRC | 14:27 | |
*** dave-mccowan has joined #openstack-nova | 14:29 | |
gibi | sean-k-mooney: good point | 14:34 |
*** xek has quit IRC | 14:34 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056 | 14:34 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 14:34 |
*** dave-mccowan has quit IRC | 14:34 | |
lyarwood | kashyap: ^ fixed it up a little and removed the SCSI CDROM change dep as it isn't helping us test this | 14:35 |
gmann | lyarwood: ^^ let's see how it run. not using the script in grenade job. I have not disable the smoke test run on base and new cloud and only run live,cold migration and resize tests | 14:35 |
*** ralonsoh has quit IRC | 14:35 | |
lyarwood | gmann: ack looking | 14:35 |
lyarwood | gmann: zuul isn't happyt | 14:36 |
lyarwood | happy* | 14:36 |
gmann | yeah some syntax things, checking | 14:37 |
*** ralonsoh has joined #openstack-nova | 14:37 | |
*** dklyle has joined #openstack-nova | 14:37 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056 | 14:38 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 14:39 |
kashyap | lyarwood: Will check shortly; thanks! | 14:45 |
lyarwood | gibi: I'm not going to be around for the meeting btw | 14:48 |
lyarwood | gibi: I don't have anything rc related or for stable | 14:48 |
sean-k-mooney | gibi: i should be around for the meeting but ill be on PTO then until wednesday just an fyi | 14:50 |
sean-k-mooney | * ill be back on wednesday | 14:51 |
gibi | lyarwood: ack, thanks for the info | 14:52 |
gibi | sean-k-mooney: OK, have a nice PTO | 14:53 |
* sean-k-mooney disparead at being 31... not really but likely will spend it doing some house work and then playing a game or two | 14:54 | |
sean-k-mooney | i think im going to repaint my bedroom or assembel some dresser drawers. such an exciting life i leed :) | 14:55 |
*** k_mouza has quit IRC | 14:59 | |
*** lpetrut has quit IRC | 15:00 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Add tests for 'AggregateImagePropertiesIsolation' https://review.opendev.org/754115 | 15:06 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Add tests for 'AggregateInstanceExtraSpecsFilter' https://review.opendev.org/754116 | 15:06 |
*** k_mouza has joined #openstack-nova | 15:06 | |
*** lpetrut has joined #openstack-nova | 15:07 | |
*** belmoreira has quit IRC | 15:24 | |
*** k_mouza has quit IRC | 15:27 | |
tosky | stephenfin: re placement PTL, couldn't the nova candidate (I assume there is just one :) also apply for placement PTL? It would be a "personal union" of the crown | 15:33 |
stephenfin | tosky: entirely possible. That's a very valid option 3 | 15:33 |
*** k_mouza has joined #openstack-nova | 15:37 | |
*** k_mouza has quit IRC | 15:37 | |
*** k_mouza has joined #openstack-nova | 15:37 | |
gibi | nova meeting starts in 10 minutes in #openstack-meeting-3 | 15:50 |
*** ozzzo has quit IRC | 15:50 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Move implemented spec https://review.opendev.org/754124 | 15:54 |
openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Move implemented specs https://review.opendev.org/754124 | 15:54 |
*** _mlavalle_1 has joined #openstack-nova | 15:54 | |
*** _mlavalle_1 has quit IRC | 15:54 | |
*** mlavalle has quit IRC | 15:57 | |
*** gyee has joined #openstack-nova | 15:59 | |
*** priteau has joined #openstack-nova | 16:00 | |
*** belmoreira has joined #openstack-nova | 16:07 | |
*** belmoreira has quit IRC | 16:15 | |
*** mlavalle has joined #openstack-nova | 16:15 | |
*** k_mouza has quit IRC | 16:15 | |
*** spatel has joined #openstack-nova | 16:16 | |
*** dtantsur is now known as dtantsur|afk | 16:17 | |
*** ralonsoh has quit IRC | 16:22 | |
*** k_mouza has joined #openstack-nova | 16:22 | |
frickler | did nova recently raise the bar on qemu? looks like opensuse is failing with "nova.exception.InternalError: Nova requires QEMU version 4.0.0 or greater." https://a5f2733c1907b1f26b90-5593d50c131879f6a486eeedbad80e3c.ssl.cf5.rackcdn.com/743800/14/check/devstack-platform-opensuse-15/91eeaf7/controller/logs/screen-n-cpu.txt | 16:24 |
*** hemna has quit IRC | 16:25 | |
*** hemna has joined #openstack-nova | 16:25 | |
gibi | frickler: yes we did bumped min version | 16:26 |
gibi | frickler: https://review.opendev.org/#/c/746981/ | 16:26 |
frickler | gibi: ah, o.k., so I guess nobody checked opensuse? I mean, personally I don't care, we can just drop that devstack job | 16:27 |
gibi | yeah it seems nobody checked the opensuse job | 16:28 |
gibi | as far as I see it is a non-voting job that was not run for the nova patch | 16:29 |
gibi | frickler: I will open a bug to track this if you haven't done that yet | 16:30 |
frickler | gibi: I've asked the suse contacts I know in #openstack-infra, feel free to create a bug, too | 16:30 |
sean-k-mooney | frickler: the latest version of opensuse shoudl have the new minium | 16:30 |
sean-k-mooney | but that does not mean that job is using the latest version | 16:31 |
gmann | i think opensuse 15.1. has qemu 3.1.1.1 | 16:31 |
sean-k-mooney | 15.2 has 4.2.0 | 16:31 |
gmann | yeah | 16:32 |
gmann | https://software.opensuse.org/package/qemu? | 16:32 |
sean-k-mooney | and tumbelweed is on 5.1 | 16:32 |
sean-k-mooney | i was looking at https://repology.org/project/qemu/versions | 16:32 |
sean-k-mooney | frickler: so idealy to fix the suse job it woudl go to 15.2 | 16:32 |
gmann | yeah, we need to move the distro version | 16:33 |
gibi | frickler, sean-k-mooney, gmann: filed a bug for tracking https://bugs.launchpad.net/nova/+bug/1897118 | 16:34 |
openstack | Launchpad bug 1897118 in OpenStack Compute (nova) "nova-compute does not start in devstack-platform-opensuse-15 job due to < 4.0.0 qemu version" [Undecided,New] | 16:34 |
gibi | I have to leave for today | 16:35 |
gibi | see you tomorrow | 16:35 |
sean-k-mooney | looking at https://github.com/openstack/devstack/blob/master/doc/source/overview.rst#base-os susue is not on the list of maintained plathforms anymore | 16:36 |
sean-k-mooney | still if its an easy fix then there is no harm | 16:36 |
gmann | sean-k-mooney: i am not sure how easy, 15.2 is not in supported version yet so it depends on how new version work- https://github.com/openstack/devstack/blob/5aa38f51b3dd0660a0622aecd65937d3c56eedc2/stack.sh#L224 | 16:36 |
sean-k-mooney | gmann: right but devstack only offically support the latest current version of suse | 16:38 |
gmann | added devstack in bug | 16:39 |
sean-k-mooney | so in theory it should have swaped when 15.2 was released like we do for fedora when a new version comes out | 16:39 |
gmann | yeah, | 16:39 |
gmann | need more active maintainer for opensuse if needed. | 16:40 |
sean-k-mooney | the first step will be ensuring dib can build 15.2 | 16:40 |
sean-k-mooney | then if it can update devstack to support it | 16:40 |
sean-k-mooney | that or use the tumbelweed job instead | 16:40 |
sean-k-mooney | * /the/a/ | 16:41 |
*** k_mouza has quit IRC | 16:46 | |
*** spatel has quit IRC | 16:50 | |
*** derekh has quit IRC | 17:01 | |
*** salmankhan has quit IRC | 17:06 | |
*** lpetrut has quit IRC | 17:06 | |
*** ociuhandu has quit IRC | 17:07 | |
*** lpetrut has joined #openstack-nova | 17:09 | |
*** hemna has quit IRC | 17:10 | |
*** hemna has joined #openstack-nova | 17:19 | |
*** aj_mailing has joined #openstack-nova | 17:24 | |
*** Yumeng has quit IRC | 17:36 | |
*** links has quit IRC | 17:38 | |
*** iurygregory has quit IRC | 17:45 | |
*** aj_mailing has quit IRC | 17:48 | |
*** aj_mailing has joined #openstack-nova | 17:48 | |
*** artom has quit IRC | 17:50 | |
openstackgerrit | Merged openstack/nova stable/ussuri: test_evacuate.sh: Stop using libvirt-bin https://review.opendev.org/753281 | 17:56 |
*** iurygregory has joined #openstack-nova | 17:58 | |
lyarwood | ~. | 17:58 |
*** hamalq has joined #openstack-nova | 18:04 | |
*** nweinber has quit IRC | 18:07 | |
*** nweinber has joined #openstack-nova | 18:08 | |
*** tosky has quit IRC | 18:22 | |
openstackgerrit | Merged openstack/nova stable/ussuri: Ensure source compute is up when confirming a resize https://review.opendev.org/748369 | 18:37 |
*** nightmare_unreal has quit IRC | 18:45 | |
*** chenhaw has quit IRC | 18:47 | |
*** maciejjozefczyk_ has quit IRC | 18:51 | |
openstackgerrit | sean mooney proposed openstack/nova master: Fix unplugging VIF when migrate/resize VM https://review.opendev.org/751642 | 18:52 |
openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | 18:52 |
*** priteau has quit IRC | 18:58 | |
*** bbowen has quit IRC | 19:05 | |
*** xek has joined #openstack-nova | 19:06 | |
*** aj_mailing has quit IRC | 19:08 | |
*** aj_mailing has joined #openstack-nova | 19:08 | |
*** openstackgerrit has quit IRC | 19:14 | |
*** nweinber has quit IRC | 19:54 | |
*** lemko has quit IRC | 19:57 | |
*** lemko has joined #openstack-nova | 19:57 | |
*** openstackgerrit has joined #openstack-nova | 20:01 | |
openstackgerrit | Merged openstack/nova stable/rocky: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/747361 | 20:01 |
*** zzzeek has quit IRC | 20:12 | |
*** zzzeek has joined #openstack-nova | 20:14 | |
*** JamesBenson has quit IRC | 20:14 | |
*** zzzeek has quit IRC | 20:18 | |
*** zzzeek has joined #openstack-nova | 20:20 | |
*** bbowen has joined #openstack-nova | 20:21 | |
*** zzzeek has quit IRC | 20:25 | |
*** zzzeek has joined #openstack-nova | 20:26 | |
*** zzzeek has quit IRC | 20:30 | |
*** zzzeek has joined #openstack-nova | 20:33 | |
*** xek has quit IRC | 20:35 | |
*** zzzeek has quit IRC | 20:43 | |
*** artom has joined #openstack-nova | 20:45 | |
*** zzzeek has joined #openstack-nova | 20:46 | |
*** zzzeek has quit IRC | 20:51 | |
*** zzzeek has joined #openstack-nova | 20:53 | |
*** slaweq has quit IRC | 21:06 | |
*** lpetrut has quit IRC | 21:30 | |
openstackgerrit | melanie witt proposed openstack/nova stable/ussuri: Test for disabling greendns https://review.opendev.org/754220 | 21:37 |
*** mriedem has left #openstack-nova | 21:40 | |
*** raildo has quit IRC | 21:42 | |
*** zzzeek has quit IRC | 22:24 | |
*** zzzeek has joined #openstack-nova | 22:25 | |
*** vishalmanchanda has quit IRC | 22:39 | |
*** zzzeek has quit IRC | 22:44 | |
*** zzzeek has joined #openstack-nova | 22:46 | |
*** rcernin has joined #openstack-nova | 22:58 | |
*** mlavalle has quit IRC | 23:02 | |
*** rcernin has quit IRC | 23:14 | |
*** rcernin has joined #openstack-nova | 23:14 | |
*** zzzeek has quit IRC | 23:43 | |
*** zzzeek has joined #openstack-nova | 23:45 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!