| *** ykarel_ is now known as ykarel | 04:52 | |
| gibi | Hi folks it seems nova-alt-configuration job is started to fail since yesterday | 07:42 |
|---|---|---|
| gibi | https://zuul.opendev.org/t/openstack/builds?job_name=nova-alt-configurations&branch=master&skip=0 | 07:42 |
| gibi | May 11 20:41:52.111833 npfa8fa206b4014 nova-compute[32688]: ERROR nova.compute.manager [instance: 0de2d300-db70-420c-a502-c6015ad70e94] libvirt.libvirtError: unsupported configuration: spice graphics are not supported with this QEMU | 07:43 |
| gibi | May 11 20:41:52.111833 npfa8fa206b4014 nova-compute[32688]: ERROR nova.compute.manager [instance: 0de2d300-db70-420c-a502-c6015ad70e94] libvirt.libvirtError: unsupported configuration: spice graphics are not supported with this QEMU | 07:43 |
| gibi | https://zuul.opendev.org/t/openstack/build/7f08461295e24d3b8f1673f29dd9f711/log/compute1/logs/screen-n-cpu.txt?severity=4 | 07:43 |
| gibi | this can be relevant https://review.opendev.org/c/openstack/devstack/+/968354 | 07:45 |
| frickler | gibi: iiuc you need to install an extra pkg for that, see also https://review.opendev.org/c/openstack/kolla/+/972441/4/docker/nova/nova-libvirt/Dockerfile.j2 | 07:49 |
| gibi | frickler: thanks for the pointer | 07:52 |
| gibi | I will push a fix in devstack then | 07:56 |
| frickler | looks like the failure was already present in the testing patch, but wasn't noticed https://review.opendev.org/c/openstack/nova/+/968892 | 07:57 |
| gibi | indeed | 08:06 |
| gibi | it is partially me who approved the devstack patch yesterday who missed that failure. | 08:06 |
| gibi | I pushed https://review.opendev.org/c/openstack/devstack/+/988214 to fix it | 08:07 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: test nova-alt-configuration https://review.opendev.org/c/openstack/nova/+/988215 | 08:09 |
| gibi | testing it in ^^ | 08:09 |
| mikal | gibi: yeah, the Debian Trixie qemu requires an additional package for SPICE. Same deal with Ubuntu FWIW. | 08:15 |
| gibi | mikal: TIL :) | 08:34 |
| gibi | elodilles: for your eyes https://review.opendev.org/q/I469688be203c319dd048d6a99057b8d98369de0e | 08:34 |
| elodilles | gibi: ACK, i'll keep an eye on them | 08:42 |
| sean-k-mooney | gibi: am i belive speice is an optional package on debian 13 and 12 + backport likely has the same pattern | 09:16 |
| sean-k-mooney | ah yep | 09:17 |
| sean-k-mooney | gibi: mikal so i have been meaning to move that job to debian 13 for a while now | 09:17 |
| mikal | sean-k-mooney: yeah fair enough. I am working on a job for Kolla-Ansible at the moment, but that's likely heavier weight than what you'd want in Nova. | 09:21 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/966479 | 09:22 |
| sean-k-mooney | was the change i worked on 6 months ago i just didnt get around to the followup for addign the spice module expelcitly | 09:22 |
| sean-k-mooney | at least not that isee upstream | 09:22 |
| mikal | I'd like to do stronger assertions that SPICE is actually working, not just getting the server "banner" back, but that's a thing I've wanted for a while and am only just getting to now. | 09:23 |
| sean-k-mooney | mikal: i mena we could do that | 09:24 |
| sean-k-mooney | it really just depends on what that looks liek and how much time it addes to the job | 09:24 |
| mikal | Well, I have most of a headless SPICE client working now. I want to have a bit more a test workload which enumerates a bit of the protocol as well though. I don't think it would make the job much slower, apart from needing to download (or cache) the client and target instance image. | 09:25 |
| mikal | But forward looking statements etc etc. | 09:25 |
| sean-k-mooney | well if you end up packagign this as a tempest/devstack plugin that we coudl teadilly enable we coudl consider addign it | 09:26 |
| sean-k-mooney | s/teadilly/readally or eaisilly/? | 09:27 |
| sean-k-mooney | oen of those words | 09:27 |
| mikal | Just as soon as I learn what a tempest plugin (or Claude tells me), I will consider that thing. | 09:29 |
| sean-k-mooney | its just a way of packaging tempst test in a seperate repo | 09:29 |
| sean-k-mooney | as an example https://opendev.org/openstack/whitebox-tempest-plugin/ | 09:29 |
| mikal | Huh, so potentially sharable between Nova and Kolla-Ansible? That would be nice. | 09:30 |
| sean-k-mooney | yes | 09:30 |
| mikal | Ok, I shall read your newsletter... | 09:30 |
| sean-k-mooney | and if you do that you could provide a devstack plugin to just set up the deps/generate the config | 09:30 |
| sean-k-mooney | https://docs.openstack.org/tempest/latest/plugins/plugin.html is proably more useful | 09:30 |
| sean-k-mooney | of course tempest (which is just stdlib unittest as the framework) may not be what you want fo rthis but i think it could be a decent fit | 09:31 |
| mikal | Ok, I will consider this all tomorrow but it sounds promising to me. | 09:41 |
| mikal | Thank you. | 09:41 |
| sean-k-mooney | if gibi has time to update teh devstack patch to make it work on debian 13 we will likely proceed with it today, if they dont we may still proceed with it today to unblock the gate and follwo up next week | 09:42 |
| mikal | Oh definitely adding the package now is the way to go. My thing will take a while to navigate I think. I'd offer to send a change for adding the patch, but I am also about to be pulled away for family duties. | 09:47 |
| sean-k-mooney | im 99% sure i have a patch also locally but the host i run my devstack vms lost a hardrive at teh weekend | 09:48 |
| sean-k-mooney | and while it can run with one failure i have it off until i have time to replace it | 09:48 |
| sean-k-mooney | so my debian 13 spice enve where i was testing that fix is ofline :) | 09:49 |
| sean-k-mooney | gibi's patch shoudl fix the current case and it would be a minor tweak to also supprot debain 13 | 09:49 |
| frickler | gibi: it there anything that can be done to make progress with https://bugs.launchpad.net/os-vif/+bug/2109396 ? there was a new pyroute2 release recently, but from the changelog it didn't look to me like it would help with this issue. the pyroute2 cap is the last remaining item we have until we can go back to having "normal" u-c updates again | 10:50 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: move nova-alt-config to debian-13 https://review.opendev.org/c/openstack/nova/+/966479 | 10:58 |
| gibi | I respin the devstack fix to include debian 13 and updated ^^ to point to that devstack change | 10:59 |
| gibi | frickler: somebody should either fix pyroute2 with eventlet or we should drop eventlet from our codebase | 11:00 |
| gibi | I have no other ideas | 11:00 |
| sean-k-mooney | if only one of those was almost complete | 11:00 |
| sean-k-mooney | frickler: i assume its not posisbel to have diffent version depending on if your using eventlet or not like we can based on python verion? | 11:01 |
| sean-k-mooney | nova i now runign nova-compute without eventlet by default so could perhaps downgrade the pyroute2 used in eventlet mode and uncap it in threaded mode? | 11:02 |
| sean-k-mooney | but that likely to requrie some toolign changes in devstack to do properly | 11:02 |
| gibi | hm if we allow downgrading to our small amount of eventlet based jobs then the rest can probably run with the latest pyroute2 | 11:03 |
| gibi | s/to/in/ | 11:03 |
| frickler | how long will the eventlet mode be needed? 2027.1 for migration purposes? | 11:03 |
| gibi | yes 2027.1 will still have eventlet as a last slurp | 11:03 |
| sean-k-mooney | and we will be deleting it in about 12 months time basiclly we will do the deletion early in 2027.2 | 11:04 |
| frickler | we could override u-c within devstack, but that won't help deployers | 11:04 |
| sean-k-mooney | 2026.2 is the first release where we are defaultign to threaded mode for nbova-compute | 11:05 |
| frickler | so let me update https://review.opendev.org/c/openstack/requirements/+/973210 and then we can see where we actually stand currently | 11:05 |
| sean-k-mooney | so this release would be compatibaly with newer pyroute2 | 11:05 |
| *** jgilaber_ is now known as jgilaber | 11:06 | |
| sean-k-mooney | frickler: we could also just say "if you want to use py3.14 you shoudl use threaded mode adn only keep the cap for 3.13 and below | 11:07 |
| sean-k-mooney | but cool lets see what the updated run shows | 11:07 |
| opendevreview | Dr. Jens Harbott proposed openstack/nova master: DNM: test pyroute2 bump https://review.opendev.org/c/openstack/nova/+/988242 | 11:14 |
| gibi | the nova-alt-configuration is the job that tests the eventlet mode. So we need a good result from that job in ^^ But that job is broken and being fixed in devstack :) | 11:20 |
| frickler | ah, I wasn't aware that that is the same job. so waiting for it to be fixed first | 11:28 |
| gibi | yeah I relized as we are chatting here :) | 11:30 |
| sean-k-mooney | gibi: so it failed but on an undrealted test the spice tests passed but some fo the live migraiton test failed in test cleanup | 13:08 |
| gibi | I will recheck the nova patch but I think the devstack one is good to go | 13:13 |
| sean-k-mooney | yep i +2'd the devstack one | 13:13 |
| sean-k-mooney | did you have a test patch on deb-12 on nova? i just check the patch that moved nova-alt-config to debin 13 | 13:14 |
| gibi | sean-k-mooney: this https://review.opendev.org/c/openstack/nova/+/988215 shows that the devstack patch unblock the nova-alt-config job | 13:15 |
| gibi | with debian 12 | 13:15 |
| sean-k-mooney | great | 13:15 |
| sean-k-mooney | then ya we shoudl proceed with the devstack patch is frickler and other are happy with it | 13:16 |
| sean-k-mooney | as i said the test failure looked unrelated on debian 13 | 13:16 |
| sean-k-mooney | gibi: while i have been in meeting i have been having ai agents test my kernel module and it apears to be workign enough for the device to show up in lspci | 13:19 |
| sean-k-mooney | im going to play around with it a bit more and see if i can get it to a porint where the VF are usabel by a nested qemu quickly | 13:20 |
| gibi | sean-k-mooney: thanks | 13:21 |
| elodilles | sean-k-mooney: sorry for pinging, can you please revisit my placement gatefix patch / check my comment on it? o:) https://review.opendev.org/c/openstack/placement/+/980761/2#message-6b13614a07d353e22186487083fe57b46b3aa02f | 13:22 |
| sean-k-mooney | elodilles if the grenade job is working im kind of ok with it existing | 13:25 |
| sean-k-mooney | but i dont really like having a know broken job | 13:25 |
| sean-k-mooney | that is non voting | 13:25 |
| sean-k-mooney | i would prefer if the skip level job was removed until it was fixed but if other agree then we can keep it | 13:26 |
| elodilles | sean-k-mooney: yes, i usually remove as well broken / unfixable CI jobs, but with this i feel some hope o:) (also, it's included via a job template) | 13:27 |
| sean-k-mooney | ill merge it for now and we can discuss it more once the cid is unblocked | 13:30 |
| elodilles | sean-k-mooney: sure, thanks \o/ | 13:31 |
| opendevreview | ribaudr proposed openstack/nova master: Add reproducer for cold migration stuck after compute restart https://review.opendev.org/c/openstack/nova/+/988257 | 13:32 |
| opendevreview | ribaudr proposed openstack/nova master: Clean up orphaned cold migration on compute restart https://review.opendev.org/c/openstack/nova/+/988258 | 13:32 |
| opendevreview | Takashi Kajinami proposed openstack/nova-specs master: libvirt: AMD SEV-SNP support https://review.opendev.org/c/openstack/nova-specs/+/983376 | 13:57 |
| opendevreview | ribaudr proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | 13:59 |
| Uggla | dansmith, sean-k-mooney as discussed yesterday ^ can you please hava a look. | 14:04 |
| sean-k-mooney | i does not look liek you adressed gibi's comments on stale comments | 14:07 |
| opendevreview | Julien LE JEUNE proposed openstack/nova master: Reproduce bug #2150616: KeyError in _rollback_volume_bdms https://review.opendev.org/c/openstack/nova/+/986754 | 14:07 |
| opendevreview | Julien LE JEUNE proposed openstack/nova master: Fix KeyError in _rollback_volume_bdms aborting live-migration rollback https://review.opendev.org/c/openstack/nova/+/988263 | 14:07 |
| sean-k-mooney | well at least particlly | 14:07 |
| sean-k-mooney | over all i think it looks ok | 14:07 |
| sean-k-mooney | just lookign quickly | 14:07 |
| opendevreview | Merged openstack/placement stable/2025.1: [CI][stable-only] Fix docs job https://review.opendev.org/c/openstack/placement/+/980761 | 15:41 |
| gibi | as https://review.opendev.org/c/openstack/devstack/+/988214 has been merged the nova-alt-configuration job should start passing again. Happy rechecking | 16:05 |
| dansmith | sweet | 16:40 |
| opendevreview | Merged openstack/nova stable/2026.1: Fix resize to unpermitted flavors https://review.opendev.org/c/openstack/nova/+/988148 | 19:08 |
| opendevreview | Merged openstack/nova master: Fix functional tests and mypy on macOS https://review.opendev.org/c/openstack/nova/+/937727 | 21:18 |
| opendevreview | Merged openstack/os-vif master: Deprecate version module https://review.opendev.org/c/openstack/os-vif/+/987636 | 22:59 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!