| opendevreview | Ghanshyam proposed openstack/nova master: Add manager graceful shutdown, timeout, and wait https://review.opendev.org/c/openstack/nova/+/975586 | 02:14 |
|---|---|---|
| *** mhen_ is now known as mhen | 02:21 | |
| *** mhen_ is now known as mhen | 03:45 | |
| *** sambork_ is now known as sambork | 06:32 | |
| LarsErikP | I feel I've asked for this "a million times" before but.. https://review.opendev.org/c/openstack/nova/+/916409 Does anyone know the guy that wrote the patch, and how to get hold of him? I've even tried to email him myself without any response. It seems so close (to me at least). We would really like to see this merged <3 | 07:08 |
| frickler | LarsErikP: looks like that would be whoami-rajat | 07:41 |
| LarsErikP | oh, he's actually here :p My bad then. Any chance you could take a new look a this, whoami-rajat ? =) | 07:43 |
| gokhan | yes thanks to whoami-rajat, he resolved Rebuild volume backed server with snapshot image issue. | 07:44 |
| LarsErikP | +1 to that! And it seems we are so close to actually getting it merged. | 07:45 |
| frickler | well I'm not sure how close this actually is, adding a functional reproducer like dansmith asked for seems reasonable, maybe you want to look at that yourself? | 07:53 |
| LarsErikP | frickler: I'm not enough of a developer to have a go at that, sadly :( | 07:54 |
| whoami-rajat | LarsErikP, frickler hello, I haven't been able to get time to follow up on the patch, and honestly I still don't have much time so if dansmith agrees to have a reproducer as a followup or another developer wants to add it, I'm happy either way but i really can't commit to doing it right now | 09:44 |
| tobias-urdin | when doing a hard reboot and we call _create_guest_with_network do we have stale network_info passed there or a recent bug related to that? if i have a neutron port, set it to admin down and hard reboot an instance it fails waiting for neutron plugged events (wait_for_vif_plugged_event_during_hard_reboot=true) even though there shouldn't be any neutron ports to plug, do we not | 12:55 |
| tobias-urdin | ignore when admin_status=down on a port and try to plug it? | 12:55 |
| tobias-urdin | *correction, wait_for_vif_plugged_event_during_hard_reboot=normal | 12:56 |
| LarsErikP | whoami-rajat: thanks for explaining. I really hope someone can find time to do what's needed here. And I'm sorry that lack the skills :P | 12:56 |
| sean-k-mooney | tobias-urdin: i think we are usign the network info cache form our db | 14:07 |
| sean-k-mooney | tobias-urdin: but to be clear seting the admin state to down shoudl have no effect on the network-vif plugged event | 14:08 |
| sean-k-mooney | that is a neutorn bug if it does | 14:09 |
| sean-k-mooney | the expected behvior is that the vm boots with the port but the traffic is blocked on the neutron side | 14:10 |
| tobias-urdin | sean-k-mooney: interesting, i'll see if i can get a 100% reproducer, thanks! | 14:18 |
| sean-k-mooney | LarsErikP: i need to rebase one fo my other patches but ill see if i can get claude to create a repoducer for whoami-rajat patch while im workign on somethign else | 15:27 |
| opendevreview | sean mooney proposed openstack/nova master: FairLockGuard: Support cross-thread sharing and nesting https://review.opendev.org/c/openstack/nova/+/973438 | 15:29 |
| sean-k-mooney | gmaan: gibi ^ can ye restore your +2w that went into merge conflciat after the libvirt event delaying patch merged | 15:30 |
| nicolairuckel | Does it make sense to restart my CI run or is there still a problem with pkg_build? | 15:44 |
| sean-k-mooney | nicolairuckel: we should have workaround enough of those for ti to pass on nova now | 16:01 |
| sean-k-mooney | so feel free to recheck it | 16:01 |
| gmaan | sean-k-mooney: done | 16:13 |
| gmaan | bauzas: not sure if you looked into my reply on this, https://review.opendev.org/c/openstack/nova/+/975242 | 16:14 |
| sean-k-mooney | thanks i tought that had alredy merged :) | 16:14 |
| opendevreview | sean mooney proposed openstack/nova master: Fix: Rebuild volume backed server with snapshot image https://review.opendev.org/c/openstack/nova/+/916409 | 16:17 |
| opendevreview | sean mooney proposed openstack/nova master: Add regression test for bug 2062127 https://review.opendev.org/c/openstack/nova/+/976654 | 16:17 |
| bryanfraschetti | Hi folks o/ | 16:41 |
| bryanfraschetti | I hope you're doing well! I believe I've found a bug where volume attachment BDM entries aren't properly cleaned up from Nova DB when the attachment fails due to an oslo timeout. I've found that this is because the exception handling to remove the entry for the failed attachment can happen before nova-compute completes the DB transaction that would otherwise need to be cleaned up. Effectively the clean up becomes a | 16:41 |
| bryanfraschetti | no-op. I've documented it in more detail at https://bugs.launchpad.net/nova/+bug/2141355 but I'm wondering if anyone has thoughts on how to address this. | 16:41 |
| sean-k-mooney | bryanfraschetti: are you usign galera in multi writer mode | 16:47 |
| sean-k-mooney | on newer release we have som logic to clean up bdms on hard-reboot if volume or are deleted on the cinder cide and on slighly older release we have nova-manage command to regenerate teh attachment tha tmight also hlep rahter then raw db surgery | 16:50 |
| bryanfraschetti | I've been using a default charmed openstack deployment (though I tore it down). I'll spin up another environment to check | 16:50 |
| sean-k-mooney | the reason i brought up galera is openstack in general an nova specficly does not supprot usign a multi write galera because of the fact that transaction can be defered or read after commit are not atomic by default | 16:51 |
| sean-k-mooney | that may not be th issue in your bug report | 16:52 |
| bryanfraschetti | That makes sense | 16:53 |
| sean-k-mooney | just poingign out that that oen common misconifugraton | 16:53 |
| bryanfraschetti | I'm not extremely familiar with database specifics but the cluster uses mysql-innodb-cluster, which seems to imply that it's not using galera, which to my understanding would be percona-cluster | 17:21 |
| sean-k-mooney | im not sure but the impartn part is we dont supprot active active writers unless the db gaurentees all client see the change after the transaction is commited | 17:22 |
| sean-k-mooney | again tha tmight not be what happenign in this sepcific cas but i think on master at least the attachments would be healed in cidner if you use the nova-mange command to regenreate tehm or the bdms would be healed in nova if you hard reboot | 17:23 |
| sean-k-mooney | hard-reboot the instnace that is | 17:24 |
| sean-k-mooney | that is not a fix just a workaround for the unable to detach issue | 17:24 |
| nicolairuckel | sean-k-mooney: thanks, I'll do that. Hopefully, it will be ready for review soon. | 17:28 |
| bryanfraschetti | Okay, thanks I'll look into that more to confirm whether or not it's some contention caused by multiple writers. I'll look into the nova-manage command too and figure out a better way to regenerate the bdms than manual db surgery then come back when I know more. | 17:30 |
| bryanfraschetti | I think if it's not caused by a misconfig of the DB, then I'd like to submit a code change so that we don't need the workaround but I'll do some more digging first | 17:33 |
| bryanfraschetti | Thanks! | 17:33 |
| sean-k-mooney | thre likly is a bug or at least a hardernign opertunity here | 17:49 |
| opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron subnets https://review.opendev.org/c/openstack/nova/+/962190 | 17:51 |
| sean-k-mooney | we may have another broken or flaky test in the devstack functional job | 18:33 |
| sean-k-mooney | https://zuul.opendev.org/t/openstack/build/7380a410c99740cba14d432559a5d75d | 18:33 |
| obre | Do I understand it right if I believe that A: os-traits 3.6.0 is released now, and B: that the issues with pkg_build is solved? How do I trigger a re-run of the tests? | 19:34 |
| sean-k-mooney | it is released https://pypi.org/project/os-traits/ but we need to merge https://review.opendev.org/c/openstack/requirements/+/976640 and to do that we need to fix a few things | 19:39 |
| obre | Ack. Then Ill wait :) | 19:40 |
| sean-k-mooney | you can add `Depends-On: https://review.opendev.org/c/openstack/requirements/+/976640` to your commit message and recheck it if you want | 19:40 |
| sean-k-mooney | well updating the commit will trigger a run | 19:40 |
| sean-k-mooney | horizon is still broken by pkg_resouces | 19:40 |
| sean-k-mooney | the xstatic packages are also breakitn the upper constratint job https://zuul.opendev.org/t/openstack/build/7642c7f731cf44d4bb5a1a07ae1c402a/log/job-output.txt#1291-1294 | 19:41 |
| nicolairuckel | sean-k-mooney: It looks like the CI passed so it would be ready for you to test whenever you have time. :) | 19:45 |
| sean-k-mooney | thanks i just finished testing live migation ill pull it into my devstack and test it quickly | 19:46 |
| nicolairuckel | good timing I guess :D | 19:46 |
| nicolairuckel | thanks! | 19:46 |
| sean-k-mooney | got the link handy | 19:47 |
| sean-k-mooney | ah | 19:47 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/959682 | 19:47 |
| nicolairuckel | right | 19:49 |
| opendevreview | Eigil Obrestad proposed openstack/nova master: Make nova recognize amx-capabilities https://review.opendev.org/c/openstack/nova/+/976390 | 19:51 |
| sean-k-mooney | (venv) debian@devstack-live-migration-1:~/repos/nova$ sudo stat -c "Access: %x | Modify: %y | Birth: %w" /var/lib/libvirt/qemu/nvram/instance-0000000b_VARS.fd | 19:58 |
| sean-k-mooney | Access: 2026-02-12 19:57:41.147164936 +0000 | Modify: 2026-02-12 19:57:45.923378683 +0000 | Birth: 2026-02-12 19:53:12.387159116 +0000 | 19:58 |
| sean-k-mooney | after 3 rebooths the creation time has not change but the acccesed and modifed do | 19:58 |
| sean-k-mooney | suspend and resume preserve the file and rebuild recreats it | 20:02 |
| sean-k-mooney | so that is what i was expecting | 20:02 |
| sean-k-mooney | rebuild might evenutlly need an api flag to choose the behviaor | 20:02 |
| sean-k-mooney | i.e. rebuild form snapshot proably shoudlnot recreate it but rebuild to a different image shoudl but we can dicusss that later | 20:03 |
| nicolairuckel | okay | 20:03 |
| nicolairuckel | You mean for a later patch or later when you did more testing? | 20:03 |
| sean-k-mooney | i mean in a later patch when we dicsuss move operations and rebuild ectra | 20:04 |
| nicolairuckel | sounds good | 20:04 |
| sean-k-mooney | live migration also seams to work although witout puting some data in the nvram its hard to verfiy the actual behavior | 20:06 |
| sean-k-mooney | i think im +2 on the patch overall | 20:07 |
| sean-k-mooney | there is more work to do in this areay but i think we can proceed with that as an incremental improvment | 20:07 |
| nicolairuckel | perfect :) | 20:10 |
| nicolairuckel | thanks again | 20:16 |
| sean-k-mooney | no worries melwitt dansmith ye may want to add https://review.opendev.org/c/openstack/nova/+/959682 to your review queue. its esscally the same as the vtpm preservation if you still have context form that | 20:17 |
| dansmith | sean-k-mooney: it's already on both of our lists, I was waiting for you to be happy with it since you had the most feedback | 20:18 |
| sean-k-mooney | ack | 20:23 |
| melwitt | ack sean-k-mooney it has been on my radar and was most interested in your vote on it for the same reason :) I will look at it soon | 20:41 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!