Monday, 2024-07-08

opendevreviewMerged openstack/nova master: Server Rescue leads to Server ERROR state if original image is deleted  https://review.opendev.org/c/openstack/nova/+/87238501:07
*** bauzas_ is now known as bauzas03:10
opendevreviewmelanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2  https://review.opendev.org/c/openstack/nova/+/87093704:07
opendevreviewmelanie witt proposed openstack/nova master: Support cross cell resize with ephemeral encryption for qcow2  https://review.opendev.org/c/openstack/nova/+/90959504:07
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS  https://review.opendev.org/c/openstack/nova/+/88431304:07
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS  https://review.opendev.org/c/openstack/nova/+/88991204:07
spatelany idea what is this error in nova_libvirt 04:26
spatel2024-07-08 04:25:59.489+0000: 2691: error : qemuDomainObjBeginJobInternal:968 : Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainCreateWithFlags)04:26
spatelThis is happening after ceph failure. Now ceph is health but nova doesn't like to talk 04:26
*** bauzas_ is now known as bauzas05:20
opendevreviewmelanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2  https://review.opendev.org/c/openstack/nova/+/87093707:18
opendevreviewmelanie witt proposed openstack/nova master: Support cross cell resize with ephemeral encryption for qcow2  https://review.opendev.org/c/openstack/nova/+/90959507:18
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS  https://review.opendev.org/c/openstack/nova/+/88431307:18
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS  https://review.opendev.org/c/openstack/nova/+/88991207:18
*** bauzas_ is now known as bauzas07:32
*** bauzas_ is now known as bauzas08:11
sean-k-mooneygibi: regarding the time out how would you feel if i only tested one block size for iso10:19
sean-k-mooneyeither 64 * units.Ki, or  1 * units.Mi10:20
sean-k-mooneyboth would get the entiry header section in one read10:20
sean-k-mooney512 is not as bad as 17 but its proably why it takes so long10:21
sean-k-mooneyim going to try that locally and see how much of a speed up that gives10:22
sean-k-mooneywe can see it timed out in chunk = wrapper.read(block_size) so its definetly in that read loop and making progress it just takes too long10:24
sean-k-mooney5.5-5.9 second with the optimization vs 18.9-19 seconds without10:30
sean-k-mooneyi think that is a reasonable trade off10:30
sean-k-mooneynone of the logic im adding for the iso inspector cares about unaliged reads, that is just generic and shared with the other parsers so im personally happy with not testing it for these formats.10:32
sean-k-mooneyim going to commit that locally and adress the other nits but if you have concerns with that let me know10:33
*** ykarel__ is now known as ykarel10:54
gibisean-k-mooney: I'm fine skipping the unaligned read test for iso11:41
sean-k-mooneyack im just adressing the other nits ill have new version up in the next 30 mins or so11:43
opendevreviewsean mooney proposed openstack/nova master: Reproduce iso regression with deep format inspection  https://review.opendev.org/c/openstack/nova/+/92350712:03
opendevreviewsean mooney proposed openstack/nova master: Add iso file format inspector  https://review.opendev.org/c/openstack/nova/+/92353312:03
sean-k-mooneygibi: dansmith: ^ when your about those are functiontally the saem just with the nits addressed12:04
sean-k-mooneyim going to stack with those now and confirm if this correctly sovles the rebuild regression12:04
sean-k-mooneyi.e. boot form qcow usin a KNOPPIX or similar rescue iso to repair a vm12:05
sean-k-mooneys/usin/then using/12:06
gibisean-k-mooney: those two patches looks good to me.12:10
sean-k-mooneyhopefully that resolves the timeouts but we will see. the fact that it was only timing out in the coverage job ment it was borderline before12:11
sean-k-mooneyespicaly give the spread between tox py11 at 11 mins to py312 at 2212:11
sean-k-mooneywhere that job runs obviouly has a non trivial impact on the job time12:13
sean-k-mooneygibi: ill comment on the patch but i was able t boot form an cirros qcow and then rescue with an alpine iso12:24
sean-k-mooneyunrescue reverted back to cirros as expected12:24
*** bauzas_ is now known as bauzas12:27
gibisounds good12:41
andrewbonneyHi. Any reviews on https://review.opendev.org/c/openstack/nova/+/919961 would be appreciated when people have time. Thanks12:41
opendevreviewRobert Hoffmann proposed openstack/nova master: Fix: clean up volume attachments  https://review.opendev.org/c/openstack/nova/+/92364613:08
spatelAny idea on nova_libvirt error : qemuDomainObjBeginJobInternal:968 : Timed out during operation: cannot acquire state change lock (held by monitor=init-instance-00000d99) 13:37
*** haleyb|out is now known as haleyb14:00
*** bauzas_ is now known as bauzas14:25
elodillessean-k-mooney dansmith : should we wait for any other patch to merge, or is it OK if I generate now stable release patches for nova? (so that we release the fixes for bug/2059809)14:43
dansmithelodilles: I think fungi would say yes, release14:44
elodillesACK, then I'm preparing the patch14:44
fungiyeah, the sooner the better14:44
dansmithelodilles, fungi: bauzas is out this week, will we need an ack?14:45
fungithe release team is able to override/bypass that requirement, if they're willing to14:46
dansmithcool14:46
elodillesdansmith: https://review.opendev.org/c/openstack/releases/+/92365614:58
elodillesUggla auniyal : ping, as you are also release liaisons ^^^ o:)14:58
elodillesrelease jobs (validation and list-changes) logs should be available in ~10-20 mins15:00
dansmithelodilles: hmm, are those the right hashes for each release?15:00
dansmithseems like A and C are reversed, unless I'm missing something15:00
dansmithwait, maybe not15:01
dansmithokay yeah, mis-copy-paste I think, nevermind15:01
*** bauzas_ is now known as bauzas15:05
mnaserhi nova friends.  https://review.opendev.org/c/openstack/nova/+/923298/2 -- can someone look at this unmaintained/zed stack, it looks good to me so far and would be nice to get going into the gate16:14
mnaser(CVE stuff)16:14
dansmithmnaser: note that nova-core doesn't even have +2 on that branch because it's unmaintained16:16
dansmithlooks like it has the only tweak we made after the first push, so I'd assume that if it works for you it's good16:18
mnaseryeah I got that in16:20
mnaserwell, elodilles did :)16:20
JayFmnaser: feel free to add me as reviewer for any security backports in unmaintained/ that need additional review.16:25
JayFmnaser: I have the one patch you've indicated opened in my window, will have a look16:25
JayFSpeaking of; mnaser has an important fix for the Ironic driver which will need to land in master + go back to Caracal, and is probably bad enough that I'd hope we could get it in a caracal point release: https://review.opendev.org/c/openstack/nova/+/923530 -- basically Ironic+Nova integrated environments with complex networking configurations will not have their metadata generated properly16:30
dansmithJayF: +2d16:33
dansmithlooks simple enough, maybe gmann or melwitt could hit it for you16:33
JayFdansmith: thank you very much \o/ (for this and for fixing the security issues over the last weeks)16:33
melwitt+Wd16:36
JayFawesome, I'll get a backport in once it lands16:37
JayFI do have an item on my todo to follow up and make sure all those tests use iter([obj1,obj2]) to catch this kinda issue16:38
dansmithsean-k-mooney: so the coverage job is still failing on your test_iso966017:19
dansmithI don't really know why though17:19
dansmithbecause it reads 10M 17 bytes at a time maybe? does it need to be that big?17:21
sean-k-mooneydidnt i fixt that already https://review.opendev.org/c/openstack/nova/+/923533/8/nova/tests/unit/image/test_format_inspector.py#21117:23
sean-k-mooneydansmith: https://review.opendev.org/c/openstack/nova/+/923507/3 still need a +w by the way17:23
sean-k-mooneythat why its not currently in the ci17:24
dansmithoh yeah, I was looking at the wrong report17:24
dansmiththat's why duh17:24
dansmithcool17:24
sean-k-mooneybut ya the 32KB system area makes it time out with 7 or 512 bytye blocks17:24
dansmithI just changed to 128k locally and it works fine at that size, so no real need for it to be larger I think17:25
sean-k-mooneybtu the 64k and 1mb block size do it in a single read17:25
dansmithbut as long as it passes now all good, it's only temporary anyway17:25
sean-k-mooneywell its reading 34KB total so the 64K read size is enough17:25
dansmithin the gate now17:27
dansmithtotes forgot about that repro patch below :D17:27
sean-k-mooneywe could perhaps replace the 7 bytes with 7k later to have an unaligend read test be a little fater but ya for now it should be fine17:27
sean-k-mooneyok those are going to take a while to go though the gate so ill likely take this time to cook dinner and ill check back after 17:29
dansmith++17:30
dansmithhmm, might have a requirements conflict: gabbi 3.0.0 depends on jsonpath-rw-ext>=1.0.018:30
dansmithjust failed in gate18:30
dansmithokay might've just been the requirements change that was in front of it18:34
dansmiththat failed (with multiple issues) and the nova restart passed that point I think18:34
fricklerthat sounds like pypi is having some issues18:40
dansmithmaybe18:41
clarkbthose sorts of failures can occur when we get served old indexes from pypi18:43
dansmithokay the requirements patch wasn't changing either of those packages, and one other job on it failed in a different way (but the nova one failed the same way the nova patch behind it did)18:48
opendevreviewMerged openstack/nova master: Reproduce iso regression with deep format inspection  https://review.opendev.org/c/openstack/nova/+/92350718:56
opendevreviewMerged openstack/nova master: Add iso file format inspector  https://review.opendev.org/c/openstack/nova/+/92353320:29
sean-k-mooneysweet ^ ill start backporting those20:32
dansmith\o/20:45

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!