Monday, 2024-08-26

opendevreviewMichael Still proposed openstack/nova master: libvirt: Add extra spec for sound device.  https://review.opendev.org/c/openstack/nova/+/92612600:06
*** bauzas_ is now known as bauzas01:26
*** bauzas_ is now known as bauzas06:15
*** elodilles_pto is now known as elodilles07:24
opendevreviewBalazs Gibizer proposed openstack/nova stable/2024.1: Functional test test_boot_reschedule_with_proper_pci_device_count  https://review.opendev.org/c/openstack/nova/+/92655607:39
opendevreviewBalazs Gibizer proposed openstack/nova stable/2024.1: Fix PCI passthrough cleanup on reschedule  https://review.opendev.org/c/openstack/nova/+/92655707:39
bauzasmorning nova08:17
fricklergood morning bauzas, in case you missed it last week: please check the os-vif release patch https://review.opendev.org/c/openstack/releases/+/926768 , we're now past the deadline already08:25
bauzasfrickler: ack, I was on PTO for the previous 3 weeks but I'll take a look at it very shortly08:26
fricklerthx, just wanted to make sure that you were aware of it with all the flapping that's still happening with your account :)08:28
frickler(I made my client ignore joins and quits, but not name changes, so these kind of stand out to me)08:29
*** ykarel_ is now known as ykarel08:36
opendevreviewribaudr proposed openstack/nova master: Amend ShareMappingStatus due to asynchronous call  https://review.opendev.org/c/openstack/nova/+/90886408:36
opendevreviewribaudr proposed openstack/nova master: Amend DB model add a unique constraint.  https://review.opendev.org/c/openstack/nova/+/91251808:36
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (manila abstraction)  https://review.opendev.org/c/openstack/nova/+/83119408:36
opendevreviewribaudr proposed openstack/nova master: Use client token when talking to manila  https://review.opendev.org/c/openstack/nova/+/92527708:36
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (drivers and compute manager part)  https://review.opendev.org/c/openstack/nova/+/83309008:36
opendevreviewribaudr proposed openstack/nova master: Mounting the shares as part of the initialization process  https://review.opendev.org/c/openstack/nova/+/88007508:36
opendevreviewribaudr proposed openstack/nova master: Deletion of associated share mappings on instance deletion  https://review.opendev.org/c/openstack/nova/+/88147208:36
opendevreviewribaudr proposed openstack/nova master: Add metadata for shares  https://review.opendev.org/c/openstack/nova/+/85050008:36
opendevreviewribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part)  https://review.opendev.org/c/openstack/nova/+/85482308:36
opendevreviewribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute manager part)  https://review.opendev.org/c/openstack/nova/+/85482408:36
opendevreviewribaudr proposed openstack/nova master: Add share_info parameter to resume method for each driver (driver part)  https://review.opendev.org/c/openstack/nova/+/86028408:36
opendevreviewribaudr proposed openstack/nova master: Support resuming an instance with shares (compute manager part)  https://review.opendev.org/c/openstack/nova/+/86028508:36
opendevreviewribaudr proposed openstack/nova master: Add helper methods to rescue/unrescue shares  https://review.opendev.org/c/openstack/nova/+/86028608:36
opendevreviewribaudr proposed openstack/nova master: Support rescuing an instance with shares  https://review.opendev.org/c/openstack/nova/+/86028708:36
opendevreviewribaudr proposed openstack/nova master: Allow to mount manila share using Cephfs protocol  https://review.opendev.org/c/openstack/nova/+/88386208:36
opendevreviewribaudr proposed openstack/nova master: Check shares support (compute manager)  https://review.opendev.org/c/openstack/nova/+/88575108:36
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (API)  https://review.opendev.org/c/openstack/nova/+/83683008:36
opendevreviewribaudr proposed openstack/nova master: Add helper methods to attach/detach shares  https://review.opendev.org/c/openstack/nova/+/88575308:36
opendevreviewribaudr proposed openstack/nova master: Add instance.share_attach notification  https://review.opendev.org/c/openstack/nova/+/85050108:36
opendevreviewribaudr proposed openstack/nova master: Add instance.share_detach notification  https://review.opendev.org/c/openstack/nova/+/85102808:36
opendevreviewribaudr proposed openstack/nova master: Add shares to InstancePayload  https://review.opendev.org/c/openstack/nova/+/85102908:36
opendevreviewribaudr proposed openstack/nova master: Add instance.share_attach_error notification  https://review.opendev.org/c/openstack/nova/+/86028208:36
opendevreviewribaudr proposed openstack/nova master: Add instance.share_detach_error notification  https://review.opendev.org/c/openstack/nova/+/86028308:36
opendevreviewribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working.  https://review.opendev.org/c/openstack/nova/+/85208608:36
opendevreviewribaudr proposed openstack/nova master: Add virt/libvirt error test cases  https://review.opendev.org/c/openstack/nova/+/85208708:36
opendevreviewribaudr proposed openstack/nova master: Docs about Manila shares API usage  https://review.opendev.org/c/openstack/nova/+/87164208:36
opendevreviewribaudr proposed openstack/nova master: Reports instance events to the DB regarding attaching and detaching a share  https://review.opendev.org/c/openstack/nova/+/92708808:36
auniyalbauzas, frickler, yes, so I was not sure of https://review.opendev.org/c/openstack/releases/+/92676808:38
auniyalso was waiting for bauzas08:38
bauzasno worries09:01
*** bauzas_ is now known as bauzas10:44
gibiIt seems we have a gate block on stable/2024.1 10:46
gibicurl -f -O https://download.ceph.com/rpm-/el9/noarch/cephadm returns 404 making the ceph job fail10:46
gibiexample https://zuul.opendev.org/t/openstack/build/3be807e5dbf843ff995fbe1fdcb7fed210:46
gibielodilles: is it something known?10:46
elodillesgibi: well, i haven't seen that yet :-o10:47
gibisorry not 2024.1 but 2023.210:48
elodillesthe link is definitely missing something after "rpm-"10:48
gibihm, good point10:49
elodillesand it doesn't seem to be a constant error, at least if we look at the job history: https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack%2Fnova&branch=stable%2F2023.2&skip=010:50
sean-k-mooneyit should be https://download.ceph.com/rpm-quincy/10:51
sean-k-mooneyor https://download.ceph.com/rpm-reef/10:51
elodillesit looks like a CEPH_RELEASE (?) variable or something like that was not set10:51
sean-k-mooneyyep10:51
sean-k-mooneyalthough that shoudl not block the gate just the c9s jobs10:51
sean-k-mooneywhich should not be voting10:52
gibihttps://review.opendev.org/c/openstack/nova/+/926555 this reproduced the error twice in a row10:52
gibiit happening in the nova-multi-cell jo10:52
gibib10:52
sean-k-mooneywhy is that not using ubuntu?10:53
sean-k-mooneyoh it is10:53
sean-k-mooney Ubuntu 22.0410:54
sean-k-mooneyso for some reason its trying to pull the rpm version when it should be pulling the deb package10:54
gibihm there was error in the script before that10:56
gibiError: writing blob: storing blob to file "/var/tmp/storage2783954232/1": happened during read: unexpected EOF10:56
gibiError: writing blob: storing blob to file "/var/tmp/storage2783954232/1": happened during read: unexpected EOF10:56
gibihttps://zuul.opendev.org/t/openstack/build/3be807e5dbf843ff995fbe1fdcb7fed2/log/job-output.txt#1146210:56
elodillesgibi: note that the 1st error is from nova-ceph-multistore and the second error is from nova-multi-cell job10:57
elodillesgibi: and the nova-multi-cell job failure is 2 test case failure10:57
elodillesso it's not the same10:58
sean-k-mooney ok cause i was looking at multi cell and not seeing any references to ceph :) 10:58
elodilles:]10:59
sean-k-mooneyoh it is11:00
sean-k-mooneyso looking at the multi store josb local.conf https://3e402c0e76741e83fc60-d00ff4f1a74cdbc5ea9d8044145b77c0.ssl.cf2.rackcdn.com/926555/2/check/nova-ceph-multistore/3be807e/controller/logs/_.localrc_auto.txt11:01
sean-k-mooneywe do not seam to be settign a release version in the job so we are using the defaults form the plugin11:01
sean-k-mooneywhich should be ok11:01
sean-k-mooneythat is defiend here https://github.com/openstack/devstack-plugin-ceph/blob/stable/2023.2/devstack/lib/ceph#L3111:04
sean-k-mooneythis is in the lib module meaing that it for internal use only otherwise it shoudl be in settings at the top of the plugin 11:04
sean-k-mooneyit should still be set but lateter so its probaly fine as is11:05
sean-k-mooneyso this is just trhe podman pull that is failing an di have seen that fail at least once in the last 2 weeks11:06
sean-k-mooneybut i didnt look into why so im not sure this is jobs specific11:07
sean-k-mooneyi think the way this is coded we download the container that ceph adm would use11:08
sean-k-mooneythen we parse teh version form that and download the corresponding cephadm binary directly11:08
sean-k-mooneyand then use that11:08
sean-k-mooneyso the fact its using el9 on ubuntu does not matter https://github.com/openstack/devstack-plugin-ceph/blob/stable/2023.2/devstack/lib/cephadm#L13211:09
sean-k-mooneyso ya this look like a generic podman pull issue with quay.io11:10
sean-k-mooneywe could perhaps retry that pull to mitigate it11:11
sean-k-mooneyalthough i think ideally we would mirror that or precache the container and cephadm in the base image at least for the default release11:12
frickleriiuc we do have a caching proxy for quay.io, so someone just would have to make that job use it (cc gouthamr)11:25
sean-k-mooneyah ok is that by configuing http_proxy ? or something else?11:25
sean-k-mooneywe have a generic setup mirrors role in the openstack-ci-jobs repo i think11:26
sean-k-mooneyi kind of assumed that also configured the http proxies11:26
fricklerthere was some discussion about this in some other channel last week, iirc that work never completed. but kolla does somehow use that proxy already11:27
sean-k-mooneyah i was thinking of https://github.com/openstack/openstack-zuul-jobs/tree/master/roles but i guess the morrior setup is in the generic zuul-jobs repo11:27
gibisorry for the mixup11:28
sean-k-mooneyi guess that does not handel the systm proxies 11:28
sean-k-mooneyhttps://opendev.org/zuul/zuul-jobs/src/branch/master/roles/configure-mirrors/defaults/main.yaml11:29
fricklersean-k-mooney: yes, afaict this is still missing for general zuul jobs. cf. https://opendev.org/openstack/kolla-ansible/src/branch/master/tests/templates/globals-default.j2#L82-L88 11:30
sean-k-mooneyso that could be a good change to put in the base devstack job11:30
sean-k-mooneydo we know why that has not been done before?11:30
sean-k-mooneyoh this is doing it via explicly using a difent registry11:32
fricklersean-k-mooney: iiuc this was started by some tripleo folks but never finished. https://meetings.opendev.org/irclogs/%23openstack-infra/%23openstack-infra.2024-08-21.log.html#t2024-08-21T11:06:16 was the discussion I was referring to11:32
sean-k-mooneyright i tought this might be settign the generic http proxy but its acatully podman/docker speciric config11:32
frickleryes, we cannot do a generic proxy since we could not properly secure it11:33
sean-k-mooneyi think the kolla approch need to have all usage of podman be parmatiarised by the registry11:33
sean-k-mooneyso we would need to modify the ceph devstack plugin11:33
sean-k-mooneybut that should not be hard11:33
frickleryes, I don't think podman is currently being used anywhere else in devstack? even when it was, that should not be a blocker11:34
sean-k-mooneyit might have been used for something else but we can always do it one plugin at a time11:35
sean-k-mooneyi dont think we use it in core devstack11:35
opendevreviewBalazs Gibizer proposed openstack/nova stable/2024.1: hardware: Correct log  https://review.opendev.org/c/openstack/nova/+/92711011:41
sean-k-mooneyfrickler: looks like we can either configure a mirror which will take precidnece and then fall back or we can remap prefixes11:50
sean-k-mooneyhttps://www.redhat.com/sysadmin/manage-container-registries11:50
sean-k-mooneyi think the mirror approch is prefered as it will try the direct connection if the mirror does not work11:50
sean-k-mooneyi can see if i can find time to code that up in core devstack11:51
*** bauzas_ is now known as bauzas13:27
opendevreviewDan Smith proposed openstack/nova master: Use format_inspector from oslo  https://review.opendev.org/c/openstack/nova/+/92502514:22
opendevreviewribaudr proposed openstack/nova master: Use client token when talking to manila  https://review.opendev.org/c/openstack/nova/+/92527714:55
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (drivers and compute manager part)  https://review.opendev.org/c/openstack/nova/+/83309014:55
opendevreviewribaudr proposed openstack/nova master: Mounting the shares as part of the initialization process  https://review.opendev.org/c/openstack/nova/+/88007514:56
opendevreviewribaudr proposed openstack/nova master: Deletion of associated share mappings on instance deletion  https://review.opendev.org/c/openstack/nova/+/88147214:56
opendevreviewribaudr proposed openstack/nova master: Add metadata for shares  https://review.opendev.org/c/openstack/nova/+/85050014:56
opendevreviewribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part)  https://review.opendev.org/c/openstack/nova/+/85482314:56
opendevreviewribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute manager part)  https://review.opendev.org/c/openstack/nova/+/85482414:56
opendevreviewribaudr proposed openstack/nova master: Add share_info parameter to resume method for each driver (driver part)  https://review.opendev.org/c/openstack/nova/+/86028414:56
opendevreviewribaudr proposed openstack/nova master: Support resuming an instance with shares (compute manager part)  https://review.opendev.org/c/openstack/nova/+/86028514:56
opendevreviewribaudr proposed openstack/nova master: Add helper methods to rescue/unrescue shares  https://review.opendev.org/c/openstack/nova/+/86028614:56
opendevreviewribaudr proposed openstack/nova master: Support rescuing an instance with shares  https://review.opendev.org/c/openstack/nova/+/86028714:56
opendevreviewribaudr proposed openstack/nova master: Allow to mount manila share using Cephfs protocol  https://review.opendev.org/c/openstack/nova/+/88386214:56
opendevreviewribaudr proposed openstack/nova master: Check shares support (compute manager)  https://review.opendev.org/c/openstack/nova/+/88575114:56
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (API)  https://review.opendev.org/c/openstack/nova/+/83683014:56
opendevreviewribaudr proposed openstack/nova master: Add helper methods to attach/detach shares  https://review.opendev.org/c/openstack/nova/+/88575314:56
opendevreviewribaudr proposed openstack/nova master: Add instance.share_attach notification  https://review.opendev.org/c/openstack/nova/+/85050114:56
opendevreviewribaudr proposed openstack/nova master: Add instance.share_detach notification  https://review.opendev.org/c/openstack/nova/+/85102814:56
opendevreviewribaudr proposed openstack/nova master: Add shares to InstancePayload  https://review.opendev.org/c/openstack/nova/+/85102914:56
opendevreviewribaudr proposed openstack/nova master: Add instance.share_attach_error notification  https://review.opendev.org/c/openstack/nova/+/86028214:56
opendevreviewribaudr proposed openstack/nova master: Add instance.share_detach_error notification  https://review.opendev.org/c/openstack/nova/+/86028314:56
opendevreviewribaudr proposed openstack/nova master: Reports instance events to the DB regarding attaching and detaching a share  https://review.opendev.org/c/openstack/nova/+/92708814:56
opendevreviewribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working.  https://review.opendev.org/c/openstack/nova/+/85208614:56
opendevreviewribaudr proposed openstack/nova master: Add virt/libvirt error test cases  https://review.opendev.org/c/openstack/nova/+/85208714:56
opendevreviewribaudr proposed openstack/nova master: Docs about Manila shares API usage  https://review.opendev.org/c/openstack/nova/+/87164214:56
dansmithbauzas: hopefully this will be quick, but here's the patch to deprecate AMI as we discussed.. people wanted to wait for you to return/approve: https://review.opendev.org/c/openstack/nova/+/92614415:10
bauzasdansmith: all good, will look at it now15:10
*** bauzas_ is now known as bauzas16:20
sean-k-mooneygibi: so as i was saying in the qa channel thanks for the reivews. i wonder wich is a higher priority to make progress on the healthch code or the eventlet removal16:48
sean-k-mooneyi put the healthcheck partly on top of the eventlet serise to avoid a merge conflcit but perhaps i should split them16:48
sean-k-mooneyim not convince we will have time to merge both or either of them so im open to input on which to proceed with first16:49
melwittsean-k-mooney: I just looked at the review comments you mention and yeah, that's the same conclusion that was reached with my patch is that there is no way to cancel a native thread (from the main thread). so it's either let it keep running (and assume it exits on its own eventually) or switch to processes. or like you said somehow run a timer in the child thread to exception exit itself, if there is a way we can do that16:58
sean-k-mooneywe might be able to do that with funtools.partil and have it create a time before callign the orginal funciton16:59
melwittthe code as-is in the patch will just move on after the timer expires and leave the worker thread 16:59
sean-k-mooneybut that feels even more hacky16:59
sean-k-mooneyi could use a process pool but that feels a bit heavy weight. if people are ok with that its easy to do i think16:59
sean-k-mooneyprocess pools are ok provide the funciton does not try to modify any shared sate17:00
sean-k-mooneyi dont think this code does but thats somethign we would need to be careful of17:00
melwittyeah. with my patch the review just kept going around in circles with nobody happy with anything so I hopefully you will have better luck17:00
sean-k-mooneyim feelign like i might be better off moving that patch to later in the series17:01
sean-k-mooneyhowever i see value in makeing the api eventlet free first17:01
sean-k-mooneysince otherwise its the simplest service to do that with17:02
*** bauzas_ is now known as bauzas17:23
opendevreviewmelanie witt proposed openstack/nova master: DNM testing  https://review.opendev.org/c/openstack/nova/+/92563519:17
opendevreviewmelanie witt proposed openstack/nova master: DNM testing  https://review.opendev.org/c/openstack/nova/+/92563519:19
sean-k-mooneymelwitt: by the way this is somehting i was playing with https://github.com/SeanMooney/os-vol19:20
melwittsean-k-mooney: that is very different :)19:30
sean-k-mooneyyep kind of intentionally19:30
sean-k-mooneyi was playing with the idea of storage backends abstracting how the storage is provisioned and allocating volumed which are then consumed19:30
sean-k-mooneywith the connection between a volume and a backend indirect though a storage pool19:31
sean-k-mooneyso you would not create a file or rbd volume19:32
sean-k-mooneyyou would ask the storage pool to allocate a volume of a given size from its backend19:32
sean-k-mooneythat volume object is the logical handel to the backing storage19:33
melwitthm ok19:33
sean-k-mooneymy idea was you could have multiple backend and have adaptor that can take data form diffen sources and import it into a volume19:34
sean-k-mooneyi have not really fleshed out the interface much as i said i was just playing with that desgin to see what it looked like19:35
melwittthat's cool19:35
sean-k-mooneyim not saying you should not consinue with your experiments by the way, the design im going for might be overly generic for what we want19:36
sean-k-mooneyon the other hand i kind of like the idea of having this in a self contianed lib or module with a stict inteface19:37
sean-k-mooneythe reason i have host_attach in the interafce for example is so we can allocate a file (lvm volume, rbd volume) 19:37
sean-k-mooneyattach it to the host19:38
sean-k-mooneycall makefs on it and unmount it19:38
sean-k-mooneywithout having to care where the data is stored19:38
melwittsure. where I was coming from is I wondered if we can pare down/simplify imagebackend as a baby step and whether or not it could make image format tracking easier in some way19:39
sean-k-mooneyyep19:39
sean-k-mooneyi think that would be valueable to do in anycase19:39
melwittI was thinking priority one is getting the format tracking done and I thought maybe removing some of the confusing layers of indirection could help facilitate that19:40
sean-k-mooneyyep i agree19:40
sean-k-mooneyi dont know if you looked at some of the tests i created https://github.com/SeanMooney/os-vol/blob/master/os_vol_tests/unit/backends/test_flat_file.py#L38-L63 but as i said this is just something im toying with form time to time19:42
sean-k-mooneyi have not considered how you could use this in nova really 19:42
sean-k-mooneyhaving a simplifed code path would help if we were ever going to consider going this way19:43
melwittdansmith: I tried a minimalistic refactor of imagebackend things with the goal of helping somehow with the image format tracking we want to do https://review.opendev.org/c/openstack/nova/+/925635 I dunno if something like that could help with the ideas you had for the format tracking19:44
dansmithmelwitt: ah cool, I'll queue up a look19:45
melwittsean-k-mooney: ah yeah, I didn't see the tests. that does make it easier to understand19:46
sean-k-mooneyi defintly have some bugs but im going to keep playing wiht it on and off and see if i can build it out to be something more useful but its just a toy for now.19:47
opendevreviewMerged openstack/nova stable/2024.1: Functional test test_boot_reschedule_with_proper_pci_device_count  https://review.opendev.org/c/openstack/nova/+/92655621:01
opendevreviewMerged openstack/nova stable/2024.1: Fix PCI passthrough cleanup on reschedule  https://review.opendev.org/c/openstack/nova/+/92655721:07
*** bauzas_ is now known as bauzas21:11
opendevreviewMerged openstack/nova master: Deprecate AMI image formats  https://review.opendev.org/c/openstack/nova/+/92614421:31

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!