| *** mhen_ is now known as mhen | 01:08 | |
| *** mhen_ is now known as mhen | 02:06 | |
| opendevreview | ribaudr proposed openstack/nova master: Reproducer for bug 2114951 https://review.opendev.org/c/openstack/nova/+/952894 | 08:05 |
|---|---|---|
| opendevreview | ribaudr proposed openstack/nova master: Fix bug 2114951 https://review.opendev.org/c/openstack/nova/+/952895 | 08:05 |
| opendevreview | ribaudr proposed openstack/nova master: Add Flamingo prelude section https://review.opendev.org/c/openstack/nova/+/959188 | 08:25 |
| opendevreview | Tobias Urdin proposed openstack/nova master: Prevent leaking host info when HostMappingNotFound https://review.opendev.org/c/openstack/nova/+/959296 | 10:00 |
| opendevreview | ribaudr proposed openstack/nova master: Add Flamingo prelude section https://review.opendev.org/c/openstack/nova/+/959188 | 10:23 |
| opendevreview | Merged openstack/nova master: Fix duplicate words https://review.opendev.org/c/openstack/nova/+/959136 | 10:24 |
| nate | Hi ! I have a question regarding compute nodes and cinder volume encryption. We use a ceph backend with RBD with LUKS encrypted volume with encryption keys stored in barbican. At volume creation, the key belong to the end-user creating the volume. But our compute node, who has a dedicated compute-nodeX openstack user, can access this key. How does this magic happen in term of | 10:24 |
| nate | rights, since our compute nodes do not have any admin role ? | 10:24 |
| nate | Our compute nodes have the service role on the service project, as well as a custom admin-compute role that only has two extra policies in neutron | 10:31 |
| opendevreview | Merged openstack/nova master: [CI]Make nova-tox-py312-threading voting https://review.opendev.org/c/openstack/nova/+/955791 | 10:40 |
| gibi | Uggla: re ceph volume creation gate issue. I filed https://bugs.launchpad.net/cinder/+bug/2121941 | 11:41 |
| gibi | it happens about 30% of the time | 11:41 |
| gibi | Uggla: as the patch making the py312-threading job voting landed I marked https://blueprints.launchpad.net/nova/+spec/eventlet-removal-part-1 as implemented and opened https://blueprints.launchpad.net/nova/+spec/eventlet-removal-gazpacho as the continuation | 11:48 |
| sean-k-mooney | nate: the comput and admin cannot access the key, we store it in libvirt using the enduser access when the vm is creatd | 12:03 |
| sean-k-mooney | its the presence of the key as a libvirt secreate that allows it to work | 12:03 |
| sean-k-mooney | so it was the user keystone token that granted access but we only have that whiel we are attachign the voluem or creating the vm based on there request | 12:04 |
| jkulik | sean-k-mooney: would that key then get transported on live-migration to another node, too? does libvirt do that automatically or is there some special mechanism in Nova? | 12:05 |
| sean-k-mooney | gibi: we debuged that cidner volume isseu a few weeks ago | 12:05 |
| sean-k-mooney | jkulik: i belivw ew do not suprot live migration with encypted volumes today | 12:05 |
| jkulik | ah. ok. I was assuming :) | 12:05 |
| sean-k-mooney | jkulik: that was a limitation intially and we plan to solve it using the same stragey for vtpm mitragion when that is complete | 12:06 |
| gibi | sean-k-mooney: it is the same? what is the solution? is there a bug I can duplicate mine to? | 12:06 |
| sean-k-mooney | for vtpm migration we are introducing 2 other modes, deployment where the encyrption keey will be owned by nova not the end user and host where the secret will be copied form host to host by reading it form libvirt and creating it | 12:07 |
| sean-k-mooney | gibi: no, i dough into it, found the error and looked at how cidner is implemtneding cpu trotteling breifly | 12:07 |
| sean-k-mooney | i brought it up on the QA channel | 12:08 |
| sean-k-mooney | but i think i found a bz for it | 12:08 |
| sean-k-mooney | ill need to check back | 12:08 |
| jkulik | very helpful. thank you. also found the spec for that: https://github.com/openstack/nova-specs/blob/master/specs/2025.2/approved/vtpm-live-migration.rst | 12:09 |
| sean-k-mooney | there is a diffent bug in teh qemu we are suing on debian 12 that i think is already fixed | 12:09 |
| sean-k-mooney | gibi: let me check the irc logs and see if i can find the details | 12:09 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/2025.1: Reproduce that only half of the PCI devs are removed https://review.opendev.org/c/openstack/nova/+/959308 | 12:10 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/2025.1: Fix pci_tracker.save to delete all removed devs https://review.opendev.org/c/openstack/nova/+/959309 | 12:10 |
| gibi | sean-k-mooney: thanks | 12:10 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/2024.2: Reproduce that only half of the PCI devs are removed https://review.opendev.org/c/openstack/nova/+/959310 | 12:12 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/2024.2: Fix pci_tracker.save to delete all removed devs https://review.opendev.org/c/openstack/nova/+/959311 | 12:12 |
| sean-k-mooney | gibi: https://meetings.opendev.org/irclogs/%23openstack-qa/%23openstack-qa.2025-08-12.log.html | 12:13 |
| sean-k-mooney | so its https://bugzilla.redhat.com/show_bug.cgi?id=2336437 | 12:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/2024.1: Reproduce that only half of the PCI devs are removed https://review.opendev.org/c/openstack/nova/+/959313 | 12:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/2024.1: Fix pci_tracker.save to delete all removed devs https://review.opendev.org/c/openstack/nova/+/959314 | 12:13 |
| sean-k-mooney | gibi: oh right i stated trying to fix it | 12:13 |
| sean-k-mooney | so i put this on hold due to the security bug then FF | 12:13 |
| sean-k-mooney | gibi: teh fix i starteed on is using a newer qemu form the backports repo | 12:14 |
| sean-k-mooney | there are 2 ways to do that i was trying the one clark suggested althouth the other way is simpler | 12:14 |
| sean-k-mooney | the fixed qemu is avaibel in debian 12 backporta dn proably debian 13 so we can jsut swap the job to 13 on master after RC1 once we clean up the devstack patch for debian 13, so that will be a month or so away | 12:15 |
| gibi | ack. I added the bz to the cinder bug | 12:15 |
| sean-k-mooney | for stable if we use qemu form the backprots repo it "should" fix it but we will need to either use -t to enabel the backports repo when isntalling qemu or update the name of the package to include /backprots or wahtever | 12:16 |
| sean-k-mooney | the latter is the harder approch i was trying but then i need to list all the package however it would be cleaner in the long rung | 12:16 |
| sean-k-mooney | the former is eaiser to do but it requires use to supprot that in the package_install comamnd | 12:17 |
| sean-k-mooney | which is doable jsut need some extra pluming to make it happen | 12:17 |
| gibi | I see | 12:18 |
| gibi | I will use the bug I filed on cinder as the target of rechecks but feel free to re-target that bug from cinder to whathever we use to track package issues | 12:18 |
| nate | sean-k-mooney: Thanks for the clarification. But how does nova create the secret in libvirt. How does it access the secret stored in barbican ? You mentioned end-user access token. How is it passed to nova-compute ? | 12:20 |
| sean-k-mooney | looks like i did a dumb and deleted my psi vm for debian 12 with the patch i have not pushed yet but i didn tget that far anyway | 12:20 |
| sean-k-mooney | nate: yes so wehn a use attaches an encypted volume or creates a vm with it the pass a keystoen token, nova uses that token to access barbican and retrive the key and store it in libvirt | 12:21 |
| sean-k-mooney | nate: but we only have that token in memory while we are processign teh api request form the user | 12:21 |
| sean-k-mooney | so its discarded and nova does nto have access beyond that intial point | 12:21 |
| sean-k-mooney | gibi: ill at this to my list to work on again next week | 12:22 |
| nate | sean-k-mooney: Ok so nova running on the compute node get this token and use it to retrieve the encryption key ? Am I correct ? | 12:23 |
| nate | sean-k-mooney: because I have a call to barbicanclient in the logs of nova on my compute node | 12:24 |
| sean-k-mooney | yes | 12:24 |
| sean-k-mooney | nova uses the toke supplied by the user to talk to other service like glance to get the image or barbican to get the secret | 12:24 |
| sean-k-mooney | nova itself does nto have the ablity to retirive the barbica seccret with the creditals in the nova.conf | 12:25 |
| sean-k-mooney | but the users token does adn like all service we are expected to use the users token where possibel to talk to other service when processing a users request | 12:25 |
| sean-k-mooney | that why a user can take the action but an admin cannot | 12:26 |
| nate | sean-k-mooney: Perfect great ! Thank you that clarifies it. | 12:29 |
| sean-k-mooney | gibi: just for contenxt this is the call that either need to be done with -t backports https://github.com/openstack/devstack/blob/master/lib/nova_plugins/functions-libvirt#L72 or we need to move thos packages to here with /backports and # dist:debian12 | 12:30 |
| sean-k-mooney | i tired claks suggestion and while i could get it to work having to specyify all the deps one by one is more fagile IMO then doign it the other way so i think im goign to try the -t approch isntead next time. | 12:31 |
| sean-k-mooney | gibi: basiclly i would have to list all the same packages you did in yoru downgrade patch for qemu-img and thats a pain | 12:32 |
| sean-k-mooney | addin a branch to install_libvirt for debian 12 i think its just simpler the refactoing the install to use files | 12:33 |
| Uggla | gibi thanks for the above info. | 12:51 |
| Uggla | the msg from qemu-img is quite strange. | 12:53 |
| opendevreview | Tobias Urdin proposed openstack/nova master: Retry more volume API functions on server error https://review.opendev.org/c/openstack/nova/+/942981 | 13:11 |
| ratailor | sean-k-mooney, gibi could you please review https://review.opendev.org/c/openstack/nova/+/946733 and https://review.opendev.org/c/openstack/nova/+/954460 | 14:02 |
| opendevreview | Rajesh Tailor proposed openstack/nova master: Fix typos https://review.opendev.org/c/openstack/nova/+/959322 | 14:03 |
| ratailor | Uggla, could you please review this clean backport to stable/2024.1 https://review.opendev.org/c/openstack/nova/+/952961 | 14:06 |
| Uggla | ratailor, sure I'm going to do it. | 14:06 |
| ratailor | Uggla, Thanks! | 14:07 |
| gibi | ratailor: https://review.opendev.org/c/openstack/nova/+/954460 is pretty featury so that probably need to wait G | 14:12 |
| gibi | I'm +2 on https://review.opendev.org/c/openstack/nova/+/946733 | 14:14 |
| ratailor | gibi, ack. Thanks! could you please glance through it and check if anything is missing. when you have time. | 14:14 |
| sean-k-mooney | ya i feel the same about 954460 | 14:22 |
| sean-k-mooney | we can revisit it after rc1 is cut in 8 days | 14:22 |
| sean-k-mooney | so ping us again in like 2 weeks | 14:22 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/946733 is ok before RC1 but ill review it properly when i get back to my desk, grabing coffee | 14:23 |
| ratailor | sean-k-mooney, sure. Thanks! I will do. | 14:27 |
| ratailor | sean-k-mooney, should I resolve conflict here https://review.opendev.org/c/openstack/nova/+/957958 with your permission, looks like you backported directly from UI. | 14:28 |
| sean-k-mooney | i did and i just have not had time to update it | 14:47 |
| bauzas | gibi: sorry forgot the eventlet meeting, do we have it ? | 14:47 |
| sean-k-mooney | i fyou want to rebase it properly feel free i was going to wait until i cleared my current backlog of patche befoer redoing it | 14:47 |
| sean-k-mooney | that why i set - workflow on the patch orginally | 14:48 |
| sean-k-mooney | i wanted to wait until after FF/RC was out of the way to finish the backport | 14:48 |
| opendevreview | Rajesh Tailor proposed openstack/nova stable/2025.1: only show standard image properties in server show. https://review.opendev.org/c/openstack/nova/+/957958 | 15:00 |
| zigo | I'm having an issue with tempest's packaging: it's having tempest/cmd which conflicts with sytem's cmd.Cmd. Here's the build log: | 15:28 |
| zigo | https://paste.opendev.org/show/bb1c8yKdZi4J7aKd4KsY/ | 15:28 |
| zigo | It's obviously failing when building the sphinx doc. Anyways, does anyone have a clue what I could do? (except a painful tempest.cmd folder rename) | 15:28 |
| clarkb | is something doing from tempest import cmd? Otherwise they shouldn't conflict right? | 15:38 |
| clarkb | hrm maybe it is part of the sphinx imports situation. Looks like your traceback has a whole import_module() routine that could be "polluting" things | 15:40 |
| sean-k-mooney | so you can have issue like that on ocation if thing do relitive imporat based on your workign dir btu this wi oen of the reason we never use relitive imports in openstack | 16:05 |
| sean-k-mooney | zigo: by the way this is proably more suitbale for #openstack-qa | 16:06 |
| sean-k-mooney | sinc ethis does not seam to be related to nova in nay way | 16:07 |
| sean-k-mooney | zigo: oh | 16:07 |
| gibi | bauzas: yeah we had one, I forgot to ping here. Bottom line we discussed a bit of scale testing, a bit of fixing sync rpc calls like volume attach and a bit of switching defaults in early G from eventlet to threading in the API | 16:07 |
| sean-k-mooney | zigo: why are you doing PYTHONPATH=. | 16:07 |
| sean-k-mooney | your adding the current workign dir into the python path | 16:07 |
| bauzas | gibi: thanks for the summary | 16:07 |
| sean-k-mooney | that could cause import issues like that | 16:07 |
| sean-k-mooney | gibi: ah sorry i ment to join i got nerd sniped by somethign else | 16:08 |
| sean-k-mooney | gibi: by the way i almost never use our fake driver, but if we wanted to test a mroe compelte fake driver eventually we coudl likely enabel the libvirt fixture in it vai a config option or siilar. it still would not actully talk to libvirt but it would be much more complete | 16:09 |
| sean-k-mooney | basiclly run it the way we do in the libvirt fucntional tests | 16:10 |
| sean-k-mooney | i dont think you can do that via config today but it could be intersting to try | 16:10 |
| sean-k-mooney | kolla has or at least had the ablity to deploy multiple copies of the nova and neutorn fake drivers per physical host in the past | 16:11 |
| sean-k-mooney | allowing you to take 3 host and run 10s to 100s of fake hosts | 16:11 |
| sean-k-mooney | i think devstack might eb able to do that too but its been 8 years or so since i actully tried to use any of that | 16:12 |
| gibi | yeah it is a good idea to try to upgrade our fake driver to use the libvirt fixture. I have an item for G to have fake driver based scale testing that can be extended with this | 16:14 |
| sean-k-mooney | https://docs.openstack.org/devstack/latest/guides/nova.html#fake-virt-driver | 16:16 |
| sean-k-mooney | so we do still have the basic supprot but i dotn knwo if this ever worked with ovn adn i suspect there coudl be other issues. | 16:17 |
| sean-k-mooney | just bit rot honstly more then anything fundemental | 16:17 |
| sean-k-mooney | but the ml2 ovsagent can run in a fake mode, the other hack is to just test without networking | 16:18 |
| opendevreview | ribaudr proposed openstack/nova master: Add Flamingo prelude section https://review.opendev.org/c/openstack/nova/+/959188 | 16:20 |
| gmaan | Uggla: you need quick fix here which is caused by rebase, btw updates lgtm now - https://review.opendev.org/c/openstack/nova/+/952894/comment/75fbb034_6c75825e/ | 16:22 |
| Uggla | gmaan, thx I did not noticed it failed. I gonna fix it right now. | 16:24 |
| opendevreview | ribaudr proposed openstack/nova master: Reproducer for bug 2114951 https://review.opendev.org/c/openstack/nova/+/952894 | 16:48 |
| opendevreview | ribaudr proposed openstack/nova master: Fix bug 2114951 https://review.opendev.org/c/openstack/nova/+/952895 | 16:48 |
| opendevreview | Merged openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994 | 16:59 |
| opendevreview | Merged openstack/nova master: Fix 'nova-manage image_property set' command https://review.opendev.org/c/openstack/nova/+/946733 | 17:29 |
| gmaan | Uggla: thanks +2 on both | 18:38 |
| zigo | sean-k-mooney: It's normal to do PYTHONPATH=., because the package isn't installed in /usr/lib/python3/dist-packages when calling sphinx, that's a way to add the module in the path. | 19:55 |
| zigo | I do that very often in my packages, with no issue. | 19:55 |
| zigo | As for my error here, the weird thing is that it happens it crashes in Unstable, but not in Trixie, so something must have changed. | 19:55 |
| zigo | I'll try to do a pip freeze and compare. | 19:55 |
| clarkb | zigo: why not add the path to the location rather than .? | 19:58 |
| clarkb | then cmd would only be resolvable as tempest.cmd or the stdlib stuff | 19:59 |
| zigo | clarkb: It produces the same result, that's not the issue. | 20:31 |
| clarkb | zigo: I think that means you're a directory level too deep? | 20:33 |
| clarkb | youre in tempest/ so cmd is there and . allows you to import cmd that way. Another approach would be to put it at the end of your path if python resolves the path in order | 20:33 |
| zigo | I'm not. | 20:33 |
| zigo | I'm on top of that folder. | 20:33 |
| clarkb | its possible that the sphinx build changes its cwd maybe? (but I would expect the rooted version instead of '.' to fix that) | 20:39 |
| zigo | Ok, the diff is because of sphinx version. | 20:40 |
| zigo | 8.1.3 in Trixie, vs 8.2.3 in Unstable. | 20:41 |
| zigo | I checked building in Trixie, adding sphinx as --extra-package in the sbuild command line. | 20:41 |
| zigo | And then it fails ... | 20:42 |
| zigo | clarkb: Is there a way I can do such demonstration with a patch in gerrit? | 20:42 |
| zigo | In conf.py, there's: | 20:46 |
| zigo | sys.path.insert(0, os.path.abspath('../../tempest')) | 20:46 |
| zigo | That's a likely guilty thingy ... :P | 20:46 |
| clarkb | it could be just run the tempest doc job with the version of sphinx that you want | 20:46 |
| clarkb | I think you might need to edit requirements upper-constraints and do a depends on for that | 20:47 |
| clarkb | its possible that should be os.path.abspath(os.path.join(__file__, '../../')) | 20:47 |
| clarkb | then you have to refer to things as tempest.cmd not just cmd | 20:47 |
| zigo | BINGO !!! | 20:49 |
| zigo | I was right. | 20:49 |
| zigo | I see no reason why the conf.py does this ... When I remove it, building the doc continues to work. | 20:51 |
| clarkb | the git log might give a hint. But if it works without it then removing it seems like a good option | 20:51 |
| zigo | Yeah, let me git blame. | 20:52 |
| clarkb | If extensions (or modules to document with autodoc) are in another directory, add these directories to sys.path here. If the directory is relative to the documentation root, use os.path.abspath to make it absolute, like shown here. | 20:52 |
| clarkb | thats from the comment in conf.py. So its related to autodoc. It is possible that it is required to make autodoc work and then that breaks sphinx trying to use cmd | 20:53 |
| clarkb | zigo: if you look at the automodule directives they all start with tempest | 20:55 |
| clarkb | so my hunch here is that the path manipulation is unnecessary (because its off by a level anyway) and that the doc build must be relying on tox installing the package before building | 20:55 |
| clarkb | from https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#ensuring-the-code-can-be-imported the code just need to be importable | 20:55 |
| clarkb | if you need to keep it then I think you just need __file__ + '../../' then tempest is in that directory and the automodule directives should find it is importable from there. But also we can stop polluting the nova channel as this is a qa problem | 20:56 |
| zigo | Let's see how it goes: https://review.opendev.org/c/openstack/tempest/+/959390 :) | 20:57 |
| zigo | clarkb: Yeah, wrong chan, appologies. | 20:58 |
| zigo | Comes from here: https://review.opendev.org/c/openstack/tempest/+/735619 | 21:08 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!