| opendevreview | Merged openstack/nova master: [doc] Adding vGPUs max_instances caveat fix for virtual-gpu https://review.opendev.org/c/openstack/nova/+/949579 | 07:40 |
|---|---|---|
| opendevreview | Kamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event https://review.opendev.org/c/openstack/nova/+/949754 | 09:34 |
| sean-k-mooney | gibi: would you mind looking at https://review.opendev.org/c/openstack/nova/+/946581/2 also it the update to the repoducer below melwitt migration fix. im going to take a look at the migration fix now | 10:01 |
| opendevreview | Serhii Ivanov proposed openstack/nova-specs master: Proposes 'Add Graceful Nova Compute Shutdown' https://review.opendev.org/c/openstack/nova-specs/+/937185 | 10:11 |
| opendevreview | Serhii Ivanov proposed openstack/nova-specs master: Proposes 'Add Graceful Nova Compute Shutdown' https://review.opendev.org/c/openstack/nova-specs/+/937185 | 10:14 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: wip https://review.opendev.org/c/openstack/placement/+/949864 | 10:16 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove installation guide for openSUSE/SLES https://review.opendev.org/c/openstack/placement/+/949864 | 10:17 |
| opendevreview | Ivan Anfimov proposed openstack/nova master: wip https://review.opendev.org/c/openstack/nova/+/945781 | 10:38 |
| *** ykarel_ is now known as ykarel | 11:41 | |
| opendevreview | Lajos Katona proposed openstack/os-vif master: VS Trunk: Add bridge_name to external_ids https://review.opendev.org/c/openstack/os-vif/+/949736 | 11:58 |
| opendevreview | Merged openstack/nova master: Amend functional reproducer for bug 1899835 https://review.opendev.org/c/openstack/nova/+/946581 | 12:46 |
| MengyangZhang[m]1 | Hi folks,... (full message at <https://matrix.org/oftc/media/v1/media/download/AQCeKS0ygNQmR5GAVx8_xminTOY9XUuccxdyP6UnUQrmb3Vz1w1JBUlNsmlKKGyklGrLpna6pA6u3Ut2PN1dGyJCeXHE8e3wAG1hdHJpeC5vcmcvdnJhVFZrZ1hHbERzQWJWVFluSVJ0bE5s>) | 13:50 |
| *** kevko6 is now known as kevko | 15:49 | |
| gibi | stephenfin: sean-k-mooney: melwitt: I think we indeed have a useless lock in https://review.opendev.org/c/openstack/nova/+/949161/comment/e72e2ca3_b8b7f24f/ | 16:43 |
| sean-k-mooney | gibi: so this was there i belvie because of mod_wsig | 16:44 |
| sean-k-mooney | but i guess your right that becasue we reinitalise the lock | 16:45 |
| sean-k-mooney | even if its reloaded it wont guard the we we expect | 16:45 |
| gibi | yeah | 16:45 |
| sean-k-mooney | we need either only create the lock if it is not None or delete it | 16:45 |
| melwitt | hm | 16:45 |
| gibi | I think there is no way to threadsafely check if the variable is None and set it atomically | 16:46 |
| sean-k-mooney | if we really want we coudl proably apply the run once decorator to init_applciation | 16:47 |
| gibi | at least not in this context | 16:47 |
| sean-k-mooney | what we are trying to do is not have 2 thread reintiallse the applction object | 16:47 |
| gibi | we would need to guard the is None check with a lock :) | 16:48 |
| sean-k-mooney | on the other hand nova does nto currently supprot using multiple thread in mod_wsgi any way for other reasons | 16:48 |
| sean-k-mooney | gibi: you normlly fix this with atomics | 16:49 |
| gibi | yeah CAS | 16:49 |
| gibi | which we don't have :) | 16:49 |
| sean-k-mooney | well actully | 16:49 |
| sean-k-mooney | how we fix it in nova | 16:49 |
| sean-k-mooney | is we use the synconise decorator | 16:49 |
| sean-k-mooney | to create a filesystem level lock | 16:49 |
| melwitt | with the guard you would just check None once outside the lock and again inside the lock before proceeding to init, no? | 16:51 |
| sean-k-mooney | you mean double check locking that only works if you declari the loc with static lifetime | 16:51 |
| melwitt | right | 16:52 |
| gibi | the lock needs to exists before any of the two threads enters, otherwise both can create a lock as both can see that the lock variable is none | 16:52 |
| sean-k-mooney | so we can test this if we just import the module twice in an inteperer | 16:52 |
| gibi | to trigger the race the thread needs to be switched right after the first thread checked that the lock is none but before set the lock variable | 16:53 |
| gibi | so it is not easy to trigger the actual race | 16:53 |
| * melwitt yeah I guess like sean said either use an external file lock with the decorator or use the run_once decorator | 16:54 | |
| sean-k-mooney | oh becase of the if applcation is None | 16:54 |
| sean-k-mooney | melwitt: right if we use a file lock or any interprocess lock form oslo utils | 16:54 |
| melwitt | yeah. seems like one of those is consistent with other stuff in nova | 16:55 |
| sean-k-mooney | then the filesytem will ensure a strict orderign for us and the handle is just that | 16:55 |
| gibi | yeah external lock works, if we can create them before any of the threads enter, we cannot let the threads create the lock after they checking for none, as the none check would be unprotected | 16:55 |
| melwitt | we already @run_once for the global data init. I guess we could just move it? dunno | 16:56 |
| sean-k-mooney | we can add a do_init function the the curent logic and decorate that | 16:56 |
| sean-k-mooney | the problem with a file lock is we want this to actully work if you have mulipel processes | 16:57 |
| gibi | I think either the module import (and therefore the module level code execution) already happens under a lock or we cannot make this thread safe. | 16:57 |
| sean-k-mooney | so the name has to include the pid the trehad is form | 16:57 |
| gibi | yepp there is an import lock... | 16:58 |
| sean-k-mooney | gibi: well there is also the gil | 16:59 |
| sean-k-mooney | but we doen really want to rely on that | 16:59 |
| gibi | I don't see who can you create a lock in the module level code that is threadsafe and ensured to be only created once | 17:00 |
| gibi | a/who/how/ | 17:00 |
| gibi | if lock not is None: lock = createLock() # is not thread safe | 17:00 |
| gibi | it is only threadsafe as there is an explicit import lock in python so we know that the module level code only run by a single thread uninterupted | 17:01 |
| gibi | but with that we can have just an applicaton is None check, no need to any other locks | 17:02 |
| gibi | https://docs.python.org/fr/3.8/library/imp.html#imp.lock_held | 17:12 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Use futurist for _get_default_green_pool() https://review.opendev.org/c/openstack/nova/+/948072 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Replace utils.spawn_n with spawn https://review.opendev.org/c/openstack/nova/+/948076 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Add spawn_on https://review.opendev.org/c/openstack/nova/+/948079 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Move ComputeManager to use spawn_on https://review.opendev.org/c/openstack/nova/+/948186 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Move ConductorManager to use spawn_on https://review.opendev.org/c/openstack/nova/+/948187 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Make nova.utils.pass_context private https://review.opendev.org/c/openstack/nova/+/948188 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Rename DEFAULT_GREEN_POOL to DEFAULT_EXECUTOR https://review.opendev.org/c/openstack/nova/+/948086 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Make the default executor configurable https://review.opendev.org/c/openstack/nova/+/948087 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Document threading mode and tuneables https://review.opendev.org/c/openstack/nova/+/949364 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: WIP: allow service to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 17:13 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: DNM:Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | 17:13 |
| cardoe | hey all. I see you don't have https://review.opendev.org/c/openstack/nova-specs/+/471815 on your roadmap for 2025.2 | 17:52 |
| cardoe | Any chance we can get that added? | 17:52 |
| cardoe | There's a number of operators that are using Ironic with Nova and need trunk port data. | 17:53 |
| cardoe | The actual implementation of that was proposed by two different companies against nova. | 17:53 |
| cardoe | I've gotten everyone to collapse down behind Vasyl's patch and he's updated the spec as asked. | 17:54 |
| cardoe | There's a push on doing this on the Ironic side as well for some standalone cases. | 17:55 |
| cardoe | Really wanting to advocate for collaboration here between the projects and to fulfill operator need. | 17:55 |
| cardoe | We're essentially blocked on reviews. | 17:55 |
| ishanwar[m] | Hey | 18:06 |
| ishanwar[m] | sean-k-mooney (@_oftc_sean-k-mooney:matrix.org) | 18:06 |
| ishanwar[m] | can you have a look for this https://review.opendev.org/c/openstack/nova/+/945204 | 18:06 |
| sean-k-mooney | cardoe: is that the vnc console one | 18:07 |
| sean-k-mooney | to provide a graphical console | 18:07 |
| cardoe | No. That's trunk port info in metadata | 18:07 |
| sean-k-mooney | cardoe: oh its not | 18:07 |
| sean-k-mooney | cardoe: so that problemantic | 18:08 |
| sean-k-mooney | cardoe: the problem is tha twhen ironci is used vai nova it relaly should not be touchign the neutorn prots at all | 18:08 |
| sean-k-mooney | cardoe: which means that to supprot port group and trunkign it shoudl be modeled in neutron | 18:09 |
| sean-k-mooney | whic hits not today | 18:09 |
| cardoe | I don't disagree. | 18:10 |
| cardoe | One of the many specs on my queue to write. | 18:10 |
| sean-k-mooney | cardoe: without deep diving into the spec we probably can supprot addign the subport info into the metadtata api or config drive | 18:11 |
| sean-k-mooney | for any virt dirver | 18:11 |
| sean-k-mooney | even ironic | 18:11 |
| sean-k-mooney | but only if the trunk port is pass by uuid as part of the boot request | 18:11 |
| sean-k-mooney | if ironic makes changes to the prots out of band however | 18:12 |
| sean-k-mooney | i.e. maping that 1 neutorn trunk port to to a port group | 18:12 |
| sean-k-mooney | your kind of out of luck | 18:12 |
| sean-k-mooney | so like you could not really combin doign bondign via port groups and this | 18:13 |
| JayF | Yeah, that's been the root pain the whole time. To model this all in network metadata on the nova side requires it to be modeled all the way through, which I think has to start with neutron not with nova | 18:14 |
| sean-k-mooney | cardoe: or rather if you did try and combind them nova can only provide the info it had ebfore we call ironci to provision in the config drive | 18:15 |
| sean-k-mooney | JayF:right we can add any info that is aviabel from neutron at the point we generate the config drive | 18:15 |
| sean-k-mooney | but after that point either ironic need to update it itsself | 18:16 |
| sean-k-mooney | or we would need a way to trigger a refresh | 18:16 |
| JayF | (patches are close to merge that have ironic regen it) | 18:16 |
| sean-k-mooney | JayF: is that to adresss the bonding bug mnaser reported | 18:16 |
| JayF | I think we addressed that in a backportable way | 18:17 |
| cardoe | which is the bug mnaser reported? | 18:17 |
| JayF | https://launchpad.net/bugs/2106073 | 18:18 |
| sean-k-mooney | https://bugs.launchpad.net/ironic/+bug/2106073 | 18:18 |
| sean-k-mooney | :)_ | 18:18 |
| sean-k-mooney | cardoe: its a similar problam statement | 18:18 |
| sean-k-mooney | the config drive is needed to be abel to bootstrap the networkign to be able to hit the metadta when you are usign static ips or bonds | 18:19 |
| sean-k-mooney | specificly any bond that require both the swtich and host to be configured in a given bonding mode for traffic to flow | 18:19 |
| sean-k-mooney | cardoe: your usecase is related but seperate and i think valid | 18:20 |
| sean-k-mooney | cardoe: ironic could enrich the metadat with the turnk port info | 18:21 |
| sean-k-mooney | btu nova can do that too | 18:21 |
| sean-k-mooney | and nova could do it for libvirt or vmware in a generic way | 18:21 |
| cardoe | I would love if neutron modeled this all. Much like I’m trying to get L2 VNI in there. | 18:34 |
| sean-k-mooney | cardoe: when we dicussed trunkports intially we also disucssed bond ports | 18:35 |
| cardoe | Which is something already patched in nova | 18:35 |
| sean-k-mooney | the latter just didnt get doen btu i dont see why you cant have the same bond port sub port relationship | 18:35 |
| sean-k-mooney | port groups in ironci are a littel differnt however then just a bond or turnk port | 18:36 |
| cardoe | Trunk boards are already modeled inside of neutron | 18:36 |
| cardoe | Ports | 18:36 |
| cardoe | It’s nova that doesn’t handle it | 18:36 |
| cardoe | So it’s not really related to that bug. | 18:37 |
| sean-k-mooney | cardoe: so i did a very quick review and asked a few questions but +1 for the over all idea | 18:41 |
| sean-k-mooney | cardoe: im not sure if we currently test turnkprots in the nova ci jobs (maybe the ovs hybrid plug one?) but i know the neutron do and have a tempest plugin to do some more advanced testing with it | 18:43 |
| sean-k-mooney | so whiel im not sure if the trunk connectivty test would be accepted in core tempest it definlly could be done in one of neutron tempest plugins | 18:43 |
| sean-k-mooney | melwitt: speaking of specs are you plannign to work on the vtpm spec and implemetion this cycle. i think that was the plan aroudn the ptg but just touching base to make sure that has not changed? | 18:49 |
| * melwitt sean-k-mooney: yes I am -- still the plan. I have re-proposed the spec and noted the changes from the last approval in the commit message | 18:50 | |
| melwitt | argh | 18:50 |
| melwitt | sean-k-mooney: yes I am -- still the plan. I have re-proposed the spec and noted the changes from the last approval in the commit message | 18:50 |
| sean-k-mooney | cool. i will proably try and put aside some time to look at the open spec next week | 18:51 |
| melwitt | sure, thanks | 18:51 |
| * melwitt seen this a few times now, nova-ceph-multistore failures due to "nova.exception.ImageDeleteConflict: Conflict deleting image. Reason: HTTP 409 Conflict: Image a34561ed-6ef7-457c-8845-677f83af10de could not be deleted because it is in use: The image cannot be deleted because it is in use through the backend store outside of Glance.." | 18:55 | |
| melwitt | seen this a few times now, nova-ceph-multistore failures due to "nova.exception.ImageDeleteConflict: Conflict deleting image. Reason: HTTP 409 Conflict: Image a34561ed-6ef7-457c-8845-677f83af10de could not be deleted because it is in use: The image cannot be deleted because it is in use through the backend store outside of Glance.." | 18:56 |
| sean-k-mooney | ya | 18:56 |
| sean-k-mooney | i noticed that too | 18:56 |
| melwitt | I wonder if anything's changed or why it's happening often recently. I haven't spent time looking into it yet | 18:57 |
| cardoe | sean-k-mooney: https://review.opendev.org/c/openstack/nova/+/941227 is the implementation. Linked you’ll find the addition of tests and tempest patches as well to test it all. | 19:10 |
| sean-k-mooney | cardoe: can you put the implemation and the spec on a gerrit topic called bp/<sepc/blueprint name> | 19:32 |
| sean-k-mooney | convetionally the topic shoudl eb "bp/expose-vlan-trunking" in thei case but "spec/expose-vlan-trunking" is also perfectly fine | 19:33 |
| cardoe | Will do. | 19:33 |
| sean-k-mooney | cardoe: we try to keep teh filename/topic/spec link all the same. | 19:33 |
| sean-k-mooney | cardoe: im goint to call it a day now but did you sway there are also existing tempest tests | 19:36 |
| sean-k-mooney | cardoe: ah tehy were abandoned https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/941355 | 19:37 |
| cardoe | sorry was replying from mobile. I'll resurrect things. | 19:55 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!