| mikal | stephenfin: I can't find an example in the nova codebase of a unit test which tests parsing for a non-string flavor extra spec. I'll do one up, but I think its a thing we've not deeply tested in the past unless every other extra spec is always a string or something. | 00:45 |
|---|---|---|
| *** mhen_ is now known as mhen | 01:23 | |
| jssfr | sean-k-mooney, regarding NVRAM, we're currently working on reviving https://review.opendev.org/c/openstack/nova/+/621646, too | 05:57 |
| jssfr | (this is somewhat re discussion in https://review.opendev.org/c/openstack/nova/+/955657, but also because I saw your other change) | 05:57 |
| opendevreview | Merged openstack/nova master: api: Add response body schemas for removed APIs https://review.opendev.org/c/openstack/nova/+/952282 | 06:11 |
| opendevreview | Ian Watson proposed openstack/nova master: Add migrations_supported methed for resource tracker https://review.opendev.org/c/openstack/nova/+/955690 | 08:32 |
| opendevreview | Ian Watson proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | 08:59 |
| opendevreview | Ian Watson proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | 09:03 |
| *** ykarel_ is now known as ykarel | 09:08 | |
| opendevreview | Ian Watson proposed openstack/nova master: Don't remove instance from cache if 409 hit https://review.opendev.org/c/openstack/nova/+/955686 | 09:16 |
| opendevreview | Ian Watson proposed openstack/nova master: Changing how placement updates are triggered in RT https://review.opendev.org/c/openstack/nova/+/955687 | 09:23 |
| opendevreview | Ian Watson proposed openstack/nova master: Add migrations_supported methed for resource tracker https://review.opendev.org/c/openstack/nova/+/955690 | 09:28 |
| rpittau | hello everyone! :) | 10:22 |
| rpittau | we've started to see an error in the ironic grenade job a couple of days ago, you can see an example report here https://e33cfc4e5ae5ade32027-ad9677b8f3b079990c4951ac9cbbd797.ssl.cf2.rackcdn.com/openstack/2f8dfc2059784e6691e13e08876ef108/controller/logs/grenade.sh_log.txt | 10:22 |
| rpittau | it looks like the issue is somehow related to nova, at least from the errors we can see there, like 2025-07-23 21:16:59.494 | ERROR nova.objects.instance [[01;36mNone req-c824579b-82f9-499c-8b59-382bdf74f47b [00;36mNone None] [01;35m[instance: 00000000-0000-0000-0000-000000000000] Unable to migrate instance because host None with node None not found[00m: nova.exception.ComputeHostNotFound: Compute host None could not be found. | 10:22 |
| rpittau | sorry for the wall of text, hoping to find someone who can help have a look here, thanks! | 10:22 |
| gibi | rpittau: was the instance being migrated here is booted successfully on a node before? Does the test tries to migrate with a target host? If so what is the target host the test sends in the API request? | 11:18 |
| gibi | does ironic supports migration at all? | 11:18 |
| gibi | :) | 11:18 |
| opendevreview | Lajos Katona proposed openstack/nova master: blueprint: iothreads-for-instances https://review.opendev.org/c/openstack/nova/+/939254 | 11:42 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | 12:50 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | 12:57 |
| noonedeadpunk | sean-k-mooney: this is basically parallel migration thing we've talked last week ^ | 12:58 |
| noonedeadpunk | just need to poke at unit tests a little bit | 12:58 |
| noonedeadpunk | really interesting thing, is that tunneled migrations does have better performance then native migrations with tls | 12:59 |
| noonedeadpunk | so the bottleneck was always not the tunnel all that time... | 13:00 |
| sean-k-mooney | you mean tunneled migration with multiple connections | 13:00 |
| sean-k-mooney | is beter then netive with 1? | 13:00 |
| noonedeadpunk | tunneled with single is better then native with single | 13:00 |
| sean-k-mooney | or is native tls with multipel connections somehow slower then tunnled with multiple | 13:00 |
| noonedeadpunk | I don't think you can do parallel with tunnel | 13:00 |
| sean-k-mooney | really... | 13:00 |
| sean-k-mooney | that does not track with the testing we did at the tiem but if that is ture | 13:01 |
| sean-k-mooney | it defintly soemthing we shoudl follow up on | 13:01 |
| sean-k-mooney | to understand why | 13:01 |
| noonedeadpunk | at least according to what I tested in a lab with EPYC 9534 as a cpu | 13:01 |
| sean-k-mooney | that a very interesting datapoint | 13:01 |
| noonedeadpunk | without tls it's way faster though | 13:01 |
| noonedeadpunk | needless to say about parallel thing | 13:02 |
| noonedeadpunk | I put some results here: https://review.opendev.org/c/openstack/nova-specs/+/955783/1/specs/libvirt-parallel-migrate.rst | 13:02 |
| noonedeadpunk | Under tls enabled/disabled I mean [libvirt]/live_migration_with_native_tls just in case | 13:06 |
| noonedeadpunk | as [libvirt]/live_migration_scheme was always `tls` | 13:07 |
| noonedeadpunk | as first option is responsible for how memory is being transferred during migration by qemu, the second is how libvirt connection/auth is performed in libvirt | 13:08 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | 13:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | 13:35 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Document native threading mode and tuneables https://review.opendev.org/c/openstack/nova/+/949364 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Allow services to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-api and -metadata in threaded mode https://review.opendev.org/c/openstack/nova/+/951957 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor https://review.opendev.org/c/openstack/nova/+/952666 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet https://review.opendev.org/c/openstack/nova/+/953436 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Run unit test with threading mode https://review.opendev.org/c/openstack/nova/+/953475 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively https://review.opendev.org/c/openstack/nova/+/953815 | 13:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [CI]Make nova-tox-py312-threading voting https://review.opendev.org/c/openstack/nova/+/955791 | 13:54 |
| opendevreview | Artem Vasilyev proposed openstack/nova master: mypy: fix type annotations for PciDevicePoolList and NUMATopologyLimits https://review.opendev.org/c/openstack/nova/+/955792 | 14:03 |
| opendevreview | Artem Vasilyev proposed openstack/nova master: Fix functional tests and mypy linter on macOS https://review.opendev.org/c/openstack/nova/+/937727 | 14:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | 14:19 |
| opendevreview | Merged openstack/nova master: Implement sound model extra spec for libvirt. https://review.opendev.org/c/openstack/nova/+/940770 | 14:24 |
| sean-k-mooney | noonedeadpunk:i did a quick pass but ya overall that in line whith what we discussed | 14:42 |
| sean-k-mooney | i think that woudl be a good improvement | 14:42 |
| noonedeadpunk | damn, I missed the directory for spec /o\ | 14:47 |
| sean-k-mooney | what matters is you have the content in the spec that is needed to get input early and agree the direction | 15:00 |
| noonedeadpunk | ++ | 15:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | 15:10 |
| noonedeadpunk | there is also a some implementation of it in ^ | 15:11 |
| noonedeadpunk | I got your comments and will adjust things accordingly | 15:11 |
| stephenfin | gibi: I've been discussing this with sean-k-mooney already, but I'm seeing an issue where I get a timeout "[w]aiting for libvirt event about the detach of device" | 16:24 |
| stephenfin | I'm seeing this on Dalmatian. I haven't tried later releases. It doesn't happen on Bobcat. Curiously, it also doesn't happen if I upgrade Bobcat to Dalmatian (bump all services, bump all dependencies in the global venv, compare all service configuration, ...) | 16:26 |
| stephenfin | Distro is identical (22.04 with latest updates), distro packages are effectively identical, OpenStack configuration should be identical (per above), so I'm mostly stumped. I'm *also* discussing with eharney_, but I'm noting here since iirc you wrote that code initially and you may have some tidbit nestled away in the back of your head :D | 16:27 |
| stephenfin | and the net effect is volume detaches take 20 seconds every single time on Dalmatian, while they happen immediately on Bobcat (upgraded to Dalmatian) | 16:28 |
| opendevreview | Biser Milanov proposed openstack/nova-specs master: Add the handling-nova-compute-restarts-during-live-migration spec https://review.opendev.org/c/openstack/nova-specs/+/955812 | 17:07 |
| sp-bmilanov | gibi: dansmith: based on your suggestion https://review.opendev.org/c/openstack/nova-specs/+/955812 I have to go now, but I hope we discuss it again on the next Nova weekly | 17:12 |
| sean-k-mooney | ... | 17:39 |
| sean-k-mooney | how is https://review.opendev.org/c/openstack/nova/+/944054 not merged already.... | 17:39 |
| sean-k-mooney | this was ment to be in epoxy | 17:39 |
| sean-k-mooney | before we released ... | 17:39 |
| opendevreview | sean mooney proposed openstack/nova master: only show standard image properties in server show. https://review.opendev.org/c/openstack/nova/+/942413 | 17:57 |
| opendevreview | sean mooney proposed openstack/nova master: [tests] Add printing of sample and template paths https://review.opendev.org/c/openstack/nova/+/944054 | 17:57 |
| sean-k-mooney | dansmith: can you readd your +2 to those... that was ment to be included before we released epoxy. i cosidered that to be a release blocker so sin tthat aprpently didnt happen im keen to get that fixed in stable as soon as possible | 17:59 |
| sean-k-mooney | i only noticed that was not there because i went to debug a test an notice my follow patch to improve the test loging was missing | 18:00 |
| gibi | sean-k-mooney: fast approved both | 18:04 |
| sean-k-mooney | thansk | 18:04 |
| sean-k-mooney | i think we just missed this in the scamble to fix all the tempest schemam issues that those micoverions caused | 18:05 |
| sean-k-mooney | i.e. missed that it failed to merge | 18:05 |
| sean-k-mooney | i was 100% certian this was in | 18:05 |
| gmaan | tempest sche3ma check key-value pattern not exact values so we are good there https://github.com/openstack/tempest/blob/master/tempest/lib/api_schema/response/compute/v2_98/servers.py#L32 | 18:14 |
| sean-k-mooney | gmaan: ya the tempest side is fine | 18:15 |
| sean-k-mooney | it was more that we discoverd the issue when trying to merge the speice stuff and hadn several patches in flight | 18:15 |
| sean-k-mooney | this just fell through the cracks | 18:15 |
| sean-k-mooney | gmaan: by the way i didnt find time to look at the testing of the manager role stuff today but ill try and do that tomorrow | 18:16 |
| gmaan | sean-k-mooney: no issue. thanks | 18:17 |
| opendevreview | Merged openstack/nova master: api: Add response body schemas for networks API https://review.opendev.org/c/openstack/nova/+/952283 | 20:19 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!