Wednesday, 2025-07-02

*** bauzas2 is now known as bauzas00:14
*** bauzas7 is now known as bauzas06:20
r-taketnHi cores. I would appreciate it if you could review the SEV-ES code(https://review.opendev.org/q/topic:%22bp/amd-sev-es-libvirt-support%22) for the 2025.2 release. The SEV-ES code looks good to me. I apologize if this is already part of your review plan (https://etherpad.opendev.org/p/nova-2025.2-status#L88).06:44
*** bauzas7 is now known as bauzas07:34
opendevreviewMasahito Muroi proposed openstack/nova-specs master: Add spec for virtio-blk multiqueue and iothreads extra_spec  https://review.opendev.org/c/openstack/nova-specs/+/95163609:12
fricklermikal: looks like we might need to evict more pkgs from our mirror. or ubuntu needs to fix this be releasing pkgs with a bumped version09:16
frickleroh, seems this was discussed in the glance channel already09:20
mikalOh, I'm not in the glance channel so I haven't seen that.09:38
mikalIs there anything you need me to do?09:38
fricklermikal: iiuc fungi was working on this, I'm not sure if the cleanup was completed already. do you still see new failures currently?09:44
mikalI haven't tried for a few hours. Let me try again again now.09:47
mikalOh I lie, I kicked off a recheck an hour ago. Its still running. Results will be at https://review.opendev.org/c/openstack/nova/+/926126 when they're done.09:48
mikalIn other questioins... I am unclear on what problem Glean is solving, apart from cloud-init being a bit big and complicated these days. Someone please to explain?09:49
opendevreviewKamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event  https://review.opendev.org/c/openstack/nova/+/94975410:01
opendevreviewKamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event  https://review.opendev.org/c/openstack/nova/+/94975410:02
fricklermikal: yes, cloud-init pulls in a lot of python dependencies at the distro level, which may conflict with dependencies for OpenStack projects, which is why we try to avoid it for CI purposes10:06
frickleralso no failures so far, so looks like things might be fixed https://zuul.opendev.org/t/openstack/status?change=92612610:07
Ugglasean-k-mooney, gibi, I know you are busy, but It would be great if you can have a look at 951636: Add spec for virtio-blk multiqueue and iothreads extra_spec | https://review.opendev.org/c/openstack/nova-specs/+/951636. masahito updated it and I reviewed it on my side. So there is still a chance to have this spec approved for this cycle.10:16
sean-k-mooneyhum ok,,, i was goign to write a differnt version of that spec(im 2/3rd of the way though) since i assuemd they were not going to update it at this point and i had some other ideas on how to do it10:21
mikalfrickler: that makes sense. So it's only intended for CI workloads? Config drive was meant to be gone by now...10:28
gibiUggla: left couple of questions ther10:31
gibie10:31
opendevreviewsean mooney proposed openstack/nova-specs master: support iothread for nova vms  https://review.opendev.org/c/openstack/nova-specs/+/95394011:03
sean-k-mooneythat is my version of how to do that ^11:03
opendevreviewBalazs Gibizer proposed openstack/nova master: FUP: Translate scatter-gather to futurist  https://review.opendev.org/c/openstack/nova/+/95333811:24
opendevreviewBalazs Gibizer proposed openstack/nova master: FUP: Use futurist for _get_default_green_pool()  https://review.opendev.org/c/openstack/nova/+/95333911:24
stephenfinbauzas: We're waiting for you on https://review.opendev.org/c/openstack/nova-specs/+/94044011:44
fricklermikal: in the current state CI is the only remaining use I'm aware of, yes. is config drive deprecated? that's news to me11:56
stephenfingmaan: I've proposed deprecating the v2.0 API for now, and removing in 2 releases. That will cause keystoneauth and Gophercloud to ignore it by default during discovery which is a good first step11:57
opendevreviewStephen Finucane proposed openstack/nova-specs master: Deprecate v2.0 API  https://review.opendev.org/c/openstack/nova-specs/+/95194912:00
opendevreviewsean mooney proposed openstack/nova-specs master: support iothread for nova vms  https://review.opendev.org/c/openstack/nova-specs/+/95394012:01
sean-k-mooneyfrickler: its not deprecated and we have no plans to remvoe it12:05
gibistephenfin: wasn't we deprecated v2.0 already in the past?12:05
sean-k-mooneyfrickler: we even turn it on by default in our downstream installer12:05
stephenfingibi: It still reports SUPPORTED from the root API doc12:05
sean-k-mooneymikal: we hyave no plans to remove config drive.12:06
stephenfinso not from the API perspective, afaict12:06
fricklersean-k-mooney: ah, thx for confirming12:11
sean-k-mooneywe did deprecate https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.config_drive_format a while ago12:11
sean-k-mooneybut keepign vfat does tno really hurt so we never remvoed it12:12
sean-k-mooneythe plan was to only supprot isos onc ethe very very old libivrt bug was resolved12:12
sean-k-mooneyits one of those thing where until it brekas the effort to remove vfat support is not woth the gain12:13
gibistephenfin: ack, that is a bummer but the I agree to deprecate12:22
fungimikal: can you expand on what "config drive was meant to be gone by now" means? is there a better metadata source that doesn't depend on flaky/inconsistent networking or hard-coded addresses? see also the latest cloud-init security fix where using configdrive is literally the recommendation for non-x86 guests now12:23
fungithe reason we use configdrive with glean is because network-supplied metadata has been pretty terrible in lots of clouds12:24
gibibrace for impact...12:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Add spawn_on  https://review.opendev.org/c/openstack/nova/+/94807912:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Move ComputeManager to use spawn_on  https://review.opendev.org/c/openstack/nova/+/94818612:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Move ConductorManager to use spawn_on  https://review.opendev.org/c/openstack/nova/+/94818712:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Make nova.utils.pass_context private  https://review.opendev.org/c/openstack/nova/+/94818812:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Rename DEFAULT_GREEN_POOL to DEFAULT_EXECUTOR  https://review.opendev.org/c/openstack/nova/+/94808612:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Make the default executor configurable  https://review.opendev.org/c/openstack/nova/+/94808712:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Print ThreadPool statistics  https://review.opendev.org/c/openstack/nova/+/94834012:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Document native threading mode and tuneables  https://review.opendev.org/c/openstack/nova/+/94936412:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Allow services to start with threading  https://review.opendev.org/c/openstack/nova/+/94831112:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Run nova-next with n-sch in threading mode  https://review.opendev.org/c/openstack/nova/+/94845012:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Do not yield in threading mode  https://review.opendev.org/c/openstack/nova/+/95099412:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet  https://review.opendev.org/c/openstack/nova/+/95343612:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Run unit test with threading mode  https://review.opendev.org/c/openstack/nova/+/95347512:24
opendevreviewBalazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively  https://review.opendev.org/c/openstack/nova/+/95381512:24
opendevreviewBalazs Gibizer proposed openstack/nova master: Do not yield in threading mode  https://review.opendev.org/c/openstack/nova/+/95099412:33
opendevreviewBalazs Gibizer proposed openstack/nova master: Run nova-api and -metadata in threaded mode  https://review.opendev.org/c/openstack/nova/+/95195712:33
opendevreviewBalazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor  https://review.opendev.org/c/openstack/nova/+/95266612:33
opendevreviewBalazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet  https://review.opendev.org/c/openstack/nova/+/95343612:34
opendevreviewBalazs Gibizer proposed openstack/nova master: Run unit test with threading mode  https://review.opendev.org/c/openstack/nova/+/95347512:34
opendevreviewBalazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively  https://review.opendev.org/c/openstack/nova/+/95381512:34
gibinow sign-offs in place :)12:34
opendevreviewKamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event  https://review.opendev.org/c/openstack/nova/+/94975412:38
sean-k-mooneyfungi: there was discssion in the past to remove it a very long time ago, the reason we have kep it is for things like ironic or where you need to use metadata form the config drive to confirgure networkign before you can actully call the metadata api12:47
sean-k-mooneythere are also envionment where the metadta api is not deployed12:48
sean-k-mooneyor where its in accapble in specific neutron network toplogies like whne you are booted directly to an external network and you are not using ovn or dhcp12:48
fungiyeah, and also (at least currently) no means for the host to signal the location of a network metadata source with non-x86 platforms12:49
sean-k-mooneyhow does x86 factor into this?12:49
fungisean-k-mooney: https://bugs.debian.org/110840312:50
sean-k-mooneyis there someting arm or power speciic that woud prevent it doing dhcp adn using the well know adress?12:50
sean-k-mooneyok so that not really nova related its a bug in cloud-init12:51
sean-k-mooneyglean or other first boot agenst are not nessiarly impacted right12:51
fungii have lost access to the more verbose explanation in the (still private) lp bug about how qemu passes information from host to guest that is x86-specific12:52
sean-k-mooneyok but the point is qemu is not ment ot be passign info form the host to the guest in general. config dirve is a way to do that12:52
funginova is able to indicate the address of the metadata server through a host-to-guest communication channel when it's x86 platform, but not on other platforms, so cloud-init had a workaround where it just assumed a specific linklocal address, but this was a security problem if booting on other cloud platforms12:53
sean-k-mooneywe dont do that however12:54
fungiwhatever trick qemu provides for that signalling is specific to emulated x86 systems12:55
sean-k-mooneyso im not aware of the mechisim yoru refering too12:55
fungiso cloud-init queries that when run in x86 guests12:55
sean-k-mooneythe metaddat discover is ment to be to a hardcoded ip adress in cloud-init or to the defautl gateway as a fall back12:55
fungireally wish they weren't continuing to keep that bug private after referring to it in the commit message of a public security fix12:56
sean-k-mooneyto be clear lookign at https://github.com/canonical/cloud-init/commit/f43937f0b462734eb9c76700491c18fe4133c8e112:56
sean-k-mooneyim not even sure that shoudl be a cve12:56
sean-k-mooneybut in any case cloud init will use the ec2 well knwo adress in an openstack env12:56
sean-k-mooneyso that would apply equally to that12:57
fungiyeah, my understanding is that they're dropping that functionality with the commit there12:57
sean-k-mooneytahts a major brakign change12:58
fungibecause they considered it a security risk if you ran cloud-init in an environment where some other bad actor on a shared network was able to spoof that well-known address12:58
fungiyes, it's a breaking change12:58
sean-k-mooneynot that the dmi interface they are now relaying on 12:58
sean-k-mooneyis not part of nova api contract to the guest runtime12:58
sean-k-mooneyits an internal impelmation detail of the livbirt driver12:58
fungiah, yes, dmi was how they were saying it worked for x86 guests, so they kept that12:59
sean-k-mooneyya so that is not a thing for ironic vmware hyperv ectra12:59
fungiand recommend that any other architectures now rely on configdrive12:59
sean-k-mooneyso that is not somthign they can use to detect if its oepnstack12:59
sean-k-mooneyit will work for libvirt with qemu but its not genrelaly correct to do12:59
fungiright, all the rationale and debate as to available options is still locked up in a private lp bug13:00
sean-k-mooneyso to me the behaivor it hasd to reach out to the link local adress was the correct behavior and not a security issue13:02
sean-k-mooneyyou can disable datasouce via the cloud init config in yoru image if you wanted too13:02
sean-k-mooneyim not ok with https://review.opendev.org/c/openstack/nova/+/953732 by the way13:05
sean-k-mooneyfungi: i guess that is why that was being rasised earlier in the week13:05
sean-k-mooneyfungi: by the way the dmi info when it is provided to vms is also configuabel via the downstream distobution https://github.com/openstack/nova/blob/2c19c07d5e29ca5445fa6dd45a6117542951714c/nova/version.py#L49-L6313:41
sean-k-mooneyi dont hink we have used this historically downstream13:44
sean-k-mooneybut its a thing that can be configured13:44
gibiI'm sorry there was a botched rebased in the bottom of the series...13:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Use futurist for _get_default_green_pool()  https://review.opendev.org/c/openstack/nova/+/94807213:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Replace utils.spawn_n with spawn  https://review.opendev.org/c/openstack/nova/+/94807613:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Add spawn_on  https://review.opendev.org/c/openstack/nova/+/94807913:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Move ComputeManager to use spawn_on  https://review.opendev.org/c/openstack/nova/+/94818613:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Move ConductorManager to use spawn_on  https://review.opendev.org/c/openstack/nova/+/94818713:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Make nova.utils.pass_context private  https://review.opendev.org/c/openstack/nova/+/94818813:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Rename DEFAULT_GREEN_POOL to DEFAULT_EXECUTOR  https://review.opendev.org/c/openstack/nova/+/94808613:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Make the default executor configurable  https://review.opendev.org/c/openstack/nova/+/94808713:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Print ThreadPool statistics  https://review.opendev.org/c/openstack/nova/+/94834013:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Document native threading mode and tuneables  https://review.opendev.org/c/openstack/nova/+/94936413:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Allow services to start with threading  https://review.opendev.org/c/openstack/nova/+/94831113:47
opendevreviewBalazs Gibizer proposed openstack/nova master: Run nova-next with n-sch in threading mode  https://review.opendev.org/c/openstack/nova/+/94845013:48
opendevreviewBalazs Gibizer proposed openstack/nova master: Do not yield in threading mode  https://review.opendev.org/c/openstack/nova/+/95099413:48
opendevreviewBalazs Gibizer proposed openstack/nova master: Run nova-api and -metadata in threaded mode  https://review.opendev.org/c/openstack/nova/+/95195713:48
opendevreviewBalazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor  https://review.opendev.org/c/openstack/nova/+/95266613:48
opendevreviewBalazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet  https://review.opendev.org/c/openstack/nova/+/95343613:48
opendevreviewBalazs Gibizer proposed openstack/nova master: Run unit test with threading mode  https://review.opendev.org/c/openstack/nova/+/95347513:48
opendevreviewBalazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively  https://review.opendev.org/c/openstack/nova/+/95381513:48
opendevreviewBalazs Gibizer proposed openstack/nova master: FUP: Translate scatter-gather to futurist  https://review.opendev.org/c/openstack/nova/+/95333813:49
opendevreviewKamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event  https://review.opendev.org/c/openstack/nova/+/94975414:31
gibieventlet sycn https://meet.google.com/bcy-uqoz-hje?authuser=014:32
fungisean-k-mooney: thanks, yeah revisiting my recollections of the original bug, it's that cloud-init is guessing whether it's booting in an openstack environment based on the dmi data, but that only works for qemu virtual machines on x86, so on e.g. arm cloud-init can't decide whether it's safe to assume the metadata source is legitimate or potentially spoofed by some attacker14:53
fungibecause it doesn't know whether it's in an openstack cloud14:53
sean-k-mooneyright but you can contol that by the cloud-init config in the guest when you build the image14:56
sean-k-mooneyadn the expect behiovr is to enumerate all data souces that are enabled in the image14:56
sean-k-mooneyso to me removign the enumeration is fundementaly breakign how this was inteded to work14:57
sean-k-mooneyno i think they are jsut chagnign there defaults in there config14:57
sean-k-mooneyrather then entirly removing it14:57
sean-k-mooneyi.e. if you expiclty customise an image and enable the openstack data souce adn upload it to glance14:58
sean-k-mooneyyou can still use ti with vmware or lxc or ironic14:58
sean-k-mooneyi would not be surpised if some distros decide to revert back when buidling there cloud images15:01
sean-k-mooneyfungi: glean just uses config drive right, ye never implemented the metadata rest api?15:01
fungii think so, we prefer dhcp/slaac+ra but if the cloud needs to set network configuration through metadata then we rely on glean to pull that from the configdrive because our experience with network metadata sources has been bad15:07
gmaanstephenfin: RE v2.0, thanks for updates. +W on the deprecation way. 15:21
stephenfinta15:22
gmaangibi: yeah, that was my main concern on v2.0 direct removal as it was not marked as deprecated or not supported. at least giving deprecation phase to users (if any) can come forward if they need it for some more time and why not using v2.115:23
opendevreviewribaudr proposed openstack/nova-specs master: Enable memfd support for shared memory backing  https://review.opendev.org/c/openstack/nova-specs/+/95168915:31
gibigmaan: cool, I agree that we need a deprecation first15:32
Ugglasean-k-mooney, if you can have a look at 951689: Enable memfd support for shared memory backing | https://review.opendev.org/c/openstack/nova-specs/+/951689 let me know if I manage to capture all you inputs.15:32
Ugglasean-k-mooney, no urgency on that one.15:33
gibifyi, Sree while verifying PCI in Placement caught a nice, long standing bug in our pci tracker https://bugs.launchpad.net/nova/+bug/2115729 15:33
opendevreviewMerged openstack/nova-specs master: Deprecate v2.0 API  https://review.opendev.org/c/openstack/nova-specs/+/95194915:34
Ugglagibi, I have seen this bug yesterday. kudos to Sree. I guess I can triage it as a valid one ?15:48
gibiI did the triage now that I found the root cause15:48
gibiI will push a fix in 30 mins...15:49
Ugglagibi, faster than light ! 15:50
gibithis is a typical bug where finding the root cause is the hard thing, fixing it is the easy15:54
ratailorgmaan, could you please provide your feedback on https://review.opendev.org/c/openstack/nova-specs/+/929780/ my replies ?16:05
opendevreviewRajesh Tailor proposed openstack/nova-specs master: Show finish_time field in instance action show  https://review.opendev.org/c/openstack/nova-specs/+/92978016:08
ratailorsean-k-mooney, gibi could you please also review ^^ as tomorrow is last date for FF. 16:09
opendevreviewBalazs Gibizer proposed openstack/nova master: Reproduce that only half of the PCI devs are removed  https://review.opendev.org/c/openstack/nova/+/95397116:12
opendevreviewBalazs Gibizer proposed openstack/nova master: Fix pci_tracker.save to delete all removed devs  https://review.opendev.org/c/openstack/nova/+/95397216:12
gmaanratailor: sure, will check today16:12
ratailorgmaan, Thanks!16:13
opendevreviewAlexey Stupnikov proposed openstack/nova master: Use updated_at to purge shadow_instance_extra  https://review.opendev.org/c/openstack/nova/+/95398418:29
opendevreviewAlexey Stupnikov proposed openstack/nova master: Use updated_at to purge shadow_instance_extra  https://review.opendev.org/c/openstack/nova/+/95398418:33
sean-k-mooneygibi: bauzas Uggla  did ye look at my verion of the iothreads spec by the wayhttps://review.opendev.org/c/openstack/nova-specs/+/953940/2/specs/2025.2/approved/dynamic-vm-iothreads.rst18:41
sean-k-mooneywe are unlikely to merge either by tomorrow but i fell like my version is closer18:42
opendevreviewMerged openstack/nova-specs master: Add flavor-search-by-name spec  https://review.opendev.org/c/openstack/nova-specs/+/94044018:50
Ugglasean-k-mooney, yep I have quickly looked at it. Tbh I think from technical point of view It looks great. I'm just annoyed about how masahito feels about it. I'd like a kind of co authoring and make sure his need is fulfill by the spec too. I will discuss with him tomorrow, because I'd like to explain, the goal is not to overwrite his spec but19:28
Ugglacombine the two.19:28
mikalfungi: my general recollection of this decade old coversation aligns with what sean-k-mooney said. Config drive v2 was always meant as a stop gap until everyone was using the metadata service. It really only existed at all because it was copied from AWS. I'm confused by your x86 address detection comments because the metadata service just uses a19:31
mikalhard coded zeroconf IP address on any of the clouds I can think of.19:31
mikalfungi: certainly a lot of the warts in config drive (like it originally not being able to be updated, although I think maybe that's fixed now) were because it was seen as a temporary thing.19:32
fungimikal: i was mis-remembering the bug, looking at the cloud-init commit it's "nova detection" really. they want to only contact the metadata server address for discovery if run in an openstack (or ec2) cloud, so are checking dmi to determine if it's safe to do so, but that check is only viable on x8619:35
fungiso when it comes to booting generic non-x86 distro cloud images with cloud-init included, the recommendation is to rely on configdrive instead19:36
mikalHonestly most of that DMI stuff is a bit dumb. That data exists in metadata, why do we ram it into two places one of which doesn't work on other architectures?19:37
fungiyeah, and so if nova didn't do that even on x86, then cloud-init would really just recommend distros go back to making openstack-specific images so they could set the override in the kernel command-line or an embedded conffile19:38
fungi(or suggest always using configdrive maybe)19:38
mikalI'm not here to hate on config drive. My point is more that its outlived its expected life, had some weird implementation choices because of the time it was done in, and probably needs some love if its really here forever.19:38
opendevreviewBrett Holman proposed openstack/nova master: Add DMI data to non-x86 architectures  https://review.opendev.org/c/openstack/nova/+/95398919:39
mikalLike for example it execs a command line to build the ISO image, solely because the only library we could find at the time to do it nicely was GPL and we refused to use GPL dependencies.19:39
fungiagreed, it has challenges, for example the earlier change at the start of this discussion about using virtio for cdrom devices by default, because debian doesn't include ahci drivers in their genericcloud kernel configs19:39
mikalFAT config drives should work too? IIRC it doesn't _have_ to be a CDROM, it just defaults to that?19:40
mikalIts been a long time.19:40
fungiit probably could be any kind of block device, i think it was virtual cd by convention so not sure if that assumption is baked into places (e.g. in cloud-init)19:40
fungigranted, forcing a different device type is still going to be a custom image property on upload or some similar override, if it's not nova's default19:42
mikalSo cloud-init will only probe for a couple of things to match -- device ID matters (config2) and type. The probing code only looks for CDROMs or FAT formatted disks IIRC.19:51
mikalHonestly, most of the people I know who really use metadata use it as a poor man's replacement for Nova not having some sort of mature out of band agent thing. They really want AWS SSM agent or equivalent -- https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html.19:53
mikalWe could do the same with either a custom agent or the qemu agent (although the qemu agent has some pretty big limitations), that bit isn't too hard. The hard bit is marshalling the requests into the instance. That likely involves some sort of per-instance queue that Nova doesn't have right now.19:54
fungiyeah, an actual communication channel from host to guest at least, if not bi-directional19:58
mikalYeah, specifically one which doesn't not require networking to be configured and working.19:59
mikalI worked for a training startup for a bit... We did all sorts of contortions to rescue VMs when students messed up their networking. This would have made it so much easier.19:59
fungia fifo-like proc interface in the kernel or something would be amazing19:59
fungiand/or pseudofiles like device drivers tend to expose20:00
mikalSo qemu-agent uses a unix domain socket on the outside and virtio-serial on the inside and speaks JSON blobs, which is a bit terrible. Serial means only one talker at a time.20:00
fungithough at boot time, one talker in the guest is generally sufficient?20:01
mikalShakenFist does this with a unix domain socket on the outside and virtio-vsock on the inside, and speaks protobufs. That's actually really cool if I don't say so myself because the in-guest agent can just run a socket server like you're used to and serve N requests at the same time because virtio-vsock does all the multiplexing for you.20:01
fungior maybe not these days, with the advent of parallel boot processing20:01
mikalSerialized requests forces you to come up with some sort of queueing / locking mechanism to handle the case of multiplem requests, even if its not used much.20:02
fungithe protobuf design sounds nice, if only protobuf libraries weren't such shifting sand these days (ut maybe they're getting better)20:03
mikalhttps://www.madebymikal.com/virtio-vsock-python-examples-of-running-the-server-in-the-guest/ is what I wrote up when I was playing with virtio-vsock.20:03
fungiyeah, i guess low-level socket access like that is probably enough for these cases, you don't need google's own protobuf libs20:06
fungioh, i see what you're saying, shakenfist tunnels protobufs over an interface like that20:09
fungiyou'd still need separate libs for the serialization i guess20:09
mikalYes, virtio-vsock is the transport, and I talk protobuf down it. Qemu talks JSON in its similar but slightly more terrible case.20:12
mikalNotably I do not talk gRPC, just the serialization protocol.20:12
mikalBut meh, its just serialization. It could be JSON for all it matters.20:12
mikalThe in-guest agent could present a REST API for example over virtio-vsock and that would not be insane.20:12
fungimakes sense, and yes grpc is a much bigger mess20:15
mikalOn reflection, I am also a bit confused about the "tangled python deependencies" answer to why glean is needed compared to cloud-init. Isn't that just a case of using venvs for the various competing components? Especially when modern pythons are super opposed to you installing directly into the system pip?20:17
fungidevstack is finally in a position to be able to isolate openstack services from the system python module search path, but that's an extremely recent change20:29
fungii recall clarkb hacked on versions of that improvement for years before it reached a state others in the community were comfortable with20:35
clarkbfrom the CI/OpenDev perspective I think there are two primary reasons we liek config drive. The first is that it is incredibly reliable. The boot error rate with metadata service is >0 and in some cases quite high. The other is that it is simple. Cloud-init configures a lot of stuff and sometimes in desctructive ways (it once decided to do bad things to volume mounts). Glean20:36
clarkbsets up your network and ssh keys and that is about it20:36
clarkbtoday we can only boot ubuntu noble in rackspace classic with config drive. Metadata does not work. This is with the upstream ubuntu noble cloud image which uses cloud init (not glean)20:37
clarkb(that is because rackspace doesn't do metadata in the way normal clouds do iirc and they don't have their custom image for noble yet (or ever) so the only way it works with standard tools is config drive)20:37
clarkbnaively I'm not sure what the issue is. From the end user standpoint config drive is simple and bullet proof20:39
clarkbthe ability to mount iso9660 or fat32 devices is near univeral, the data is small and static. Using a complicated api service that requires hosts know how to magically negotiate a protocal more complicated than `mount` is always going to be more error prone.20:40
clarkbiirc there are layers of proxies involved ont he backend too (which I want to say was the assumed source of the flakyness when we shifted all our ci nodes to config drive)20:40
mikalI am not sure there is an issue, I am just surprised given config drive was meant to be a temporary fix which seems to have "rusted on". If it's genuinely around forever it could probably do with some love.21:00
clarkbI think the primary downside to config drive is that it consumes one of your disk slots. I know this is a problem for Xen. Not sure about KVM21:07
clarkbwith Xen you only get something like 16 disks. One of which is implicitly owned by the root disk then config drive consumes a second. This limits you to 14 devices which may limit the total storage attachable to the instance if you also have volume size limits21:08

Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!