| opendevreview | Merged openstack/sushy-tools master: Replace hardcoded BiosVersion with an updatable field https://review.opendev.org/c/openstack/sushy-tools/+/909487 | 01:09 |
|---|---|---|
| opendevreview | cid proposed openstack/ironic master: Self-Service via Runbooks https://review.opendev.org/c/openstack/ironic/+/922142 | 02:20 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Fix anacaonda boot interface https://review.opendev.org/c/openstack/ironic/+/923701 | 04:09 |
| opendevreview | Jacob Anders proposed openstack/sushy-tools master: [WIP] Add support for BIOS update emulation https://review.opendev.org/c/openstack/sushy-tools/+/909500 | 06:37 |
| rpittau | good morning ironic! o/ | 06:49 |
| rpittau | TheJulia JayF, re virtualpdu, I should be available during my afternoon today, I have only a 30 minutes call at some point | 07:04 |
| maxh[m] | TheJulia: I opened an issue and layed down the details: https://bugs.launchpad.net/ironic-python-agent/+bug/2072544 | 08:18 |
| iurygregory | good morning Ironic | 10:58 |
| shajizad | Good morning, dtantsur I was just checking your comments for the VIF patch, specifically about calling microversion twice. Will storing the required microversion based on whether optional params were included, and then calling assert_microversion for that value solve the issue? | 11:14 |
| shajizad | Something like this:... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/bWyLIhbXdXJPiMeIqUoEtuBh>) | 11:16 |
| dtantsur | shajizad: yes, that's exactly what I have in mind. | 11:22 |
| shajizad | Alright thank you | 11:26 |
| shajizad | I pushed the changes 👍🏻 | 11:26 |
| iurygregory | tks shajizad o/ | 11:34 |
| opendevreview | Jacob Anders proposed openstack/sushy-tools master: [WIP] Add support for BIOS update emulation https://review.opendev.org/c/openstack/sushy-tools/+/909500 | 12:30 |
| TheJulia | good morning | 13:05 |
| iurygregory | good morning TheJulia | 13:11 |
| TheJulia | rpittau: how is your afternoon looking over the next couple of hours? | 13:12 |
| sylvr | Hello ! I opened this bug report https://bugs.launchpad.net/kayobe/+bug/2072550 but I'm wondering if I should make it affect Ironic/Bifrost too | 13:13 |
| TheJulia | sylvr: just because a machine has an IPMI bmc doesn't mean the address is discoverable (requires inband bmc channel is enabled, often it is disabled) or qualified for discovery (dhcp + bmc is not a great idea) | 13:16 |
| sylvr | TheJulia: it managed to work a few weeks back, ipmi_driver is my only option and a dhcp server to give IP to the BMC is necessary as it is the only option that don't require to manually set tens or hundreds of bmc IPs... | 13:20 |
| sylvr | is there a solution I missed ? | 13:20 |
| dtantsur | I wonder if Kayobe changed something in-between. (Also, this feature is not 100% compatible with the newer inspection, so may be that.) | 13:21 |
| sylvr | well, I had an issue with the version 2023 for a while | 13:22 |
| TheJulia | sylvr: a well managed bmc network does really require address entry, as much overhead it creates, it saves you way more chaos of the dhcp server loosing/changing assignments | 13:23 |
| TheJulia | Likely need to get someone working with kayobe to chime in, really not much has chnaged on those release branches | 13:25 |
| sylvr | well, maybe it's more a kayobe issue, but if kayobe is capable of assigning IP for inspection/introspection and then set a node IP with a static one, that could be done on the oob network : zero touch and then more security as it don't need to rely on the DHCP when the nodes are registered correctly ? | 13:26 |
| rpittau | TheJulia: I'm free in ~30 minutes | 13:26 |
| TheJulia | rpittau: Okay, then I'll quickly take mr corgi for a walk then | 13:27 |
| rpittau | alright :) | 13:27 |
| TheJulia | JayF if your up in 30m | 13:28 |
| TheJulia | sylvr: if kayobe has such functionality, it really was not communicated to us. I guess the best troubleshooting step you can take is see if you can capture logs from the introspection or get a temporary ramdisk/OS on that host and see if "ipmitool lan print" works | 13:33 |
| TheJulia | bbiam | 13:33 |
| TheJulia | well at least 10m | 13:33 |
| dtantsur | TheJulia: good morning! Something occurred to me during the downstream network boot discussion... in case of virtual media, we can pass some information to IPA as configuration since IPA can mount the resulting "device". This won't work in case of UEFI HTTP boot, will it? | 13:36 |
| dtantsur | As in: can IPA mount an ISO that has been provided via UEFI HTTP boot? | 13:36 |
| JayF | TheJulia: rpittau: ack on 7am pt | 13:49 |
| TheJulia | dtantsur: highly unlikely, since as soon as the kernel begins booting, excess memory not part of the initial ramdisk is considered free for overwriting by the kernel. | 13:51 |
| TheJulia | dtantsur: I'd consider it like classical ramdisk boot of an ISO. | 13:51 |
| TheJulia | The only way for extra data to get in is for the data to get patched into the ramdisk | 13:52 |
| TheJulia | realistically | 13:52 |
| JayF | ipa-configdrive=base64stringonkernelcommandline (/s or maybe not?) | 13:54 |
| TheJulia | That is a super tight length limit | 13:54 |
| JayF | Oh, I figured there'd be some issue with actually doing it that way, but was just pondering on the various oob ways we have to get data in | 13:54 |
| JayF | and that's the most reliable in a weird way | 13:54 |
| TheJulia | ?2k chars *total* on x86_64 if I'm recalling correctly | 13:55 |
| TheJulia | yeah, gets recorded on the cmdline | 13:55 |
| JayF | if it was just for network information, I bet we could get there | 13:55 |
| TheJulia | dhcp is a functional requirement for httpboot cases | 13:56 |
| * TheJulia makes coffeeeeeee | 13:56 | |
| TheJulia | https://meet.google.com/iwu-gmus-vhy | 13:58 |
| TheJulia | JayF: rpittau: ^ | 13:58 |
| sylvr | TheJulia : I'm going to check the logs on my IPA as you recommended, also gave me ideas on how to (hopefully) handle "dynamic" BMC addresses without a DHCP server, thanks ! | 13:59 |
| rpittau | TheJulia: yep I'm in | 13:59 |
| cid | Mind if I slipped in too, | 14:03 |
| * cid joining in anyways :D | 14:04 | |
| dtantsur | TheJulia: got it, thanks. So we cannot do stuff like agent tokens or TLS certificates | 14:05 |
| JayF | agent token is via kernel command line iirc | 14:09 |
| JayF | but tls, yes | 14:10 |
| dtantsur | Interestingly, I'm in the middle of a way "need to stop using command line eventually" discussion right now | 14:14 |
| TheJulia | wheeeeeeeee | 14:19 |
| iurygregory | thank god I'm not in this discussion | 14:20 |
| dtantsur | so wondering what our options are if the people pushing UKI do not start to look outside of their bubble | 14:20 |
| iurygregory | what UKI means? .-. | 14:21 |
| dtantsur | do you really want to know? ;) | 14:21 |
| TheJulia | UniKernel | 14:21 |
| iurygregory | ohhh | 14:21 |
| dtantsur | https://uapi-group.org/specifications/specs/unified_kernel_image/ | 14:21 |
| TheJulia | somehow have a signed artifact (how?!?!!??!!?!?!?!?!) | 14:21 |
| TheJulia | how long until we have a windows registry? | 14:22 |
| dtantsur | Realistically, we need a way to pass configuration that is not tied to anything affecting the core of the operating systems (== does not compromise secure boot) | 14:23 |
| dtantsur | Which, unfortunately, is not something we can realistically invent | 14:23 |
| TheJulia | Ahh, Interesting, it is building a EFI application image | 14:24 |
| JayF | yeah, UKIs are pretty cool. Except for the part where it breaks our whole model by removing the concept of a command line for the most part | 14:25 |
| TheJulia | see, this is why we need a time machine to go back to 2018-2019 and hire someone to make EFI IPA | 14:26 |
| JayF | it's why the people working at one layer need to have a %#^#$%ing conversation that cuts through the stack | 14:27 |
| JayF | before someone's bright idea breaks half of the world | 14:27 |
| TheJulia | but it is not breaking if it is innovation! | 14:27 |
| dtantsur | \o/ | 14:27 |
| TheJulia | </sarcasm> | 14:27 |
| JayF | time to rewrite ironic in go and compile with GOARCH=efi /s | 14:28 |
| TheJulia | it would need to be a EFI runtime using all EFI interfaces | 14:28 |
| TheJulia | zero drivers | 14:28 |
| TheJulia | and even then, it would need to be signed | 14:29 |
| TheJulia | which is a whole huge hurdle | 14:29 |
| dtantsur | yeah, the signing part is the biggest problem | 14:29 |
| TheJulia | in the UKI world, are they anticipating MSFT to sign for every kernel update? | 14:30 |
| TheJulia | or are they somehow embedding shim into it? | 14:30 |
| dtantsur | I cannot find any information on that | 14:35 |
| dtantsur | I suspect we're still going to have shim as a separate binary... | 14:36 |
| dtantsur | "This file can either be directly invoked by the UEFI firmware (which is useful in particular in some cloud/Confidential Computing environments) or through a boot loader (which is generally useful to allow multiple kernel versions with interactive or automatic selection of version to boot into)." | 14:38 |
| dtantsur | My bet is on shim/grub still present. | 14:38 |
| opendevreview | cid proposed openstack/ironic master: Self-Service via Runbooks https://review.opendev.org/c/openstack/ironic/+/922142 | 14:40 |
| tkajinam | https://www.redhat.com/ja/blog/rhel-confidential-virtual-machines-azure-technical-deep-dive | 14:41 |
| tkajinam | Red Hat Enterprise Linux 9.2 uses a simpler boot scheme for an Azure CVM. It boots a UKI directly from shim: | 14:41 |
| opendevreview | Himanshu Roy proposed openstack/ironic master: add virtual media GET api to fetch all details related to virtual media devices for a given node https://review.opendev.org/c/openstack/ironic/+/921657 | 14:41 |
| dtantsur | yeah, definitely shim, thank you tkajinam | 14:42 |
| tkajinam | that's the one I saw when I was investigating UKI some time ago | 14:42 |
| tkajinam | though I've not yet tried it actually | 14:42 |
| opendevreview | Himanshu Roy proposed openstack/ironic master: add virtual media GET api to fetch all details related to virtual media devices for a given node, add unit tests https://review.opendev.org/c/openstack/ironic/+/921657 | 14:42 |
| TheJulia | direct uki from shim does seem likely from what I've heard from some of the boot folks | 15:02 |
| rpittau | bye everyone see you tomorrow o/ | 15:20 |
| shajizad | Hello dtantsur, I noticed for virtual media you said that the detach function should have an optional parameter. But this is not specified on https://docs.openstack.org/api-ref/baremetal/ | 15:39 |
| iurygregory | shajizad, I just checked our code seems like our api-ref is missing the information. https://opendev.org/openstack/ironic/src/branch/master/ironic/api/controllers/v1/node.py#L2182 https://opendev.org/openstack/ironic/src/branch/master/ironic/tests/unit/api/controllers/v1/test_node.py#L8801 https://opendev.org/openstack/ironic/src/branch/master/ironic/tests/unit/api/controllers/v1/test_node.py#L8812 | 16:02 |
| shajizad | Is device types a list? How is that passed into the request? | 16:04 |
| dtantsur | shajizad: ?device_types=a,b,c | 16:05 |
| iurygregory | dtantsur, from the tests I think it pass ?device_types=a&device_types=b https://opendev.org/openstack/ironic/src/branch/master/ironic/tests/unit/api/controllers/v1/test_node.py#L8813 | 16:06 |
| iurygregory | now I'm puzzled, since ?device_types=a,b would make more sense to me | 16:07 |
| dtantsur | iurygregory: both are now possible. the first form is something I only added recently to our API (in general, not just for this case) | 16:07 |
| iurygregory | oh, ok! | 16:07 |
| iurygregory | makes sense | 16:07 |
| shajizad | dtantsur: Thank you | 16:07 |
| shajizad | Can I do this manually is there something in the sdk code I should use | 16:08 |
| dtantsur | shajizad: there should be, Ironic's "fields" argument is treated the same way | 16:08 |
| dtantsur | okay, it's more complex https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/baremetal/v1/node.py#L97 | 16:10 |
| dtantsur | but you should be able to implement it quite easily reusing https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/baremetal/v1/_common.py#L126 | 16:10 |
| opendevreview | Merged openstack/sushy-tools master: Fix httpboot handling and unit tests https://review.opendev.org/c/openstack/sushy-tools/+/923592 | 16:50 |
| TheJulia | JayF: so I think the root cause of the issue is virtualpdu's binding of udp transport in virtualpdu/pdu/pysnmp_handler.py, unfortnately I think the cleanest way is a rewrite of it but I'm not sure. If we can get it so virtualpdu is in a different venv, that would at least allow us to unwedge ironic snmp support and virtualpdu to make them independent problems | 17:35 |
| JayF | isolating it to a root cause makes it tempting to fix directly :D | 17:35 |
| JayF | just use defer pysnmp_handles.close() | 17:36 |
| JayF | lolsob | 17:36 |
| * JayF currently auditing the nova driver for more missed iter() in the unit tests | 17:37 | |
| JayF | once I'm done with that, I'm going to try to draft guest-metadata | 17:37 |
| JayF | then I'll get to vpdu | 17:38 |
| TheJulia | Yeah, I'm out of time to burn on it. did try wiring up stop() to tear everything down, but I'm likely just too far down the rabbit hole | 17:38 |
| JayF | I just have trouble understanding that code, it's very misdirected | 17:39 |
| JayF | being written that way is probably part of how this kinda bug emerges | 17:39 |
| TheJulia | indeed | 17:40 |
| JayF | found exactly one remaining unit test that needed an iter([nodes]) in that nova driver, not bad, now if adding the iter breaks the tests :( | 17:44 |
| JayF | fyi https://review.opendev.org/c/openstack/nova/+/923781 is landing, fixes that ^ instance | 18:11 |
| JayF | the more I think about it, the more I think we should split the venvs for ironic & ironic-bmc-emulators because that's the only way to be certain we can change *only one of them* while testing still be somewhat valid | 18:15 |
| JayF | so I will head down that route when this bubbles to the top of my list | 18:15 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Fix anacaonda boot interface https://review.opendev.org/c/openstack/ironic/+/923701 | 18:33 |
| opendevreview | cid proposed openstack/ironic master: Follow-up to change; #922951 breaks RAID https://review.opendev.org/c/openstack/ironic/+/923570 | 18:39 |
| cid | o/ | 18:59 |
| iurygregory | someone asked me how to debug "InspectionError -> Failed to inspect hardware. Reason: unable to start inspection: The attribute Links/ManagedBy is missing from the resource /redfish/v1/Systems/1" .. I told them do check the redfish response using curl "curl -ksu "<user>:<password>" https://10.16.10.91/redfish/v1/Systems/1 | jq ."... there is no ManagedBy YAY! | 20:07 |
| iurygregory | X-Fusion 2288H V5 | 20:11 |
| iurygregory | https://xkcd.com/927/ \o/ | 20:11 |
| hjensas | TheJulia: reported bug on inspection tempest validating against flavor - https://bugs.launchpad.net/ironic-inspector/+bug/2072589. I guess a somewhat simple way, would be to allow overriding the flavor used by inspection tests. | 20:20 |
| TheJulia | Yeah, I'm not super convinced that a flavor is really the right thing to use, but I'll take a look. Thanks! | 20:21 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Fix anaconda boot interface https://review.opendev.org/c/openstack/ironic/+/923701 | 21:11 |
| JayF | I' | 21:58 |
| JayF | **I'm implementing https://blueprints.launchpad.net/nova/+spec/ironic-guest-metadata and I'm wondering, would it be valuable to send over image *name*? | 21:59 |
| JayF | seems to me like it might be? | 21:59 |
| TheJulia | Seems reasonable | 22:14 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: set tftp folder permission https://review.opendev.org/c/openstack/ironic/+/923590 | 23:13 |
| opendevreview | Julia Kreger proposed openstack/ironic-tempest-plugin master: Permit boot_interface to be updated :( https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/923591 | 23:16 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!