Wednesday, 2023-04-12

vanougood morning ironic00:51
rpittaugood morning ironic! o/07:06
samuelkunkel[m]Good morning07:17
samuelkunkel[m]cmon HPE.... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/HWySPIunTBWdXzcjKaHcTUnc>)07:21
dtantsursamuelkunkel[m]: some BMCs allow disabling IPMI entirely or the host interface07:37
samuelkunkel[m]I feared the day I have handle that07:44
samuelkunkel[m]But I think the new ILO6 is just confusing currently. Lets first check if there are some settings ;)07:45
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Move ubuntu jobs to jammy  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/87953807:46
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Move ubuntu jobs to jammy  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/87953807:58
samuelkunkel[m]Did anyone ever encountered this issue? (deploying arm based node)09:57
samuelkunkel[m]The ipa reports during deploying09:57
samuelkunkel[m]2023-04-12 09:49:36.251 1269 ERROR ironic_python_agent.efi_utils [-] Could not configure UEFI boot on device /dev/nvme0n1: Unexpected error while running command.09:57
samuelkunkel[m]Command: efibootmgr -b 000F -B09:57
samuelkunkel[m]Exit code: 1509:57
samuelkunkel[m]Stdout: ''09:57
samuelkunkel[m]Stderr: 'Could not delete variable: Invalid argument\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.09:57
samuelkunkel[m]the imaeg itself was written properly from what I can see. Going to boot the node into ipa and trying some stuff manually09:58
opendevreviewVerification of a change to openstack/ironic bugfix/21.0 failed: Wipe Agent Token when cleaning timeout occcurs  https://review.opendev.org/c/openstack/ironic/+/87740310:51
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Collect disk usage info post CI jobs run  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/87982611:20
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Add the option to not use tmpfs to build dib images  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/88014611:35
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Move ubuntu jobs to jammy  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/87953811:58
opendevreviewMerged openstack/ironic bugfix/20.2: Wipe Agent Token when cleaning timeout occcurs  https://review.opendev.org/c/openstack/ironic/+/87740412:32
opendevreviewMerged openstack/ironic bugfix/19.0: Wipe Agent Token when cleaning timeout occcurs  https://review.opendev.org/c/openstack/ironic/+/87740512:44
TheJuliasamuelkunkel[m]: To grok that, we would likely need the general efibootmgr logging output13:32
TheJuliadtantsur: I think HPE has had the option to disable the interface as a setting for longer than any other general hardware vendor13:33
TheJuliaalso, good morning everyone13:35
samuelkunkel[m]Well, if there would be some of use...... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/lLrFeAQDlUyiFYtHhsOTFILA>)13:36
samuelkunkel[m]I have also contacted our HPE Rep.13:36
* TheJulia looks down at the cup of coffee13:36
samuelkunkel[m]So I can not even modify any setting (set active, inactive, delete) via efibootmgr13:36
* TheJulia suspects stronger coffee is needed13:36
TheJuliaoh wow....13:36
samuelkunkel[m]so something with the /sys/firmware/efi is amiss13:36
samuelkunkel[m](but I can read the files)13:37
samuelkunkel[m]so this is super strange13:37
samuelkunkel[m]and my guess its something with the Firmware of this HPE RL30013:37
TheJuliayeah13:38
samuelkunkel[m]But I let you know once HPE answers13:39
samuelkunkel[m]thanks anyway!13:39
TheJuliathe files are representation of the raw data, but I thought efibootmgr interacted with ioctls to read/write directly to nvram13:43
TheJuliaI recently had a bug where one of the records exposed had a UTF-16 character.... that was fun.13:44
samuelkunkel[m]I think you are correct. From what I see in the strace part it uses ioctl.13:49
samuelkunkel[m]You can still read this partly obscure sysfs files.13:49
samuelkunkel[m]root@debian:/home/ipa# cat /sys/firmware/efi/efivars/Boot000F-8be4df61-93ca-11d2-aa0d-00e098032b8c13:49
samuelkunkel[m]`ironic1*0�y\�F��N�:��K2\EFI\BOOT\BOOTAA64.EFI�13:49
TheJuliaout of curiosity, what do you see if you hexdump it?13:50
rpittaugood night! o/14:54
samuelkunkel[m]root@debian:/home/ipa# hexdump /sys/firmware/efi/efivars/Boot000F-8be4df61-93ca-11d2-aa0d-00e098032b8c... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/WcUkdDkYRlIAzZqctBmczjiQ>)14:57
opendevreviewMaksim Malchuk proposed openstack/bifrost master: [DNM] test linters  https://review.opendev.org/c/openstack/bifrost/+/88016315:27
opendevreviewChris Krelle proposed openstack/ironic master: Add ablity to power off nodes that enter clean failed.  https://review.opendev.org/c/openstack/ironic/+/88016516:01
NobodyCamGood morning Openstack Folks16:03
opendevreviewChris Krelle proposed openstack/ironic master: Add ablity to power off nodes in clean failed  https://review.opendev.org/c/openstack/ironic/+/88016516:12
TheJuliaGood morning16:12
TheJuliasamuelkunkel[m]: that looks like utf-16 at a glance16:13
NobodyCamo/16:13
TheJuliasamuelkunkel[m]: with some actual mutlibyte chars... :\16:14
samuelkunkel[m]I saw that you did some stuff regarding utf-1616:14
samuelkunkel[m]hmm16:14
TheJuliafwiw, that is valid16:14
samuelkunkel[m]ok16:14
samuelkunkel[m]thanks16:14
TheJuliajust... I don't know how we would decode it off-hand, the data in nvram is supposed to be utf-16, but the weirdest thing I've seen before today is 0x00FF being used for a bios reboot trigger as a nvram entry....16:15
TheJuliathe character itself was not the trigger, it was in the description16:15
TheJuliaÿ16:17
TheJuliaso, making child nodes execute steps seems... surprisingly simple although we likely need guard rails documented i.e. "don't expect the agent"... and I'm suspecting anything that returns with CLEANWAIT needs to error things out possibly16:18
TheJulia... actually that makes sense to do since cleanwait means RPC action on the parent to re-trigger things, meaning agent *on* the child node.16:19
opendevreviewChris Krelle proposed openstack/ironic master: Add ablity to power off nodes in clean failed  https://review.opendev.org/c/openstack/ironic/+/88016516:48
TheJulia$ ~/.local/bin/tox -epep8|grep "Use TODO(NAME" |wc -l17:28
TheJulia219 <-- What have I done17:28
*** tzumainn__ is now known as tzumainn17:33
NobodyCamTheJulia: Thank you, trying to see what failed in the grenade job..19:51
NobodyCamoh my there are a lot of logs in that job19:52
TheJuliayeah, just recheck on grenade 19:58
TheJuliait is a known issue we're hoping they will approve our fix for19:58
TheJulia... I mean, the alternative is remove the grenade job19:58
NobodyCamahh okay19:58
NobodyCamlet me see if I can get a test going 19:59
TheJuliaw/r/t double quotes, *shrugs*, we likely need to clean up the files anyway. It was totally just a nit, the real aspect is a reno and a unit test :)19:59
NobodyCamoh yea19:59
NobodyCamlooking at tests now... 19:59
TheJuliaugh, I didn't realize the step valiation stuff worked the way it does21:51
TheJuliathat complicates some things :(21:51

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!