Tuesday, 2025-02-18

shermanmso, I was trying out the `ipa-ntp-server` boot param to fix clock skew on a server, but it seems like the conductor is failing to verify TLS and shutting the server back down before the the ntp server gets a chance to run? I tried on a second server (that was already up to date), and it seemed like the ntp update happened towards the end of provisioning.00:02
shermanmAm I misunderstanding something, or is this not helpful for fixing an already-skewed clock?00:02
shermanmedit: rather, it seems like there's some race between the initial time sync and the TLS cert generation, but I'll need to dig into which specific flavor of time sync we've got in our image. (kinda jumped the gun before debugging enough :) )00:24
vsaienkohello ironic community, can you please vote https://review.opendev.org/c/openstack/devstack/+/939578 patch enables atop installation by default, should help to debug performance issues and unstable ci07:57
opendevreviewDmitry Tantsur proposed openstack/ironic bugfix/28.0: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206008:02
opendevreviewDmitry Tantsur proposed openstack/ironic bugfix/27.0: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206108:03
opendevreviewDmitry Tantsur proposed openstack/ironic bugfix/26.0: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206208:03
rpittaugood morning ironic! o/08:08
opendevreviewRiccardo Pittau proposed openstack/ironic master: Mention removing codespell job in bugfix branches  https://review.opendev.org/c/openstack/ironic/+/94206709:13
opendevreviewRiccardo Pittau proposed openstack/ironic bugfix/26.0: The codespell test is not supposed to run on bugfix branches  https://review.opendev.org/c/openstack/ironic/+/94206909:15
opendevreviewDmitry Tantsur proposed openstack/ironic bugfix/26.0: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206209:15
opendevreviewMerged openstack/ironic bugfix/27.0: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206109:28
opendevreviewVerification of a change to openstack/ironic bugfix/28.0 failed: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206009:45
opendevreviewVasyl Saienko proposed openstack/networking-generic-switch master: Adds vlan aware VMs support for Cumulus NVUE and DellOS10  https://review.opendev.org/c/openstack/networking-generic-switch/+/94096610:25
opendevreviewMerged openstack/ironic stable/2023.2: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94191710:53
opendevreviewMerged openstack/ironic bugfix/28.0: Pass agent token to get command results  https://review.opendev.org/c/openstack/ironic/+/94206011:02
mnasiadkaJayF: Vasyl made really great progress on NGS trunking patch (https://review.opendev.org/c/openstack/networking-generic-switch/+/928490) - we're still puzzled with what docs should be added - can you reply with some ,,requirements'' in Gerrit?13:09
*** tkajinam is now known as Guest944913:10
opendevreviewKaifeng Wang proposed openstack/ironic-python-agent-builder master: Change to maximum compression level when using lzma  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/94060213:38
opendevreviewJulia Kreger proposed openstack/ironic-python-agent master: Lockout agent command results if a token is received  https://review.opendev.org/c/openstack/ironic-python-agent/+/94201014:32
opendevreviewJulia Kreger proposed openstack/ironic-python-agent master: Remove pre-victoria cycle agent token transition upgrade support  https://review.opendev.org/c/openstack/ironic-python-agent/+/94156514:35
opendevreviewJulia Kreger proposed openstack/ironic-python-agent master: Remove agent_token_required upgrade knob  https://review.opendev.org/c/openstack/ironic-python-agent/+/94156614:36
TheJuliaHey folks, ironic/ipa is 3 small patches away from being able to support oci://urls by the agent for final asset streaming (at least, when not zstd compressed. https://review.opendev.org/c/openstack/ironic/+/941252 https://review.opendev.org/c/openstack/ironic-python-agent/+/941300 https://review.opendev.org/c/openstack/ironic-python-agent/+/941494 All of the patches have one +2 at this time.14:43
opendevreviewcid proposed openstack/ironic master: API/Testing: Inspection rules migration  https://review.opendev.org/c/openstack/ironic/+/93921714:59
opendevreviewcid proposed openstack/ironic master: [WIP] Follow-up to Inspection Rules  https://review.opendev.org/c/openstack/ironic/+/94211214:59
opendevreviewVerification of a change to openstack/ironic master failed: trivial: lock dnsmasq check to ubuntu  https://review.opendev.org/c/openstack/ironic/+/94171015:00
opendevreviewMerged openstack/ironic bugfix/26.0: The codespell test is not supposed to run on bugfix branches  https://review.opendev.org/c/openstack/ironic/+/94206915:02
TheJuliastevebaker[m]: https://review.opendev.org/c/openstack/ironic/+/939191 2x+2. It would be awesome for some further clarity around auth and and maybe some theory level of doc at some point in the admin guide to provide a clear picture, but as-is LGTM.15:27
TheJuliaSome nits though, but fine to follow-up. :)15:27
JayFI had a question on that I never sent through15:47
JayFdon't let it be considered blocking, I'm just curious15:47
* JayF found it in drafts when he clicked the link15:47
opendevreviewMerged openstack/ironic stable/2024.1: ci: automatically set the branch for IPA to match TARGET_BRANCH  https://review.opendev.org/c/openstack/ironic/+/94192015:49
-opendevstatus- NOTICE: nominations for the OpenStack PTL and TC positions are closing soon, for details see https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/7DKEV7IEHOTHED7RVEFG7WIDVUC4MY3Z/15:58
opendevreviewMerged openstack/ironic stable/2023.2: ci: automatically set the branch for IPA to match TARGET_BRANCH  https://review.opendev.org/c/openstack/ironic/+/94192116:00
opendevreviewMerged openstack/ironic master: OCI: Send the auth header to IPA  https://review.opendev.org/c/openstack/ironic/+/94125216:03
shermanmhey, I wanted to ask if anyone else had a use for a NGS feature we're working on internally; using vlan translation to use a switch as a "l2gw" between some neutron-managed vlan network, and some externally managed provider vlan, so we can separate the lifecycles of each16:17
opendevreviewJulia Kreger proposed openstack/ironic master: deprecate shellinabox  https://review.opendev.org/c/openstack/ironic/+/94212416:20
TheJuliashermanm: seems interesting to upstream, not sure if anyone has a use for it directly at the moment, but perhaps in the future16:30
opendevreviewJulia Kreger proposed openstack/ironic master: deprecate shellinabox  https://review.opendev.org/c/openstack/ironic/+/94212416:35
rpittaugood night! o/16:50
TheJuliaJayF: w/r/t https://review.opendev.org/c/openstack/ironic/+/942124 I feel like it might need to be 2026.1 at the earliest but I've lost track of that model and where we are.17:28
cardoeshermanm: so like VLAN remapping?17:43
shermanmCardoe: exactly, the underlying implementation in our case is vlan translation in a juniper switch, just working out what kind of API is sane to wrap around it17:48
TheJuliaframing it that way, people might be totally on board for such a capability becuase I know one of the huge giant nightmare problems is the concept neutron has that it is the network infrastucture owner17:51
TheJuliawhich is... laughable17:51
TheJuliabut... its the world they live in17:51
TheJuliaWith baremetal, we live in a different world revolving around kWh17:52
shermanmFor us, we had initially statically configured these external networks as provider networks, but it became frustrating for us to manage as it got more dynamic, and frustrating for end users who'd need to rebuild / reattach instances to new ones as vlan ids changed. We solved our frustration by wrapping an API around "create a specific provider net for me", but it's still fairly brittle17:57
cardoeshermanm: yeah so literally working on that right now as well.17:59
cardoeBut with our VXLAN setup.17:59
cardoeSo I've only really cared about the trunking side.17:59
cardoeSo when the user creates a subport and they specify their segmentation-id. We set that as the user visible VLAN.18:01
shermanmIn some ways I think we only need the most minimal case, because it's restricted to a 1-1 connection between internal vlan net, and external vlan id.18:06
shermanmBut I'd love to also support vlan trunks and vxlan, it's been on our wishlist.18:07
shermanmWhen brainstorming, I was looking at extending networking-l2gw to support vlan-vlan, instead of only vxlan-vlan, just because it already modeled the " l2 gateway" idea, and then plumbing into NGS to execute the "l2gw create/update/delete...." operations 18:09
cardoeYeah I guess I need to look at the l2gw bits some more.18:10
TheJuliaRegarding networking: some more reviews on https://review.opendev.org/c/openstack/ironic/+/941343 would be appreciated18:17
JayFTheJulia: My thought was: you must have at least 1 SLURP release with it deprecated before it can be removed. I think that means we could pull it in 2025.2 since from a SLURP perspective: 2025.1 deprecated it; 2026.1 removed it.18:19
TheJuliawfm18:19
TheJuliaI'll rev it18:19
* JayF commented that in gerrit, too18:20
JayFit gets funky with .2, e.g. if we deprecated in 2025.2 we couldn't remove until 2026.2 (the deprecation has to hit a SLURP release)18:21
opendevreviewJulia Kreger proposed openstack/ironic master: deprecate shellinabox  https://review.opendev.org/c/openstack/ironic/+/94212418:25
JayF+218:30
TheJuliawheee, do a thing, find a bug18:43
opendevreviewMerged openstack/ironic-python-agent master: oci: Enable embedded authentication passing  https://review.opendev.org/c/openstack/ironic-python-agent/+/94130019:22
opendevreviewVerification of a change to openstack/ironic-python-agent master failed: oci: permit an 'unknown' but valid image  https://review.opendev.org/c/openstack/ironic-python-agent/+/94149419:22
opendevreviewJulia Kreger proposed openstack/ironic master: centos devstack support: fix VM permissions  https://review.opendev.org/c/openstack/ironic/+/94213419:31
opendevreviewJulia Kreger proposed openstack/ironic master: Don't log PXE when your not doing PXE.  https://review.opendev.org/c/openstack/ironic/+/94213519:31
opendevreviewDoug Goldstein proposed openstack/sushy master: migrate sushy_oem_idrac to sushy  https://review.opendev.org/c/openstack/sushy/+/94055719:35
opendevreviewDoug Goldstein proposed openstack/sushy master: fix extension loading test to use correct namespace  https://review.opendev.org/c/openstack/sushy/+/94213619:35
opendevreviewDoug Goldstein proposed openstack/sushy master: fix extension loading test to use correct namespace  https://review.opendev.org/c/openstack/sushy/+/94213619:36
opendevreviewDoug Goldstein proposed openstack/sushy master: migrate sushy_oem_idrac to sushy  https://review.opendev.org/c/openstack/sushy/+/94055719:36
stevebaker[m]good morning20:07
stevebaker[m]TheJulia: cool thanks. I've added a todo to write some context docs. Could we maybe +1 that change?20:10
TheJuliastevebaker[m]: you mean workflow it?20:11
stevebaker[m]ya20:11
TheJuliaAny objections out there in IRC land to beginning to workflow graphical console work?!20:15
* TheJulia suspects the folks who care have chimed in already ;)20:16
TheJulia(besides, need to start getting stuff merged anyway)20:16
JayF🛬20:41
TheJuliaheh20:57
cardoeGo for it.20:57
cardoeLand my first commit in the hooks stuff to fix the crasher in preprocess() of a hook.20:58
cardoehttps://review.opendev.org/c/openstack/ironic/+/940332 that guy20:59
TheJuliaa crasher() you say? ;)21:01
TheJuliaIs this a feature ?! Inquiring minds want to know ;)21:01
cardoeWell if you throw an exception of any kind, we attempt to log it but our logger line is wrong. So conductor then itself hits an exception.21:08
cardoeI added more test coverage when I moved the code around and hit it.21:08
cardoeSame way the merging of the idrac code into sushy is exposing badness in the sushy tests.21:08
JayFI'm sorry, but we can't remove the spacebar heating feature without leaving a toggle /s  https://xkcd.com/1172/21:09
TheJuliaJayF: don't tell one of my neighbors, her thing is heated mouses and keyboards21:13
TheJuliacardoe: shouldn't there be a test on https://review.opendev.org/c/openstack/ironic/+/940333/ ?21:26
cardoeThat's fair. I'll add one.21:30
cardoeThe more concerning one to me was the crasher that causes conductor to bail out and cleaning up the node is less than fun.21:31
opendevreviewcid proposed openstack/ironic master: [WIP] Follow-up to Inspection Rules  https://review.opendev.org/c/openstack/ironic/+/94211221:34
opendevreviewcid proposed openstack/ironic master: API/Testing: Inspection rules migration  https://review.opendev.org/c/openstack/ironic/+/93921721:34
opendevreviewMerged openstack/ironic master: OVN UEFI IPv6 CI job  https://review.opendev.org/c/openstack/ironic/+/94135221:46
JayFDid someone (rpittau?) sign up Ironic for the PTG?21:50
opendevreviewSatoshi Shirosaka proposed openstack/ironic-python-agent master: Missing 'interface' in Clean Step  https://review.opendev.org/c/openstack/ironic-python-agent/+/94215221:57
TheJuliaJayF: he said he was going to22:16
JayFokay cool, just checking since the last call just went out22:17
TheJuliacardoe: yeah, fair enough as well :)22:17
JayFand that often is a "next PTL" thing :D 22:17
TheJuliayup22:17
TheJulialets see, where did I put my brain22:17
JayFI just had roofers arrive 15 minutes ago and start cleaning off my roof, and it's the loudest thing ever22:19
JayFI'm saying, your brain might be having a party with my attention somewhere three towns over22:19
opendevreviewcid proposed openstack/ironic master: [WIP] Follow-up to Inspection Rules  https://review.opendev.org/c/openstack/ironic/+/94211222:29
opendevreviewcid proposed openstack/ironic master: API/Testing: Inspection rules migration  https://review.opendev.org/c/openstack/ironic/+/93921722:29
TheJuliaUgh, roofers22:30
opendevreviewVerification of a change to openstack/ironic master failed: Add ironic-novncproxy service  https://review.opendev.org/c/openstack/ironic/+/93919122:34
cardoewell... nova.exception.ValidationError: Ironic node: 90b75aae-bd7b-4ad8-98f0-230968738d2c failed to validate. (deploy: Image 5c54ce37-0d26-4c0a-9704-f8791c590eaa is missing the following properties: stage2_id, power: No Error, storage: No Error) with my patch to nova22:51
cardoeEven though that image has "stage2_id" on it.22:51
JayFI wonder if the user nova is using to call glance can see it22:51
cardoeAll of the pieces were created at the same time by the same user.22:53
cardoeThe owner is the same for them all and they're all shared.22:53
cardoeBut more debugging22:53
opendevreviewDoug Goldstein proposed openstack/sushy master: fix extension loading test to use correct namespace  https://review.opendev.org/c/openstack/sushy/+/94213623:08
opendevreviewDoug Goldstein proposed openstack/sushy master: migrate sushy_oem_idrac to sushy  https://review.opendev.org/c/openstack/sushy/+/94055723:08
cardoeI guess that needs a release note... and I also need to figure out the right way for OpenStack to express an install conflict.23:15
opendevreviewDoug Goldstein proposed openstack/sushy master: fix extension loading test to use correct namespace  https://review.opendev.org/c/openstack/sushy/+/94213623:20
opendevreviewDoug Goldstein proposed openstack/sushy master: migrate sushy_oem_idrac to sushy  https://review.opendev.org/c/openstack/sushy/+/94055723:20
cardoehrmmm that extension loading mock is still behaving oddly.23:33
opendevreviewMerged openstack/ironic master: move hooks execution into shareable module  https://review.opendev.org/c/openstack/ironic/+/94033223:36
opendevreviewDoug Goldstein proposed openstack/sushy master: fix extension loading test to use correct namespace  https://review.opendev.org/c/openstack/sushy/+/94213623:36
opendevreviewDoug Goldstein proposed openstack/sushy master: migrate sushy_oem_idrac to sushy  https://review.opendev.org/c/openstack/sushy/+/94055723:36
cardoeLike I don't get how that test ever passed.23:37

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!