Monday, 2026-03-02

opendevreviewMerged openstack/networking-generic-switch master: Log warning when `backend_url`` is not configured  https://review.opendev.org/c/openstack/networking-generic-switch/+/97831602:00
rpittaugood morning ironic! o/07:53
jandershey rpittau o/08:04
rpittauhey janders :)08:11
opendevreviewMerged openstack/sushy-tools master: novadriver: Fix boot mode and secure boot detect  https://review.opendev.org/c/openstack/sushy-tools/+/97742609:24
opendevreviewJacob Anders proposed openstack/ironic master: fix redfish BIOS settings update to not require IPA  https://review.opendev.org/c/openstack/ironic/+/97756910:50
opendevreviewLéonard Suslian proposed openstack/ironic master: Use ServiceRoot.Vendor for detect_vendor() instead of System.Manufacturer  https://review.opendev.org/c/openstack/ironic/+/97843911:42
opendevreviewLéonard Suslian proposed openstack/ironic master: Use ServiceRoot.Vendor for detect_vendor() instead of System.Manufacturer  https://review.opendev.org/c/openstack/ironic/+/97843911:43
opendevreviewLéonard Suslian proposed openstack/ironic master: Use ServiceRoot.Vendor for detect_vendor() instead of System.Manufacturer  https://review.opendev.org/c/openstack/ironic/+/97843913:20
opendevreviewLéonard Suslian proposed openstack/ironic master: Use ServiceRoot.Vendor for detect_vendor() instead of System.Manufacturer  https://review.opendev.org/c/openstack/ironic/+/97843913:22
opendevreviewMerged openstack/networking-generic-switch master: docs: Reorder docs to be more sane  https://review.opendev.org/c/openstack/networking-generic-switch/+/97801514:10
rpittauwe're just missing one nb patch for the release https://review.opendev.org/c/openstack/networking-baremetal/+/97707414:16
TheJuliarpittau: perhaps just workflow it, its just logging changes14:17
rpittauyeah, donw14:18
cardoehttps://review.opendev.org/q/hashtag:ironic-week-prio+status:open should be cleaned up and ready for folks to review14:37
rpittaucardoe: thanks!14:38
rpittauI think we're right in time for the releases14:38
rpittauI'm going to review CI status for ipa this week14:38
TheJuliavery cool14:39
TheJuliaI'm updating my quarterly goals and setting my next quarter's goals at the moment, once I'm done with that we'll likely be in meeting time  :)14:40
JayFwiki is a little slow to load the agenda and I'm a little slow to load coffee14:59
JayFwill get the meeting starting a minute or two lATE14:59
JayF#startmeeting Ironic15:00
opendevmeetMeeting started Mon Mar  2 15:00:19 2026 UTC and is due to finish in 60 minutes.  The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'ironic'15:00
JayFWelcome to the Ironic meeting everyone! I'm going to refresh the coffee and wait for quorum :)15:00
alegacy_o/15:00
rpittauo/15:00
TheJuliao/15:01
dtantsuro/15:01
* TheJulia presses the coffee machine button15:01
Mahnooro/15:02
JayFokay caff has begun to flow :)15:02
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent master: [DNM] TEST CI  https://review.opendev.org/c/openstack/ironic-python-agent/+/97848615:02
JayF#topic Announcements/Reminders15:02
JayFPlease review patches hashtagged "ironic-week-prio" and hashtag your ready to review patches15:02
JayF#link https://tinyurl.com/ironic-weekly-prio-dash15:02
JayFThe release is LOOMING. It's R-4. Cycle highlights are due this week. 15:03
JayF#link https://releases.openstack.org/gazpacho/schedule.html15:03
JayF#topic Working Group Updates15:03
JayFStandalone networking is first up. alegacy_?15:03
alegacy_Only outstanding change is the bifrost patch: https://review.opendev.org/c/openstack/bifrost/+/96239415:04
cido/15:04
alegacy_other than that, I owe a couple of minor follow-ups which I plan to get to in the next week or so15:04
cardoealegacy_: it's failing CI so I removed ironic-week-prio on it.15:04
alegacy_ok, i'll check.  thanks.15:05
JayFCongrats on a successful project \o/15:05
JayFWhat about AsyncIO? Any updates there? 15:05
cardoeI believe the spec has been pushed and needs some feedback from everyone else.15:05
dtantsurYep, collecting more feedback15:06
TheJuliaI think a review jam to review/discuss would be good15:06
dtantsurOtherwise, I'm really short on time to code anything... :(15:06
cardoehttps://review.opendev.org/c/openstack/ironic-specs/+/97275415:06
JayFWho wants to own scheduling said jam?15:06
JayF[touches nose[15:06
cardoeOnce we cut 2026.1 from master of ironic, I'm gonna try and bring sushy into the ironic code base.15:06
TheJuliaI can own scheduling it15:06
JayF#action TheJulia to coordinate review jam for asyncio spec15:07
TheJuliafolks, when would we like to meet, tomorrow? Wednesday?15:07
TheJuliaFriday?15:07
JayFLast but not least, VXLAN Networking ... is this another story of celebration time?15:07
JayFTheJulia: tomorrow after 11am (or at 10am PST if we wanna roll it into GR-OSS review jam downstream)15:07
cardoeBiggest item for VXLAN networking are the docs... https://review.opendev.org/c/openstack/ironic/+/977102 Jay and I have +2'd it but he's got valid concerns in comments.15:07
cardoeI suspect we'll shake out those items as follow on changes however but I wanted to let TheJulia weigh in.15:08
JayFTheJulia: or thursday with a couple of 30 minute chunks in the way, or friday after noon15:08
TheJuliaThe whole kit and caboodle of vxlan stuffs, except for a docs patch or two has landed. Direction is likely to shift to testing/sharp edge removal next.15:08
JayFI'll call it mostly done \o/ 15:08
rpittauTheJulia, alegacy_: I'm holding the bifrost release until end of this week for OCI registry support and standalone networking15:08
JayFIs there anyone specifically doing cycle highlights yet?15:08
JayFIf not; I'll take the action 15:08
TheJuliaI have not, I'm trying to prepare a blog post for ironicbaremetal.org15:08
JayFsounds like a good segmentation of work15:09
TheJuliaI think we'll want to do *several* posts15:09
JayFwe can each see if we miss something15:09
JayF#action JayF to draft cycle highlights15:09
JayFMoving on to 15:09
JayF#topic Discussion Topics15:09
JayFtwo are listed here; first is PTG15:09
JayF#link https://etherpad.opendev.org/p/ironic-ptg-2026.215:09
JayFI-personally haven't taken the time I should on this yet; I'll dig into it some today15:09
TheJuliaWe're still in early idea phase, that is sure based upon the state of the etherpad15:10
JayFI suggest the team make a special effort to populate PTG etherpad further this week so maybe we can have a bigger chat about it next week-ish?15:10
TheJuliaI agree15:11
JayFSecond item for discussion is https://review.opendev.org/c/openstack/ironic/+/973279/4/ironic/conductor/deployments.py15:11
JayFsome weird behavior came to light in this change:15:11
JayFif we ABORT a state that's on the last step, that last step can't be aborted ... we just let it finish15:11
JayFthis is existing behavior in CLEANING/SERVICING and was proposed for DEPLOYING there when I got confused and raised a flag15:12
JayFmy preference, after talking with dtantsur is that we land his change with consistent behavior, then make the behavior abort in all cases (or at least configurable)15:12
JayFit seems very operator-expectation-breaking that ABORT on a DEPLOYWAIT node could end in ACTIVE15:12
dtantsuryeah, it may be a nightmare to program the state machine for a consumer15:13
TheJuliaI'm onboard with that plan. It feels like a bit of a hyper-optimizaiton, but also likley best if we take the most recently requested action15:13
TheJuliaas it obviously superceeds the original request15:13
JayFI'm thinking when this matters, it really matters -- like abuse preventation trying to block builds in flight15:13
TheJuliaBut, we don't have to iterate to get it right on that patch, it can be a follow-up since it really is a bug fix15:13
JayFexactly, it's actually better if we make it consistently gross15:13
JayFthen fix it15:13
JayFthan if we make it inconsistent and leave a loose thread15:14
opendevreviewAllain Legacy proposed openstack/bifrost master: Add support for standalone ironic networking  https://review.opendev.org/c/openstack/bifrost/+/96239415:14
JayFat least then it's a full consistent story15:14
TheJuliaOr, even if we document it and then fix it later15:14
TheJuliaI'm just wanting to make sure we don't forget it15:14
JayFyeah, my comment on the review was to JFDI fix it or bug it15:14
JayFI think a bug is the right route15:14
JayFAny other thoughts/input on this before moving on?15:15
TheJuliaa bug gives us a basis for fixing in backport as well which we can weigh once we have a patch15:15
JayFmoving on15:16
dtantsurI can file a bug or do you want to JayF?15:16
* dtantsur would appreciate a W+1 now15:16
JayFif you wanna do it r/n I won't complain15:16
JayFand if you link it I can approve15:16
dtantsurack15:16
JayF#topic Bug Deputy Updates15:16
JayFWho was bug deputy?15:17
cidI was bug deputy15:17
cidI found two newly reported bugs and two possible RFEs15:17
cidLink on the wiki page15:17
cidFirst RFE is already somewhat discussed15:18
cidhttps://bugs.launchpad.net/networking-generic-switch/+bug/2142771 - Cache and reuse ssh connections15:18
JayFseems a little like an obvious one, but I'd warn the implementor about sessino limits15:18
TheJuliaI'm good with the idea, I've not sat down and thought through the fine details there15:18
JayFthere are some switches which those peristent connections might choke out other connections to the switch15:18
JayFe.g. if you can only have one or two incoming ssh sesh at a time15:18
TheJuliaJayF: they are already aware and have set it for a per-switch knob15:18
JayFI am +1 to approving 214227115:19
TheJuliaSame15:19
TheJuliaI think the value benefit makes a nice and strong case15:19
cidCool, cool. 15:19
cidThe second is only an RFE if we determine it to be15:20
cidhttps://bugs.launchpad.net/networking-generic-switch/+bug/2142786 15:20
cidThe question is if we should honor `ngs_max_connections` limit with a file locks fallback when `backend_url` is not provided.15:20
cardoeSo this is almost similar to the NGS agent idea.15:20
TheJuliaYeah, I'm honestly kind of split on that one. I think its the right move to do it (i.e. as proposed in the lp item), but the basic challenge is we're talking older style OSes versus more modern multi-user switch OSes15:22
TheJuliaIts a super weird line for us to walk as well15:22
TheJuliabecause if we artificially force all OSes into the oldest OS pattern, we actually force sub-optimal behavior15:22
TheJuliaso... its a weird one I guess.15:22
JayFsounds like as much a docs bug as a bug-bug15:23
JayFsince fixing it would maybe add a decoder ring for folks to figure out to "fix" performance15:23
TheJuliaYeah, kind of is. Requires some context and "know thy network gear"15:23
cidYeah!15:23
cidFWIW, I already pushed a patch over the weekend to just log a warning, now merged (thanks to TheJulia, cardoe) which I believe is a good enough immediate addressing of the original report.15:23
cidThat's all the updates I have. Also happy to take up the role for next week unless someone else volunteers15:24
TheJuliaYeah, I concur, I think doing anything more than just adding clarifying documentation does become a bit problematic. But shermanm did say most of the switches are dell OS9 and OS10, OS9 being the more problematic ones.15:24
JayFDoes anyone want to ensure there's not a cid-shaped rut in our Bug Deputy station?15:25
JayF:D 15:25
TheJuliaI'll take it on this week15:25
JayF#action TheJulia as bug deputy15:25
JayFNext topic is RFE review, which we did inline. Skipping it.15:26
JayF#topic Open Discussion15:26
dtantsurSpeaking of bugs, https://bugs.launchpad.net/ironic/+bug/2143060 is the requested bug about aborting actions15:26
JayFNo premade agenda items here; anyone have anything for open discussion?15:26
TheJuliaI was assuming open discussion would be a giant party?15:26
dtantsurparty \o/15:26
TheJuliaSeriously everyone, we've made some huge strives this cycle. Three major networking pushes all at once15:27
dtantsurSpring is here \o/ (except for those on the bottom half of the Earth - sorry folks!)15:27
TheJuliaspring++15:27
JayFWho wants to chair the next meeting on March 9?15:27
JayFI'll do it again. I probably owe a few :D 15:28
JayF#action JayF to chair March 9 meeting15:28
TheJuliaheh15:28
JayFThat's going to bring the meeting to a close. Please stay for an afterparty, apparently.15:28
JayF#endmeeting15:28
opendevmeetMeeting ended Mon Mar  2 15:28:21 2026 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:28
opendevmeetMinutes:        https://meetings.opendev.org/meetings/ironic/2026/ironic.2026-03-02-15.00.html15:28
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/ironic/2026/ironic.2026-03-02-15.00.txt15:28
opendevmeetLog:            https://meetings.opendev.org/meetings/ironic/2026/ironic.2026-03-02-15.00.log.html15:28
TheJuliaSo, in a few weeks we are going to need to figure out how we want to present all the network stuff15:28
JayFI was wondering15:28
JayFshould we ask the foudnation for our own openinfra live this release?15:29
JayFmaybe do a very truncated 3-5 minutes on the main chat to plug an ironic-specific talk15:29
JayFwhere we go in depth on the added features15:29
TheJuliaWouldn't be a bad idea15:29
TheJuliaOr! if we are all in a general project updates, we would need to have very concise slides15:29
JayFtbh I don't hate the spotlight we'd get from having out own party :)15:30
TheJuliaWhich is also doable, I don't know how much momentum we might loose in doing that but we should celebrate and spread the good word of everyone's work this cycle.15:30
TheJuliaJayF: Then sounds like you might want to ask :)15:30
JayFI don't wanna ask until I'm sure I can draft enough people to do the networking talking15:31
JayFif it ends up being me trying to bumble through vxlan for 30 minutes it'll be entertaining but not useful :P 15:31
TheJuliaI can talk vxlan, and if needed TBN context15:31
TheJuliaor even TBN on some details if I go back and brush up on stuffs15:31
JayFI figured I could do TBN if clif didn't feel up to it15:31
TheJuliaSGTM15:32
JayFwe'd need someone for standalone, someone for vxlan15:32
JayFand then a someone for "everything else" I suspect15:32
JayFif we don't want it to be just handing the mic back and forth between two/three people15:32
clifI don't mind talking about TBN some but idk what openinfra live is15:32
TheJuliaI think that would be good overall, could we convince cardoe to speak!?15:32
JayFclif: youtube show the foundation puts on15:33
cardoeSorry this meeting overlaps with a Teams call I've got so bouncing back and forth.15:33
cardoeI can speak.15:33
* TheJulia imagines cardoe bouncing back and forth literally and gets dizzy15:33
JayFclif: e.g. https://www.youtube.com/watch?v=cdqL6umlM1w (although we're pitching an ironic-specific one)15:33
TheJuliaso, we should then likely start drafting some slides!15:34
JayFaround 35:00 for Ironic in that link15:34
TheJuliaWho wants to start/drive that process15:34
JayFme15:34
JayFwho to ask?15:34
JayFKendall?15:34
* TheJulia looks at JayF 15:34
opendevreviewMerged openstack/networking-baremetal master: vxlan: follow-up on trunk reconciler  https://review.opendev.org/c/openstack/networking-baremetal/+/97697715:34
TheJuliauhhh Allison most likely, blame me but note blame comes from Julia as an IC15:34
TheJuliaIf its not allison, she will redirect to the right person. She is awesome that way15:35
TheJuliaJayF: I'll try to follow up on your doc comments today15:39
JayFI don't remember said comments but I appreciate it :)15:40
JayFIf I did anything for anyone last week, or promised to do something, please don't assume I remembered it. It was a pretty taxing week personally and I was not at my best by any stretch.15:40
TheJuliaIt was all failure cases claude noted out, and there are some mitigating details to note in and all15:43
TheJuliaand the real "solution" to it all was never completed so might be good to try and deal with this next cycle15:43
JayFoh, this was the list of things which gets nodes stuck in states15:44
JayFwhich made me go !#$% like a sims character15:44
JayFqbert reviewing code up in here and stepped on ... whatever those enemies are15:44
TheJuliaI never played the sims, but I know some folks who got way too addicted to it and second life15:48
JayFThere was a guy in support at my first job who literally broke his computer (overheating) from playing second life while working. I think that was a RGE for him.15:49
JayFSecond Life, Everquest, those old school MMO games really would hook people15:50
opendevreviewMerged openstack/ironic stable/2025.2: fix inspection rules loaded from a file to not fail in logging  https://review.opendev.org/c/openstack/ironic/+/97828016:15
opendevreviewMerged openstack/networking-baremetal master: vxlan: trunk-reconciler: Dial back logging  https://review.opendev.org/c/openstack/networking-baremetal/+/97707416:15
opendevreviewMerged openstack/sushy-tools master: novadriver: rescue-based net-boot support  https://review.opendev.org/c/openstack/sushy-tools/+/97728416:16
TheJuliaJayF: please take a quick look at https://review.opendev.org/c/openstack/ironic/+/977102 as I replied to your concerns. I'm thinking of revving the change at this point, but if there is any additional or focused concern just let me know16:22
opendevreviewMerged openstack/ironic master: inspection rules tester returns non-zero code on failure  https://review.opendev.org/c/openstack/ironic/+/97826016:53
opendevreviewLéonard Suslian proposed openstack/ironic master: Use ServiceRoot.Vendor for detect_vendor() instead of System.Manufacturer  https://review.opendev.org/c/openstack/ironic/+/97843917:20
opendevreviewMerged openstack/ironic master: Allow aborting deployments in "wait call-back" state  https://review.opendev.org/c/openstack/ironic/+/97327917:47
opendevreviewJulia Kreger proposed openstack/ironic master: vxlan: Add docs to Ironic to cover the VXLAN use case  https://review.opendev.org/c/openstack/ironic/+/97710218:01
TheJuliaJayF: please take a look at ^ and lmk if that reads/conveys context better for you18:02
JayFoooh yeah18:02
JayFbinding failure option was the piece I was missing18:02
JayFmy downstream was bitten by that behavior not existing and so it got me going WTF when I saw the previous wording18:03
TheJuliaI still want to get successful binding callbacks wired eventually18:03
TheJuliaI'm semi-making room to try and work on that as a rough edge this next cycle18:03
JayFoh 1000% but this draws the line in a more understandable place18:04
TheJuliayeah18:04
JayFfor stuff like this, where I feel I have a tenuous grasp on the tech details18:04
JayFI review the docs like I'm evaluating it for production18:04
JayFwhich sometimes leads me to get hung up on wording burrs instead of actual issues18:04
TheJuliaEh, its all good to call out, I took another pass on it and was like "well... we should be more clear here"18:23
JayFTheJulia: real docs job failure on that: https://review.opendev.org/c/openstack/ironic/+/977102/5#message-ae925adeea936a8fce3c25f3fb3c7d4c923495e118:32
JayFcardoe: https://review.opendev.org/c/openstack/ironic/+/978361 if you need this further back, it needs a manual cherry pick rebase (MissingAttributeError in retrieve_nic_components)18:34
TheJuliasadness18:35
opendevreviewJulia Kreger proposed openstack/ironic master: vxlan: Add docs to Ironic to cover the VXLAN use case  https://review.opendev.org/c/openstack/ironic/+/97710218:37
cardoeIt shouldn't need to go farther back18:41
cardoeThe nic only firmware was part of 2025.218:41
opendevreviewMichael Sherman proposed openstack/networking-generic-switch master: Allow reusing ssh connections  https://review.opendev.org/c/openstack/networking-generic-switch/+/97814118:44
shermanmheh, thanks for addressing both of my bug reports so fast :)  I should be able to test out the multiple connection behavior on a real OS10 switch later this week, just need to get my hands on a "non-production" one18:45
TheJuliashermanm: the simulator is also available. I believe its much happier18:51
opendevreviewMerged openstack/ironic stable/2025.2: Redfish: guard retrieve_nic_components against MissingAttributeError  https://review.opendev.org/c/openstack/ironic/+/97836119:05
opendevreviewMerged openstack/ironic master: Add section about TBN to networking doc  https://review.opendev.org/c/openstack/ironic/+/97833319:08
opendevreviewMichael Sherman proposed openstack/networking-generic-switch master: Allow reusing ssh connections  https://review.opendev.org/c/openstack/networking-generic-switch/+/97814121:21
shermanmI've got initial support in for using a connection pool now, but the multi-process case gets a bit hairy if we need to limit total (active+idle) connections across all processes.21:23
TheJuliaOnly for much older switches, which includes os9 unfortunately21:35
janderscardoe when you have time, WDYT about https://review.opendev.org/c/openstack/ironic/+/977569/comment/cb5f8a8d_93939196/ ?21:43
JayFCycle highlights draft: https://review.opendev.org/c/openstack/releases/+/978597 22:47
shermanmTheJulia  I can also confirm that, at least on the OS10 switches we have, we start to get random failures fairly consistently above ngs_max_connections=3 (for active changes, pretty sure it doesn't care about idle sessions until a much higher #)23:05
TheJuliaInteresting!23:05
shermanmit might actually be workable, the CLI takes longer and longer to run as the concurrency goes up, so i had to modify netmiko's "read_timeout", which could be worth exposing as a per switch knob if it's not already23:27
shermanmOf course, this only verifies that the CLI didn't throw an obvious error, I haven't tested that the ports are actually configured, and that the config is consistent. (which feel's like the cli's job, but...)23:27
JayF> which could be worth exposing as a per switch knob if it's not already < I'd +1 such a change, if neccessary23:31
shermanmlooks like that knob already exists under `read_timeout_override` from netmiko. 23:56
JayFyou can highlight it in the docs if you think it's a decoder ring23:58
JayFhonestly, probably could use an operational faq for those switches generally23:59
JayFand that might be a good first entry23:59
JayFI'm thinking in the style of https://docs.openstack.org/ironic/latest/admin/troubleshooting.html23:59

Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!