| rpittau | good morning ironic! o/ | 07:21 |
|---|---|---|
| abongale | good morning o/ | 09:03 |
| opendevreview | cid proposed openstack/ironic master: Support `disable_ramdisk` during servicing https://review.opendev.org/c/openstack/ironic/+/973016 | 10:06 |
| opendevreview | cid proposed openstack/ironic-python-agent master: Fix lookup proceeding with incomplete data https://review.opendev.org/c/openstack/ironic-python-agent/+/973776 | 10:06 |
| dtantsur | cardoe: hey, what about we merge https://review.opendev.org/c/openstack/ironic/+/966344 and ask janders to follow-up? The issue with largely broken upgrades is quickly heating up for us.. | 11:08 |
| opendevreview | Elod Illes proposed openstack/ironic master: Adapt bindep for debian-trixie https://review.opendev.org/c/openstack/ironic/+/973818 | 12:44 |
| opendevreview | Merged openstack/bifrost master: Provide a means to disable the registry service install https://review.opendev.org/c/openstack/bifrost/+/973673 | 12:54 |
| janders | cardoe sorry for not getting back to you earlier, been down with COVID last week | 13:08 |
| janders | will get onto the follow-up this week for sure | 13:08 |
| cardoe | dtantsur: yeah that works | 13:31 |
| cardoe | janders: sorry to hear but glad you are doing better | 13:32 |
| janders | cardoe thank you! yeah back to the living this week | 13:36 |
| janders | last week was a bit of a shutdown -h now | 13:36 |
| opendevreview | Verification of a change to openstack/ironic master failed: Make post-firmware-update reboot conditional on component https://review.opendev.org/c/openstack/ironic/+/966344 | 13:57 |
| opendevreview | Verification of a change to openstack/ironic master failed: Fix firmware update failure when task remains in RUNNING state https://review.opendev.org/c/openstack/ironic/+/970998 | 13:57 |
| opendevreview | Verification of a change to openstack/ironic master failed: Fix BIOS firmware not flashing when task completes before first poll https://review.opendev.org/c/openstack/ironic/+/970999 | 13:57 |
| opendevreview | Verification of a change to openstack/ironic master failed: Fix BMC resource validation to catch all sushy exceptions https://review.opendev.org/c/openstack/ironic/+/971000 | 13:57 |
| TheJulia | good morning | 14:01 |
| dtantsur | morning TheJulia | 14:04 |
| dtantsur | huh, what does CI hate today? | 14:04 |
| TheJulia | don't know, is it us today? | 14:10 |
| dtantsur | Looks like it's DNS (tm) | 14:13 |
| opendevreview | Verification of a change to openstack/ironic master failed: Make post-firmware-update reboot conditional on component https://review.opendev.org/c/openstack/ironic/+/966344 | 14:34 |
| opendevreview | Jacob Anders proposed openstack/python-ironicclient master: Add support for node health status field https://review.opendev.org/c/openstack/python-ironicclient/+/967055 | 14:53 |
| TheJulia | dtantsur: fwiw, I +2'ed the py313 patch, We can always follow it up | 14:55 |
| TheJulia | cardoe: will you have at least a little time to discuss vxlan today? | 14:56 |
| kubajj | o/ | 15:00 |
| janders | o/ | 15:01 |
| TheJulia | meeting time? | 15:02 |
| TheJulia | #startmeeting ironic | 15:03 |
| opendevmeet | Meeting started Mon Jan 19 15:03:00 2026 UTC and is due to finish in 60 minutes. The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:03 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:03 |
| opendevmeet | The meeting name has been set to 'ironic' | 15:03 |
| rpittau | o/ | 15:03 |
| * TheJulia hands everyone coffeeeeee | 15:03 | |
| dtantsur | I think we need to get back to asking the question "who is chairing the meeting next time?" :) | 15:03 |
| dtantsur | o/ | 15:03 |
| kubajj | o/ | 15:03 |
| TheJulia | dtantsur: ++ c oncur | 15:03 |
| TheJulia | Greetings everyone, welcome to our weekly Ironic meeting. Our agenda can bef ound at https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_January_19.2C_2025 | 15:03 |
| cid | o/ | 15:03 |
| TheJulia | err, be found | 15:05 |
| TheJulia | Clearly, I need more coffee | 15:05 |
| * TheJulia sips coffee and proceeed | 15:05 | |
| TheJulia | s | 15:05 |
| TheJulia | #topic Announcements / Reminders | 15:06 |
| TheJulia | As a standing reminder to review #ironic-week-prio items. | 15:06 |
| TheJulia | #link https://tinyurl.com/ironic-weekly-prio-dash | 15:06 |
| TheJulia | We have 30 items, lets get it below 25 this week. | 15:06 |
| TheJulia | (preferably 20) | 15:06 |
| cid | ++ | 15:07 |
| rpittau | if CI collaborates I think it's doable :) | 15:07 |
| TheJulia | This is week R-10 on the release cycle. | 15:07 |
| TheJulia | #link https://releases.openstack.org/gazpacho/schedule.html | 15:07 |
| TheJulia | As a reminder, we should check the cycle priorities as well. If you get a chance this week, please take a quick glance at that list. | 15:07 |
| TheJulia | Does anyone have anything else to remind us of or announce this week | 15:08 |
| rpittau | yup, I'm planning a round of releases btw | 15:08 |
| TheJulia | Does anyone have anything else? | 15:08 |
| TheJulia | Onward | 15:09 |
| TheJulia | #topic Working Group Updates | 15:09 |
| cardoe | TheJulia: via msg? Iām driving home today | 15:10 |
| TheJulia | First up standalone networking. alegacy you around today? | 15:10 |
| TheJulia | cardoe: could work | 15:10 |
| TheJulia | My impression is we're largely just in review phase for the standalone networking work. | 15:11 |
| TheJulia | dtantsur: An update for Async IO ? | 15:11 |
| dtantsur | Not much new on the async front. Still finishing the spec. | 15:12 |
| dtantsur | (re standalone networking, I've left some comments on the first patch) | 15:12 |
| TheJulia | okay then | 15:13 |
| TheJulia | As for VXLAN networking, I think we're starting to make progress. I've gotten bit of a development test lab up and we have patches posted for networking-generic-switch to handle quite a bit of switch side configuration as it relates to the topic. | 15:14 |
| TheJulia | The spec still needs reviews, but I think we're in a state of consensus at this point | 15:14 |
| TheJulia | Does anyone else have anything working group related? | 15:14 |
| cid | Not at the moment | 15:14 |
| TheJulia | Since we have no defined discussion topics, lets go onward to the Bug Deputy Updates | 15:15 |
| TheJulia | cid, the microphone is yours | 15:16 |
| cid | Sure | 15:16 |
| cid | There were a total of 6 new bugs, and 2 RFEs. | 15:16 |
| cid | - Allow abort verb on DEPLOYWAIT | 15:16 |
| cid | - Deferred Tasks - Have a way to centralize periodics and minimize sleep usage | 15:16 |
| cid | I don't think these need reviews (?) | 15:16 |
| TheJulia | The deploywait one, I think we already agreed to | 15:16 |
| TheJulia | Deferred tasks is for the proposed spec document which I think I updated last week | 15:16 |
| cid | Is it ready for reviews yet? | 15:17 |
| TheJulia | Yes, it is up for review in ironic-specs | 15:17 |
| cid | +++, cool cool | 15:17 |
| TheJulia | is that 6 new bugs across all ironic projects? | 15:17 |
| cid | Indeed, Bifrost, Ironic, Sushy, ... | 15:18 |
| TheJulia | okay | 15:18 |
| cid | 12th - 19th January, 2026 | 15:18 |
| cid | - https://bugs.launchpad.net/ironic/+bug/2138310 (in Progress, no review necesssary) | 15:18 |
| cid | - https://bugs.launchpad.net/bifrost/+bug/2137376 | 15:18 |
| cid | - https://bugs.launchpad.net/sushy-tools/+bug/2137387 | 15:18 |
| cid | - https://bugs.launchpad.net/ironic/+bug/2137571 | 15:18 |
| cid | - https://bugs.launchpad.net/ironic/+bug/2138129 | 15:18 |
| cid | - https://bugs.launchpad.net/python-ironicclient/+bug/2138466 | 15:18 |
| cid | - https://bugs.launchpad.net/ironic-python-agent/+bug/2138540 | 15:18 |
| cid | - https://bugs.launchpad.net/ironic/+bug/2137571 (RFE but already approved, no review necessary) | 15:18 |
| TheJulia | Impressive given launchpad was down for ?2? days last week | 15:18 |
| cid | Oh, I didn't notice. | 15:19 |
| TheJulia | There would be 7 or 8 if it had been up all week because cardoe hit the inspection issue and I think there was another thing patches got propsoed to just fix because we couldn't file a bug | 15:19 |
| TheJulia | no worries :) | 15:19 |
| TheJulia | Okay, So who whall be the deputy next week? | 15:19 |
| cid | I'm always open to it, in the absence of any other interests. I wouldn't want to have all the fun :D | 15:20 |
| TheJulia | We should really try to alternate it out | 15:21 |
| TheJulia | but you do a really good job at it :) | 15:21 |
| TheJulia | I'll give it a shot this week | 15:22 |
| cid | tks! | 15:22 |
| TheJulia | Since we have no RFE's requiring review this week, onward to Open Discussion | 15:22 |
| TheJulia | #topic Open Discussion | 15:22 |
| TheJulia | Does anyone have anything they would like to discuss | 15:22 |
| TheJulia | ? | 15:22 |
| janders | I have a couple topics | 15:22 |
| TheJulia | janders: go ahead | 15:23 |
| janders | first - I hit this in CI an hour ago, running recheck, hopefully will clear but giving everyone a heads-up: | 15:23 |
| janders | | controller | fatal: unable to look up thekelleys.org.uk (port 9418) (Temporary failure in name resolution) | 15:23 |
| janders | looks like DNS failure downloading DNS package :) | 15:23 |
| janders | (same one dtantsur mentioned before the meeting) | 15:23 |
| janders | no action required, unless we start seeing it more | 15:23 |
| janders | it failed one of my patches in gate | 15:23 |
| TheJulia | janders: btw, I did comment on the Redfish forum item we discussed briefly last week | 15:23 |
| janders | TheJulia yeah, I saw it, ++ | 15:24 |
| janders | do you expect you'll be attending tomo? we can try raise this issue for another discussion | 15:24 |
| abongale | cid: o/ I could take this one https://bugs.launchpad.net/python-ironicclient/+bug/2138466 and https://bugs.launchpad.net/ironic/+bug/2138129, it will be fun | 15:24 |
| janders | prob best if we're both around for the discussiobn | 15:25 |
| TheJulia | janders: I will try to join tomorrow. If you want to try and raise it on the agenda then I think it might make sense. The gap really needs to be bridged somehow | 15:25 |
| janders | ++ | 15:25 |
| janders | thanks TheJulia | 15:25 |
| TheJulia | And we're sort of in a unique position there | 15:25 |
| janders | yeah | 15:25 |
| janders | it feels we do have a valued voice; lets use it | 15:25 |
| TheJulia | ++ | 15:25 |
| cid | abongale. That would be nice. o/ | 15:26 |
| janders | my second topic: I would appreciate reviews on https://review.opendev.org/c/openstack/ironic/+/971152/6 and https://review.opendev.org/c/openstack/ironic/+/971150/6 - last two patches in the big firmware upgrade chain | 15:26 |
| janders | right now I am re-running CI on the initial, big patch but once that's good it would be great to tick this off | 15:26 |
| janders | I marked them as weekly prio | 15:26 |
| TheJulia | abongale: it might be breaking, but could make sense to change the pattern of behavior, if a field value is requested with json, to return the real field name | 15:27 |
| janders | both have 1x +2 | 15:27 |
| kubajj | I also have two things | 15:27 |
| kubajj | A question about the possibility of implementing a node evacuation feature - i.e. have a stand-by node, if a node goes down, a user could bring over the disks and start the node with the same parameters on the other node (would probably be mostly changes on nova) | 15:27 |
| kubajj | My question regarding this topic is - do yo u think this is even possible? Would anybody else want this? | 15:27 |
| TheJulia | janders: I'll trade you if you review https://review.opendev.org/c/openstack/ironic/+/973560 and https://review.opendev.org/c/openstack/ironic/+/973413 :) | 15:28 |
| TheJulia | (actually, I'll try to look at those this week) | 15:28 |
| dtantsur | abongale, TheJulia, anything around https://bugs.launchpad.net/ironic/+bug/2138129 should really consider https://bugs.launchpad.net/ironic/+bug/2049913 | 15:29 |
| TheJulia | kubajj: oh wow. That could be a big discussion. Like user physically moves the platters? | 15:29 |
| dtantsur | because it's essentially a different angle of the same discussion | 15:29 |
| janders | kubajj would it be expected for the image to "just work" or is there assumption there'll be manual NIC/storage/... massaging? | 15:30 |
| TheJulia | dtantsur: hmm, likely yes | 15:30 |
| kubajj | TheJulia: yep, they operate mission critical nodes, like cryogenics, and stuff | 15:30 |
| dtantsur | So very like an RFE that needs a spec | 15:30 |
| TheJulia | Yeah, a spec most likely | 15:30 |
| janders | kubajj first thing that came to mind is it's borderline touching baremetal snapshots | 15:30 |
| TheJulia | The... big thing which comes to mind is NVRAM records and as previously mentioned network configuration | 15:31 |
| janders | although I do get the difference | 15:31 |
| kubajj | janders: we don't have those, right? | 15:31 |
| TheJulia | we don't really have a good way to handle that | 15:31 |
| janders | kubajj I don't believe so | 15:31 |
| TheJulia | kubajj: we've talked about it, a lot over the years. (That being snapshots...) but the real issue is the complexity versues the variety of lower level interfaces | 15:32 |
| TheJulia | I *do* think it would be relatively easy for us to have a means to move VIFs and ironic side configuration over | 15:32 |
| kubajj | janders: yes, ideally, it would "just work" but a good starting point would be just something which would "kinda work" | 15:32 |
| TheJulia | I think a spec would be a really good starting point to enumerate the requirements | 15:33 |
| janders | kubajj 100% agreed | 15:33 |
| kubajj | TheJulia: undestandable, it does sound complex | 15:33 |
| kubajj | TheJulia: ok, we can try to come up with one | 15:33 |
| TheJulia | its almost a "move x node's config to y node" | 15:33 |
| janders | just throwing it out there: with devices in general, where I have mission critical stuff I just deploy more than I need to | 15:33 |
| TheJulia | and then go move the disks in order | 15:33 |
| kubajj | TheJulia: yep, exactly | 15:33 |
| janders | so if some die, L1 can come and do something simple-enough to keep the lights on | 15:34 |
| janders | since hard work has been done preemptively | 15:34 |
| TheJulia | yeah | 15:34 |
| janders | now I wonder how this maps into this concept | 15:34 |
| janders | curious about any quick thoughts, otherwise happy to take this to the spec discussion | 15:34 |
| janders | TLDR is - what if for some reason moving the disk isn't enough | 15:34 |
| kubajj | janders: yeah, for some stuff, we do have LBs, but I do not know what exactly is the plan here, was just asked, if something like this is possible | 15:34 |
| janders | kubajj makes sense | 15:35 |
| janders | it's a clever idea TBH | 15:35 |
| janders | lightweight cold spare | 15:35 |
| janders | kinda | 15:35 |
| kubajj | janders: afaik, this is just what they do now, we are just overtaking their mini-datacentre (~600 nodes) with Ironic and they want all procedures to work | 15:36 |
| kubajj | we can explore other ways of handling this later | 15:36 |
| TheJulia | So | 15:37 |
| kubajj | I will discuss it with them and we can come up with a spec | 15:37 |
| janders | it's hard to dispute some of those battle-learned-and-tested tricks from veteran ops folks | 15:37 |
| TheJulia | One of the vendors did propose something like this as a value add years ago, but it was always using the backend SAN | 15:37 |
| TheJulia | But yeah, move the disks and if it is setup for dhcp \o/ | 15:37 |
| TheJulia | Although, if you want to force network reconfiguration... that is a bit more work | 15:37 |
| TheJulia | and could require the agent to going and nuking cloud-init and then metadata would need to be updated. Regardless, there is a very interesting idea here | 15:38 |
| janders | ++ | 15:38 |
| janders | I wonder about BMC config implications | 15:38 |
| janders | (network and other) | 15:38 |
| kubajj | TheJulia: I think we do not need network reconfiguration yet (still didn't manage to convince our networking team to let us touch their switches) | 15:38 |
| TheJulia | Well, you evacuate to another node, so the BMC would absolutely change | 15:39 |
| TheJulia | just not the config for the BMC itself | 15:39 |
| TheJulia | The way evacuate works is I think you functionally say "take this, put on this" | 15:39 |
| kubajj | what would be interesting is if they had different bios settings, would complicate it even more š | 15:39 |
| TheJulia | kubajj: well, I was more meaning workload side | 15:39 |
| TheJulia | Yeah, the real question is what is the resolution of the configuration changes | 15:40 |
| janders | btw none of my comments are to hint it's a bad, idea, im just thinking out loud about painpoints | 15:40 |
| TheJulia | ditto | 15:40 |
| TheJulia | Does anyone else have anything else to raise/discuss/plot? | 15:40 |
| kubajj | janders: that's what I wanted to hear, if it was straightaway unimplementable, I would not want to start doing stuff :D | 15:41 |
| kubajj | The second thing was that I wanted to thank mostepha[m] for his work upstream, he is heading back to university next week. It was great to have him around, especially for his work on the IPA | 15:41 |
| TheJulia | Thank you mostepha[m]! | 15:41 |
| cid | Thank you mostepha[m] | 15:42 |
| janders | Thank you mostepha[m]! | 15:42 |
| TheJulia | Anything else, anyone? | 15:43 |
| TheJulia | #topic Who is going to run our next meeting? | 15:45 |
| TheJulia | Would anyone like to volunteer? | 15:45 |
| TheJulia | I guess I'll run the next meeting | 15:46 |
| TheJulia | Okay folks, thanks everyone! Have a great week! | 15:47 |
| TheJulia | #endmeeting | 15:47 |
| opendevmeet | Meeting ended Mon Jan 19 15:47:11 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:47 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/ironic/2026/ironic.2026-01-19-15.03.html | 15:47 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/ironic/2026/ironic.2026-01-19-15.03.txt | 15:47 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/ironic/2026/ironic.2026-01-19-15.03.log.html | 15:47 |
| janders | thanks everyone | 15:47 |
| TheJulia | cardoe: so, vxlan! I ran into fun with OVN on saturday where it looks like port binding never completes because OVN thinks the ports need to be shutdown because $something needs to be done by the local ovn agent, that it doesn't know/comprehend | 15:48 |
| TheJulia | cardoe: I'm curious, if your $other mech driver might be completing the L2 binding? | 15:48 |
| cardoe | Ah it must be. | 15:49 |
| TheJulia | Or, if we should be expecting OVN to do so, specifically it has me wondering if I have an incorrect configuration OR if I need to begin to craft another mech driver | 15:50 |
| TheJulia | FWIW, I don't see OVN do anything regarding br-ex or the otherwise defined bridge | 15:50 |
| TheJulia | but again, it could entirely be my config or a area where we need to highlight "correct" configuration | 15:50 |
| TheJulia | FWIW, I did find a couple minorish items on the posted mech driver code and in the NGS patches. The NGS patches do the proper binding, the mech driver with minor fixes when loading *does* allocate the segment | 15:51 |
| TheJulia | Code in ironic presently stalls on the port binding process because of the outstanding L2 binding record, because part of the code in OVN gets trigger to think that it needs to apply L2 binding for the local agent, I've got a workaround identified but it likely needs to live in the other mech driver if its not a misconfiguration | 15:55 |
| opendevreview | Jacob Anders proposed openstack/ironic master: Add hardware health monitoring via management interface https://review.opendev.org/c/openstack/ironic/+/966946 | 16:11 |
| TheJulia | cardoe: I guess what is confusing to me is you stressed the tagging on the physical bridge meaning the ovn/ovs side of the attachment is happening automatically when that doesn't really seem to be the case, but I could totally see a mech driver maybe doing a little bit more to make it happen. That or I have a persistent misconfiguration I've not discovered | 16:24 |
| opendevreview | Jacob Anders proposed openstack/python-ironicclient master: Add support for node health status field https://review.opendev.org/c/openstack/python-ironicclient/+/967055 | 16:28 |
| janders | cardoe working through conditional reboot patch. It seems you and Dmitry disagree on this point: https://review.opendev.org/c/openstack/ironic/+/966344/comment/69bb04f3_b189dc3b/ - first wanted to confirm you're against constants here? | 16:42 |
| janders | (if so we can work out a solution while I work on the follow up) | 16:42 |
| cardoe | I want constants | 19:05 |
| janders | cardoe TY, I thought that may be the case | 19:06 |
| cardoe | I was fixing another bug last week where a string was slightly different between files. | 19:08 |
| janders | ++ | 19:09 |
| janders | folks, I think we have a genuine CI issue. Nothing to do with the dnsmasq package this time, "missing" (pip I think) packages this time | 19:09 |
| janders | it's blocking my approved change https://review.opendev.org/c/openstack/ironic/+/966344 from merging but is also affecting another unrelated change https://review.opendev.org/c/openstack/openstacksdk/+/973264 in a similar way | 19:10 |
| janders | logs: | 19:10 |
| janders | https://zuul.opendev.org/t/openstack/build/1ac3a2d3cf9245aab76a5a8c1feb51cc : No package matching 'mysql-client' is available | 19:10 |
| janders | https://zuul.opendev.org/t/openstack/build/ae483aebb4ae4b719459e8c6e5b53768: No package matching 'libpcre3-dev' is available | 19:11 |
| janders | to rule out some weird problem created by the environment on my laptop, has anyone managed to get anything pass CI in the last few hours? | 19:11 |
| janders | I would appreciate any pointers. 966344 is critical for a downstream bugfix | 19:12 |
| janders | (to be specific it became critical over the weekend) | 19:13 |
| janders | If I were to guess I suspect a dodge ubuntu repo update (or package) | 19:14 |
| opendevreview | Julia Kreger proposed openstack/networking-baremetal master: WIP: l2vni baremetal mech driver https://review.opendev.org/c/openstack/networking-baremetal/+/973889 | 19:24 |
| TheJulia | figures, the base OSes have likely changed and broken the plugin/bindep runs | 19:24 |
| TheJulia | cardoe: ^^^ first pass at moving the ml2 plugin to networking-baremetal | 19:25 |
| TheJulia | with some extra logging a known workaround (for now) | 19:26 |
| opendevreview | Julia Kreger proposed openstack/networking-baremetal master: WIP: l2vni baremetal mech driver https://review.opendev.org/c/openstack/networking-baremetal/+/973889 | 22:34 |
| TheJulia | cardoe: I'll be giving ^^^ a spin tomorrow, but I suspect that might handle the local port side as well in the model, which leaves a "bind a subport" model on the local attached switch to be a thing which is still needed | 22:49 |
| opendevreview | Merged openstack/bifrost master: CI: Make bifrost-upgrade-keystone-centos-10 voting https://review.opendev.org/c/openstack/bifrost/+/973463 | 22:58 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!