| opendevreview | Merged openstack/ironic master: fix loading of built-in inspection rules https://review.opendev.org/c/openstack/ironic/+/971266 | 07:37 |
|---|---|---|
| rpittau | good morning ironic! o/ | 08:11 |
| kubajj | good morning rpittau o/ | 08:43 |
| rpittau | hey kubajj | 08:52 |
| opendevreview | Milan Fencik proposed openstack/ironic master: fix: iPXE boot interface PXE capability detection https://review.opendev.org/c/openstack/ironic/+/971173 | 12:36 |
| mfencik | TheJulia: cardoe: I made the changes as discussed for 971173, I did think about improving the tests, but the only way I see atm would be to test all boot interfaces individually for add_ports_to_network, which I can do, it will just take a bit more work | 12:44 |
| opendevreview | Pavlo Shchelokovskyy proposed openstack/networking-generic-switch master: Use upper constraints in pep8 job https://review.opendev.org/c/openstack/networking-generic-switch/+/971421 | 14:04 |
| cardoe | we don't use pre-commit in NGS? | 14:09 |
| TheJulia | good morning | 14:18 |
| TheJulia | cardoe: not intentionally, afaik | 14:18 |
| TheJulia | err, unintentionally | 14:18 |
| cardoe | yeah I figured cause I forgot. | 14:19 |
| cardoe | so with you're morning coffee if ya wanna lazily help me fix the docs for 2025.2... :D | 14:19 |
| cardoe | https://review.opendev.org/c/openstack/ironic/+/971370 | 14:19 |
| cardoe | Cause the actual docs fix deps on that one. | 14:20 |
| TheJulia | my coffee just finished brewing, so retrieving | 14:21 |
| opendevreview | Julia Kreger proposed openstack/bifrost stable/2025.2: Centos: set masked: false for firewalld https://review.opendev.org/c/openstack/bifrost/+/971378 | 14:25 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: add an OCI artifact registry https://review.opendev.org/c/openstack/bifrost/+/961388 | 14:25 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: bifrost-registry-install: install the ORAS client https://review.opendev.org/c/openstack/bifrost/+/968355 | 14:25 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: Upload a disk image to OCI https://review.opendev.org/c/openstack/bifrost/+/968416 | 14:25 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: Upload to and use artifact from OCI registry https://review.opendev.org/c/openstack/bifrost/+/968417 | 14:25 |
| TheJulia | cardoe: if you have a moment, take a glance on my commen at https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/971236/3/dib/ironic-python-agent-ramdisk/static/etc/systemd/system/ironic-agent-resolve-config-drive.service -> I guess that leads to a thought if we should just quite literally break glean from being able to execute. This would rely upon basically "dhcp_all_interfaces" element style behavior to | 14:32 |
| TheJulia | work which is what the ramdisk has done for years, where as glean otherwise explicitly forces dhcp if nothing is found. Given nearby scripting already cripples and replaces glean's start, maybe its a noop and I should stop over-thinking it | 14:32 |
| cardoe | hmm yeah I wanted to actually build all this up locally this afternoon and test this. | 14:34 |
| TheJulia | So, turns out one of the non-voting jobs gets super close to full reproduction of alegacy's case | 14:38 |
| TheJulia | the only difference, we don't change the subnet in the test | 14:38 |
| TheJulia | because it runs with "flat" | 14:38 |
| TheJulia | (at least, it was enough for me to verify via job logs, but yeah.) | 14:39 |
| iurygregory | TheJulia, oh you were referring to unit tests in the other patches in chain <facepalm> I need more coffee XD | 14:53 |
| iurygregory | I'm working on them right now o/ | 14:53 |
| TheJulia | Yeah, I got to the last one and didn't even look at the logic because without a test or a really good explaination why we don't need one in base_manager or manager... I'm honestly not going to review it. | 14:55 |
| TheJulia | it became more of a "skim" pattern of review | 14:56 |
| iurygregory | yeah totally understand | 14:56 |
| TheJulia | iurygregory: btw, it would be awesome if you could spend a few looking at the bifrost changes regarding OCI registry support, so I can ensure steve is unblocked come january :) https://review.opendev.org/q/topic:%22oci-registry%22 | 15:04 |
| TheJulia | rpittau: is there an open bug yet for VM UEFI firmware (i.e. edk2) crashes with centos10 yet? | 15:05 |
| iurygregory | ack | 15:05 |
| iurygregory | will add to my list, I will be working normally since we only have 4 days of holidays in Dec. | 15:05 |
| rpittau | TheJulia: not aware of it, I've opened recently one for ubuntu but it got fixed already | 15:15 |
| TheJulia | I've seen the centos10 jobs blow up a few times so far, unfortunately | 15:16 |
| opendevreview | Matt Crees proposed openstack/bifrost master: Document IPA image download options https://review.opendev.org/c/openstack/bifrost/+/964145 | 15:16 |
| TheJulia | a number of them are running now, so there may be a failure in all of that | 15:17 |
| TheJulia | just time will tell | 15:17 |
| rpittau | I haven't noticed changes in the package recently, but I could've definitely just missed it | 15:18 |
| opendevreview | Pavlo Shchelokovskyy proposed openstack/networking-generic-switch master: Use upper constraints in pep8 job https://review.opendev.org/c/openstack/networking-generic-switch/+/971421 | 15:26 |
| sdmitriev1 | HI there, looking for advice on ironic node servicing, can't make it work. The ironic-python-agent can't perform a node lookup, getting this error "Lookup is not allowed for node <NODE> in the provision state service wait" which makes sense since servicing is not in the allowed states - https://github.com/openstack/ironic/blob/6362fa466368794d9fba50caa82f424d3556b154/ironic/common/states.py#L271 | 15:28 |
| sdmitriev1 | So I was wondering if there is a way to make it work without allowing unsafe node lookups https://github.com/openstack/ironic/blob/6362fa466368794d9fba50caa82f424d3556b154/ironic/conf/api.py#L77 ? | 15:28 |
| JayF | sdmitriev1: that appears to be a correct bug | 15:31 |
| JayF | but I'm confused how it ever worked if that is a bug | 15:31 |
| JayF | so wondering what I'm missing | 15:31 |
| sdmitriev1 | JayF: this was exactly my chain of thoughts, it looked really logical to have servicing state in the allowed list but it is not there; so has it been working then? | 15:35 |
| opendevreview | Takashi Kajinami proposed openstack/ironic master: Use common request_id middleware https://review.opendev.org/c/openstack/ironic/+/970978 | 15:43 |
| JayF | sdmitriev1: or option 2, which I hope isn't the case: nobody tested servicing with in-band steps | 15:44 |
| opendevreview | Takashi Kajinami proposed openstack/ironic master: Use common request_id middleware https://review.opendev.org/c/openstack/ironic/+/970978 | 15:45 |
| sdmitriev1 | JayF: would it fine if I propose the fix to include SERVICING, SERVICEWAIT, SERVICEHOLD into _LOOKUP_ALLOWED_STATES ? | 15:47 |
| JayF | I think so, ideally with a unit test that fails without those values in there | 15:48 |
| JayF | well, sans hold | 15:48 |
| JayF | just the ING and the WAIT states | 15:48 |
| sdmitriev1 | yeah, make sense, all right, will look into it today, thanks for the help! | 15:50 |
| opendevreview | Merged openstack/ironic stable/2025.2: update inspection rules docs and code to the same order https://review.opendev.org/c/openstack/ironic/+/971370 | 15:54 |
| dtantsur | sdmitriev1: in fact, only SERVICEWAIT, we don't expect IPA to be booting in SERVICING | 16:06 |
| dtantsur | JayF: well, it worked because all service steps that we've actually tested work without IPA | 16:06 |
| dtantsur | in fact, Jacob removed the dependency on IPA from firmware upgrades (and possibly settings too) | 16:06 |
| dtantsur | sdmitriev1: (okay, since we have ING states in the constant already, you can add SERVICING too) | 16:07 |
| opendevreview | Takashi Kajinami proposed openstack/ironic master: Use common request_id middleware https://review.opendev.org/c/openstack/ironic/+/970978 | 16:08 |
| sdmitriev1 | dtantsur: understood | 16:14 |
| JayF | dtantsur: we (and I'm pointing at me too, I've done this as well) have to get better at testing the full surface area of features. This is one example, OCI is maybe another :( | 16:17 |
| dtantsur | True. I don't know how to do it though. The previous idea was to use standalone jobs to gradually cover most features, but they come with their own issues. | 16:21 |
| TheJulia | rpittau: I don't think its package changes much as more contents in the firmware packages because it is pretty clear edk2 is what is blowing up | 16:23 |
| rpittau | ah ok then | 16:24 |
| TheJulia | I think part of the condundrum is the amount of work to get testing fully in place in all possible cases is just a huge amount of work, mix in CI instability, and just reviewer apathy to some of the cases, its almost discouraging. I'm not saying this to beat anyone up, but at some point when you get into a "revise when I have time pattern", things just slide off the radar completely | 16:26 |
| TheJulia | especially when it comes to tests/testing | 16:26 |
| TheJulia | Also, patterning. Service was originally viewed as all async, so everything we have done there is all really without the agent, makes sense to boot the agent, but it is again a super easy thing to miss as well until you get to that point of saying "i want to service to rebuild" or something along those lines. | 16:28 |
| TheJulia | I guess what I'm trying to say is we shouldn't beat ourselves up, its going to happen, all we can do is roll with the punches and try to be better about reviewing stuff in flight and follow-up work. | 16:29 |
| dtantsur | Speaking of OCI testing, we could use some reviews on our Bifrost patch chain ;) | 16:30 |
| TheJulia | Yes, I've been trying to get people to do that | 16:31 |
| TheJulia | :) | 16:31 |
| opendevreview | Merged openstack/ironic master: OCI: Add an option to fallback to HTTP https://review.opendev.org/c/openstack/ironic/+/969277 | 16:31 |
| opendevreview | Verification of a change to openstack/ironic master failed: OCI Basic Auth/Tag access https://review.opendev.org/c/openstack/ironic/+/969314 | 16:31 |
| * TheJulia glares at zuul | 16:32 | |
| TheJulia | rutro, looks like the metal3 job might be fail happy for a little bit, gpg check failures :\ | 16:36 |
| TheJulia | ?key rotation?" | 16:36 |
| TheJulia | (or bad mirror?) | 16:37 |
| opendevreview | Doug Goldstein proposed openstack/ironic stable/2025.2: fix: inspection rules docs to match the code https://review.opendev.org/c/openstack/ironic/+/971453 | 16:51 |
| cardoe | I've noticed that in the past with yum repos separately from this. metal3 seems to be hitting that a bunch lately. | 16:52 |
| dtantsur | I wonder if we can add pre-baked images for Metal3 jobs to our infra | 16:53 |
| cardoe | What are we even building with yum? | 16:53 |
| dtantsur | I haven't looked at the job, but it could be the testing dependency (think devstack) or the ironic image | 16:57 |
| dtantsur | So I've looked, and it's building ironic-image | 17:09 |
| dtantsur | I wonder if we can somehow have two-stage Ironic builds, where we first build (and publish) the image was all dependencies, then build a new image with scripts and Ironic content out of it | 17:11 |
| dtantsur | cc rpittau | 17:11 |
| JayF | TheJulia: this is 100% the cycle I end up in, but realistically we can't billboard features that don't work with a massive second feature. I don't know how to square the whole circle but status quo is very, very painful :( | 17:11 |
| JayF | me: "We can't let X happen!" also me: [lets X happen all the time] | 17:12 |
| TheJulia | I do feel like part of that is the conundrum of open source when we don't have an immediate consumption side user testing the variety of cases | 17:23 |
| TheJulia | Also, we do work, we say "it works in x case", doesn't mean we are sure it will work in y variation case until that is found, that right there is the pain | 17:23 |
| rpittau | dtantsur: the multi stage ironic-image is something I was also wondering about, I did some tests, it's not trivial (all that python stuff.... and more) | 17:24 |
| rpittau | consider that to keep the image slim we're removing packages at the end, because we don't want to keep builds dependencies, so dnf will still hit issues with repos | 17:24 |
| rpittau | well, I haven't dedicated a lot of time to that, so maybe there's a solution | 17:24 |
| dtantsur | rpittau: I wonder if we can build all Python deps into wheels in advance | 17:31 |
| opendevreview | Merged openstack/bifrost master: ILO driver retirement https://review.opendev.org/c/openstack/bifrost/+/965177 | 17:40 |
| cardoe | So that's something we've done in OSH as well. | 17:43 |
| cardoe | I was hoping to come up with a smaller generic image and talk to kolla about sharing. | 17:43 |
| cardoe | https://review.opendev.org/c/openstack/ironic/+/971453 if anyone has a moment... I'll +W with just 1 core since its a backport to make the docs correct. | 17:44 |
| opendevreview | Stanislav Dmitriev proposed openstack/ironic master: Allow node lookup for in-band servicing https://review.opendev.org/c/openstack/ironic/+/971459 | 17:48 |
| sdmitriev1 | JayF: ^ let me know if you had something different in mind for the unit test part | 17:48 |
| opendevreview | Merged openstack/ironic master: fix: iPXE boot interface PXE capability detection https://review.opendev.org/c/openstack/ironic/+/971173 | 17:54 |
| opendevreview | Doug Goldstein proposed openstack/ironic stable/2025.2: fix: iPXE boot interface PXE capability detection https://review.opendev.org/c/openstack/ironic/+/971463 | 18:13 |
| opendevreview | Doug Goldstein proposed openstack/ironic bugfix/33.0: fix: iPXE boot interface PXE capability detection https://review.opendev.org/c/openstack/ironic/+/971464 | 18:14 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Support filtering portgroups by shard https://review.opendev.org/c/openstack/ironic/+/970560 | 18:16 |
| cardoe | sdmitriev1: I don't understand the test. | 18:16 |
| JayF | https://review.opendev.org/c/openstack/ironic/+/966140 could use a review | 18:16 |
| opendevreview | Doug Goldstein proposed openstack/ironic bugfix/33.0: fix loading of built-in inspection rules https://review.opendev.org/c/openstack/ironic/+/971466 | 18:18 |
| cardoe | JayF: done | 18:20 |
| cardoe | trade ya https://review.opendev.org/c/openstack/ironic/+/971453 so I can backport the last one on top. | 18:20 |
| JayF | cardoe: feel free to review the one next in the line :) | 18:23 |
| cardoe | +2'd | 18:35 |
| cardoe | https://review.opendev.org/c/openstack/ironic/+/971142 is another one that I'll fire off backports once it merges. | 18:35 |
| cardoe | The "name" field is the the name of the Redfish Object | 18:38 |
| cardoe | I'll end up fixing up sushy-tools as a follow up. | 18:38 |
| opendevreview | Merged openstack/ironic stable/2025.2: fix: inspection rules docs to match the code https://review.opendev.org/c/openstack/ironic/+/971453 | 18:52 |
| opendevreview | Doug Goldstein proposed openstack/ironic stable/2025.2: fix loading of built-in inspection rules https://review.opendev.org/c/openstack/ironic/+/971471 | 18:53 |
| opendevreview | Doug Goldstein proposed openstack/ironic stable/2025.2: fix loading of built-in inspection rules https://review.opendev.org/c/openstack/ironic/+/971471 | 18:56 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Support filtering portgroups by shard https://review.opendev.org/c/openstack/ironic/+/970560 | 19:24 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove ovn vtep support https://review.opendev.org/c/openstack/ironic/+/966140 | 19:38 |
| opendevreview | Verification of a change to openstack/ironic master failed: OCI Basic Auth/Tag access https://review.opendev.org/c/openstack/ironic/+/969314 | 19:47 |
| opendevreview | Nahian Pathan proposed openstack/ironic master: Reduce API calls when collecting sensor data with redfish https://review.opendev.org/c/openstack/ironic/+/955484 | 20:01 |
| opendevreview | Iury Gregory Melo Ferreira proposed openstack/ironic master: Fix firmware update failure when task remains in RUNNING state https://review.opendev.org/c/openstack/ironic/+/970998 | 20:09 |
| cardoe | TheJulia: you still around? | 20:25 |
| cardoe | Anyone else that plays with OVN/OVS around? | 20:26 |
| opendevreview | Merged openstack/ironic master: fix redfish inspect system product name https://review.opendev.org/c/openstack/ironic/+/971142 | 21:01 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove ovn vtep support https://review.opendev.org/c/openstack/ironic/+/966140 | 21:05 |
| opendevreview | Doug Goldstein proposed openstack/ironic stable/2025.2: fix redfish inspect system product name https://review.opendev.org/c/openstack/ironic/+/971487 | 21:32 |
| opendevreview | Doug Goldstein proposed openstack/ironic bugfix/33.0: fix redfish inspect system product name https://review.opendev.org/c/openstack/ironic/+/971488 | 21:33 |
| TheJulia | cardoe: I had to run out to deal with a broken down car | 22:31 |
| TheJulia | cardoe: and now I have a migraine | 22:31 |
| cardoe | :( sorry to hear. | 22:31 |
| TheJulia | it is what it is | 22:32 |
| opendevreview | Mikkel Vestergård proposed openstack/ironic master: Provide pw_file to ipmitool-socat console https://review.opendev.org/c/openstack/ironic/+/971492 | 22:34 |
| cardoe | Just wanted to bounce my "ovn-baremetal" extension off of someone. | 22:35 |
| TheJulia | cardoe: could that be tomorrow? | 22:35 |
| cardoe | Of course. | 22:35 |
| TheJulia | ok then, I'm going to go lay down and hope meds and food kick in quickly | 22:36 |
| cardoe | I'll just link it in here so it doesn't get lost. | 22:39 |
| cardoe | https://docs.openstack.org/neutron/latest/admin/ovn/external_ports.html#scheduling-and-database-information | 22:39 |
| cardoe | https://docs.openstack.org/ironic/latest/admin/ovn-networking.html#chassis-as-gateway | 22:40 |
| cardoe | https://docs.openstack.org/ironic/latest/admin/ovn-networking.html#if-you-need-to-attach-to-the-network | 22:40 |
| cardoe | https://docs.openstack.org/openstack-ansible-os_neutron/latest/app-ovn.html | 22:42 |
| cardoe | https://docs.openstack.org/charm-guide/latest/admin/networking/ovn/external-connect.html | 22:42 |
| cardoe | Those are the only places I really see how to configure OVN correctly. | 22:44 |
| cardoe | Which is kinda surprising to me. | 22:44 |
| cardoe | The Ironic doc is the most accurate but too vague as far as the commands go. But you can take what the Ironic doc says and go to the OVN.org docs and figure it out from there. | 22:45 |
| cardoe | My doc describing things https://review.opendev.org/c/openstack/neutron/+/965415 hasn't had traction. | 22:46 |
| cardoe | But ultimately today you've gotta configure something like ovn-bridge-mappings="physnet1:br-ex" on your OVN | 22:48 |
| cardoe | Your br-ex needs to be configured on your network node to a physical NIC or bond and on your switch that needs to be plugged into a port that's a trunk port and allows the range of VLANs you'd be messing with. | 22:49 |
| cardoe | Not an ideal setup for folks that don't wide open settings for VLAN networks | 22:51 |
| cardoe | But won't work for VXLAN. | 22:51 |
| cardoe | If we want NGS in the mix we need to get local_link_connection info in. | 22:52 |
| opendevreview | Stanislav Dmitriev proposed openstack/ironic master: Allow node lookup for in-band servicing https://review.opendev.org/c/openstack/ironic/+/971459 | 22:54 |
| cardoe | Today Neutron needs OVN reads that bridge mapping here... https://opendev.org/openstack/neutron/src/commit/1a0e4115995497ac0df9ed5353fabca483a7afb1/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py#L88 | 22:54 |
| opendevreview | Merged openstack/ironic master: OCI Basic Auth/Tag access https://review.opendev.org/c/openstack/ironic/+/969314 | 22:57 |
| cardoe | If I patch that to also read "ovn-port-mapping=br-ex:switchname1:portname1,br-ex:switchname2:portname1" (duplicated for a bond) | 22:58 |
| cardoe | Then my ovn-baremetal just listens for the special ports like DHCP or router or whatever (there's already a helper function to check if this port is one that's gonna land on a network node). The agents that we're landing on are already referenced. And it's a one liner lookup of the agent object with that port mapping in it. | 23:04 |
| cardoe | I create a port of vnic_type baremetal:network (just to make something different) and set the device-id to the actual port that neutron made | 23:06 |
| cardoe | Then when the neutron created port gets modified or deleted, I listen for that and look up my baremetal:network port and perform the matching operation on it. | 23:07 |
| cardoe | Oh when I create it I set the local_link_information from that ovn-port-mapping like Ironic does in its attach VIF flow. | 23:07 |
| cardoe | That's enough for NGS to be called and the port object NGS has looks no different than a port that it operates on today. | 23:08 |
| opendevreview | Merged openstack/ironic master: Remove ovn vtep support https://review.opendev.org/c/openstack/ironic/+/966140 | 23:10 |
| cardoe | If I cannot patch neutron then I could still require that same field and use the ovsdb API inside of neutron to read the data myself. | 23:11 |
| cardoe | If I cannot do that then I could still make it work but require the cloud admin to create a port that'll have to be configured a certain way with the local_link_info already set. | 23:13 |
| cardoe | I don't want it to be a config file thing because that has problems as agents come and go. | 23:16 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Simplify ovn vtep microversion logic https://review.opendev.org/c/openstack/ironic/+/971494 | 23:40 |
| opendevreview | Takashi Kajinami proposed openstack/ironic master: Use common request_id middleware https://review.opendev.org/c/openstack/ironic/+/970978 | 23:54 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!