| opendevreview | Steve Baker proposed openstack/ironic-python-agent master: bootc install: Treat SELinux Permissive like Enforcing https://review.opendev.org/c/openstack/ironic-python-agent/+/972809 | 02:10 |
|---|---|---|
| -opendevstatus- NOTICE: Zuul is up and running again and should report back to Gerrit successfully. Changes can also merge if there are no other failures, but we expect that publication jobs like docs and tarballs updates will not work currently. | 05:02 | |
| rpittau | good morning ironic! happy friday! o/ | 07:37 |
| kubajj | good morning Ironic! o/ | 09:06 |
| Sandzwerg[m] | Morning ironic. Was there ever a discussion about using redfish disk secure erase as out-bound cleaning instead of the current in-bound way? | 12:47 |
| opendevreview | Jacob Anders proposed openstack/python-ironicclient master: Add support for node health status field https://review.opendev.org/c/openstack/python-ironicclient/+/967055 | 13:48 |
| dtantsur | Sandzwerg[m]: I personally was not aware of this feature, even though some vendor BMCs did offer it for a long time. Do you have a reference? | 13:53 |
| TheJulia | good morning | 14:14 |
| TheJulia | AFAIK its only been offered by HPe | 14:21 |
| TheJulia | and I *think* that was as an OEM action | 14:22 |
| TheJulia | If there is a standardized way, that might be interesting. The only concern I might have is does it cover, or how do we ensure all devices are actually cleaned in case of, for example, a thumb drive in the back of the chassis | 14:23 |
| TheJulia | the device that should not be there but for some reason be it accident or spycraft, is present. | 14:23 |
| dtantsur | It might be a compromise that the operator makes | 14:25 |
| TheJulia | Yeah, as a risk aware conscious decision, I'm totally onboard for it. | 15:06 |
| TheJulia | Its sort of an inverse problem from bmc, because for a BMC to see a device, it has to have drivers and support context for the use pattern. Like raid controllers have to be in the "supported" list to configure RAID. for OS side, the OS generally has the bulk of the drivers and is also similar to the usage surface so has more drivers out of the box because the other pattern wise from a BMC, the drivers are supposed to come | 15:08 |
| TheJulia | with the device, so if you put a non-oem device in, odds are it won't have compatible drivers | 15:08 |
| TheJulia | (as devices bring a lot of their own "drivers" | 15:08 |
| TheJulia | Can anyone tell the coffee has woken me up? | 15:09 |
| dtantsur | I cannot, nothing has woken *me* up | 15:30 |
| dtantsur | :) | 15:30 |
| dtantsur | Otherwise, yeah, it's a bit of a "know your hardware" situation (and have some security guards in your DC lol) | 15:30 |
| JayF | Sandzwerg[m]: we use that method in my downstream as a custom cleaning inband thing since the hpe tools will let you do the work in band | 15:57 |
| JayF | And iirc it still needs a reboot to take effect because the wipe actually happens on the next boot, so I'm not sure how much you save by doing it out of band | 15:58 |
| Sandzwerg[m] | dtantsur: I also only heard from it now because a colleague mentioned it. It's mentioned here: https://www.dmtf.org/sites/default/files/standards/documents/DSP2046_2017.3.pdf or what kind of reference do you mean? | 15:59 |
| dtantsur | #Drive.SecureErase ah cool | 16:01 |
| dtantsur | Do you have hardware that actually implements that? | 16:01 |
| Sandzwerg[m] | JayF: the colleague mentioned it as a possible alternative to normal cleaning as we have some trouble with network during cleaning that is partly the fault of the constraints the network hardware we use has and partly of the driver we written internally but it means that we frequently run into situations where network and thus cleaning fails. That's why I also found https://bugs.launchpad.net/ironic/+bug/2121702 | 16:02 |
| Sandzwerg[m] | I must say I haven't checked yet. Apparently HPE isn't the only vendor to support this. We have HPE, Dell and Lenovo I could check if any of the stuff we have supports this | 16:03 |
| Sandzwerg[m] | IMHO for us it's short term not an alternative to fix our cleaning mess but it might be useful and interesting for ironic so I thought I ask if anyone even heard of it before or has any experience | 16:04 |
| TheJulia | I think I remember seeing it in Lenovo | 16:43 |
| * TheJulia makes a giant pot of coffee and offers coffee to everyone in IRC | 17:01 | |
| * Sandzwerg[m] just got fresh tea | 17:01 | |
| TheJulia | I feel like I'm the person in charge of trying to push discussion along | 17:07 |
| TheJulia | at least, that feels like my role today | 17:07 |
| Sandzwerg[m] | with a websearch for redfish secure erase I found https://github.com/dell/iDRAC-Redfish-Scripting/blob/master/Redfish%20Python/SecureEraseDevicesREDFISH.py and took a look at a R770 we have. There I see the SecureErase action at least for the disks that are in the part of the BOSS card and for something attached to the "CPU" as well, that are probably the NVMes that are also there. Need to check if it's both or only one though. | 17:47 |
| Sandzwerg[m] | But first look seems okay. So yeah there seem to exist hardware that supports this. I might take another look next week but for now it's weekend for me. Have a good one everyone o/ | 17:47 |
| JayF | dtantsur: you are correct in your initial comment that AFAICT the support for this in any gear I've seen has been in oem | 18:06 |
| JayF | dtantsur: and may have been part of my general push to try and make it easier to do $oem_things in OOB Cleaning :) | 18:07 |
| JayF | clif: are you still working actively on Sandzwerg[m]'s bug? https://bugs.launchpad.net/ironic/+bug/2121702 | 18:08 |
| opendevreview | Merged openstack/ironic stable/2025.2: fix: report a better error in validate-interfaces https://review.opendev.org/c/openstack/ironic/+/972511 | 18:12 |
| cardoe | Rough first draft of network node attachment. https://gist.github.com/cardoe/ca985d5592b9ff1e07db1728eb5472af | 18:19 |
| cardoe | Even if you haven't read through the VXLAN specs that's okay because that approach is generic. Even if you use VLAN networks today with Neutron you've gotta manually configure your network node's switch ports. With this approach you don't have to. | 18:25 |
| cardoe | But if you have read the VXLAN specs , the end conclusion I hope you come to is "a Neutron network node is no differently treated than an Ironic node and the same code and behaviors apply to both" | 18:27 |
| cardoe | Sandzwerg[m]: yeah they've supported secure erase for a while. | 18:38 |
| cardoe | tear in my eye for ISE | 18:39 |
| cardoe | ISE brought to you by https://en.wikipedia.org/wiki/Hainan_Island_incident | 18:39 |
| cardoe | its the same secure erase that Ironic already does when having IPA do it. It's just triggerable out of band. | 18:43 |
| clif | JayF: Not actively working on it atm, should I prioritize it? | 19:34 |
| TheJulia | cardoe: TIL | 19:50 |
| JayF | clif: just unassign if you're not actively working on it, I might have a glance depending on how sticky it is | 19:53 |
| clif | gotcha, sorry about it | 19:53 |
| JayF | no problem just noticed it when it was mentioned in channel and I figured it had been swept away in the TBN tide :D | 19:56 |
| TheJulia | cardoe: so, basically the key really being, for example, ironic-neutron-agent possibly reconciling and or mech driver going "oh, this also needs to be labeled/tagged" | 20:03 |
| TheJulia | I guess, a *plus* side is the agent could have a bit more logic and cross-cutting awareness | 20:04 |
| TheJulia | cardoe: oh, so your proposing a mech driver to do it as it relates | 20:17 |
| cardoe | Gimme a little bit. | 20:20 |
| TheJulia | I guess, it makes sense and sort of scales depending on underlying network capabilities, the question might be how you separate between the 1 or more physical networks | 20:28 |
| cardoe | Heading back to my computer now. I made https://bugs.launchpad.net/ironic/+bug/2138129 as well for inspection. I forget if someone else was interested in that. | 20:47 |
| cardoe | TheJulia: So the Neutron Agent in the OVN world is fake. There is no actual agent. But there is.... it's complicated like much of Neutron. | 21:08 |
| TheJulia | Yeah, I'm trying to think of the right positioning | 21:09 |
| TheJulia | because it feels like there needs to be some weird level of physcial network awarenss blended in | 21:09 |
| cardoe | So they need to get events from OVN so they spawn a thread that calls the ovn CLI which is dumping events as JSON objects and they're reading that. That's the "OVN agent". | 21:09 |
| cardoe | There already is a level of physical network awareness blended in. | 21:09 |
| TheJulia | that is just the ovn "agent" though, right? | 21:12 |
| cardoe | Yes. | 21:13 |
| cardoe | There must be 1 "agent" per chassis | 21:13 |
| TheJulia | so basically the mech driver necessary can sit anywhere IF it has the configuration necessary to be able to map the aspects across | 21:14 |
| cardoe | Yep. | 21:14 |
| TheJulia | and it can create the ports, but it needs to know the local link data | 21:14 |
| TheJulia | and in theory, physnet awareness should still be present mapping wise | 21:14 |
| TheJulia | because that gets pre-determined upon first bind | 21:14 |
| cardoe | You actually have to specify it. It cannot even be empty. | 21:15 |
| TheJulia | I think it can be if others are known, but it resets if I'm recalling correctly and then has to exist | 21:15 |
| TheJulia | there is a known weird thing there | 21:15 |
| TheJulia | so mental step back, wherever the mech driver is attached and running, it needs to have full config context, that can just be a file or config, but then it needs to look at the attached hosts for routers and bind the ports. Alternatively we could look at going router-less, but its less of a direct hint to the physcial mappings (i guess.....) | 21:16 |
| * TheJulia gets out red yarn | 21:17 | |
| cardoe | https://opendev.org/openstack/neutron/src/commit/944f3a93260fac8edbb63c844cfdf99fa970a81c/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py#L88 | 21:17 |
| cardoe | If ya want I'll google meet | 21:17 |
| JayF | I would lurk such a chat just like I'm lurking this ;) | 21:17 |
| cardoe | But essentially each of those ports I mention at the top of the file (and even some more) get scheduled to a OVN chassis (or multiple for HA) | 21:18 |
| cardoe | I've yet to implement this for HA fwiw. | 21:18 |
| cardoe | But I don't see how it would be any different. | 21:18 |
| TheJulia | yeah, shouldn't be any different really | 21:19 |
| cardoe | So let's take the DEVICE_OWNER_ROUTER_INTF as an example. It's the internal side of a router. OVN is only going to schedule that port to an agent_type=OVN_CONTROLLER_GW_AGENT | 21:19 |
| cardoe | https://www.irccloud.com/pastebin/8XJw7t19/ | 21:20 |
| cardoe | Sorry hope a snippet is okay. | 21:20 |
| TheJulia | digesting mentally | 21:21 |
| cardoe | You know... rough first draft missed some pre-reqs. | 21:21 |
| TheJulia | well, I'm wonder how we really trigger this cleanly so I need to mentally wrap that in my head | 21:22 |
| cardoe | Yep understood. | 21:22 |
| cardoe | So we walk through each gateway_agent. Right now I'm not playing with AZs but you can further filter on those as well. Cause the port will have the AZ mentioned in the create_post_postcommit() already. | 21:23 |
| cardoe | https://cardoe.com/neutron/evpn-vxlan-network/admin/data-center-networks.html#physical-layout | 21:28 |
| cardoe | Let's say my neutron network node is server1 in that diagram | 21:29 |
| cardoe | So in the above call gateway_agents is a list of NeutronAgent objects and the list has 1 item in it and that item references server1 | 21:29 |
| cardoe | agent1_mappings = gateway_agents[0].chassis.other_config.get("ovn-bridge-mappings", "") | 21:30 |
| cardoe | agent1_mappings = agent1_mappings.split(",") | 21:31 |
| TheJulia | okay, so if that info is available, the decision makin can be determined outside of the individual node itself, that is as long as we don't need to do anything else on the node itself locally that we can't orchustrate via the bindings | 21:34 |
| cardoe | https://www.irccloud.com/pastebin/gzc3FKNy/ | 21:34 |
| cardoe | I forgot to include the physical network. | 21:35 |
| cardoe | But assume that I did. | 21:35 |
| cardoe | This is where https://review.opendev.org/c/openstack/ironic/+/964570 comes into play. | 21:36 |
| TheJulia | yes | 21:36 |
| cardoe | And https://review.opendev.org/c/openstack/neutron/+/965053 _bind_port_segment() | 21:37 |
| * JayF starts pondering how this would intersect with TBN in this kinda design. I guess we'd send a different physical_network to neutron based on what goes where? | 21:37 | |
| TheJulia | blarg, we just got buzzed by a jet | 21:37 |
| cardoe | Cause it'll see "oh you're making a VNIC_BAREMETAL port which has a physical_network set. Let me see if theres a VLAN segment for that physical network... if not let me create it." | 21:38 |
| TheJulia | hmm | 21:38 |
| cardoe | So now it creates it and we ultimately bind_port() to the VLAN segment | 21:38 |
| TheJulia | perhaps jumping on a call might be better | 21:38 |
| cardoe | Yep. | 21:38 |
| cardoe | JayF: I really want this to work with TBN. I need to play with the simulator. | 21:39 |
| TheJulia | https://meet.google.com/yvd-zznx-yxs | 21:39 |
| JayF | nope, I saw the timer countdown in the corner lol | 22:40 |
| cardoe | whelppp that's that. :-D | 22:40 |
| TheJulia | https://meet.google.com/mjd-qavb-epr | 22:40 |
| TheJulia | nooooooooooooo, we must finish! | 22:40 |
| cardoe | Anyway. I'm all for explicit over implicit. But I understand compat. | 22:40 |
| TheJulia | we are sooooo close | 22:40 |
| cardoe | ironic-prometheus-exporter is broken cause tinyipa has gone away... https://zuul.opendev.org/t/openstack/build/f11e9d7824044bbbb1bb0eecc9513a29 https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/971072 | 23:08 |
| opendevreview | Julia Kreger proposed openstack/ironic-specs master: VXLAN networking https://review.opendev.org/c/openstack/ironic-specs/+/959401 | 23:45 |
| TheJulia | le sigh | 23:45 |
| TheJulia | okay, I'll take a look at that tomorrow | 23:45 |
| TheJulia | cardoe: revised spec document above ^ | 23:45 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!