| TheJulia | alegacy: I've tagged https://review.opendev.org/c/openstack/ironic/+/965039 with ironic-week-prio, I'll just need another reviewer to give a thumbs up and we can merge it and hopefully get that CI job to be happy moving forward | 00:00 |
|---|---|---|
| cardoe | TheJulia: https://review.opendev.org/c/openstack/neutron/+/965053 that's the most minimal partial binding mechanism I could come up with. Spent WAYYYYYYYY too long reading through Neutron code. | 03:39 |
| opendevreview | cid proposed openstack/ironic master: Add upgrade check for iLO/iLO5 driver deprecation https://review.opendev.org/c/openstack/ironic/+/965014 | 06:27 |
| opendevreview | Merged openstack/ironic master: CI: make multinode runtime check sane https://review.opendev.org/c/openstack/ironic/+/965039 | 07:26 |
| rpittau | good morning ironic! o/ | 08:10 |
| janders | hey rpittau o/ | 08:17 |
| rpittau | hey janders :) | 08:17 |
| rpittau | hey all when someone has a moment I'm looking for reviews for https://review.opendev.org/c/openstack/bifrost/+/964752 to be able to move the upgrade jobs on master to DIB and remove tinyipa | 08:36 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: Deprecate tinyipa usage in CI https://review.opendev.org/c/openstack/bifrost/+/964404 | 08:37 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 08:39 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 08:40 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 08:44 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 08:48 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 09:33 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 09:35 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 09:39 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 09:42 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 09:47 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 09:50 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 10:27 |
| Ayo[m] | Good morning ironic! :) | 10:36 |
| rpittau | hey Ayo[m] :) | 10:36 |
| Ayo[m] | hello rpittau :) | 10:44 |
| Ayo[m] | mostepha: i added the tag as you adviced and one of my merge request got accepted while the other is still being reviewed | 10:44 |
| Ayo[m] | thank you mostepha | 10:45 |
| Ayo[m] | i have a merged work on openstack!!!! :) | 10:45 |
| mostepha[m] | Ayo: Glad to hear it worked out! :) | 10:46 |
| Ayo[m] | * hi rpittau :) | 10:46 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Add Release Schedule to release how to https://review.opendev.org/c/openstack/ironic/+/965090 | 11:22 |
| alegacy | TheJulia... thanks for the multinode job fix. Very much appreciated! | 11:24 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: Remove tinyipa support https://review.opendev.org/c/openstack/bifrost/+/964404 | 11:28 |
| dtantsur | JayF: JFYI, we got rid of ilo drivers in Metal3, all clear on this front. | 11:34 |
| iurygregory | good morning ironic | 12:18 |
| iurygregory | TheJulia, just to double check, we had 3 extra topics yesterday? | 12:19 |
| TheJulia | yes | 12:19 |
| iurygregory | ack | 12:20 |
| iurygregory | PTG starting in ~5min | 12:54 |
| TheJulia | Pre-PTG coffee session has started | 12:56 |
| iurygregory | \o/ | 12:56 |
| iurygregory | I'm grabbing my coffee :D | 12:56 |
| TheJulia | And we're going to get started.... | 13:02 |
| * TheJulia gets out the tour guide flag | 13:02 | |
| *** masghar is now known as Mahnoor | 13:08 | |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Remove sushy-oem-idrac from driver requirements https://review.opendev.org/c/openstack/ironic/+/965114 | 13:31 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Remove sushy-oem-idrac from driver requirements https://review.opendev.org/c/openstack/ironic/+/965114 | 13:36 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Remove sushy-oem-idrac from driver requirements https://review.opendev.org/c/openstack/ironic/+/965114 | 13:39 |
| cardoe | I hope I didn't miss any good chats this morning. Been bouncing between other PTG sessions. | 14:19 |
| opendevreview | Merged openstack/ironic master: Add upgrade check for iLO/iLO5 driver deprecation https://review.opendev.org/c/openstack/ironic/+/965014 | 14:29 |
| JayF | you've missed/are missing redfish stuff you were interested in | 14:37 |
| cardoe | Yeah :( | 14:53 |
| opendevreview | Verification of a change to openstack/ironic-tempest-plugin master failed: Add stable/2025.2 job https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/961390 | 15:31 |
| cardoe | what topic are we gonna pick up? | 16:01 |
| cardoe | Just trying to balance calls. | 16:01 |
| cardoe | or sessions I should say. | 16:01 |
| opendevreview | Takashi Kajinami proposed openstack/networking-generic-switch master: Drop remaining logic for linuxbridge-agent https://review.opendev.org/c/openstack/networking-generic-switch/+/965149 | 16:06 |
| dtantsur | frickler: hey, re https://bugs.launchpad.net/ironic/+bug/2119212: do you have regular access to this hardware? | 16:17 |
| dtantsur | we can develop code but we don't have anything to test it on | 16:17 |
| frickler | dtantsur: not directly, but via a customer | 16:18 |
| dtantsur | frickler: are they open to testing things? | 16:19 |
| dtantsur | (also is it only NFS or CIFS too?) | 16:20 |
| frickler | dtantsur: depends on details, but in general I'd assume so, yes | 16:20 |
| dtantsur | cid: ^^^ | 16:20 |
| frickler | I have to check but I think they were only interested in nfs | 16:20 |
| cid | ++ | 16:21 |
| JayF | I'll note that our (GR-OSS) interest in this is basically baked in the assumption the no-http-vmedia hardware will be visited upon us soon enough lol | 16:21 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Remove sushy-oem-idrac from driver requirements https://review.opendev.org/c/openstack/ironic/+/965114 | 16:26 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/2025.2: Add upgrade check for iLO/iLO5 driver deprecation https://review.opendev.org/c/openstack/ironic/+/965158 | 16:49 |
| dtantsur | JayF: damn it, bifrost enables ilo by default | 16:56 |
| dtantsur | I can take a look later this week if nobody beats me to it | 16:56 |
| JayF | cid: ^ | 16:58 |
| JayF | dtantsur: cid is handling ilo retirement, that can be a part of it. It's all 'ironic' | 16:58 |
| dtantsur | that would be nice, I'm overwhelmed this week | 16:58 |
| JayF | just make sure that gets as review feedback on the change, espeically if not ack'd here | 16:58 |
| rpittau | I was going to check that as I've noticed it when removing tinyipa bits, I can review the change if cid has time to propose the patch, or I can even do it myself | 16:59 |
| cid | Will definitely pick that up. | 16:59 |
| rpittau | ack | 16:59 |
| janders | cardoe TheJulia looking at tomorrow's PTG topics, I'd like to try educate myself on how VXLAN/EVPN work in the context of baremetal networking. Would you be able to point me to a good resource to get started? I used to know a fair bit about baremetal SDN from the ovs/vlan networking era but I feel I need to fast-forward to 2025 state of things. | 17:03 |
| janders | Thanks in advance! | 17:03 |
| cardoe | janders: uhhh. I struggle at explaining it to the Neutron folks. | 17:03 |
| JayF | VXLAN is VLAN except with a bigger int and different header field | 17:11 |
| JayF | education finished! /s | 17:12 |
| * tkajinam has been hoping that vxlan is that simple | 17:21 | |
| JayF | me too :) | 17:24 |
| *** ubuntu is now known as Guest30073 | 17:27 | |
| TheJulia | janders: I guess a starting point is vxlan as a concept then pile on the couple specs on the topic? | 17:39 |
| TheJulia | janders: there is still a block of question marks in terms of vendors, but I think the idea is have n-g-s do the switch side needful which is beyond the base fabric interactions | 17:48 |
| TheJulia | I added another item to our bonus section, sounds likewe might need to swap our byte encoding to 4 byte utf in the database which if I remember correctly is an alter table universe of unhappiness | 17:51 |
| TheJulia | i.e. not online upgrade friendly. we can discuss tomorrow or thursday | 17:51 |
| janders | Thank you TheJulia | 17:54 |
| TheJulia | The high level idea is every compute node has vxlan, and you end up with mesh of tunnels, and those tunnels need to be extended out to the switches, and then inside the switches the vni needs to be attached to a vtep which is bound to a vlan, which is then attached to a port | 17:55 |
| janders | complex, but powerful! | 17:56 |
| TheJulia | somewhat, yeah. There are a few different moving pieces there | 18:09 |
| cardoe | So I piddled on our cross-sessions for tomorrow a little bit... (sorry). | 18:29 |
| cardoe | I met with cinder just now for the past hour about driver improvements. | 18:29 |
| cardoe | But I think I primed the convo around what we want to do. | 18:29 |
| TheJulia | cool, I also primed that convo | 18:43 |
| TheJulia | or at least, tried to. | 18:43 |
| opendevreview | Verification of a change to openstack/networking-generic-switch master failed: devstack: Drop explicit etcd api version https://review.opendev.org/c/openstack/networking-generic-switch/+/957207 | 19:05 |
| opendevreview | cid proposed openstack/bifrost master: ILO driver deprecation warning https://review.opendev.org/c/openstack/bifrost/+/965177 | 19:28 |
| cardoe | janders: So not sure if you're still around? | 20:03 |
| cardoe | TheJulia: So I spoke with jbernard and his cinder crew during the PTG earlier. One of the things I brought up with them was the fact that different backends and different connectors require different properties. e.g. https://github.com/openstack/nova/blob/30bf8c1025ea24823ebde3eacede4ea34ec6be8c/nova/virt/ironic/driver.py#L2058 means this only works with iSCSI and not with NVMe | 21:51 |
| TheJulia | cardoe: we've already discussed that | 21:52 |
| cardoe | They told me they use os-brick with nova to abstract this. Like https://github.com/openstack/os-brick/blob/a98490788cd2cc5bf7b48e19dedf4f4fd8658c14/os_brick/initiator/connectors/iscsi.py#L64 | 21:53 |
| cardoe | Yep. I'm rehashing what you and I spoke about. | 21:53 |
| cardoe | Just trying to level set for tomorrow so we don't have to cover it with them. | 21:53 |
| TheJulia | ahh, okay | 21:53 |
| TheJulia | yeah, we can easily house an initiator value in ironic afaik | 21:54 |
| TheJulia | err | 21:54 |
| TheJulia | initiator/client data | 21:54 |
| * TheJulia needs to go lay down | 21:54 | |
| cardoe | The issue I brought up is that this os-brick code won't work for us. Because in all the cases it runs a binary that's expected to work with the storage. They were telling me "just run it on your compute" | 21:54 |
| * TheJulia facepalms | 21:55 | |
| cardoe | So I had to make them understand that my compute is really not related at all to where my machine is so running "nvme get_host_nqn" won't work like ever... cause I don't have the nvme binary installed in my nova-compute-ironic pod and whatever it returns is gonna be the wrong thing. | 21:55 |
| TheJulia | you'd think after all these years, people would understand there is more than a VM hypervisor | 21:56 |
| TheJulia | yeah | 21:56 |
| cardoe | I suggested that what they need are "defines" like neutron-lib. | 21:56 |
| cardoe | Which they asked for that to live in os-brick. Which I had agreed to. | 21:56 |
| TheJulia | could be reasonable for us to house some translation/remodeling logic and import os-brick in the grand scheme of things | 21:57 |
| TheJulia | dunno | 21:57 |
| cardoe | However locally on my Mac (yeah I'm difficult) os-brick is a beast. | 21:58 |
| TheJulia | ewww | 21:58 |
| cardoe | I was aiming for "from os_brick.initiators.connectors import constants" where I'm writing constants.py | 21:59 |
| TheJulia | ++ | 21:59 |
| cardoe | So just trying to do that import where constants.py is an empty file... oslo_privsep is being called with a C FFI call to libc.so | 21:59 |
| cardoe | and making some Linux-y syscall | 22:00 |
| cardoe | Feels like our RAM usage is gonna go in the wrong direction with that. | 22:00 |
| TheJulia | So, executing calls in __init__.py someplace? | 22:00 |
| cardoe | yep | 22:00 |
| TheJulia | Ugh | 22:00 |
| TheJulia | How… awful | 22:01 |
| TheJulia | Maybe different plan? | 22:01 |
| * TheJulia recommends cat cuddling | 22:01 | |
| cardoe | I'm gonna finish off that constants.py and my thought was a follow on to cinder to switch all the free form strings to that. | 22:03 |
| TheJulia | I think that is fine as long as we can import without __init__.py executing quietly | 22:04 |
| cardoe | Yeah I figure that can be solved and I'm sure nobody is gonna be upset at solving that. | 22:05 |
| cardoe | What I'm not sure about is that nova needs to know which type of backend is in play to send the right properties | 22:06 |
| TheJulia | Regardless of documented strings or a constant values or set of them, the nova.virt.ironic code still needs to know what to choose. But that is sort of a simpler problem | 22:06 |
| cardoe | --type <type> Type of the volume connector. Can be 'iqn', 'ip', 'mac', 'wwnn', 'wwpn', 'port', 'portgroup'. (from ironic's CLI) | 22:06 |
| TheJulia | We can add imitator….. and just abstract | 22:06 |
| TheJulia | Just an idea | 22:07 |
| cardoe | ? I was thinking we need some kind of mapping. | 22:07 |
| TheJulia | I’m not entirely sure we do, actually | 22:07 |
| cardoe | Ultimately iSCSI and NVMe can both use "iqn" | 22:07 |
| TheJulia | Well we use it | 22:07 |
| TheJulia | Ugh | 22:07 |
| TheJulia | Oh wait | 22:08 |
| TheJulia | Some drivers do and some don’t | 22:08 |
| TheJulia | Nqn or iqn | 22:08 |
| TheJulia | But seriously I’m fine with changing the ironic data shape too | 22:08 |
| cardoe | So if I force Ironic to allow "nqn" in there. I can make iSCSI and NVMe work. | 22:09 |
| TheJulia | Peaches wants me to take a nap https://usercontent.irccloud-cdn.com/file/Yrx53VEs/1761689394.JPG | 22:10 |
| cardoe | Cause when nova hits Ironic to ask, it gets back iqn then I use the iSCSI properties. If it gets back nqn, I use the NVMe properties. | 22:10 |
| TheJulia | Indeeeeed | 22:10 |
| cardoe | Anyway yeah good time to walk away. cat nap sounds good. | 22:10 |
| TheJulia | So the thing I’m sort of wondering is dynamically plugging interfaces over | 22:10 |
| TheJulia | To name, but! We reattach | 22:10 |
| cardoe | Just wanted to brain dump before I forgot | 22:11 |
| TheJulia | So nova can do a thing and we’ll reattach it | 22:11 |
| cardoe | Nova is calling Cinder directly to get the connection data from the storage device which it then passes to Ironic | 22:11 |
| cardoe | So on the storage device you have to effectively whitelist the "iqn" or "nqn" that the server will connect as. | 22:12 |
| cardoe | So Nova queries Ironic for the connector details and Ironic returns it back in the "iqn" field but if Cinder is running with an NVMe backend, Cinder won't give Nova the connection data if you send it in the "initiator" field (which is where iqn gets copied to) and will instead throw and error and Nova will just drop the whole thing on the ground. | 22:14 |
| TheJulia | Yup | 22:14 |
| cardoe | There's all this abstraction away from your backend but things are way too coupled to the backend. | 22:14 |
| TheJulia | Yeah, exactly | 22:15 |
| cardoe | It'd be great if Cinder would just you call it saying "host X is gonna connect to volume Y, gimme what I need to know" | 22:15 |
| cardoe | Cause it's literally the same data and same call between NVMe and iSCSI | 22:15 |
| TheJulia | Yup, but sans like metal is hard | 22:16 |
| cardoe | It's just Cinder requires the data in the "initiator" or the "nqn" field. | 22:16 |
| TheJulia | Initiator would be a better field to standardize on, tbh | 22:16 |
| cardoe | Agreed. So that might be something to ask them. | 22:19 |
| cardoe | Last little funky thing I realized that I just hand waved my way past... | 22:24 |
| cardoe | I patched my Ironic to always create a volume connector for the bare metal server that's got an IQN of "nqn.2014-08.org.nvmexpress:uuid:$INSTANCE_UUID" | 22:25 |
| TheJulia | … hmmm | 22:27 |
| TheJulia | We need a “bake some volume connectors real quick” | 22:28 |
| TheJulia | API call | 22:28 |
| cardoe | But that's not all... I also then push that into the settings. and I've also gotta hand wave to tell Ironic that changing these settings does not require a host reboot. | 22:28 |
| TheJulia | Oh to fit into the spawn flow of the virt driver | 22:29 |
| cardoe | Yes. Cause the virt driver expects the volume connector to exist. | 22:30 |
| TheJulia | I mean, ideally the imitator id is not instance dependent or related | 22:30 |
| TheJulia | Yeah | 22:30 |
| cardoe | So the default for all my Dell's are related to the service tag. I can reliably compute it from inspection. | 22:31 |
| TheJulia | I’d expect the imitators to be pre-baked | 22:31 |
| TheJulia | At least that is the way it is modeled | 22:31 |
| cardoe | So that was my other thought. So I jotted down the idea of an inspection hook. | 22:31 |
| cardoe | I had forgotten about these patches cause they're small and I did them ages ago. | 22:32 |
| TheJulia | That and or and (yes, intentionally) if we have a clean step to reconfigure imitators | 22:32 |
| TheJulia | Err initiators | 22:32 |
| cardoe | Actually that's probably better. | 22:33 |
| cardoe | reconfigure_initiators would read the current settings from the BMC and create the appropriate volume connectors. | 22:33 |
| TheJulia | ++++++++++++ | 22:33 |
| cardoe | Cause the BMC has settings for like multi-path as well which can be put into the extra field on the volume connector and Nova reads that and passes that along to Cinder and it might work. I've not tried actually failing it over. | 22:34 |
| TheJulia | Now your cooking with oil! | 22:35 |
| TheJulia | Cat gravity winning, nap time. | 22:35 |
| cardoe | And being yelled at by my terrier so I think we'll pick it up tomorrow. | 22:35 |
| cardoe | Good timing | 22:35 |
| opendevreview | David Nwosu proposed openstack/ironic master: Add Getting Started guide https://review.opendev.org/c/openstack/ironic/+/965189 | 23:28 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
