| JayF | TheJulia: it's not an execute | 00:39 |
|---|---|---|
| JayF | TheJulia: it's doing an open on a file in /sys to see if it's ro, similar kinda pattern in the other one | 00:39 |
| TheJulia | doh! | 00:40 |
| TheJulia | that is not *that* bad, but yeah | 00:40 |
| JayF | when I say scary I just mean like, the dread when I saw it erroring *trying to erase /dev/sda* because my *real /dev/sda* is r/o | 00:45 |
| JayF | nothing is that scary once you know what's really going on :D | 00:45 |
| dtantsur | TheJulia, yeah, I know, that's great progress (re VirtualMedia in Systems). Now we only need to implement that.. | 07:34 |
| opendevreview | Verification of a change to openstack/ironic stable/2023.1 failed: redfish_address - wrap_ipv6 address https://review.opendev.org/c/openstack/ironic/+/895795 | 09:25 |
| opendevreview | Merged openstack/ironic-inspector stable/zed: Handle bracketed IPv6 redfish_address https://review.opendev.org/c/openstack/ironic-inspector/+/896324 | 09:39 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Enable the new inspection implementation https://review.opendev.org/c/openstack/bifrost/+/896925 | 10:38 |
| dtantsur | folks, looking for bifrost reviews: https://review.opendev.org/c/openstack/bifrost/+/896143 and https://review.opendev.org/c/openstack/bifrost/+/891693 | 10:40 |
| dtantsur | both are pretty important and pretty simple | 10:41 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: [PoC] Life without Inspector https://review.opendev.org/c/openstack/bifrost/+/887934 | 10:47 |
| alaysd | hi everyone | 10:53 |
| alaysd | is there a way we can use IRONIC / Bifrost to PXE boot an ESXi VM without OS ? | 10:53 |
| alaysd | I donot have access to any BMC, I want to PXE boot it, is it possible? | 10:53 |
| dtantsur | alaysd, hi. you can probably make it work by manually configuring https://docs.openstack.org/ironic/latest/admin/agent-power.html | 10:54 |
| dtantsur | It's not going to be a straightforward out-of-box experience though. | 10:54 |
| alaysd | okay, I shall check | 10:55 |
| alaysd | But do we need "driver" to PXE booting a ESXi VM? Can we somehow bypass that? | 10:56 |
| alaysd | for physical nodes I do have IPMI so that option seems to work and can sucessfully install OS | 10:56 |
| dtantsur | please check the link, it mentions the manual-management driver | 10:57 |
| alaysd | ok, thanks | 10:57 |
| alaysd | but how is the device discovered by Bifrost machine? | 11:12 |
| alaysd | does my bifrost server need to be in the same subnet as my ESXi VM? Also, what ACL or next-server configurations do I need to set on Subnet so that PXE boot happens from bifrost ? | 11:13 |
| alaysd | Dtantsur, I was just seeing your video Bifrost 101 | 11:14 |
| dtantsur | Bifrost provides a DHCP/PXE server, which has to be on the same L2 network as the VM | 11:14 |
| dtantsur | Essentially, everything is similar to a bare-metal node, just without power control | 11:15 |
| dtantsur | (you may also need to enable DHCP traffic in your hypervisor, it's often blocked) | 11:15 |
| alaysd | I guess the DHCP traffic is enabled as the VM is trying to look for PXE server and is erroring out saying "No DHCP or proxyDHCP offers were received" | 11:17 |
| dtantsur | "No offers received" does not sound like enabled to me | 11:18 |
| dtantsur | It may be called something like promiscuous mode for the vNIC | 11:18 |
| alaysd | are you asking if the VMs can take DHCP IPs? | 11:20 |
| dtantsur | I'm asking if DHCP traffic is allowed to the VMs | 11:20 |
| dtantsur | At least some hypervisors block it by default | 11:20 |
| dtantsur | If it is, you need to use tcpdump to see why DHCP requests or offers are not received by the other side | 11:21 |
| alaysd | I m sorry to sound dumb, very new to OpenStack | 11:22 |
| alaysd | do I need to take the tcpdump on the ESXi VM ? | 11:23 |
| alaysd | @dtantsur, please help :( | 11:28 |
| dtantsur | alaysd, probably start with tcpdump on the bifrost node. And check the networking settings for the thing I mentioned. | 11:37 |
| dtantsur | I'm trying to help, but you're doing something very non-standard with, as you say yourself, lacking knowledge. That won't be easy. You'll need to do a lot of experimenting and researching. | 11:37 |
| iurygregory | good morning Ironic | 11:38 |
| mallik | dtantsur, ping | 11:51 |
| dtantsur | o/ | 11:51 |
| mallik | dtantsur, wanted to check quickly about inspector in iornic. I have setup master devstack around 2weeks back. I tried doing inband inspection using inspector in ironic. . | 11:54 |
| mallik | dtantsur, I am seeing this error Failed to start inspection: ['internal', 'public'] endpoint for | | 11:54 |
| mallik | | | baremetal-introspection service in RegionOne region not found | 11:54 |
| Nisha_Agarwal | dtantsur, is there any documnetation for latest inspector usage in ironic? | 11:55 |
| Nisha_Agarwal | Do we need to create inspector endpoint seperately even if it is merged with ironic? | 11:57 |
| dtantsur | are you two asking the same question or just two different questions? :) | 11:57 |
| Nisha_Agarwal | same | 11:57 |
| Nisha_Agarwal | :) | 11:57 |
| dtantsur | right | 11:57 |
| dtantsur | DevStack has not been changed to use the new inspection approach. | 11:57 |
| dtantsur | You won't need to crate an inspector endpoint because there will be no inspector. But our devstack plugin still uses inspector. | 11:58 |
| Nisha_Agarwal | ok so if we have to use inband inspection in latest ironic how do we proceed? | 11:58 |
| dtantsur | like before. nothing has changed so far. | 11:58 |
| Nisha_Agarwal | so inspector service has to be brought up? | 11:59 |
| iurygregory | yup ^ | 11:59 |
| Nisha_Agarwal | ok if we have to use latest inspection code merged in ironic, how can we do that? | 12:01 |
| dtantsur | Why are you trying to do it? We'll provide documentation, devstack changes, etc in time. | 12:01 |
| Nisha_Agarwal | :) | 12:01 |
| mallik | dtantsur, is the new inspection not targeted to be completed in bobcat release? | 12:04 |
| dtantsur | It was not targeted and has not been completed. | 12:04 |
| mallik | dtantsur, ok thanks. | 12:04 |
| dtantsur | It's ready for experiments already, but that's it. The big opening is expected in Caracal. | 12:04 |
| mallik | ok | 12:05 |
| alaysd | how to know the PXE server from which it has booted? | 12:34 |
| alaysd | dtantsur please | 12:40 |
| alaysd | :((( | 12:40 |
| alaysd | I know its non standard but if possible and you have some time then please tell me | 12:40 |
| alaysd | i want to know atleast the PXE boot happened then it is from the server I want | 12:42 |
| dtantsur | alaysd, I can give you some directions, but I cannot debug your environment for you. You can use tcpdumps on DHCP ports, you can check the logs of the appropriate services (dnsmasq, for example), you can look at the VM's virtual console during boot. | 12:49 |
| TheJulia | Good morning | 12:50 |
| * TheJulia sips coffee and reads email | 12:50 | |
| dtantsur | morning TheJulia | 12:52 |
| dtantsur | TGIF! | 12:52 |
| TheJulia | indeed! | 12:54 |
| TheJulia | I think today, I'm going to hack on sushy-tools | 12:56 |
| dtantsur | Enjoy! | 12:56 |
| TheJulia | plus a customer meeting which I really just need not attend | 12:57 |
| dtantsur | I hope to finish Outreachy business this week | 12:57 |
| dtantsur | heh | 12:57 |
| TheJulia | dtantsur: w/r/t virutalmedia in systems, would it be helpful to at least raise visibility of the link in sushy? I was hacking on sushy yesterday for uri booting | 12:59 |
| TheJulia | and added BootProgress | 12:59 |
| TheJulia | I'm not sure the on system options object is... well defined based upon what I glanced at yesterday | 12:59 |
| dtantsur | it would be helpful, I just never had time to do it | 13:10 |
| TheJulia | ack, I'll see what I can do | 13:11 |
| * dtantsur is going through the history of our Outreachy participation and that is making him upset | 13:14 | |
| TheJulia | :( | 13:16 |
| TheJulia | err, maybe worth waiting. Dell only has the config object with a single field, HPE has not implemented v1.18 | 13:38 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Do not store ramdisk logs as part of the inventory https://review.opendev.org/c/openstack/ironic/+/896941 | 13:42 |
| TheJulia | If possible, I'd appreciate a few quick reviews on metalsmith branches so I can un-wedge ipa. Specifically https://review.opendev.org/q/Id258b294972c2f5ea816cb274773b8947f92e110 | 13:43 |
| drannou1 | TheJulia : Thx for the Rescue info, I'm building my own IPA, debian based, and discover that the "wheel" group was not created and so no 'rescue' user was created. When you say that the DHCP should be automatically trigger, how is it done ? neutron should make a shut/noshut on TOR config ? or something trigger by the IPA ? | 14:05 |
| TheJulia | drannou1: I believe the basis is on on-exit script to be triggered upon IPA's exit | 14:20 |
| TheJulia | err, an on-exit | 14:20 |
| TheJulia | I'm in the middle of a email, I can look when I'm done | 14:20 |
| TheJulia | okay! chairy email sent | 14:24 |
| TheJulia | drannou1: you figured out the issue with the user... hmm... what is the dhcp path taken | 14:31 |
| TheJulia | I think you may have found a bug! | 14:38 |
| TheJulia | so, the way it is *supposed* to work, is that upon exit, dhcp is supposed to be re-triggered | 14:38 |
| TheJulia | We did that in the days of original coreos, we do that in tinyipa, I don't see that happening with the DIB based ipa images | 14:39 |
| TheJulia | ... I'm wondering if everyone has been testing them on the same flat network, so it just works | 14:40 |
| TheJulia | drannou1: so, for my context, I'm guessing your using the "neutron" network_interface, static l2 network, different l3 networks between your tenant workload and your rescue network where the ramdisk boots? I'm guessing no ML2 switch port driver as well? | 14:46 |
| TheJulia | drannou1: for building, are you using ironic-python-agent-builder at all? | 14:47 |
| drannou1 | TheJulia: We are using our own neutron driver implementation, which vlan based. So the "customer" is using his own vlan, and we are forcing a specific rescue vlan, which host the TFTP server (so a "classical" infrastructure I would say) | 15:08 |
| drannou1 | For the build we are using ironic-python-agent-builder --release bullseye debian | 15:09 |
| TheJulia | drannou1: Interesting, When we tear down the vif, we expect the port to go down, I guess line carrier is held up in your case? | 15:09 |
| TheJulia | the line carrier dropping *would* normally force dhcp to re-trigger | 15:10 |
| drannou1 | Yeah that's what we where thinking about, but We don't see neutron asking for a "down" | 15:10 |
| TheJulia | so we did it in the neutron ml2 plugin if memory serves | 15:10 |
| drannou1 | give me a few tens of minutes (booting a server is sooooo long), I'm testing it | 15:13 |
| TheJulia | no worries | 15:13 |
| TheJulia | so looking at networking-generic-switch, we shutdown the port from the unbind operation of the prior vif when we move it to the rescue network | 15:14 |
| TheJulia | then we re-setup the port | 15:14 |
| drannou1 | we don't saw that on the other drivers, but may be we missed it, we are checking | 15:16 |
| TheJulia | Yeah, it is sort of super well hidden deep inside the base driver code in networking-generic-switch, and afaik some of the vendor SDN stuffs out there work the same way | 15:23 |
| drannou1 | ok so Here we are : We manually made the down up on the switch (shut/noshut), I see in the host VNC the link going down and UP, but no DHCP renew append | 15:24 |
| drannou1 | of course if I manually restart dhclient, it will make the DHCP request | 15:26 |
| TheJulia | I guess there is not something like NetworkManager running.... | 15:27 |
| TheJulia | hmmm | 15:27 |
| drannou1 | but I fill that it would be better if IPA force it | 15:27 |
| TheJulia | So I guess the path forward is *likely* code to explicitly bounce dhcp by default if visible like we do with tinycore testing | 15:28 |
| TheJulia | we need something here https://github.com/openstack/ironic-python-agent-builder/blob/master/dib/ironic-python-agent-ramdisk/static/usr/local/bin/ironic-python-agent-create-rescue-user.sh | 15:29 |
| drannou1 | yes there or directly in the rescue driver | 15:30 |
| drannou1 | to better separe user creation and restart of the network | 15:30 |
| TheJulia | you'd basically need a similar pattern, on ramdisk exit do the needful | 15:30 |
| TheJulia | IPA doesn't explicitly trigger dhcp, it expects the OS to do the needful because we don't want IPA's API to ever possibly touch an untrusted network, but since we made that decision, we've implemented additional controls so we might be willing to accept a patch to toggle dhcp before completing teardown of the agent | 15:32 |
| drannou1 | going back on the switch side, it seems that this 'down/up' is optional: https://opendev.org/openstack/networking-generic-switch/src/branch/master/networking_generic_switch/devices/__init__.py#L43 ex : https://opendev.org/openstack/networking-generic-switch/src/branch/master/networking_generic_switch/devices/netmiko_devices/__init__.py#L283 | 15:34 |
| drannou1 | Yeah but I also agree that if there is this "down/up" on the host POV, it should trigger back the DHCP. so may be a DIB wrong config that need to be check | 15:36 |
| TheJulia | zigo: have you ever tried rescue, out of curiosity? | 15:36 |
| TheJulia | I do agree, there are many variables, unfortunately | 15:37 |
| iurygregory | facepaml, I just noticed the api-ref is not showing information about v1/nodes/{node_ident}/firmware ... | 16:59 |
| TheJulia | it happens | 17:00 |
| TheJulia | api-ref is not the easiest thing | 17:00 |
| * TheJulia takes the corgi overlord for a walk he is demanding | 17:00 | |
| opendevreview | Iury Gregory Melo Ferreira proposed openstack/ironic master: api-ref for nodes/{node_ident}/firmware https://review.opendev.org/c/openstack/ironic/+/896958 | 17:51 |
| TheJulia | sushy-tools makes my head hurt | 18:10 |
| *** awb_ is now known as awb | 18:20 | |
| JayF | Do not expect me to be around today; got in a small fender bender during my morning volunteering. I am physically OK but dealing with the logistical fallout. | 18:40 |
| TheJulia | ugh, ack | 18:45 |
| iurygregory | JayF, take care | 19:48 |
| JayF | Yeah I'm physically OK; truck damage appeared to be superficial but it's got a shimmy in the front end so something probably got bent | 19:55 |
| JayF | thank you for the concern | 19:56 |
| opendevreview | Julia Kreger proposed openstack/sushy-tools master: WIP: Support testing for HttpBootUri https://review.opendev.org/c/openstack/sushy-tools/+/896963 | 21:04 |
| opendevreview | Merged openstack/metalsmith stable/yoga: Stable-Only: CI: Force install an appropriate openstacsdk version https://review.opendev.org/c/openstack/metalsmith/+/896763 | 22:47 |
| opendevreview | Merged openstack/metalsmith stable/xena: Stable-Only: CI: Force install an appropriate openstacsdk version https://review.opendev.org/c/openstack/metalsmith/+/896765 | 22:47 |
| opendevreview | Merged openstack/metalsmith stable/wallaby: Stable-Only: CI: Force install an appropriate openstacsdk version https://review.opendev.org/c/openstack/metalsmith/+/896766 | 22:47 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!