| opendevreview | Jacob Anders proposed openstack/sushy-tools master: Translate VM name to UUID to avoid redirect issue https://review.opendev.org/c/openstack/sushy-tools/+/924297 | 00:39 |
|---|---|---|
| janders | cardoe CC TheJulia I think IPA booting on BIOS settings change is expected (as much as it may have limited sense for the IPA functionality itself). BIOS setting change will create a lifecycle controller job and a reboot is needed for that to run. | 00:47 |
| TheJulia | It occurs to me, part of a beer and salad into dinner before my flight, that there is an explicit option *not to* boot ipa | 00:48 |
| janders | that sounds about right (and will likely be the future direction) but for now while working on https://review.opendev.org/c/openstack/sushy-oem-idrac/+/915092 I was putting up with IPA boots (which slowed the process a little bit adding a reboot on top of all the LC-triggered reboots) but wasn't a problem | 00:50 |
| janders | that auto-attach thing however is curious | 00:50 |
| janders | looking into it now | 00:50 |
| janders | I do not recall seeing this before | 00:50 |
| janders | it did exist in 6.10 and 7.00 iDRAC firmwares (and seems set to AutoAttach by default) | 00:52 |
| TheJulia | That is sort of what my brain was thinking, the don’t attach ipa thing for the step, I would need to look at the code. I suspect it is under documented | 00:55 |
| janders | cardoe have a look at your iDRAC config in the area referenced here: https://www.dell.com/support/manuals/en-au/idrac9-lifecycle-controller-v3.0-series/idrac_3.00.00.00_ug/configuring-remote-file-share-using-web-interface?guid=guid-7cdb4a3e-4649-44de-9c70-0207d200674e&lang=en-us and see if you have attach mode set to detached by any chance - if so, | 00:58 |
| janders | try change it, see what happens | 00:58 |
| janders | I had a look at a couple machines and they all had AutoAttached set, seems like the default | 00:58 |
| janders | also see if it's not already mounted (also check for stale mounts) | 00:59 |
| janders | GET /redfish/v1/Managers/iDRAC.Embedded.1/VirtualMedia/CD should give you an instant answer. If it is in some half-configured state try to reset the iDRAC (may be a good idea in any case for a clean start). is the iDRAC running recent firmware? | 01:00 |
| cid | cardoe, :( not judging at all. Was just waving an EOD goodbye. | 04:28 |
| rpittau | good morning ironic! o/ | 06:59 |
| TheJulia | Good morning from Chicago’s airport at 2 AMm | 07:01 |
| rpittau | Oh gosh :/ | 07:01 |
| sylvr | good morning :) | 07:06 |
| TheJulia | This week is definitely a rough week for me :( | 07:17 |
| janders | :( | 07:19 |
| sylvr | Hello ! I got this error "Deploy step deploy.prepare_instance_boot failed" | 07:23 |
| sylvr | Seems that "Command: efibootmgr -v -c -d /dev/sda -p 1 -w -L rocky -l \EFI\rocky\shimx64.efi" is missing some quote to escape the \ | 07:23 |
| dtantsur | sylvr: we don't put these \ there verbatim, I think they come from somewhere and don't need escaping | 07:24 |
| dtantsur | check the logs for while the command actually failed | 07:24 |
| sylvr | Command: efibootmgr -v -c -d /dev/sda -p 1 -w -L rocky -l \EFI\rocky\shimx64.efi | 07:30 |
| sylvr | Exit code: 5 | 07:30 |
| sylvr | Stdout: '' | 07:30 |
| sylvr | Stderr: 'Could not prepare Boot variable: Invalid argument\n' | 07:30 |
| TheJulia | I am curious as to why it is not running from the csv file…. | 07:31 |
| sylvr | is that what you wanted me to send ? | 07:31 |
| TheJulia | Unless that is sourced from the csv file | 07:31 |
| dtantsur | it must be, how else could we get "rocky"? | 07:32 |
| TheJulia | Dunno, running on like very little sleep during my 3.5 hour flight from San Diego | 07:32 |
| * TheJulia is going to be a disaster | 07:32 | |
| dtantsur | TheJulia: managed to get going? | 07:32 |
| TheJulia | Yeah, sitting in Chicago O’hare, landed around 1:15 AM, next flight boards at 5:30 AM | 07:33 |
| dtantsur | oh, brutal. good luck! | 07:33 |
| * TheJulia is watching United Airlines technical operations folks inspect aircraft | 07:34 | |
| sylvr | I'm using old machines for my tests, could it be something to do with a bios/uefi config ? I know I had loots of issue with boot order | 07:35 |
| TheJulia | When you say old, what kind of old? | 07:36 |
| sylvr | PowerEdge R720 running idrac7 | 07:36 |
| dtantsur | pretty old kind of old indeed | 07:36 |
| sylvr | My boss says anything older than like 5 years to be old, so technically I have only old machines ^^ | 07:37 |
| TheJulia | I know UEFI on idrac8 generation hardware is a little weird, but does work. 7 may be a whole different story | 07:38 |
| dtantsur | yeah. a metal3 person uses ilo4 machines for his testing, and he has to switch to legacy boot. | 07:38 |
| TheJulia | I recently shattered someone’s dreams of idrac8 hardware being new and happy….. I pointed out the end of support date for that hardware. | 07:39 |
| TheJulia | Not surprising | 07:39 |
| dtantsur | time flies, yeah? | 07:39 |
| TheJulia | Idrac8 gear really wants to think of UEFI booting like classic bios booting and it feels “wedged in” | 07:40 |
| rpittau | idrac7 is considered ancient history now :P | 07:40 |
| TheJulia | Not quite like the scrolls of Pythia | 07:40 |
| sylvr | so I should look for an archeologist ? xD | 07:40 |
| TheJulia | All this has happened before, all this will happen again. | 07:40 |
| dtantsur | better a necromancer - you need to keep the beasts alive | 07:40 |
| sylvr | TheJulia : is this from Dark ? | 07:41 |
| TheJulia | Battlestar Galactica - the early 2000s rendition | 07:41 |
| sylvr | well, at this point I'm okay with completely wiping boot entries to hopefully get a clean slate for my (hopefully final) deployment | 07:42 |
| sylvr | I really should start watching more sci-fi | 07:42 |
| dtantsur | sylvr: is legacy boot on the table? (I cannot believe I'm suggesting that) | 07:42 |
| TheJulia | dtantsur: your being a realist ;) | 07:43 |
| dtantsur | happens to me sometimes, especially after a vacation :-P | 07:43 |
| TheJulia | For sale, kid on other side of the bench who is kicking where I am trying to relax and maybe fall asleep a little. | 07:43 |
| sylvr | if it can be configured just for this node, that could be ok ! | 07:43 |
| dtantsur | TheJulia: do the United technicians have a large empty box? just asking | 07:44 |
| dtantsur | sylvr: it's definitely configurable per node, regardless of what the default is | 07:44 |
| TheJulia | dtantsur: secure doors, unfortunately | 07:45 |
| dtantsur | le sigh | 07:45 |
| dtantsur | these security people! only produce closed doors and BSOD's! | 07:45 |
| rpittau | lol | 07:46 |
| sylvr | dtantsur: well, let's go, I'm using kayobe, should I check the documentation over there for singular node config ? | 07:46 |
| sylvr | if you have documentation/guide on how to do that through ironic/bifrost I'll take it | 07:46 |
| TheJulia | So speaking of secure boot, I was hoping to chat with some of rh’s secure boot folks around key injection. I’m thinking we might want to look at making/acquiring/forking of sorts mokutil to make it easier so we can strip the keyboard req out and maybe build in a song pipeline/process/step | 07:47 |
| dtantsur | sylvr: in pure ironic https://docs.openstack.org/ironic/latest/install/advanced.html#boot-mode-support | 07:47 |
| dtantsur | if you use the bifrost inventory, you should be able to update 'properties' as well | 07:47 |
| sylvr | thanks, I'll check that, and try to see what can I do | 07:48 |
| TheJulia | I don’t know if I’ll be able to have that chat when I’m in Boston now since I’ve lost a day. Thanks crowdstrike! | 07:48 |
| dtantsur | TheJulia: I've thought of this, but I guess we're going to hit a chicken-and-egg problem when trying to boot $something to run mokutil? | 07:48 |
| dtantsur | I'd rather look the direction of Redfish at this stage | 07:48 |
| TheJulia | dtantsur: I think if base kernels are still signed we should be able to iso boot something minimalistic without modification, thinking less about IPA and more about the common more complex workloads | 07:49 |
| TheJulia | To help folks get that workload initially up | 07:50 |
| dtantsur | I guess we could.. I just find the Redfish approach more future-looking and potentially less fragile. | 07:50 |
| TheJulia | We just have folks who are not on redfish BMCs for everything.. | 07:51 |
| dtantsur | Sure, we still do, but we're talking about the future, and the future seems to be red and fishy | 07:52 |
| TheJulia | Not for everyone :) | 07:53 |
| TheJulia | But I do generally agree. Now if folks would appear regarding snmp stuffs…. | 07:54 |
| sylvr | redfish seemed so cool but I'm stuck with ipmi for now | 07:58 |
| TheJulia | It is far better to have a rest api… that is for sure. | 08:00 |
| dtantsur | sylvr: yeah, you don't have good options for idrac 7, unfortunately | 08:04 |
| dtantsur | IPMI is not going away as the baseline | 08:05 |
| sylvr | and idrac8 don't work either, they were never updated and got some nasty issue with ssl :) | 08:05 |
| dtantsur | ahhh, can be | 08:05 |
| dtantsur | I remember we needed to ask HPE people to fix iLO 5 for newer FIPS in RHEL | 08:05 |
| dtantsur | otherwise, many people just opt out of TLS validation | 08:06 |
| rpittau | sylvr: AFAIK the very last update to idrac8 was in February, but I guess you gave that a try already? | 08:07 |
| sylvr | I'll look into it, honestly got so many compatibility issue I've gone the easiest way possible | 08:08 |
| sylvr | rpittau: yes! but it didn't work and Dell support told me to update version by version :) | 08:08 |
| rpittau | mmm yeah | 08:08 |
| sylvr | so 12 nodes times like 10 versions x) I'll need to look into automating this process but I need to preserve my sanity | 08:09 |
| dtantsur | in the bright future, when you get new hardware, we'll be able to automate firmware upgrades for you | 08:10 |
| TheJulia | that assumes the vendors update works properly | 08:10 |
| TheJulia | Too soon to remind us of crowdstrike? | 08:11 |
| dtantsur | LOL | 08:11 |
| dtantsur | yeah, I think iurygregory already saw some hardware that cannot be normally updated | 08:11 |
| sylvr | don't forget McAfee back 2010 | 08:11 |
| TheJulia | 3 hours until next flight departs… | 08:13 |
| sylvr | good luck! | 08:13 |
| TheJulia | I think it will be fine, the big question is how late will I be for the meeting which will have already started and will I look like slightly pan fried death | 08:16 |
| dtantsur | Depending on the meeting, it may be a useful role :) | 08:16 |
| TheJulia | I mean, my filters will likely be gone | 08:17 |
| TheJulia | That could be a good thing | 08:18 |
| opendevreview | Merged openstack/ironic master: idrac driver doc should mention all dependencies https://review.opendev.org/c/openstack/ironic/+/924678 | 08:23 |
| iurygregory | good morning ironic | 10:35 |
| TheJulia | Good morning! | 10:37 |
| TheJulia | Time for zombie Julia’s next flight. | 10:37 |
| iurygregory | safe travels TheJulia o/ | 10:43 |
| *** iurygregory__ is now known as iurygregory | 12:17 | |
| opendevreview | Jacob Anders proposed openstack/sushy-tools master: Translate VM name to UUID to avoid redirect issue https://review.opendev.org/c/openstack/sushy-tools/+/924297 | 12:24 |
| cardoe | @cid I was just kidding with ya. | 13:40 |
| cid | Oh, :D | 13:43 |
| rpittau | good night! o/ | 16:01 |
| masghar | o\ | 16:01 |
| surbhi | TheJulia: Do you know anyone who can help, I am not able to run devstack for more than few hours, After sometime any openstack command gives error for - Internal server error 500. I am trying for almost a week but no luck only issues, If there is anyone who can help pelase let me know. | 16:41 |
| clarkb | surbhi: you probably need to check server logs to find the underlying cause of the 500 errors | 16:43 |
| clarkb | if it is happening after some time though I would be suspicious of things like running out of disk space | 16:44 |
| surbhi | clarkb: I don't think so It has enough space but Let me double check and will update here | 16:46 |
| clarkb | surbhi: it is just one hunch. The best thing to do is to check your server logs to see why the 500 errors occur | 16:49 |
| surbhi | clarkb: Doing that | 16:49 |
| opendevreview | cid proposed openstack/ironic master: Self-Service via Runbooks https://review.opendev.org/c/openstack/ironic/+/922142 | 20:40 |
| opendevreview | cid proposed openstack/python-ironicclient master: Add CLI support for runbooks https://review.opendev.org/c/openstack/python-ironicclient/+/924101 | 20:40 |
| cid | o/ | 20:54 |
| JayF | \o | 20:56 |
| cardoe | janders: sorry for the slow reply. Running 7.00.60.00 on those boxes it seems. So under the settings, it's enabled Attach Mode is AutoAttach. Max sessions is 1. Current sessions is 0. Connection status is not connected. Floppy and Boot Once are disabled. | 22:33 |
| cardoe | https://paste.opendev.org/show/bUZln4wKQ1eINnjpTscw/ that's from the CD endpoint you mentioned | 22:38 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!