| cbouchar | TheJulia: TY | 00:00 |
|---|---|---|
| *** tkajinam is now known as Guest5435 | 07:14 | |
| *** Guest5435 is now known as tkajinam | 07:36 | |
| rpittau | good morning ironic! o/ | 08:16 |
| smachi | Hello ironic | 08:58 |
| smachi | Resolved the conflicts for the path https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/845356 and also received +2 in code review | 09:02 |
| smachi | waiting for it to be merged | 09:02 |
| timeu | Did anybody use Packer for building wholedisk images for baremetal deployments ? I have been using disk-image-builder but was wondering if by using Packer I could make it less dependend on the host (build the image in OpenStack VMs instead) | 09:26 |
| ftarasenko | timeu: I found no option to create EFI partitions with packer, have you solved this issue or not affected? | 09:30 |
| timeu | yes that was what I was wondering too. I haven't really started to play around but after some research the EFI partition setup is a bit of a pain in Packer | 09:31 |
| ftarasenko | I use VM in OpenStack with gitlab runner and disk-image-builder installed to create images.) | 09:35 |
| timeu | ftarasenko: Good to know. Maybe there is a way to integrate disk-image-builder into packer. | 09:36 |
| janders | hey rpittau timeu ftarasenko and Ironic o/ | 10:47 |
| iurygregory | good morning Ironic o/ | 11:01 |
| opendevreview | Uemit Seren proposed openstack/sushy master: Revert workaround for weak Etags https://review.opendev.org/c/openstack/sushy/+/850345 | 12:42 |
| iurygregory | ◕_◕ | 12:45 |
| TheJulia | timeu: would that patch above happen to be you? | 12:47 |
| TheJulia | I'm going to guess yes, thanks /whois | 12:50 |
| TheJulia | :) | 12:50 |
| timeu | yeah ;-) | 12:51 |
| timeu | my handle is my first name reverse ;-) | 12:52 |
| TheJulia | Ahh! | 12:52 |
| timeu | so I tested it. Seems like the newest XCC firmware fixes my original patch of stripping the \W tag | 12:52 |
| timeu | so now the etag can be returned as is and it works. | 12:52 |
| TheJulia | I guess I'm wondering what was the behavior that was being observed with the weak etag workaround? I guess someone needs to go hunt down the original why it was done as well | 12:52 |
| timeu | I verified that it was still an issue with the older firmware | 12:53 |
| timeu | I did the patch originally | 12:53 |
| timeu | in the older Lenovo firmware I got an weak etag and if I passed it as is, the Lenovo box complained about it | 12:53 |
| timeu | so I stripped it | 12:53 |
| TheJulia | ahh okay | 12:53 |
| timeu | which fixed the issue. Seems that the recent Lenovo firmware fixed the issue that they accept the etag that they send us in the header | 12:54 |
| TheJulia | nice | 12:54 |
| * TheJulia wonders if we should check the DMTF docs w/r/t etags just to be on the safe side | 12:54 | |
| timeu | According to https://storyboard.openstack.org/#!/story/2009680#comment-192175 both the HTML standard as well as the redfish implementation requires strong Etags | 12:56 |
| timeu | Lenovo for some reason doesn't adhere to it. So this PR should not break any vendors that do | 12:56 |
| timeu | because they will send a strong Etag | 12:56 |
| timeu | and we just return it | 12:56 |
| iurygregory | I think we had a problem with weak etag in another hardware... not only Lenovo | 12:59 |
| iurygregory | going to double check with others to be sure about that | 12:59 |
| TheJulia | iurygregory: your memory is the same as mine | 13:01 |
| TheJulia | I seem to think there were etags issues on a particularly popular whitebox hardware manufacturer | 13:01 |
| iurygregory | should we say yay? | 13:02 |
| timeu | iurygregory: would be great if you could test the change on that hardware | 13:03 |
| timeu | at least sending both etags breaks the virtual media function on the Lenovo one | 13:03 |
| iurygregory | interesting | 13:04 |
| TheJulia | iurygregory: https://www.youtube.com/watch?v=XJYmyYzuTa8 | 13:04 |
| TheJulia | I mean, you can stop after the first 20 seconds :) | 13:04 |
| iurygregory | timeu, thanks for the heads up! this would explain the new BZ janders was working on *I think* (since it was a Lenovo machine) | 13:05 |
| iurygregory | TheJulia, perfect! | 13:05 |
| timeu | iurygregory: do you have a link or this is a non-public BZ ? | 13:06 |
| iurygregory | timeu, let me see if I can find and check if is public or not give me some minutes =) | 13:07 |
| timeu | thanks ! | 13:07 |
| timeu | I think arne_wiebalck should probably also test my patch with the SuperMicro nodes as I am reverting his patch more or less but I think the SuperMicro behave fine ;-) | 13:09 |
| mraineri | With regards to ETags and Redfish, the general guidance I've given clients is to treat the ETag value as an opaque string regardless of it being strong or weak | 13:14 |
| mraineri | Just copy it as-is in the If-Match header, some database of cached resources to monitor when things change, etc | 13:15 |
| iurygregory | timeu, https://bugzilla.redhat.com/show_bug.cgi?id=2103710 | 13:34 |
| opendevreview | Riccardo Pittau proposed openstack/networking-baremetal stable/wallaby: Set agent_type in tests https://review.opendev.org/c/openstack/networking-baremetal/+/850049 | 13:54 |
| iurygregory | Reminder - Revise Ironic Vision meeting in 1hr | 14:00 |
| iurygregory | https://etherpad.opendev.org/p/revise-ironic-vision | 14:00 |
| opendevreview | Merged openstack/ironic-tempest-plugin master: Set minimal tempest version to 27.0.0 https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/845285 | 14:02 |
| timeu | iurygregory: Thanks. that's actually the error message I got until I created the PR. But this requires a relatively recent XCC (Lenovo's BMC) firmware | 14:14 |
| iurygregory | timeu, by any chance you remember the version had when the workaround was required? | 14:24 |
| iurygregory | and which version you have now | 14:24 |
| timeu | so for the SD530 model (Type7X21) the current XCC version that doesn't need the workaround is 5.20 (Build ID: TEI3C8M). The one that still needed the workaround was at 4.80 (Build ID: TEI3C0K) | 14:26 |
| timeu | looking at the changelog between those two versions I couldn't find antying that would suggest that this was changed tough | 14:28 |
| timeu | There are version 5.00 and 4.83 in between | 14:29 |
| opendevreview | Aija Jauntēva proposed openstack/ironic stable/xena: Fix Redfish RAID to update raid_config https://review.opendev.org/c/openstack/ironic/+/850356 | 14:43 |
| iurygregory | JayF, rloo https://etherpad.opendev.org/p/revise-ironic-vision | 15:01 |
| iurygregory | we started the meeting feel free to join | 15:03 |
| rloo | iurygregory: last minute meeting... will try to get there later. | 15:04 |
| iurygregory | rloo, ack =) | 15:04 |
| JayF | iurygregory: I'm in a meeting now, can join ~830 | 15:19 |
| JayF | (11 minutes) | 15:19 |
| iurygregory | JayF, ack | 15:19 |
| iurygregory | JayF, basically now we are sharing where we see ironic in 2026 | 15:29 |
| iurygregory | this is the etherpad https://etherpad.opendev.org/p/revise-ironic-vision | 15:29 |
| * JayF joins meeting | 15:30 | |
| * JayF hears "no need for IPA at all | 15:30 | |
| JayF | lol | 15:30 |
| iurygregory | LOL | 15:59 |
| TheJulia | noooooooooooo! | 15:59 |
| rpittau | cut short | 15:59 |
| iurygregory | WTF GOOGLE | 15:59 |
| rloo | 59 min only... we can count... | 15:59 |
| iurygregory | https://meet.google.com/bvo-rxhi-dka | 15:59 |
| iurygregory | -.-' | 15:59 |
| rpittau | no no, makes sense, it's my punishment for being distracted! | 15:59 |
| iurygregory | I didn't know they would kill the meeting if you schedule for 1hr | 16:00 |
| iurygregory | my mistake | 16:00 |
| iurygregory | -.-' | 16:00 |
| JayF | the message that popped said | 16:00 |
| TheJulia | next meeting, thanks everyone! | 16:01 |
| JayF | 1hr limit on free meetings | 16:01 |
| rpittau | I'll add something to the etherpad | 16:01 |
| iurygregory | JayF, ohhh | 16:01 |
| JayF | so I don't think it was a scheduling problem, it's just free services being limited | 16:01 |
| iurygregory | that's what happen I use my personal account LOL | 16:01 |
| iurygregory | next time I will try to use bluejeans probably... or my redhat email | 16:01 |
| iurygregory | sorry for that everyone, please update things in the etherpad we can schedule another meeting o/ | 16:02 |
| rpittau | iurygregory: or discord :P | 16:05 |
| iurygregory | ++ | 16:06 |
| rpittau | alright, time to go seek refuge in the fridge, see you tomorrow, if I'm not melted o/ | 16:07 |
| JayF | discord for work is something I've managed to avoid so far | 16:07 |
| timeu | iurygregory: I tested my patch with a real deployment and it works for the SD530 with XCC 5.20. Also I noticed that the IPA that comes with OSP16.2 now works with virtual media. Not sure if the size is < 500 or the XCC 5.20 fixed the limitation | 16:29 |
| timeu | well I celebrated too early. I run into a different error now at the end of the deployment when changing the boot device to disk | 16:35 |
| timeu | did anybody see this error message before: ""The value 'Continuous' for the property BootSourceOverrideEnabled is not in the list of acceptable values." | 16:42 |
| iurygregory | hummmm | 16:43 |
| * iurygregory checks if there was any BZ like that | 16:43 | |
| timeu | I will create a story and dig into it via direct sushy calls. Will try to find out what the exceptable values are. I also get a SQLalchemy exception when sushy throws this error btw | 16:44 |
| timeu | I get following ironic error: 2022-07-19 16:31:22.782 7 ERROR ironic.drivers.modules.agent_base_vendor [req-8d98ffa2-d3d8-407f-9dbb-c813212aba44 - - - - -] Failed to change the boot device to disk when deploying node d77e4414-6a9b-41c5-9ecd-32b | 16:46 |
| timeu | 1920c73b2. Error: (builtins.TypeError) Object of type 'BootSource' is not JSON serializable | 16:46 |
| iurygregory | I thought Ironic would set to Once since is virtual media deployment | 16:48 |
| timeu | this is OSP16.2 so Train release not sure if this is also the case with that release | 16:49 |
| iurygregory | not JSON serializable doesn't sound good lol | 16:49 |
| timeu | yeah I guess the error message that the lenovo node returns might be malformed | 16:51 |
| iurygregory | I would try running the command via curl for the BMC to see | 16:51 |
| timeu | yeah will do I think the DB error happens in this call: 2022-07-19 16:31:26.029 7 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python3.6/site-packages/ironic/drivers/modules/deploy_utils.py", line 555, in set_failed_state | 16:53 |
| timeu | 2022-07-19 16:31:26.029 7 ERROR oslo_db.sqlalchemy.exc_filters manager_utils.deploying_error_handler(task, msg, msg, clean_up=False) | 16:53 |
| timeu | not sure if I should open a story also for ironic | 16:54 |
| iurygregory | feel free to open one under ironic =) | 17:19 |
| opendevreview | Merged openstack/networking-baremetal stable/wallaby: Set agent_type in tests https://review.opendev.org/c/openstack/networking-baremetal/+/850049 | 17:42 |
| JayF | What is OSP? | 17:52 |
| TheJulia | OSP == short hand for RH OSP or Open Stack Platform | 18:12 |
| TheJulia | timeu: have definitely seen the continious override thing before, that originated in a miscommuniation along the way in one of the ?2019? dmtf redfish revisions | 18:14 |
| TheJulia | ahh, so we're trying to handle the error and the response... is so unexpected we're actually raising a different exception. That is awesome | 18:15 |
| timeu | According to this documentation: https://sysmgt.lenovofiles.com/help/index.jsp?topic=%2Fcom.lenovo.systems.management.xcc.restapi.doc%2Fupdate_next_onetime_bootconfig_patch.html and also querying the XCC itself Continous is not allowed: 'BootSourceOverrideEnabled@Redfish.AllowableValues': ['Once', 'Disabled']}, | 18:27 |
| timeu | so I think it's rather an issue of ironic which should probably check if Continuous is supported | 18:27 |
| TheJulia | ugh, that change rearing it's head again three years later | 18:28 |
| timeu | well we are eageraly waiting for the OSP 17.x release so we can move off Train to Wallaby at least | 18:29 |
| TheJulia | well, I don't think that specifically is going to be in there | 18:29 |
| timeu | I guess 90% of the issues are fixed anyways in OSP 17.x | 18:29 |
| timeu | the fix ? | 18:29 |
| TheJulia | well, there really is not a fix for the lenovo issue your hitting | 18:30 |
| TheJulia | except maybe they fix their firmware | 18:30 |
| TheJulia | give me a few and I can pivot and hunt down the docs | 18:30 |
| timeu | sure interestingly it worked with the older firmware. I will check again if the Continous string is part of the allowed values in the older firmware | 18:30 |
| TheJulia | likely is | 18:31 |
| TheJulia | there was this version where there was a suggestion on eway, and a couple vendors went to removing the value ability | 18:31 |
| TheJulia | and yeah :( | 18:31 |
| timeu | aha it's also not in the old firmware: 'BootSourceOverrideEnabled@Redfish.AllowableValues': ['Once', 'Disabled'] so it means that they started to check it | 18:34 |
| TheJulia | heh | 18:34 |
| timeu | in the newer firmware | 18:34 |
| TheJulia | i bet it still worked though | 18:34 |
| TheJulia | I guess we're going to need a story. Maybe there is a path there... | 18:35 |
| timeu | sushy or ironic story ? | 18:37 |
| TheJulia | sushy most likely | 18:37 |
| timeu | sushy story I created https://storyboard.openstack.org/#!/story/2010161 | 18:37 |
| TheJulia | k | 18:37 |
| iurygregory | added ironic-triaged | 18:38 |
| timeu | one approach would be to check if the BootSourceOverrideEnabled value passed to sushy is in the AllowedValues and if not fallback to Once | 18:38 |
| iurygregory | sounds reasonable I would say | 18:39 |
| timeu | I can come up with a PR draft if you like | 18:39 |
| iurygregory | happy to review it o/ | 18:39 |
| timeu | ok will have a stab at it tomorrow | 18:39 |
| TheJulia | that woudl be much appreciated | 18:40 |
| TheJulia | I've got the session cache issue thing I'm kind of trying to figure out what is going on there :( | 18:41 |
| JayF | TheJulia: can I help? | 18:55 |
| JayF | TheJulia: If you need someone to quack at I can :) | 18:55 |
| TheJulia | JayF: based upon the email thread, I think https://review.opendev.org/c/openstack/ironic/+/850259 might do the trick... Got logs like a half hour ago and haven't looked yet | 19:07 |
| JayF | ack | 19:10 |
| TheJulia | depending on what I see in the logs, will govern the storyboard entry | 19:11 |
| TheJulia | ewww | 19:27 |
| TheJulia | sushy thinks it was able to refresh the session | 19:27 |
| JayF | TheJulia: want me to fix rpittau's issues with https://review.opendev.org/c/openstack/ironic-python-agent/+/839084 so it can land? | 20:04 |
| TheJulia | JayF: by all means, wrapping my head around this sushy issue :\ | 20:05 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent master: Guard shared device/cluster filesystems https://review.opendev.org/c/openstack/ironic-python-agent/+/839084 | 20:24 |
| JayF | TheJulia: rpittau: ^ fyi, minimal changes but I did reword a couple of sentences instead of directively applying the review feedback | 20:25 |
| TheJulia | thanks | 20:31 |
| TheJulia | so looking at a this sushy issue, it *looks* like reauth just never occurs properly | 20:32 |
| TheJulia | but that should fail | 20:32 |
| TheJulia | *or* it just works. :\ | 20:32 |
| opendevreview | Julia Kreger proposed openstack/sushy master: WIP: Add more debugging to session auth process https://review.opendev.org/c/openstack/sushy/+/850425 | 20:54 |
| opendevreview | Uemit Seren proposed openstack/sushy master: Revert workaround for weak Etags https://review.opendev.org/c/openstack/sushy/+/850345 | 21:22 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!