| opendevreview | cid proposed openstack/ironic master: wip: Allow to provision arm64 than x86_64 architecture fake-bare-metal-vms https://review.opendev.org/c/openstack/ironic/+/915441 | 00:56 |
|---|---|---|
| opendevreview | cid proposed openstack/ironic master: wip: Allow to provision arm64 than x86_64 architecture fake-bare-metal-vms https://review.opendev.org/c/openstack/ironic/+/915441 | 01:00 |
| rpittau | good morning ironic! o/ | 07:01 |
| adam-metal3 | Hello Ironic | 07:02 |
| dtantsur | o/ | 07:17 |
| opendevreview | Jacob Anders proposed openstack/ironic master: Add states.SERVICING and SERVICEWAIT to _FASTTRACK_HEARTBEAT_ALLOWED https://review.opendev.org/c/openstack/ironic/+/915389 | 07:45 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic stable/2024.1: Fix servicing clean-up https://review.opendev.org/c/openstack/ironic/+/915944 | 07:48 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic stable/2024.1: Fix the confusion around service_reboot/servicing_reboot https://review.opendev.org/c/openstack/ironic/+/915945 | 07:48 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic stable/2023.2: Fix servicing clean-up https://review.opendev.org/c/openstack/ironic/+/915946 | 07:49 |
| Continuity | Morning Ironic o/ | 08:02 |
| rpittau | TheJulia: when you have a moment can you please have another look at https://review.opendev.org/c/openstack/sushy-tools/+/875366 ? thanks! | 08:49 |
| Sandzwerg[m] | Morning | 08:51 |
| opendevreview | Jacob Anders proposed openstack/sushy-oem-idrac master: [WIP] Wait for BIOS configuration job to complete https://review.opendev.org/c/openstack/sushy-oem-idrac/+/915092 | 10:13 |
| opendevreview | Merged openstack/ironic stable/2023.2: Fix servicing clean-up https://review.opendev.org/c/openstack/ironic/+/915946 | 11:31 |
| opendevreview | Merged openstack/ironic stable/2024.1: Fix servicing clean-up https://review.opendev.org/c/openstack/ironic/+/915944 | 11:32 |
| opendevreview | Tudor Domnescu proposed openstack/ironic-python-agent master: destroy_disk_metadata: support 4096 sector size https://review.opendev.org/c/openstack/ironic-python-agent/+/915983 | 12:02 |
| adam-metal3 | hey folks I have to recently pushd changes and both of them has issues with the metalsmith CI job metalsmith-integration-ipa-src-uefi, the changes are: https://review.opendev.org/c/openstack/ironic-python-agent/+/915825 and https://review.opendev.org/c/openstack/ironic-python-agent/+/915858 I am not a metalsmith user so I am not sure how relevant these failures are for my test | 12:05 |
| adam-metal3 | could someone please help me with these metalsmith CI things? | 12:06 |
| opendevreview | Tudor Domnescu proposed openstack/ironic-python-agent master: destroy_disk_metadata: support 4096 sector size https://review.opendev.org/c/openstack/ironic-python-agent/+/915983 | 12:11 |
| iurygregory | good morning Ironic | 12:11 |
| rpittau | adam-metal3: I think metalsmith is really not very happy at the moment, I don't think the failures depend on your patch | 12:43 |
| rpittau | we have a very low passing rate | 12:44 |
| rpittau | yep this doesn't look right https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_440/915858/1/check/metalsmith-integration-ipa-src-uefi/4408a50/controller/ironic-bm-logs/node-0_console.log | 12:48 |
| adam-metal3 | rpittau: thanks for the info ! | 12:52 |
| adam-metal3 | rpittau: is there anything I should do to help should I just wait until the CI get's fixed? | 12:52 |
| adam-metal3 | *or | 12:53 |
| adam-metal3 | gets* fixed | 12:54 |
| rpittau | adam-metal3: need to see what's wrong with metalsmith deployment, all the rest looks good | 12:54 |
| rpittau | probably need help myself :) | 12:54 |
| adam-metal3 | rpittau: sure, then good luck | 12:54 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic stable/2023.2: Fix the confusion around service_reboot/servicing_reboot https://review.opendev.org/c/openstack/ironic/+/915993 | 14:20 |
| dtantsur | TheJulia: interesting case in https://ff7549a1676b7e0abf4f-a8638dce9de2f2c46b7008346a40a9c5.ssl.cf5.rackcdn.com/915358/1/check/ironic-tempest-uefi-redfish-vmedia/40c5c44/controller/logs/ironic-bm-logs/node-0_console_2024-04-16-12:37:06_log.txt. It is testing a partition image with UEFI and somehow not finding any EFI files? | 14:27 |
| opendevreview | Merged openstack/ironic stable/2023.2: Fix get_async_step_return_state to account for servicing https://review.opendev.org/c/openstack/ironic/+/915527 | 15:47 |
| rpittau | dtantsur, TheJulia, what do you make of https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_440/915858/1/check/metalsmith-integration-ipa-src-uefi/4408a50/controller/ironic-bm-logs/node-0_console.log ? | 15:50 |
| dtantsur | absolutely cursed | 16:03 |
| rpittau | yeah, and I really don't have enough brain power left to dig more into that now...... | 16:11 |
| rpittau | see ya tomorrow! o/ | 16:11 |
| TheJulia | Freaky | 18:07 |
| TheJulia | dtantsur: Curious, we've seen some failures like that quite a bit recently with no real understanding of why :( | 19:18 |
| opendevreview | Julia Kreger proposed openstack/ironic stable/2024.1: Remove deprecation warning by setting schema https://review.opendev.org/c/openstack/ironic/+/915953 | 19:21 |
| opendevreview | Julia Kreger proposed openstack/ironic stable/2023.2: Remove deprecation warning by setting schema https://review.opendev.org/c/openstack/ironic/+/915954 | 19:36 |
| opendevreview | Julia Kreger proposed openstack/ironic stable/2023.1: Remove deprecation warning by setting schema https://review.opendev.org/c/openstack/ironic/+/915955 | 19:36 |
| JayF | Have we considered trying to build a partition image artifact and using that during our testing. It seems like we're detecting failures of image builds more than almost anything else these days. | 19:48 |
| TheJulia | We've been trying to use cirros and massage things to just work | 19:59 |
| stevebaker[m] | The base centos9-stream image may have just switched over to being a UEFI capable image, I'm downloading now to verify that. I wonder if that is related to this regression | 20:01 |
| stevebaker[m] | Also I'd never recommend using the grub2 diskimage-builder element, always bootloader :) | 20:12 |
| TheJulia | I have to look at what that job uses at this point | 20:15 |
| opendevreview | Steve Baker proposed openstack/metalsmith master: Use bootloader element instead of grub2 https://review.opendev.org/c/openstack/metalsmith/+/916018 | 20:18 |
| stevebaker[m] | just to see what it does | 20:19 |
| TheJulia | stevebaker[m]: your all on top of it! you must be nice and awake | 20:20 |
| TheJulia | :) | 20:20 |
| stevebaker[m] | I need a coffee, brb | 20:21 |
| *** dmellado03 is now known as dmellado0 | 20:22 | |
| iurygregory | coffee ++ | 20:25 |
| TheJulia | I, too, need coffee | 20:28 |
| TheJulia | It has been a long day, alreaady | 20:28 |
| TheJulia | Well legacy failed | 21:22 |
| TheJulia | Stderr: 'munmap_chunk(): invalid pointer\n' | 21:23 |
| TheJulia | ... that seems like the tinycore issue | 21:23 |
| TheJulia | stevebaker[m]: https://9d1795cef9362d286550-1c984d0af10b187b603f4cc0f4cd986e.ssl.cf2.rackcdn.com/916018/1/check/metalsmith-integration-glance-centos9-uefi/84de062/controller/ironic-bm-logs/node-0_console.log | 21:44 |
| TheJulia | I wonder, if centos just changed the source image, maybe we don't need to do some of the image prep stuff in that pipeline? | 21:45 |
| stevebaker[m] | Yes, CentOS-Stream-GenericCloud-x86_64-9-latest.x86_64.qcow2 is now an EFI capable image, it has just landed. Maybe we need to fix the bootloader element to handle that | 21:46 |
| stevebaker[m] | TheJulia: Is that boot failure the whole-disk image or the partition one? | 21:47 |
| TheJulia | I guess we should give it a spin locally just to verify what exactly it could be | 21:48 |
| TheJulia | stevebaker[m]: good question, one moment | 21:48 |
| TheJulia | rutro, partition image | 21:49 |
| TheJulia | ... i wonder if it is grubby config | 21:51 |
| TheJulia | so yeah, the whole disk image works | 21:52 |
| stevebaker[m] | Maybe we should disable just the partition image in the test now, and re-enable with the fix later | 21:52 |
| TheJulia | building an image now | 21:55 |
| TheJulia | found it | 22:01 |
| TheJulia | https://www.irccloud.com/pastebin/Fd7yIekw/ | 22:02 |
| TheJulia | so it is a hold over from the source image which dib likely needs to change, but for partition images to work right, ironic likely needs to fix it when laying it out on disk | 22:06 |
| TheJulia | Given we've not had this exact problem before, maybe fixing it is not the right path? | 22:07 |
| TheJulia | to be specific, I'm thinking it is time for us to just say "partition images *must* be in a state with appropriate configuration, that it is not ironic's job to endlessly massage partition images to be bootable. Regardless of that, a fix lands in the agent to go "okay, I see this is going to be a problem, and pre-emptively fixes it | 22:13 |
| TheJulia | dunno | 22:13 |
| TheJulia | stevebaker[m] rpittau JayF dtantsur ^^^ | 22:14 |
| stevebaker[m] | oh yeah, in centos, /boot/efi/EFI/centos/grub.cfg is a redirect to /boot/grub2/grub.cfg, whereas we ensure both grub.cfg files are identical. I did want to have a discussion about going with the centos approach because then we don't have to worry about regenerating /boot/efi/EFI/centos/grub.cfg ever | 22:17 |
| TheJulia | I guess my worry is we would have to sort of teach ironic to know the difference and sort through it | 22:18 |
| stevebaker[m] | Also, the test actually does UEFI boot with a partition image? Should we switch it to legacy bios instead | 22:19 |
| TheJulia | UEFI boot with a partition image | 22:19 |
| TheJulia | intentionally, the job does both wholedisk and partition | 22:19 |
| stevebaker[m] | but can we do wholedisk+uefi and partition+legacy | 22:20 |
| TheJulia | well, we were talking about getting rid of the legacy job altogether | 22:21 |
| stevebaker[m] | OK, another angle. the bootloader element should ensure /boot/efi/EFI/centos/grub.cfg is regenerated and identical to /boot/grub2/grub.cfg | 22:24 |
| TheJulia | yes, sure | 22:24 |
| TheJulia | *but* it is looking for the /boot partition's UUID | 22:24 |
| TheJulia | which means, ironic still has to reconcile it | 22:24 |
| TheJulia | well, | 22:27 |
| stevebaker[m] | That is the /boot/efi/EFI/centos/grub.cfg from the base image, not a regenerated onw | 22:27 |
| TheJulia | we copy if memory serves | 22:27 |
| TheJulia | yeah, indeed | 22:27 |
| TheJulia | because we find efi artifacts | 22:27 |
| TheJulia | we just copy them | 22:28 |
| stevebaker[m] | TheJulia: can you try building a partition image with the bootloader element instead of grub2? | 22:29 |
| diablo_rojo | Hello Excellent Ironic people :) I have an opportunity for some students 3-5 to work on a project in Ironic over the upcoming (Northern Hemisphere) summer semester (they shouldn't be taking other classes to take up time so they should be able to focus on Ironic). Anyone have ideas or would be willing/interested to mentor? | 22:29 |
| TheJulia | stevebaker[m]: sure, looking up one bit of syntax I'm drawing a blank on | 22:30 |
| TheJulia | running, making a raw image out of the gate | 22:30 |
| * TheJulia gets out the hypnosis pinwheel and convinces everyone to mentor | 22:31 | |
| stevebaker[m] | hrm, will this block even be entered for partition image? https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/elements/bootloader/finalise.d/50-bootloader#L208 | 22:31 |
| TheJulia | stevebaker[m]: same | 22:31 |
| TheJulia | doesn't look like it | 22:33 |
| stevebaker[m] | diablo_rojo: what timezone? I'm UTC+12, so europe would be difficuly | 22:33 |
| stevebaker[m] | difficult | 22:33 |
| iurygregory | diablo_rojo, depending on the timezone, I'm interested =) (UTC-3 here) | 22:36 |
| diablo_rojo | stevebaker: iurygregory: the students will be in Qatar - Doha specifically. | 22:36 |
| diablo_rojo | So thats UTC +3 I think? | 22:38 |
| iurygregory | 6hrs diff for me, I can help during the time I'm online (but wouldn't have a lot of overlap) | 22:38 |
| iurygregory | yeah | 22:38 |
| iurygregory | rpittau, dtantsur , masghar fyi in case you are interested in mentoring students ^ | 22:39 |
| diablo_rojo | iurygregory: the instructor was saying there will be a focus on peer mentoring and they will be 3-4 years into their degrees so they shouldn't need tons of support after they get rolling? | 22:39 |
| stevebaker[m] | TheJulia: This may help https://review.opendev.org/c/openstack/diskimage-builder/+/916021 | 22:39 |
| diablo_rojo | And a lot of them will have had 1-2 internships before. | 22:39 |
| iurygregory | diablo_rojo, got it, a few things can be async and I think our community will help if they have questions and the mentor is not around | 22:40 |
| opendevreview | Steve Baker proposed openstack/metalsmith master: Use bootloader element instead of grub2 https://review.opendev.org/c/openstack/metalsmith/+/916018 | 22:42 |
| TheJulia | stevebaker[m]: no difference it looks like | 22:43 |
| diablo_rojo | iurygregory: exactly. | 22:47 |
| opendevreview | Adam McArthur proposed openstack/ironic master: Small Doc Fix: Moved user export for devstack guide. https://review.opendev.org/c/openstack/ironic/+/916022 | 22:48 |
| diablo_rojo | I you and others want to share the mentor title that is totally fine - leveraging the community in the end and getting the students to be a part of it is the goal anyway. | 22:48 |
| iurygregory | ++ yeah | 22:48 |
| diablo_rojo | I think they are normally scared to jump in from the start, but are willing to do it with a little push :D | 22:48 |
| iurygregory | :D perfect! | 22:48 |
| diablo_rojo | iurygregory: I can give you a day or two to find a mentor buddy and circle back :) Alternatively, I can start a thread with you and the instructor now and you can add whomever later on. | 22:49 |
| diablo_rojo | Whatever is your preference I am happy to facilitate :) | 22:49 |
| TheJulia | gah, it woudl help if I had the right element on my command line | 22:50 |
| iurygregory | diablo_rojo, let's wait till tomorrow to see if someone would like to join the mentoring =) | 22:50 |
| TheJulia | stevebaker[m]: okay, that actually looks good and makes sense | 22:51 |
| diablo_rojo | iurygregory: sounds good :) I will make note to myself to circle back | 22:51 |
| iurygregory | diablo_rojo, awesome! Thank you! | 22:52 |
| JayF | diablo_rojo: always willing to include other folks in the various open pairing/review sessions we've had, and to continue, as much as is possible with TZ overlap | 23:01 |
| JayF | diablo_rojo: so +1 to "community mentoring" even if someone puts their name down | 23:01 |
| TheJulia | :) | 23:10 |
| diablo_rojo | JayF: exactly - thats what I want to happen - the community works together to do it, but professors want a name or two usually lol | 23:48 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!