| iurygregory | does anyone know if we have a problem in the mirrors providing the cirros images? | 00:08 |
|---|---|---|
| iurygregory | ironic jobs can't find them ... | 00:08 |
| * iurygregory checking if we have something hardcoded... | 00:09 | |
| fungi | do we even mirror those? i thought we just copied the ones devstack wants into the node images and prepopulated its cache | 00:09 |
| iurygregory | yeah not 100% sure | 00:10 |
| iurygregory | https://opendev.org/openstack/ironic/src/branch/master/zuul.d/ironic-jobs.yaml#L73 | 00:10 |
| iurygregory | seems like we always want to use 0.6.1 | 00:11 |
| iurygregory | and now it can't find .-. | 00:11 |
| fungi | https://opendev.org/openstack/project-config/src/branch/master/nodepool/elements/cache-devstack/source-repository-images copies them into /opt/cache/files/ | 00:12 |
| iurygregory | interesting that grenade also pin the version | 00:13 |
| iurygregory | but is working .-. | 00:13 |
| iurygregory | ok, we do have the info, so it should work.. it's probably something else .-. | 00:14 |
| fungi | have an example build that failed? | 00:14 |
| fungi | maybe a prior task is wiping the contents of /opt or you're doing something inside a chroot where /opt isn't mounted in, or... | 00:15 |
| iurygregory | https://zuul.opendev.org/t/openstack/build/3db0d75c34274ba68c86ea0858cb1cd4/log/job-output.txt#33887 | 00:15 |
| iurygregory | basically all ironic jobs with devstack except grenade are showing this | 00:16 |
| fungi | so probably something merged to master and not antelope | 00:17 |
| fungi | guessing your stable/2023.1 jobs are also fine | 00:17 |
| fungi | since cloud setup happens in the n-1 branch | 00:18 |
| fungi | for grenade | 00:18 |
| fungi | so master branch grenade jobs are using stable/2023.1 devstack to set up images and whatnot | 00:19 |
| iurygregory | hummm | 00:19 |
| fungi | iurygregory: https://review.opendev.org/c/openstack/devstack/+/886162 merged roughly 13 hours ago and changed the default cirros version in devstack | 00:20 |
| fungi | could that be related? | 00:20 |
| iurygregory | it could | 00:21 |
| iurygregory | but I do see we have pin to a specific version for our jobs... | 00:21 |
| fungi | maybe your 0.6.1 override isn't how you tell devstack which version to upload | 00:21 |
| iurygregory | hummm right | 00:21 |
| fungi | though the only thing that change updated was the fallback value for CIRROS_VERSION, which is what you're setting | 00:23 |
| fungi | iurygregory: that failure you linked was for a stable/2023.1 job not master | 00:26 |
| fungi | so i guess they're failing the same way | 00:26 |
| fungi | though stable/2023.1 devstack got a similar change updating to cirros 0.5.3 by default | 00:27 |
| fungi | no, i'm wrong, the stable branch change hasn't merged yet | 00:29 |
| iurygregory | fungi, my bad, I had a patch from stable and one from master | 00:30 |
| fungi | also, ironic-tempest-bfv seems to have been passing as recently as 8 hours ago | 00:30 |
| iurygregory | yeah | 00:30 |
| iurygregory | https://zuul.opendev.org/t/openstack/build/faa331a6f85c44b2832ab9c4bf16d258/log/job-output.txt#33486 | 00:31 |
| fungi | anyway, if it's also happening in stable/2023.1 then it's unlikely to be anything that changed in the master branch of devstack | 00:31 |
| iurygregory | yeah | 00:31 |
| iurygregory | for some reason the IRONIC_IMAGE_NAME is evaluated to non-existent-image | 00:33 |
| iurygregory | lol | 00:33 |
| fungi | i saw that but assumed it was intentional | 00:34 |
| * fungi knows comparatively little about these jobs | 00:34 | |
| fungi | iurygregory: the only relevant search hit for that is https://opendev.org/openstack/tempest/src/branch/master/zuul.d/base.yaml#L21 | 00:36 |
| iurygregory | yeah | 00:36 |
| iurygregory | I just saw that | 00:36 |
| iurygregory | but this is 1yr old lol | 00:37 |
| fungi | yeah, but something elsewhere has presumably caused $DEFAULT_IMAGE_NAME to end up getting used | 00:37 |
| iurygregory | yup | 00:38 |
| iurygregory | https://zuul.opendev.org/t/openstack/build/faa331a6f85c44b2832ab9c4bf16d258/log/job-output.txt#2785-2787 | 00:38 |
| iurygregory | but the logic seems correct except for the fact it got the non-existent-image.. | 00:39 |
| fungi | iurygregory: the https://review.opendev.org/c/openstack/tempest/+/831018 change to zuul.d/base.yaml looks suspicious | 00:45 |
| fungi | though that merged a day ago, so maybe not | 00:45 |
| iurygregory | hummm | 00:47 |
| fungi | no, i'm looking back across midnight utc. that timing lines up perfectly | 00:47 |
| fungi | around 6 hours ago | 00:48 |
| iurygregory | yay... | 00:48 |
| iurygregory | yeah | 00:48 |
| iurygregory | that's when I started to see the failures | 00:48 |
| fungi | okay, this might be a thread to tug at | 00:48 |
| iurygregory | I'm wondering if we can push a workaround in place in ironic before pushing the thread | 00:49 |
| fungi | i'm not entirely sure what the expectation is, with the various layers of inheritance between the devstack/tempest job definitions. folks in #openstack-qa might have a better idea | 00:50 |
| iurygregory | yeah, asking in openstack-qa to see | 00:51 |
| fungi | but maybe try removing your image name override and see if things work? | 00:51 |
| pmatulis | can i get some docs.openstack.org frontpage links published for this charms project?: | 01:28 |
| pmatulis | https://review.opendev.org/c/openstack/openstack-manuals/+/886407 | 01:28 |
| fungi | pmatulis: we're not the openstack documentation maintainers. i think #openstack-tc is your best bet, since the tc members took on documentation maintenance when the docs team/tech-writing sig evaporated | 01:30 |
| pmatulis | fungi, okay apologies | 01:30 |
| fungi | no apologies needed | 01:31 |
| fungi | it's not particularly obvious | 01:31 |
| *** amoralej|off is now known as amoralej | 06:13 | |
| opendevreview | Caio Cesar Ferreira proposed openstack/project-config master: Add Power Metrics app to StarlingX https://review.opendev.org/c/openstack/project-config/+/886814 | 11:36 |
| *** amoralej is now known as amoralej|lunch | 11:58 | |
| *** amoralej|lunch is now known as amoralej | 12:36 | |
| opendevreview | Merged openstack/project-config master: wheel builds : move to individual releases https://review.opendev.org/c/openstack/project-config/+/879722 | 14:35 |
| *** amoralej is now known as amoralej|off | 17:15 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!