| -@gerrit:opendev.org- Roja Eswaran proposed: [openstack/diskimage-builder] 990920: Bug in one of the gated tests: centos_10-stream https://review.opendev.org/c/openstack/diskimage-builder/+/990920 | 13:35 | |
| -@gerrit:opendev.org- Roja Eswaran proposed: [openstack/diskimage-builder] 987717: Replace debootstrap with mmdebstrap https://review.opendev.org/c/openstack/diskimage-builder/+/987717 | 13:50 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Douglas Lopes Pereira: [openstack/project-config] 990730: Add PTP Metrics Exporter app to StarlingX https://review.opendev.org/c/openstack/project-config/+/990730 | 14:39 | |
| -@gerrit:opendev.org- Roja Eswaran proposed: [openstack/diskimage-builder] 990920: Bug in one of the gated tests: centos_10-stream https://review.opendev.org/c/openstack/diskimage-builder/+/990920 | 14:41 | |
| -@gerrit:opendev.org- Roja Eswaran proposed: [openstack/diskimage-builder] 987717: Replace debootstrap with mmdebstrap https://review.opendev.org/c/openstack/diskimage-builder/+/987717 | 14:43 | |
| @fungicide:matrix.org | i need to go on an errand and will grab lunch while i'm out, so expect to be a couple of hours, but can help/run the etherpad upgrade as soon as i'm back (likely around 16:30 utc) | 14:43 |
|---|---|---|
| @fungicide:matrix.org | also looks like infra-prod-manage-projects just failed on another project addition, https://review.opendev.org/990730 | 14:47 |
| @clarkb:matrix.org | sounds good. I'll work on weeky local upgrades while you eat lunch | 14:47 |
| @fungicide:matrix.org | i can look at that when i get back if nobody beats me to it | 14:47 |
| @fungicide:matrix.org | https://zuul.opendev.org/t/openstack/buildset/ce74f837c3d94f648cc4e64827dd7b97 | 14:47 |
| @clarkb:matrix.org | I'll see if I can find a log to that shortly | 14:48 |
| @clarkb:matrix.org | want to do local system updates first so that I don't have to load ssh keys twice | 14:52 |
| @clarkb:matrix.org | fungi: https://paste.opendev.org/show/b6mgeUsCfQ4AdLQQPHBi/ it couldn't pull the docker image due to an image version incompatibility? that command is run by system-config/playbooks/roles/gerrit/tasks/manage-projects.yaml | 15:09 |
| @clarkb:matrix.org | I have a couple held nodes for gerrit upgrade testing that may be useful here. Can job on and cross check docker/podman package versions and try to pull | 15:09 |
| @clarkb:matrix.org | though gemini in a google search says this is a misleading error message indicating v2 failed so it fell back to v1 which also failed which likely indicates a network or service level problem | 15:10 |
| @clarkb:matrix.org | https://status.redhat.com/ ok I think that confirms it | 15:11 |
| @clarkb:matrix.org | it should be safe to us to reenqueue the deployment or let dailies run it later today if/when quay is happy again | 15:11 |
| @clarkb:matrix.org | we might also consider dropping the pull in that task list? We theoretically have a running gerrit anytime we're running manage projects which means we have a gerrit image we probably don't need to pull again? | 15:12 |
| @clarkb:matrix.org | I suspect that is there due to wanting to pull in updates to jeepyb quickly but we can do that manually maybe? | 15:12 |
| @clarkb:matrix.org | since jeepyb update rate is low | 15:12 |
| @mordred:waterwanders.com | what a lovely misleading error | 15:21 |
| @clarkb:matrix.org | its like pip saying foo===5.0.0 cannot be installed because it doesn't match foo>=3.0.0. The issue is the python version constraint removes 5.0.0 as a valid version not the package version range | 15:27 |
| @mnasiadka:matrix.org | Yeah, quay has been unstable for hours | 16:36 |
| @fungicide:matrix.org | i guess the "yay" is because of only "hours" | 16:37 |
| @fungicide:matrix.org | oh, you said "yeah" not "yay" | 16:39 |
| @clarkb:matrix.org | I think etherpad is hosted on docker still. So it may not matter that quay is down for etherpad. But I'm not 100% certain of that, we do pull mariadb from quay | 16:50 |
| @clarkb:matrix.org | so maybe things will still have a sad | 16:50 |
| @fungicide:matrix.org | also we talked about mirroring etherpad to quay at some point in the past, i guess we haven't done that yet | 16:57 |
| @clarkb:matrix.org | fungi: correct because the server is still jammy and running docker not podman on the backend for the runtime | 17:01 |
| @fungicide:matrix.org | oh right | 17:05 |
| @clarkb:matrix.org | Quays status says pushes are disabled but pulls are available so in theory the etherpad upgrade would be fine since we don't push that image to quay but do pull mariadb | 17:37 |
| @fungicide:matrix.org | makes sense, should i go ahead and approve it then? | 17:43 |
| @clarkb:matrix.org | https://review.opendev.org/c/opendev/system-config/+/989022 for some reason I thought this change and its child had merged after the fix for the adns deployment. I guess we just keep rechecking it? The LE (most recent) failure is likely due to the LE test env being overwhelmed | 18:00 |
| @clarkb:matrix.org | happy for others to weigh in on alternative approaches too | 18:00 |
| @clarkb:matrix.org | also I'm going to put together a meeting agenda in a bit. Feel free to mention any updates here and I'll get them in. I'm already going to edit the etherpad upgrade depending on how that goes if we do it etc, the new backup server will get noted, and then reminders for Gerrit upgrade happening Friday | 18:01 |
| @clarkb:matrix.org | and https://review.opendev.org/c/opendev/system-config/+/988993 is another one that we should we consider if we want to apply this config change to executors long term | 18:11 |
| @fungicide:matrix.org | i've reenqueued the failed project-config deploy from earlier | 18:16 |
| @clarkb:matrix.org | fungi: looks like that succeeded: https://zuul.opendev.org/t/openstack/build/02e01089fee6480d95c4af1ea991d271 ? | 19:43 |
| @clarkb:matrix.org | also is there anything else we can think of that should be checked/tested/covered in prep for the Gerrit upgrade Friday? I followed our usual process for runnign through things and I think it is looking good, but please let me know if something appears missing as the sooner that gets done the better | 19:44 |
| @fungicide:matrix.org | the plan in the pad looked good to me | 19:48 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [opendev/project-config] 987943: Update pipeline config based on agent behavior https://review.opendev.org/c/opendev/project-config/+/987943 | 20:33 | |
| @mordred:waterwanders.com | corvus: ^^ there we go, that has both things | 20:33 |
| @mordred:waterwanders.com | fungi, Clark over the weekend I pushed up a stack of drizzle changes that never got any results back from zuul. corvus looked in to it and it turns out it's because the CR+1 were getting dropped in out of order, which caused there to be a non-eligible parent change so nothing got enqueued. yay for learnings with vouched! :) | 20:34 |
| @fungicide:matrix.org | out of order? | 20:36 |
| @fungicide:matrix.org | oh, across the series | 20:36 |
| @fungicide:matrix.org | as in child before parent? | 20:36 |
| @mordred:waterwanders.com | yeah | 20:36 |
| @jim:acmegating.com | mordred: ++ i think the main thing to note is that this is a difference from "vouched" vs what we generally think of as a "trusted-check" pipeline. we would want to keep that requirement in for trusted-check, but is okay to remove for vouched since the only thing we're concerned about is efficiency, not security. | 20:36 |
| @fungicide:matrix.org | i wonder if it would make sense for "vouched" to ignore the vote for its ancestors | 20:37 |
| @mordred:waterwanders.com | ++ in this case, the agent review harness is event driven, and each change/review is independent. so there's no expectation or even context that there are other changes. but - one could imagine a human reviewing out of sequence as well, I know I do it backwards sometimes :) | 20:38 |
| @mordred:waterwanders.com | fungi: yah - that's basically what 987943 is doing | 20:38 |
| @clarkb:matrix.org | is that caveat worth a comment in the pipeline config? | 20:38 |
| @fungicide:matrix.org | for sure, sometimes i review backward up the stack and then vote forward down the stack once i've got complete context | 20:38 |
| @fungicide:matrix.org | depending on the nature of the series, it can help to look at the intended end state first to see what it's trying to accomplish, then up through the steps needed to get there | 20:39 |
| @jim:acmegating.com | Clark: couldn't hurt. maybe to avoid future-us being confused, or someone copying that assuming it's the same thing. | 20:39 |
| @jim:acmegating.com | * Clark: couldn't hurt. maybe to avoid future-us being confused, or someone copying that assuming it's the same thing (as trusted-check) | 20:40 |
| @fungicide:matrix.org | to be fair, future me will always be more confused than present me | 20:40 |
| @fungicide:matrix.org | i think that's a factor of meat-based aging | 20:40 |
| @clarkb:matrix.org | corvus: ya I'm worried about potential cargo culting | 20:40 |
| @mordred:waterwanders.com | basically a comment saying "we're omitting approval: ['Code-Review: 1'] on purpose because of out-of-order reviews coming in. This is safe here but should not be done in trusted-check circumstances" ? | 20:42 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [opendev/project-config] 987943: Update pipeline config based on agent behavior https://review.opendev.org/c/opendev/project-config/+/987943 | 20:45 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | 20:48 | |
| @fungicide:matrix.org | progress! | 20:48 |
| @clarkb:matrix.org | finally | 20:51 |
| @mordred:waterwanders.com | woot! | 20:51 |
| -@gerrit:opendev.org- Zuul merged on behalf of Monty Taylor https://matrix.to/#/@mordred:inaugust.com: [opendev/project-config] 987943: Update pipeline config based on agent behavior https://review.opendev.org/c/opendev/project-config/+/987943 | 20:52 | |
| @fungicide:matrix.org | so can we move forward with https://review.opendev.org/c/opendev/system-config/+/988698 now that it's unblocked? | 20:53 |
| @clarkb:matrix.org | fungi: did we want to approve https://review.opendev.org/c/opendev/system-config/+/988698 with just my review? | 20:53 |
| @clarkb:matrix.org | heh ya same question. I think we can proceed with that now if you're comfortabel with it. It should be easy to revert if necessary | 20:53 |
| @fungicide:matrix.org | i feel like these questions may be interrelated | 20:53 |
| @fungicide:matrix.org | sure, i can self-approve and keep an eye on it | 20:53 |
| @clarkb:matrix.org | sounds good. I have to do a school run in about 10 minutes then I'll work on getting tomorrows meeting agenda sent | 20:55 |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 21:01 | |
| @clarkb:matrix.org | with that merging I'm going to drop ansible 9 from the meeting agenda as all of the followups are done | 21:48 |
| @clarkb:matrix.org | ok my meeting agenda edits are in. Anything else to add? | 21:59 |
| @fungicide:matrix.org | thanks! nothing for me, no | 22:35 |
| @clarkb:matrix.org | heh good because I send the agenda just a fwe minutes before your comment | 22:54 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!