| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: | 06:59 | |
| - [zuul/zuul] 959319: Fix provider node tag formatting https://review.opendev.org/c/zuul/zuul/+/959319 | ||
| - [zuul/zuul] 960345: Continue request/node processing on exception https://review.opendev.org/c/zuul/zuul/+/960345 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 960340: Set event state of nodeless requests to completed https://review.opendev.org/c/zuul/zuul/+/960340 | 06:59 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 959321: Fix provider selection for image validation https://review.opendev.org/c/zuul/zuul/+/959321 | 09:01 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 960679: Pull base python images from quay.io/opendevorg https://review.opendev.org/c/zuul/zuul/+/960679 | 19:19 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 960682: Build container images on Debian Trixie https://review.opendev.org/c/zuul/zuul/+/960682 | 19:33 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 960682: Build container images on Debian Trixie https://review.opendev.org/c/zuul/zuul/+/960682 | 21:09 | |
| @clarkb:matrix.org | something caused the test job to fail which caused quickstart to fail on first trixie iteration. I've updated quickstart in that change to enable debug logging on the scheduler and executor in hopes the reason is clear | 21:09 |
|---|---|---|
| @clarkb:matrix.org | fungi: any thoughts on why `packaging` isn't available in a python3.11 install: https://zuul.opendev.org/t/zuul/build/7a37b49cf6ae4c95b7eab49c6bf853b5 | 21:22 |
| @clarkb:matrix.org | for some reason I thought it was bundled but maybe we need to install it explicitly? | 21:23 |
| @clarkb:matrix.org | though it doesn't seem to affect all jobs or jobs consistently so I'm confused | 21:23 |
| @fungicide:matrix.org | i don't think packaging is part of stdlib | 21:28 |
| @clarkb:matrix.org | maybe something was pulling it in and now we're not because they updated and dropped the dep | 21:28 |
| @fungicide:matrix.org | if you have a build log from where that was getting installed we can probably sus it out | 21:29 |
| @fungicide:matrix.org | but yes that seems a likely explanation | 21:29 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/zuul/build/f61586de449043d9a026263108d9877a/log/job-output.txt success https://zuul.opendev.org/t/zuul/build/64349508434745aa9301d2c72861639e/log/job-output.txt failure. They both install packaging as part of the nox install | 21:31 |
| @fungicide:matrix.org | in other news, all three afsdb servers are upgraded to noble now and back in the cluster | 21:32 |
| @clarkb:matrix.org | but then the zuul ansible manage command would be in a different venv. nox venv has packaging then it makes a new venv run zuul things in | 21:32 |
| @fungicide:matrix.org | so that just leaves afs02.dfw to upgrade once the rw volume moves are done, and then residual cleanup and removing the afs servers from the emergency list | 21:32 |
| @clarkb:matrix.org | so maybe virtualenv updated? | 21:32 |
| @clarkb:matrix.org | virtualenv hasn't updated for almost a month | 21:33 |
| @fungicide:matrix.org | oops, thought i was in another channel | 21:33 |
| @fungicide:matrix.org | 3s/// | 21:34 |
| @fungicide:matrix.org | that also doesn't work, d'oh! | 21:35 |
| @clarkb:matrix.org | nox also hasn't updated since may so I don't think it changed how it creates virtualenvs | 21:37 |
| @fungicide:matrix.org | Clark: https://zuul.opendev.org/t/zuul/build/64349508434745aa9301d2c72861639e/console#2/0/26/ubuntu-jammy says it came in via `pip install nox` | 21:37 |
| @fungicide:matrix.org | but yeah, possible one of those dependencies was pulling it in, pip used to be more verbose about that, though i think everything changed with the new-style depsolver | 21:38 |
| @clarkb:matrix.org | fungi: yes the failing job does the same thing with packaging via nox. But that is into the nox venv not the venv created by nox to run the tests | 21:39 |
| @fungicide:matrix.org | oh, right | 21:39 |
| @clarkb:matrix.org | but I don't see evidence that anything was installing packaging into the venv created by nox to run the tests in either the successful or failed job | 21:39 |
| @fungicide:matrix.org | i guess nox doesn't have separate logs for its install work like tox does? | 21:40 |
| @fungicide:matrix.org | or do we just not collect it? | 21:40 |
| @clarkb:matrix.org | its in the job output file | 21:40 |
| @clarkb:matrix.org | for example starting here: https://zuul.opendev.org/t/zuul/build/f61586de449043d9a026263108d9877a/log/job-output.txt#886 | 21:41 |
| @fungicide:matrix.org | doesn't seem to log doing that though | 21:43 |
| @fungicide:matrix.org | doesn't log that it installed packaging, i mean | 21:44 |
| @fungicide:matrix.org | new theory, new version of prometheus_client assumes packaging will be installed but doesn't declare a run-time dependency on it | 21:44 |
| @clarkb:matrix.org | oh that could be | 21:45 |
| @clarkb:matrix.org | you win https://pypi.org/project/prometheus-client/#history | 21:45 |
| @clarkb:matrix.org | "Unlisted dependency accidentally included" | 21:45 |
| @fungicide:matrix.org | so packaging was never installed, but now it needs to be | 21:45 |
| @clarkb:matrix.org | ya or just wait for all the caches to catch up with the package removal from pypi | 21:46 |
| @fungicide:matrix.org | even better since they seem to be aware | 21:46 |
| @clarkb:matrix.org | meanwhile the quickstart job fails due to `Ansible output: b"bwrap: Can't find source path /var/lib/zuul/builds/c87df79ffe754bd4a99cb478b989de4e/work: Permission denied` | 21:46 |
| @clarkb:matrix.org | so we're going to have to investigate why bwrap isn't working on trixie | 21:46 |
| @fungicide:matrix.org | namespace something something | 21:47 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/zuul/build/c41843aa20be484ba26a755c9d4fb4e5/log/container_logs/executor.log#410 is the bwrap log error location | 21:48 |
| @clarkb:matrix.org | we're doing a bind mount I wonder if uids have changed or something | 21:52 |
| @clarkb:matrix.org | `/var/lib/zuul` in the container is a volume maybe new bwrap isn't liking bind mounting something that is already bind mounted. Or could be specific to using the volumes rather than a true bind mount in quickstart | 21:54 |
| @jim:acmegating.com | i think there's enough potential kernel interaction to warrant going ahead and holding a node | 21:57 |
| @clarkb:matrix.org | corvus: ok I'll go ahead and do that now | 21:57 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 960417: Add subnode support to launcher https://review.opendev.org/c/zuul/zuul/+/960417 | 21:58 | |
| @clarkb:matrix.org | autohold is in place and I haev rechecked the change. I'm going to pop out for a bike ride nowish so feel free to jump on the held node if there is interest. Otherwise I'll dig into that later today or more likely tomorrow morning | 22:00 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 960417: Add subnode support to launcher https://review.opendev.org/c/zuul/zuul/+/960417 | 22:26 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 23:48 | |
| - [zuul/zuul] 960417: Add subnode support to launcher https://review.opendev.org/c/zuul/zuul/+/960417 | ||
| - [zuul/zuul] 960695: Add reuse label option to launcher https://review.opendev.org/c/zuul/zuul/+/960695 | ||
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!