| @harbott.osism.tech:regio.chat | infra-root: seems we have a regression caused by the py3.14 bump blocking pypi releases (see the discussion in #openstack-releases), sample failure is https://zuul.opendev.org/t/openstack/build/7b4d6684e5eb47688681649edbd3f764 | 10:51 |
|---|---|---|
| @harbott.osism.tech:regio.chat | not sure what to do about it, one of the underlying issues seems to be that we shouldn't be using `ensure-*` in post if we did `remove-sudo` earlier. but I'm also not sure how or whether installing stuff on the executor should be working at all and how to debug it | 10:53 |
| @harbott.osism.tech:regio.chat | (just as a short recap of my findings: `python3 -m wheel --help` which is used as a check for whether pip is available in ensure-pip role is now failing, triggering an attempt to install python3-pip, which fails because sudo is no longer allowed) | 11:07 |
| @abelur:matrix.org | Jens Harbott: Looked at this - I think it's the check, not the image. In the failing build pip3 is found fine and only `python3 -m wheel --help` fails, which is what sends us down the apt path into remove-sudo. wheel is absent from python-base 3.12-trixie as well as 3.14-trixie (and from upstream python:3.12/3.13-slim-trixie), so it was never coming from the base image - it came from zuul's own deps and 3.14 no longer pulls it in... | 11:44 |
| @abelur:matrix.org | Given pip hasn't needed a global wheel since PEP 517, and this playbook only needs pip to install twine, dropping that check in ensure-pip seems like the fix - but I don't know the history of it, so would want a second opinion before trying to fix this. | 11:45 |
| @harbott.osism.tech:regio.chat | Anil Belur: yes, the code has been in the role for so long that it requires some thinking before fixing it | 13:18 |
| @harbott.osism.tech:regio.chat | corvus: there were some issues reported in #openstack-infra that look to be related to the log-streaming changes. I'm not sure if is bad enough to trigger the big revert right away, but maybe you can take a closer look. sample https://zuul.openstack.org/build/c83f42b86fdb4f5ea3811ffab058472b | 13:19 |
| @jim:acmegating.com | Jens Harbott: ack, I'll look into it. let's hold off on the revert. | 13:43 |
| @jim:acmegating.com | regarding wheel: i'm inclined to agree with Anil right now. I think we'd be happy to add wheel to the executor environment if we need it, but if it's not really needed and it's just an overactive check, maybe fixing the check is the best thing? | 13:44 |
| @jim:acmegating.com | basically in zuul we try to make useful tools available on the executor for the typical types of jobs that we expect to run there | 13:44 |
| @fungicide:matrix.org | i guess the benefit of doing the ensure-* roles in post for things a subsequent task will rely on is that even if sudo is revoked we get a clearer error, it's not just about installing a tool but ensuring the tool is present, so if it isn't then we discover that faster | 13:59 |
| @fungicide:matrix.org | but if we're not going to even use a tool then ensuring its presence is indeed unnecessary | 14:00 |
| @jim:acmegating.com | i'm going to follow up in #zuul:opendev.org regarding the "worker in a dead state" issue that Jens Harbott mentioned | 14:01 |
| @mordred:waterwanders.com | I agree. I'm pretty sure wheel is being installed for hysterical raisin at this point | 14:38 |
| @fungicide:matrix.org | at least for openstack's python release jobs, we invoke pyproject-build now and let it be a black box for creating sdists and wheels | 14:41 |
| @fungicide:matrix.org | so if it does rely on the wheel module, that's an implementation detail it's going to take care of | 14:41 |
| @jim:acmegating.com | we would expect that to happen on the build node, right? only the upload part should happen on the executor, does that uses twine directly? | 14:42 |
| @jim:acmegating.com | * we would expect that to happen on the build node, right? only the upload part should happen on the executor, does that use twine directly? | 14:42 |
| @fungicide:matrix.org | correct | 14:50 |
| @fungicide:matrix.org | pyproject-build and twine are being run in separate places anyway | 14:51 |
| -@gerrit:opendev.org- Roja Eswaran proposed: [openstack/diskimage-builder] 999989: debootstrap: add DIB_MMDEBSTRAP_EXTRA_ARGS support https://review.opendev.org/c/openstack/diskimage-builder/+/999989 | 14:52 | |
| @fungicide:matrix.org | i need to go run some errands and grab lunch, but won't be long | 14:55 |
| @clarkb:matrix.org | the issue is that the new python3.14 based images don't have a pip installed in them? | 14:59 |
| @clarkb:matrix.org | I'm a bit confused as to why any of this would run on the executor in the first place | 14:59 |
| @jim:acmegating.com | i think it's just the twine upload which is run on the executor from a trusted playbook if memory serves | 15:01 |
| @clarkb:matrix.org | oh and we're installing twine there in order to do that | 15:02 |
| @clarkb:matrix.org | https://people.kernel.org/monsieuricon/creepy-crawlies is a good summary of what we've all been seeing with the Internet crawlers | 15:03 |
| @clarkb:matrix.org | The one thing that post doesn't get into is how despite being able to solve the challenges the bots do appear to be slowed down by the challenges. So its still valuable even if it doesn't block them entirely | 15:04 |
| @clarkb:matrix.org | `But no, let's in fact choose the stupidest possible way of doing it — by rendering everything as HTML commit by commit and then parsing it.` | 15:10 |
| @jim:acmegating.com | the log streaming revert has merged, but i don't think this is an urgent situation; the actual failures seem load induced so we probably have until the next periodic runs before it becomes very serious. i'm still looking at the code and coming up with avenues to explore. i'd like to continue with that just a bit before i start restarting executors, just in case there's something else to be learned before we stop the current executors. | 16:06 |
| @clarkb:matrix.org | Sounds like a reasonable plan | 16:07 |
| @clarkb:matrix.org | corvus: does the yappi profiler include memory use stats? | 16:07 |
| @jim:acmegating.com | yeah, plus we have objgraph, so i suspect getting a snapshot will be useful | 16:10 |
| @tkajinam:matrix.org | I'm hoping that these problems with log upload are transient ... https://review.opendev.org/c/openstack/nova/+/1002937 | 16:11 |
| @tkajinam:matrix.org | There are a few strange erros I found this evening. I'll see if these appears consistently but am recording these here in case someone else hit the same. | 16:13 |
| https://review.opendev.org/c/openstack/telemetry-tempest-plugin/+/999155 | ||
| psutil.ZombieProcess: PID still exists but it's a zombie (pid=120767) | ||
| https://review.opendev.org/c/openstack/ceilometer/+/1001683 | ||
| ERROR! A worker was found in a dead state | ||
| @clarkb:matrix.org | tkajinam: I think the scroll back just above will explain it | 16:13 |
| @tkajinam:matrix.org | ohh sorry I had to scroll it back by a few more lines... | 16:15 |
| @clarkb:matrix.org | corvus: I'm trying to get some travel stuff sorted out today, but should generally be around (except for maybe a bike ride before it gets hot?) let me know if I can help with further debugging | 16:21 |
| @jim:acmegating.com | okay i ran the stack dump handler on ze03; i'm not sure it's interesting right now (i'm currently eyeing the new subprocess, and it doesn't cover that), but it's there if we need it. | 17:02 |
| @jim:acmegating.com | i think we can roll the executors now. should i do a hard or graceful stop? the hard stop should be minimally disruptive since jobs will be retried.... but it's still a little disruptive.... | 17:03 |
| @jim:acmegating.com | i'm looking at the openstack status page and i'm thinking hard stop is probably okay... | 17:03 |
| @jim:acmegating.com | lmk if you think otherwise | 17:04 |
| @clarkb:matrix.org | I don't see any release jobs running and those are maybe broken anyway? | 17:07 |
| @clarkb:matrix.org | Not sure if fungi is back from lunch yet but I agree that a hard stop is probably ok | 17:07 |
| @jim:acmegating.com | i'm going to hard restart ze03 | 17:11 |
| @jim:acmegating.com | it seems to be doing ansibly things | 17:14 |
| @fungicide:matrix.org | Clark: yeah back now, catching up | 17:19 |
| @jim:acmegating.com | #status log restarted all zuul executors with log streaming update reverted (3.14 still in place) | 17:21 |
| @status:opendev.org | @jim:acmegating.com: finished logging | 17:21 |
| -@gerrit:opendev.org- Dr. Jens Harbott proposed: [openstack/project-config] 1003091: Run ensure-twine early in Python publish job https://review.opendev.org/c/openstack/project-config/+/1003091 | 17:37 | |
| @harbott.osism.tech:regio.chat | so I think ^^ should be done regardless of un-wheeling the ensure-pip role. is someone already working on the latter? | 17:38 |
| @fungicide:matrix.org | Jens Harbott: i haven't seen anyone say they were working on it yet | 17:42 |
| @harbott.osism.tech:regio.chat | also, since I just saw another github download error, reviews on https://review.opendev.org/c/opendev/zuul-providers/+/1000965 would be nice | 17:43 |
| @harbott.osism.tech:regio.chat | looking at https://opendev.org/zuul/zuul-jobs/raw/branch/master/roles/ensure-pip/README.rst which explicitly states `This role will also install ``wheel`` components sufficient to run ``bdist_wheel`` builds or ``pip wheel`` on a source tree.` and that we likely still need to support python2 with that role, I think the best we might do is add a rolevar that would allow to disable it? | 17:46 |
| @harbott.osism.tech:regio.chat | or maybe then we'd rather move ensure-twine to do something special? or have a ensure-twine-on-executor role? | 17:47 |
| @fungicide:matrix.org | modern setuptools releases (in the past ~2 years) have deprecated `setup.py bdist_wheel` as a possibility, maybe longer | 17:52 |
| @fungicide:matrix.org | maybe a toggle makes sense, but at this point it's so late on the scene that i'd announce the behavior change and add the toggle as a fallback for anyone who needs to access the deprecated behavior | 17:54 |
| @clarkb:matrix.org | ok popping out for a bit | 17:56 |
| -@gerrit:opendev.org- Dr. Jens Harbott proposed: [openstack/project-config] 1003091: Run ensure-twine early in Python publish job https://review.opendev.org/c/openstack/project-config/+/1003091 | 18:28 | |
| @harbott.osism.tech:regio.chat | corvus: can the OOM at the wrong moment also explain jobs getting marked failed without any failure seen in the job log? https://zuul.opendev.org/t/openstack/build/ec8a26195b8f4119a35b0d6f587bad55 | 18:41 |
| -@gerrit:opendev.org- Zuul merged on behalf of Michal Nasiadka: [openstack/project-config] 1001093: Add tenks to openstack-kolla notifications https://review.opendev.org/c/openstack/project-config/+/1001093 | 18:42 | |
| @jim:acmegating.com | Jens Harbott: yes, possibly between when the playbook ended and the log output handlers were dealing with it | 18:48 |
| @cardoe:matrix.org | So just wondering about that openstacksdk release on pypi? Is there something I need to do? | 18:56 |
| @cardoe:matrix.org | or is that https://review.opendev.org/c/openstack/project-config/+/1003091 that needs to land first? | 18:57 |
| @fungicide:matrix.org | Doug Goldstein: yes, when the time comes to reenqueue the release one of the openstack release managers will be on top of that and let you know that it's done | 18:57 |
| @fungicide:matrix.org | Doug Goldstein: 1003091 is the current work on addressing it and needs to merge first | 18:59 |
| -@gerrit:opendev.org- Julia Kreger proposed on behalf of Steve Baker: [openstack/diskimage-builder] 1001740: Add dib-init-image tool for creating empty disk images https://review.opendev.org/c/openstack/diskimage-builder/+/1001740 | 19:08 | |
| -@gerrit:opendev.org- Steve Baker proposed: [openstack/diskimage-builder] 1001740: Add dib-init-image tool for creating empty disk images https://review.opendev.org/c/openstack/diskimage-builder/+/1001740 | 19:09 | |
| -@gerrit:opendev.org- Roja Eswaran proposed: [openstack/diskimage-builder] 999989: debootstrap: add DIB_MMDEBSTRAP_EXTRA_ARGS support https://review.opendev.org/c/openstack/diskimage-builder/+/999989 | 20:00 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Takashi Kajinami: [openstack/diskimage-builder] 1000692: Add minimum pyproject.toml https://review.opendev.org/c/openstack/diskimage-builder/+/1000692 | 20:07 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Takashi Kajinami: [openstack/diskimage-builder] 1000680: Bump upper version of flake8 https://review.opendev.org/c/openstack/diskimage-builder/+/1000680 | 20:25 | |
| @fungicide:matrix.org | since https://review.opendev.org/c/openstack/project-config/+/1003091 should only be affecting the release-openstack-python job i'm tempted to single-core approve it, Jens Harbott and i are both openstack release managers anyway. my only reservation is that it shares the `playbooks/publish` folder with a bunch of other stuff and isn't organized the way the `zuul/zuul-jobs` repo is for example | 20:27 |
| @fungicide:matrix.org | the affected playbook doesn't appear to be called by any other jobs, so i think it's safe | 20:30 |
| @clarkb:matrix.org | ya that seems reasonable. Sorry I'm juggling lunch and zuul reviews and travel prep so I'm not as helpful as I probably could be | 20:32 |
| @fungicide:matrix.org | no worries, i just want to get that merged and retry the failing release job because this is entering a critical period for openstack's release cycle and the current bug is blocking all release activity | 20:33 |
| @fungicide:matrix.org | the main wierdness is that 1003091 is adding a `playbooks/publish/pre.yaml` that's really only related to the `pypi.yaml` playbook in that same directory and unrelated to all the publish playbooks there for other jobs | 20:36 |
| @fungicide:matrix.org | maybe it would have been clearer as `pypi-pre.yaml` or something, but that's easy enough to rename in a later change if anyone finds it problematic | 20:36 |
| -@gerrit:opendev.org- Zuul merged on behalf of Dr. Jens Harbott: [openstack/project-config] 1003091: Run ensure-twine early in Python publish job https://review.opendev.org/c/openstack/project-config/+/1003091 | 20:39 | |
| -@gerrit:opendev.org- Julia Kreger proposed: [opendev/glean] 1003156: Add pyproject.toml for modern build compatibility https://review.opendev.org/c/opendev/glean/+/1003156 | 21:10 | |
| @jim:acmegating.com | fungi: Jens Harbott huh, i thought we discussed relaxing the check in the ensure-wheel role like Anil suggested? | 21:27 |
| @jim:acmegating.com | i ask because revoke-sudo has nothing to do with the executor, so if the idea of 3091 is to do something before revoke-sudo is run on the remote node, i don't think it's going to have any effect | 21:28 |
| @fungicide:matrix.org | i think we can do that in parallel, but not installing something if wheel isn't found could result in a regression so we'll ideally spend at least a week with advance notification of a behavior change for zuul-jobs, while we can in theory mitigate it in the interim in opendev | 21:29 |
| @fungicide:matrix.org | it also seems realistic to not be running ensure-* tasks in post if we expect them to result in some action in jobs which don't allow running as root | 21:31 |
| @jim:acmegating.com | it doesn't matter where you run it, you never have root on the executor | 21:31 |
| @jim:acmegating.com | * it doesn't matter when you run it, you never have root on the executor | 21:31 |
| @fungicide:matrix.org | oh, fair point, then 1003091 isn't going to fix anything | 21:32 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [openstack/project-config] 1003159: Revert "Run ensure-twine early in Python publish job" https://review.opendev.org/c/openstack/project-config/+/1003159 | 21:33 | |
| @fungicide:matrix.org | i'll self-approve that and see if i can work out a patch for the alternative suggestion | 21:34 |
| @fungicide:matrix.org | yes i get that | 21:35 |
| @fungicide:matrix.org | we'd rather not roll back to python 3.13 if we can help it either | 21:36 |
| @jim:acmegating.com | i'm digging through history to try to understand the current state of ensure-pip | 21:40 |
| @jim:acmegating.com | https://review.opendev.org/736402 | 21:40 |
| https://review.opendev.org/736185 | ||
| are relevant | ||
| @fungicide:matrix.org | the approach i'm trying to put together is to check for wheel on python 3 by default if a certain flag is set, so that we can merge that today as a no-op, and then turn that off from the openstack release job | 21:43 |
| @fungicide:matrix.org | untangling the python 2 vs 3 logic in that role is interesting, i'm guessing we should assume that it should continue to always look for wheel if on python 2.x and make the toggle specific to python 3.x | 21:46 |
| @fungicide:matrix.org | so that we can later turn it to non-default only for 3.x | 21:46 |
| @jim:acmegating.com | it looks like the premise of 736402 "wheel is very closely linked with pip install operations" may no longer be true (was it ever?). the work to enhance that role seemed to have been focused on remote nodes, then i guess at some point, someone decided that ensure-pip was the easiest way to get twine installed on an executor. | 21:47 |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [openstack/project-config] 1003159: Revert "Run ensure-twine early in Python publish job" https://review.opendev.org/c/openstack/project-config/+/1003159 | 21:48 | |
| @jim:acmegating.com | slowly backing that out by way of opt-in sounds like a good way to handle it | 21:48 |
| @clarkb:matrix.org | ok I think I'm mostly caught up on everything else and will be working on meeting agenda stuff. Anything to add/edit/remove? | 21:49 |
| @jim:acmegating.com | https://review.opendev.org/940074 | 21:55 |
| that is relevant because that switched ensure-twine to using ensure-pip (it did not before) | ||
| @fungicide:matrix.org | yeah, i guess we wanted to make sure we had pip available in order to be able to install it | 21:56 |
| @fungicide:matrix.org | looks like ensure-pip is how we get the venv module | 21:58 |
| @jim:acmegating.com | so far i'm not seeing anything that suggests that we're on the wrong track; i think making "ensure-pip" be "ensure-pip" not "ensure-pip-and-also-wheel" is the right solution, the only question is "how fast should we do that" and a variable seems like a good way. | 21:58 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 21:58 | |
| @fungicide:matrix.org | https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 "ensure-pip: toggle to disable checking for wheel" is a naive approach | 21:59 |
| @clarkb:matrix.org | I want to say I ran into pip just not working if wheel wasn't installed when working on https://review.opendev.org/c/openstack/pbr/+/1002358 | 22:00 |
| @clarkb:matrix.org | if the package does not already have a wheel available then modern pip does not include wheel and will fail to build a package that it can install | 22:00 |
| @jim:acmegating.com | Clark: if that's the case, then should we put it into the executor env? | 22:00 |
| @jim:acmegating.com | > <@abelur:matrix.org> Given pip hasn't needed a global wheel since PEP 517, and this playbook only needs pip to install twine, dropping that check in ensure-pip seems like the fix - but I don't know the history of it, so would want a second opinion before trying to fix this. | 22:02 |
| I was taking this as suggesting that we should be moving forward with not having wheel on the executor | ||
| @mordred:waterwanders.com | It seems like we're generally unclear on whether or not wheel is needed or not needed. That feels like a thing that would be good to know one way or the other. | 22:03 |
| @clarkb:matrix.org | corvus: fwiw I think we can probably say that building packages is out of scope for the executor please use a test node for that | 22:03 |
| @clarkb:matrix.org | But my understanding is wheel is necessary if pip cannot find a wheel and needs to build one | 22:03 |
| @jim:acmegating.com | of course, we don't really want people building packages on the executor, so maybe it's okay to need wheel on remote nodes but not the executor? | 22:03 |
| @mordred:waterwanders.com | yeah. like - building on a node, copying back and publishing, or building on a node and publishing from a node - or building from a node, saving as an artifact and then having a publish job fetch the built artifact and publish it are all patterns we do places | 22:05 |
| @jim:acmegating.com | mordred: Clark fungi i am happy to either approve fungi's change or go add wheel to the executor env... i think all 3 of you are closer to the ground on the question of "is wheel always needed for pip?" than i am, so i think the 3 of you should vote on that and we'll do whichever that vote decides. :) | 22:09 |
| @fungicide:matrix.org | in this case we're installing pip so that we can create a venv in which to install twine to upload files which were built elsewhere | 22:09 |
| @clarkb:matrix.org | for that particular use case wheel should not be necessary (as creating a virtualenv and installing twine should work without building any new wheels) | 22:11 |
| @clarkb:matrix.org | fungi: I don't think this is true: `Modern pip implementations no longer expect the wheel module to be present` | 22:12 |
| @clarkb:matrix.org | I think modern pip expects wheel to be present if pip needs to build a wheel from an sdist to install a package. Maybe this only impacts non pep517 packages but modern pip will still use wheel in at least some circumstances | 22:12 |
| @clarkb:matrix.org | I think using the toggle to say "we know we don't need wheel for our use case" is fine though so I Think fungi's change is ok but maybe we update the docs to be more clear | 22:13 |
| @jim:acmegating.com | Clark: sounds like you're saying this change is the right change (remove the assumption from ensure-pip) but for a different reason (because we don't want builds happening on executors) | 22:13 |
| @clarkb:matrix.org | corvus: yes | 22:13 |
| @jim:acmegating.com | that wfm too | 22:13 |
| @clarkb:matrix.org | I +2'd the change with a note on the doc update. I think its ok to land that and followup with documentation updates if we're wanting to keep moving forward | 22:17 |
| @clarkb:matrix.org | but I didn't approve to give fungi the chance to opt into updating the change (or disagreeing with me and approving it as is which I think is fine) | 22:17 |
| @fungicide:matrix.org | i can update the wording in the readme, just a sec | 22:19 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 22:25 | |
| @fungicide:matrix.org | Clark: ^ better? | 22:25 |
| @fungicide:matrix.org | i adjusted the approach on the assumption it won't be getting defaulted off later after all | 22:26 |
| @jim:acmegating.com | should that be "ensure_pip_check_wheel" or something to maintain a strict "ensure_pip" prefix? | 22:27 |
| @clarkb:matrix.org | I think we have tried to do that to namespace things and avoid collisions. A collision here seems unlikely but that may be a good improvement | 22:28 |
| @clarkb:matrix.org | the change lgtm otherwise | 22:28 |
| @fungicide:matrix.org | i can do that, it reads awkwardly as english, but i suppose that's not the primary naming purpose | 22:28 |
| @jim:acmegating.com | yeah -- the prefix convention is definitely a documented thing... i think the only question is whether there's a reason to disregard it in this case (sounds like no) | 22:29 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 22:29 | |
| @fungicide:matrix.org | done | 22:30 |
| @jim:acmegating.com | (such a reason might be something like "there's an ensure_wheel role that works in tandem with this one"; but i don't think that's the case here) | 22:30 |
| @clarkb:matrix.org | I have approved it | 22:32 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 22:46 | |
| - [opendev/system-config] 1000367: Add optional wait condition to selenium screenshot utility https://review.opendev.org/c/opendev/system-config/+/1000367 | ||
| - [opendev/system-config] 1000379: Add selenium wait conditions to lists screenshots https://review.opendev.org/c/opendev/system-config/+/1000379 | ||
| @clarkb:matrix.org | Anil Belur: I went ahead and implemented your suggestion there^ fungi you reviewed the child (and approved it) but hadn't reviewed the parent yet so I figured I should just update the parent and a rereview would be easy | 22:46 |
| @clarkb:matrix.org | fungi: corvus https://zuul.opendev.org/t/zuul/build/34d5c363db264558a81862bd6b9ee49b does this failure mean we need to set that flag on that specific test? | 22:47 |
| @clarkb:matrix.org | hrm that says sudo not found | 22:48 |
| @clarkb:matrix.org | but is that only happening because the check for wheel and pip failed? so ya maybe setting the flag would fix the test | 22:48 |
| @fungicide:matrix.org | i guess zuul-jobs-test-ensure-pip-localhost could stop requiring the wheel module as well | 22:51 |
| @fungicide:matrix.org | i can add that to the change if everyone's on board with that solution | 22:52 |
| @clarkb:matrix.org | yup I think that makes sense to me | 22:52 |
| @jim:acmegating.com | yep, also yay we have a test for this :) | 22:54 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 22:55 | |
| @fungicide:matrix.org | tested! ^ | 22:55 |
| @clarkb:matrix.org | ok I've got the meeting agenda updated and am just about ready to send it out. New topics include the office hour thing I'm trying to start, service coordinator election results, and the zuul executor oom situation | 22:56 |
| @clarkb:matrix.org | last call before I send it out | 22:56 |
| @clarkb:matrix.org | fungi: that didn't fix it because we run pip wheel on localhost :/ | 23:01 |
| @clarkb:matrix.org | fungi: so the test needs to skip pip wheel things if we're skipping wheel | 23:01 |
| @fungicide:matrix.org | hrm... okay | 23:02 |
| @fungicide:matrix.org | i think we just need to not run that | 23:02 |
| @jim:acmegating.com | did we add that because we thought running wheel on localhost was important? or was that accidental? | 23:03 |
| @clarkb:matrix.org | corvus: I think it is reusing the test cases from the non localhost scenario | 23:04 |
| @clarkb:matrix.org | my assumption (without looking at git logs) is that in that case it is accidental | 23:05 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 23:09 | |
| @fungicide:matrix.org | i removed the task that's exercising `pip wheel` in that job | 23:10 |
| @fungicide:matrix.org | the alternative is to start testing it on a separate node instead | 23:10 |
| @clarkb:matrix.org | I think we are testing it on a separate node too | 23:13 |
| @clarkb:matrix.org | and those jobs succeed. So we're removing that task for all of the jobs (probably fine I guess) | 23:13 |
| @clarkb:matrix.org | fungi: the "easy" way to fix this without removing the check for the other jobs would be to copy the playbook and remove it only in the copy and set that playbook as the playbook for the localhost job | 23:14 |
| @fungicide:matrix.org | i'm open to further suggestions | 23:14 |
| @clarkb:matrix.org | test-playbooks/ensure-pip/ensure-pip-localhost.yaml includes the role you modified | 23:15 |
| @fungicide:matrix.org | that that playbook already includes quite a few tasks. maybe i could instead split it into multiple plays in one playbook? | 23:15 |
| @clarkb:matrix.org | so either we can use a different copy with the bits removed or make that oncidtional as well as set the condition flag in test-playbooks/ensure-pip/ensure-pip-localhost.yaml | 23:15 |
| @fungicide:matrix.org | and have a play that covers the tasks that run everwhere and another play that covers the tasks that only run on a remote node? | 23:15 |
| @clarkb:matrix.org | when: ensure_pip_test_wheel or something | 23:15 |
| @clarkb:matrix.org | and then set that when calling the role from the localhost playbook | 23:16 |
| @fungicide:matrix.org | oh, i could make it conditional on `ensure_pip_check_wheel` i guess | 23:16 |
| @clarkb:matrix.org | ya | 23:17 |
| @fungicide:matrix.org | though won't that just end up being the same as removing it? will be disabled for the whole job in that case i guess | 23:18 |
| @fungicide:matrix.org | mmm | 23:18 |
| @clarkb:matrix.org | the problem with removing it like you did is you removed it for all of the other jobs too | 23:19 |
| @clarkb:matrix.org | so making it conditional would still cover pip wheel where we do expect wheel to be present | 23:19 |
| @fungicide:matrix.org | oh, i thouhgt it was only used by one job | 23:19 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/zuul/buildset/3ba997a32b7a4ac99ddbf02ab1edb1cd no all of these other ensure pip jobs appear to use the same role | 23:19 |
| @clarkb:matrix.org | and those are platform specific not localhost | 23:20 |
| @clarkb:matrix.org | so emulate a remote node aiui | 23:20 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 23:21 | |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [openstack/project-config] 1003189: Stop checking for wheel during release uploads https://review.opendev.org/c/openstack/project-config/+/1003189 | 23:22 | |
| @jim:acmegating.com | fungi: i suspect tests will fail on 3163; i left a comment | 23:36 |
| @fungicide:matrix.org | mmm | 23:38 |
| @fungicide:matrix.org | corvus: the role sets a default, so there should always be a value right? | 23:42 |
| @fungicide:matrix.org | under what conditions would the default not get applied? | 23:42 |
| @jim:acmegating.com | but that's in the playbook for the test | 23:42 |
| @fungicide:matrix.org | if the playbook can be conditional on the variable when it's set from a job, why wouldn't the default value from the same role also be present otherwise? i guess i just don't understand how rolevars work in ansible | 23:43 |
| @jim:acmegating.com | the scope of the role variable default is only within that role, but this is a playbook that runs outside of that scope | 23:44 |
| @fungicide:matrix.org | oh, defaults don't propagate outside of role scopes, but values do propagate inward to the role scopes? okay i think i can see the reason for that | 23:45 |
| @jim:acmegating.com | yep... to oversimplify: role is like a function with a default parameter value, and you're setting a global variable on the test job. the test playbook is a different function, so can use the variable, but is not affected by the role | 23:46 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul-jobs] 1003163: ensure-pip: toggle to disable checking for wheel https://review.opendev.org/c/zuul/zuul-jobs/+/1003163 | 23:46 | |
| @fungicide:matrix.org | thanks! | 23:46 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!