| frickler | we do cache some older etcd tarballs, seems it was missed to update the cache when the version was bumped (iirc that's in devstack) | 05:39 |
|---|---|---|
| frickler | see https://opendev.org/opendev/zuul-providers/src/branch/master/dib-elements/cache-devstack/source-repository-images#L16-L18, I'm doing a patch now | 05:40 |
| frickler | funnily(?) the bump was merged exactly 1y ago, weird that the issue seems to be noticed only now https://review.opendev.org/c/openstack/devstack/+/952755 | 05:43 |
| frickler | probably it is time to go both for a newer patch release and to also test the next minor release before the current one goes eol | 05:49 |
| opendevreview | Michal Nasiadka proposed openstack/project-config master: Add mnasiadka to accessbot admins https://review.opendev.org/c/openstack/project-config/+/1000797 | 06:10 |
| opendevreview | Merged openstack/project-config master: Add mnasiadka to accessbot admins https://review.opendev.org/c/openstack/project-config/+/1000797 | 06:28 |
| opendevreview | Michal Nasiadka proposed openstack/project-config master: Add check-sources propose job for kolla https://review.opendev.org/c/openstack/project-config/+/998419 | 07:01 |
| opendevreview | Michal Nasiadka proposed openstack/project-config master: Add check-sources propose job for kolla https://review.opendev.org/c/openstack/project-config/+/998419 | 07:05 |
| stephenfin | clarkb: fungi: I think something is off with the new pbr release. I'm seeing openstack-requirements-related devstack failures that coincide with 7.1.0's release https://zuul.opendev.org/t/openstack/builds?job_name=openstacksdk-functional-devstack&skip=0 | 12:36 |
| frickler | "ModuleNotFoundError: No module named 'openstack_requirements.cmds'" | 12:43 |
| stephenfin | I know there's a mistake in the package configuration (fix is here https://review.opendev.org/c/openstack/requirements/+/1000833) but setuptools has fallback logic (with a big ugly warning) to work aorund that. I'm wondering if we broke that somehow with the pbr 7.1.0 changes | 12:48 |
| frickler | hmm, if I'm not mistaken, we are using pbr-7.0.3 for the reqs .venv still, because it is using u-c.txt as constraints | 12:56 |
| stephenfin | I don't think u-c is respected for build time constraints | 12:58 |
| frickler | ah, right | 13:00 |
| frickler | I cannot reproduce this locally so far, might be specific to py3.12 or some older setuptools version in ubuntu? | 13:04 |
| stephenfin | frickler: try this https://paste.opendev.org/show/bR5JBHxM7H6vckC2OIsD/ | 13:08 |
| stephenfin | compare the size of the wheels in /tmp/new-pbr and tmp/old-pbr | 13:09 |
| opendevreview | Stephen Finucane proposed openstack/pbr master: Fix hook ordering race that drops subpackages from wheels https://review.opendev.org/c/openstack/pbr/+/1000836 | 13:17 |
| stephenfin | I think that should fix it. We may want to yank 7.1.0 in the interim, unless we can get that out soonish | 13:17 |
| auniyal | hello, any ideas ---- | 13:33 |
| auniyal | + inc/python:_setup_package_with_constraints_edit:405 : /opt/stack/requirements/.venv/bin/edit-constraints /opt/stack/requirements/upper-constraints.txt -- keystone | 13:33 |
| auniyal | Traceback (most recent call last): | 13:33 |
| auniyal | File "/opt/stack/requirements/.venv/bin/edit-constraints", line 5, in <module> | 13:33 |
| auniyal | from openstack_requirements.cmds.edit_constraint import main | 13:33 |
| auniyal | ModuleNotFoundError: No module named 'openstack_requirements.cmds' | 13:33 |
| auniyal | + inc/python:_setup_package_with_constraints_edit:1 : exit_trap | 13:33 |
| auniyal | I think several jobs wil be blocked with this | 13:34 |
| auniyal | does this fixes it https://review.opendev.org/c/openstack/requirements/+/1000833 ? | 13:44 |
| fungi | auniyal: it was just discussed before you came into the channel | 13:47 |
| fungi | i'm reviewing https://review.opendev.org/c/openstack/pbr/+/1000836 right now which should fix it in theory | 13:51 |
| fungi | #status log Yanked PBR 7.1.0 from PyPI due to a hook ordering race that drops subpackages from wheels | 13:52 |
| opendevstatus | fungi: finished logging | 13:52 |
| fungi | stephenfin: frickler: ^ | 13:52 |
| stephenfin | fungi: I should probably file a bug, I guess? | 13:53 |
| stephenfin | thanks for quickly yanking that | 13:54 |
| fungi | yw | 13:54 |
| fungi | i think the description in the commit message is sufficient, but i don't know if other reviewers will agree | 13:55 |
| stephenfin | okay, I'll leave it for now and do it if required. It's pretty simple at least, and easily reproducible/verifiable | 13:55 |
| stephenfin | launchpad is also being super slow for me... | 13:56 |
| frickler | for me the new-pbr wheel is larger than old-pbr, that isn't what I expected | 13:57 |
| frickler | also, yeah, launchpad is struggling with "ai" stuff like many others | 13:58 |
| stephenfin | frickler: I assume yes, but did you clean all the build artifacts left in the repo between steps? | 13:59 |
| fungi | keep in mind that whl files are just zip with a different extension, i often unzip two side-by-side and do recursive diffs if in doubt | 14:02 |
| frickler | stephenfin: ah, between: yes, but I didn't clean before I ran all your paste. retried now and that looks more like the expected result (old about half the size) | 14:02 |
| stephenfin | okay, sweet | 14:03 |
| stephenfin | fungi: yeah, you can see the missing directories if you unzip it, but the filesize is a dead easy heuristic | 14:03 |
| frickler | sadly the fix is failing in CI. not sure if retrying now that 7.1.0 is pulled will be enough? | 14:04 |
| fungi | seems like an iffy job to speculatively test pbr if it's using released pbr from pypi | 14:06 |
| fungi | i had thought about suggesting we do a prerelease again and then exercise it with dnm changes in some projects, but i'm still not sure we'd have caught this | 14:07 |
| stephenfin | Yeah, I've rechecked it now that 7.1.0 has been yanked | 14:10 |
| fungi | looking at the pbr-installation-openstack-resolute failure, it's unclear to me where it's breaking, one of the last things that happens is the speculative pbr 7.1.1.dev1 build gets uninstalled and replaced by 7.0.3 from pypi | 14:18 |
| mordred | well that doesn't seem like the right thing | 14:19 |
| fungi | which is also not the broken 7.1.0 ftr | 14:19 |
| mordred | yeah. but - perhaps that'b because 7.1.1.dev1 is a dev release and we're asking pip to install nova without the "use dev releases please" flag to it down-resolves? I mena, that's something ot fix in the integration test, but is almost certainly unrelated to the fix | 14:21 |
| fungi | right, but i'm also unclear on why that's failing if it had previously been working | 14:22 |
| fungi | as in, i'm not confident that this recheck is going to help | 14:23 |
| fungi | guess we'll wait and see | 14:23 |
| mordred | well, the failure is alembic.ini not being found, which seems unrelated to either issue | 14:24 |
| mordred | didn't we change how pbr generates pre-release version numbers? | 14:24 |
| mordred | is that perhaps more or differently aligned with pip now so it's changing the behavior in the integration test, but we just didn't notice before because nothing else was breaking? | 14:25 |
| fungi | seems more likely something earlier in the test got built with the broken 7.1.0 from pypi instead of using the git version, and then i yanked it from pypi mid-run | 14:26 |
| fungi | at least i hope that's the reason | 14:28 |
| mordred | ++ the version change patch I remember actually was just supporting tags with in them. no behavior change. so, fingers crossed it's your thing | 14:31 |
| fungi | looks like it passed this time, so i guess that was it | 15:50 |
| frickler | the question now is what this actually tells us about the patch under review. I did repeat the test from stephenfin's earlier paste with it though, and that seems to produce a reasonable wheel afaict | 16:06 |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Update openstack-github-mirroring API token https://review.opendev.org/c/openstack/project-config/+/1000868 | 16:23 |
| mordred | it looks like the pbr patch did not replace the pbr under test with 7.0.3 looking through the logs. So I'm inclined to believe fungi's hypothesis | 18:52 |
| opendevreview | Merged openstack/pbr master: Fix hook ordering race that drops subpackages from wheels https://review.opendev.org/c/openstack/pbr/+/1000836 | 20:27 |
| *** skandix222198 is now known as skandix22219 | 22:18 | |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Replace 2026.2/Hibiscus key with 2027.1/Indri https://review.opendev.org/c/openstack/project-config/+/1000918 | 22:37 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!