| *** ralonsoh_out is now known as ralonsoh | 05:25 | |
| priteau | Good morning. Epoxy release of Kayobe is ready to approve: https://review.opendev.org/c/openstack/releases/+/950450 | 09:25 |
|---|---|---|
| priteau | Thank you elodilles | 09:28 |
| elodilles | np :) | 09:28 |
| opendevreview | Merged openstack/releases master: kayobe: final release for 2025.1 Epoxy https://review.opendev.org/c/openstack/releases/+/950450 | 09:38 |
| *** sfinucan is now known as stephenfin | 11:19 | |
| stephenfin | fungi: clarkb: Have either of you seen errors like this before? | 12:10 |
| stephenfin | pkg_resources.DistributionNotFound: The 'oslo.utils>=4.7.0' distribution was not found and is required by designate, oslo.versionedobjects, tooz | 12:10 |
| stephenfin | Seen when attempting to deploy designate with DevStack from the dalmatian (stable/2024.2) branch https://github.com/gophercloud/gophercloud/pull/3108#issuecomment-2897313058 | 12:10 |
| stephenfin | I've reproduced locally. The oslo.utils package is installed in the venv and I can import just fine. It's very odd. | 12:11 |
| stephenfin | $ /opt/stack/data/venv/bin/python -m pip freeze | grep oslo.utils | 12:14 |
| stephenfin | oslo.utils==7.3.1 | 12:14 |
| frickler | stephenfin: yes, we have seen that in periodic jobs a couple of days ago | 12:14 |
| frickler | likely related to the fix we needed to get the reqs bump in https://review.opendev.org/c/openstack/requirements/+/947558 + parent | 12:15 |
| stephenfin | aha https://review.opendev.org/c/openstack/designate/+/949094 | 12:16 |
| frickler | https://zuul.openstack.org/builds?job_name=openstacksdk-functional-devstack-networking&project=openstack%2Fneutron&branch=stable%2F2024.2&skip=0 | 12:16 |
| frickler | stephenfin: I think that that designate change is not related, just bad timing | 12:17 |
| frickler | there are also similar failures on master reqs now https://review.opendev.org/c/openstack/requirements/+/907665 | 12:18 |
| frickler | maybe consumers need to change from "oslo.utils" to "oslo_utils" in their requirements.txt? | 12:20 |
| frickler | or maybe the packaging we do is broken? I'm not sure yet | 12:20 |
| elodilles | stephenfin: frickler: the problem is that pkg_resources is deprecated and it has the requires() method that does not find oslo.utils anymore as with the new PEP it is listed as oslo_utils | 12:30 |
| elodilles | so it's not related to the reqs fix, rather it is related to pip itself | 12:30 |
| elodilles | (unfortunately importlib.metadata.require() is not compatible with pkg_resources.requires() as metadata.require() just gives back the list of required packages but does not check them compared to pkg_resources.requires()) | 12:32 |
| elodilles | ((importlib.metadata is meant to be the new module that somewhat replaces pkg_resources)) | 12:33 |
| elodilles | and note that since around mid January we create packages according to the new PEP rules (if i remember correctly) | 12:34 |
| elodilles | (see tarball names for example: https://tarballs.opendev.org/openstack/oslo.utils/ ) | 12:36 |
| stephenfin | So do we need to change imports from `oslo.utils` to `oslo_utils`? | 12:44 |
| stephenfin | elodilles: ^ | 12:44 |
| elodilles | not sure tbh | 12:45 |
| elodilles | idk whether that would make the current pkg_resources work | 12:46 |
| elodilles | my guess is not :/ | 12:46 |
| fungi | yeah, if it's easy to test then worth a try, but i wouldn't get my hopes up | 12:53 |
| frickler | ok, so that would be another issue with pbr, then? | 12:54 |
| fungi | pip wasn't why the sdist names got normalized recently, that happened automatically when we switched from building with `setup.py sdist` to using the pyproject-build utility which started pulling in newer setuptools | 12:55 |
| fungi | but also, distribution package names and import package names are entirely different things, they just happen to share/overload the same glossary term "package" | 12:57 |
| fungi | there is no (and has never been) expectation in python that the package you pip install has the same name as the package you import in your python code | 12:58 |
| fungi | there's even a pep currently in the works to allow distribution packages to add metadata declaring the import names they provide | 12:59 |
| fungi | precisely because they're completely disconnected (which has traditionally led to user confusion) | 13:00 |
| fungi | does anyone have a link to the full context/traceback for the pkg_resources.DistributionNotFound exception quoted above? | 13:02 |
| fungi | was it a release job failure? (guessing so as the topic was brought up here in the release cannel) | 13:03 |
| fungi | aha, https://zuul.opendev.org/t/openstack/build/847d2d4cc9be4be2bf9e2c7b9aa065bd shows it's cropping up in https://opendev.org/openstack/requirements/src/branch/master/openstack_requirements/cmds/check_conflicts.py#L51 so a random script of ours explicitly calling into pkg_resources | 13:17 |
| fungi | doesn't seem pbr-related at first look | 13:17 |
| frickler | fungi: this is the designate 2024.2 example https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e83/openstack/e831a9a209eb4fae81121e1a2625de93/controller/logs/screen-designate-api.txt | 13:19 |
| frickler | and iiuc the wsgi script is generated by pbr | 13:19 |
| fungi | odd since pbr even has a test to check that there is no pkg_resources reference in generated sgi scripts, and that test was added a decade ago | 13:22 |
| fungi | s/sgi/wsgi/ | 13:22 |
| fungi | copied from its earlier generic entrypoint script tests which checked for the non-presence of package resources back to before oslo.packaging was extracted from openstack/common | 13:26 |
| fungi | s/package resources/pkg_resources/ | 13:27 |
| fungi | okay maybe not quite that far back... mordred added the test in mid-2013 with https://review.openstack.org/38000 which is when pbr started to generate its own scripts | 13:29 |
| fungi | it looks like the setuptools built-in entrypoint script generation relied on pkg_resources but pbr's replacement did not, so the presence of pkg_resources in the script was used as a canary to indicate it had been mistakenly generated by setuptools instead of pbr, i think? | 13:31 |
| frickler | oh, so then this may be related to the CI failures in pbr, which also include failures that miss the "generated by PBR" tag | 13:33 |
| clarkb | frickler: fungi: more importantly setuptools entrypoint scripts using pkg_resources are the entire reason pbr has its own implementation. The use of pkg_resources there makes script startup extremely slow as pkg_resources has to scan the entire python path for all installed packages and sort them by version | 15:37 |
| clarkb | so its a canary but its also the thing we were intentionally trying to avoid so that scripts could be faster if they didn't rely on pkg_resources internally | 15:38 |
| fungi | i wonder if part of the migration to projects shipping their own wsgi scripts has resulted in copying the old setuptools ones? | 15:41 |
| clarkb | as a heads up opendev just approved a change to update gerrit from 3.10.5 to 3.10.6. We'll plan to restart gerrit on that later today after the images have published. (This is prep work for ensuring we're running and testing the latest versiosn of gerrit before we upgrade to 3.11) | 15:41 |
| clarkb | fungi: frickler: https://opendev.org/openstack/designate/commit/5bfc35864c28a97ded6104dea05fa9691a7f14d8 I think the issue is that its listed as a scripts | 15:45 |
| clarkb | which are installed by setuptools using pkg_resources so thats just a bug in designate imo | 15:46 |
| clarkb | if it were a console script it would use pbrs install method | 15:46 |
| clarkb | I think | 15:46 |
| clarkb | for the designate example anyway it appears to be a combo of using setuptools scripts (which gets you pkg_resources (bad)) and then the setuptools script using pkg_resources doesn't know how to import the thing without using its canonical name? | 15:47 |
| clarkb | so two bugs tripping over themselves | 15:47 |
| clarkb | stephenfin: elodilles ^ fyi | 15:48 |
| fungi | bingo | 15:48 |
| fungi | timing lines up too | 15:49 |
| johnsom | But this job on the master branch (which does not have that script) is failing too: https://zuul.opendev.org/t/openstack/build/0f67afccdc864ed2b1118bb933205738 | 15:50 |
| stephenfin | clarkb: I was trying to find out what version of pkg_resources was in use in my environment to ensure that there wasn't a regression there, but yes, that's my thinking also now | 15:50 |
| clarkb | johnsom: that looks like a different error? | 15:51 |
| stephenfin | different bug I think | 15:51 |
| stephenfin | jinx | 15:51 |
| johnsom | Oh, you are talking about the oslo util issue | 15:52 |
| clarkb | the failure on master looks similar to the setuptools bug with copying eggs because it didn't detect eggs properly. However the failur emode is different | 15:53 |
| clarkb | the python-designateclient failure seems to occur when running python setup.py egg_info against python-designateclient and in the code that installs setup requires (pbr) | 15:58 |
| clarkb | which is very similar to the issue mnaser reported but that blue up when trying to copy a dir over a dir becuse it didn't detect the package was already installed (they fixed taht upstream in setuptools) | 15:59 |
| clarkb | there is a new setuptools 80.8.0 as of May 20 (yesterday) which is the day that build failed | 15:59 |
| clarkb | its possible this is a new regression in setuptools in those code paths that desginateclient is triggering that for some reason most other things do not trigger. Maybe bcause python-designateclient has a - in its own name? | 16:00 |
| stephenfin | johnsom: Could you restore https://review.opendev.org/c/openstack/designate/+/948799? I'd like to try something | 16:04 |
| clarkb | johnsom: for designateclient I think you can test python setup.py egg_info with both setuptools 80.7.1 and 80.8.0 and see if the first works and the second breaks. And then take it from there | 16:05 |
| frickler | stephenfin: done | 16:11 |
| stephenfin | frickler: Thanks. fungi, clarkb, johnsom: I _think_ this ought to do the trick. sean-k-mooney suggested it and previous testing looked promising https://review.opendev.org/c/openstack/designate/+/950569 | 16:12 |
| clarkb | stephenfin: the shape of that looks right to me | 16:13 |
| stephenfin | The key is that the object pointed to *must* be called application | 16:13 |
| stephenfin | I really don't want us doing this on master either. It's not a valid script and shouldn't be installed in a `bin` directory | 16:14 |
| stephenfin | ...but needs must for stable branches | 16:15 |
| clarkb | why isn't it valid? | 16:16 |
| clarkb | if uwsgi can run it dosen't that make it valid? | 16:17 |
| clarkb | if the issue is taht you can't `./foo` it so it shouldn't go in bin/ that seems fine | 16:17 |
| clarkb | but I guess console scripts only go to bin/? | 16:17 |
| opendevreview | Stephen Finucane proposed openstack/releases master: python-openstackclient 8.1.0 https://review.opendev.org/c/openstack/releases/+/950572 | 16:18 |
| johnsom | The key with that script is that it must work with Apache mod_wsgi as some people are still deploying that way | 16:40 |
| johnsom | using stable branches | 16:40 |
| -opendevstatus- NOTICE: Gerrit is being updated to the latest 3.10 bugfix release as part of early prep work for an eventual 3.11 upgrade. Gerrit will be offline momentarily while it restarts on the new version. | 17:34 | |
| opendevreview | JaromÃr Wysoglad proposed openstack/releases master: Release telemetry-tempest-plugin https://review.opendev.org/c/openstack/releases/+/950604 | 20:59 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!