| opendevreview | Takashi Kajinami proposed openstack/oslo.cache master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.cache/+/1001997 | 02:20 |
|---|---|---|
| opendevreview | Takashi Kajinami proposed openstack/oslo.cache master: Fix mypy errors https://review.opendev.org/c/openstack/oslo.cache/+/1001998 | 02:25 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.reports master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.reports/+/1002001 | 03:18 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.log master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.log/+/1002002 | 03:19 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.privsep master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.privsep/+/1002003 | 03:20 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.concurrency master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.concurrency/+/1002004 | 03:23 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.db master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.db/+/1002006 | 03:25 |
| *** elodilles_pto is now known as elodilles | 07:26 | |
| opendevreview | Takashi Kajinami proposed openstack/oslo.log master: Surpress mypy errors https://review.opendev.org/c/openstack/oslo.log/+/1002034 | 09:41 |
| opendevreview | Merged openstack/oslo.db master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.db/+/1002006 | 10:25 |
| opendevreview | Merged openstack/oslo.cache master: Fix mypy errors https://review.opendev.org/c/openstack/oslo.cache/+/1001998 | 10:31 |
| opendevreview | Merged openstack/oslo.concurrency master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.concurrency/+/1002004 | 10:37 |
| opendevreview | Merged openstack/oslo.privsep master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.privsep/+/1002003 | 10:44 |
| opendevreview | Merged openstack/oslo.reports master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.reports/+/1002001 | 10:55 |
| opendevreview | Adam Harwell proposed openstack/tooz master: Add Consul coordination driver using native services https://review.opendev.org/c/openstack/tooz/+/997145 | 11:05 |
| opendevreview | Adam Harwell proposed openstack/oslo.messaging master: HTTP-RPC driver: core transport, server, and dedup https://review.opendev.org/c/openstack/oslo.messaging/+/997470 | 11:20 |
| opendevreview | Adam Harwell proposed openstack/oslo.messaging master: HTTP-RPC driver: tooz service discovery and fanout broadcaster https://review.opendev.org/c/openstack/oslo.messaging/+/997471 | 11:20 |
| opendevreview | Adam Harwell proposed openstack/oslo.messaging master: HTTP-RPC driver: DevStack plugin, CI jobs, and documentation https://review.opendev.org/c/openstack/oslo.messaging/+/997472 | 11:20 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.cache master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.cache/+/996918 | 11:48 |
| opendevreview | Stephen Finucane proposed openstack/oslo.log master: Address Python 3.13 issue https://review.opendev.org/c/openstack/oslo.log/+/1001841 | 11:52 |
| opendevreview | Adam Harwell proposed openstack/tooz master: Add leader election to Consul driver https://review.opendev.org/c/openstack/tooz/+/1000914 | 12:19 |
| opendevreview | Merged openstack/oslo.cache master: Remove duplicated optional dependencies from test requirements https://review.opendev.org/c/openstack/oslo.cache/+/996918 | 14:15 |
| stephenfin | cardoe: fungi: on the pbr issue, that hook is in setuptools since setuptools v64.0.0, and pbr's own pyproject.toml explicitly requires that (though its requirements.txt doesn't) | 15:10 |
| stephenfin | https://github.com/pypa/setuptools/commit/821d664b7a9fe9779061812d540b5cd7a7386f5e | 15:10 |
| stephenfin | so I'm guessing the setuptools there is old enough that it doesn't support pyproject.toml, meaning the min pyproject.toml doesn't get picked up | 15:12 |
| stephenfin | I think we're going to have to yank 7.1.1 again and do yet another release with the min in requirements.txt | 15:13 |
| opendevreview | Stephen Finucane proposed openstack/pbr master: Set setuptools min in requirements.txt https://review.opendev.org/c/openstack/pbr/+/1002140 | 15:18 |
| stephenfin | cardoe: fungi: That should do the trick? ☝️ idk if we want to yank 7.1.1 or just get 7.1.2 out soon enough since the problem is constrained to older envs | 15:19 |
| fungi | stephenfin: we want pbr to "just work" regardless of whether a project has a pyproject.toml file, but if we need a newer setuptools then that's probably okay | 15:54 |
| fungi | clarkb: ^ | 15:54 |
| fungi | that said, if the path forward for 7.1.2 is to set a higher setuptools minimum, we'll need to yank 7.1.1 anyway because the dep solver will pick it for anyone requiring older setuptools | 15:55 |
| fungi | arguably 7.1.1 "requires" newer setuptools but doesn't declare that | 15:56 |
| stephenfin | I think what I've proposed is a better option than the alternative, which is checking the setuptools version at runtime and exposing/not exposing the hook point depending on that | 15:56 |
| stephenfin | fungi: I think we'd be okay? The requirement for a newer setuptools only exists on Python 3.7, and setuptools 64.0.0 supports Python 3.7 https://pypi.org/project/setuptools/64.0.0/ | 15:57 |
| stephenfin | afaik pip won't choose an older (< latest) version of a package based on existing packages. It will only do so based on Python version and platform specifiers | 15:58 |
| clarkb | stephenfin: PBR is an install requires so there is no pip involved | 16:05 |
| clarkb | its some easy_install shim thing (was once upon a time easy install itself) | 16:05 |
| stephenfin | oh I know. I was just responding to fungi's comment about pip's dep solver potentially picking an older pbr. We're talking about the pip (vs. easy_install) path there, naturally | 16:06 |
| clarkb | ya I guess I was trying to simplify the problem to remove pip and the dep solver | 16:07 |
| clarkb | the latest version of pbr needs to support the oldest platform versions we expect to see realistically in the wild | 16:07 |
| clarkb | so if 7.1.1 can't do that then it should probably be yanked | 16:07 |
| stephenfin | yes, it fails if the distro provides setuptools < 64.0.0 but the user has installed a newer pip (greater than whatever version added pep-517 support) | 16:09 |
| stephenfin | my argument is that 7.1.2 will fix that and will still install, so yanking is optional | 16:09 |
| clarkb | true that becomes latest and overrides the old latest installation path | 16:11 |
| fungi | ah, okay so if the toolchain is new enough to be using pip to install pbr then it won't have such an old setuptools that pip 7.1.1 breaks | 16:12 |
| stephenfin | yeah, exactly: I'm just trying to reduce the work needed from fungi/you. But if you would rather yank it, far be it from me to tell you what to do :) | 16:13 |
| fungi | i agree that makes thr 7.1.1 yanking at most an option for temporary relief | 16:13 |
| stephenfin | yep, exactly | 16:13 |
| stephenfin | given the limited impact, I would personally get 7.1.2 out nice and soon, give it a few hours to bake, and then yank if neutron issues don't disappear | 16:14 |
| stephenfin | I'm confident that the fix will address that issue. I just hope there are no more lurking underneath | 16:15 |
| opendevreview | Takashi Kajinami proposed openstack/tooz master: Use etcd from distribution https://review.opendev.org/c/openstack/tooz/+/997919 | 16:48 |
| cardoe | fungi: stephenfin: https://review.opendev.org/c/openstack/pbr/+/1002140 failed due to "disk quota exceeded installing the package" | 17:04 |
| fungi | wow | 17:04 |
| fungi | we don't set quotas in those jobs as far as i know | 17:04 |
| fungi | mmm, i wonder if it ran out of space on /tmp and it's mounted as tmpfs and that's how the kernel deals with enospc | 17:06 |
| fungi | the command output mentions a destination in /tmp and https://zuul.opendev.org/t/openstack/build/3c84cb3849d840fa8636de5f370acda8/log/zuul-info/zuul-info.ubuntu-resolute.txt#133 indicates it's indeed tmpfs there | 17:07 |
| fungi | approximately 4gb allowed | 17:08 |
| cardoe | that's a chunky /tmp usage | 17:08 |
| clarkb | the error is cloning openstack/cinder into /tmp | 17:22 |
| clarkb | fungi: is that a build isolation behavior to use /tmp? | 17:23 |
| clarkb | pip install is being run against the file:/// path to cinder and that apparently triggers a clone into /tmp | 17:24 |
| fungi | i believe pip follows $TMPDIR | 17:24 |
| fungi | and falls back to /tmp if unset | 17:24 |
| clarkb | yup google seems to confirm. If you pip install against a git repo it clones it to tmpdir | 17:25 |
| fungi | my guess is that the noble job comes in close to the limit but squeaks by, and the newer python on resolute is resulting in more packages built from sdist on the fly for deps that are pinned and don't have wheels for the newer python | 17:26 |
| clarkb | ya that makes sense. More build artifacts filling the "disk" | 17:27 |
| clarkb | fungi: also maybe resolute changed its tmpfs settings | 17:29 |
| clarkb | er /tmp settings. Is it tmpfs on the old release maybe not? | 17:29 |
| fungi | oh maybe, checking... | 17:29 |
| fungi | aha! good call https://zuul.opendev.org/t/openstack/build/899a486883b841d8bcd6d2a157986dd8/log/zuul-info/zuul-info.ubuntu-noble.txt#118-126 | 17:30 |
| fungi | no separate /tmp there | 17:30 |
| clarkb | tmpfs will swap but still enforce its configured limit which is 50% of the memory size | 17:30 |
| clarkb | so either we're at 4GB of /tmp and failing or we're running out of memory earlier than that and don't have swap configured (I don't think these jobs configure swap but I'm not positive of that) | 17:31 |
| fungi | so on noble i think the job is spooling those tempfiles onto the rootfs | 17:31 |
| clarkb | anyway an easy solution would be to make a tmpdir on the rootfs and have the jobs use that or do it test case by test case? | 17:32 |
| frickler | this failure looks a bit different from the neutron one, but I assume the proposed fix should work for it as well? https://zuul.opendev.org/t/openstack/build/8609121c6f3346408b9e337876bf3ff3 No module named 'setuptools.extern.tomli' | 17:37 |
| fungi | frickler: indeed that seems like it's assuming newer setuptools functionality | 17:56 |
| fungi | so probably the same fix | 17:57 |
| opendevreview | Takashi Kajinami proposed openstack/tooz master: Use etcd from distribution https://review.opendev.org/c/openstack/tooz/+/997919 | 18:10 |
| opendevreview | Takashi Kajinami proposed openstack/oslo.db master: Do not install code to build release notes https://review.opendev.org/c/openstack/oslo.db/+/1002180 | 18:15 |
| mnasiadka | Ok, so what’s the plan? pbr 7.1.1 goes to /dev/null? | 20:20 |
| fungi | looks like pbr-installation-openstack-pip-dev-resolute is also intermittently failing on the same /tmp problem as pbr-installation-openstack-resolute | 20:38 |
| fungi | mnasiadka: we are fairly confident that once 1002140 merges and we tag 7.1.2 anything that's currently unable to use 7.1.1 will start working, and since it's only impacting older stable branches the urgency seems insufficient to warrant yanking 7.1.1 from pypi for a little temporary relief there | 20:39 |
| mnasiadka | fungi: makes sense, ack | 20:46 |
| fungi | basically in situations with too-new pip and too-old setuptools, pbr 7.1.1 goes sideways. if you have old pip+setuptools or new pip+setuptools then pbr 7.1.1 works as intended | 20:50 |
| * cardoe stares at broken neutron testing with sadness | 21:07 | |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!