| *** cschwede_afk is now known as cschwede | 06:52 | |
| *** elodilles_pto is now known as elodilles | 08:49 | |
| opendevreview | Stephen Finucane proposed openstack/pbr master: Drop Python < 3.11 https://review.opendev.org/c/openstack/pbr/+/988753 | 11:32 |
|---|---|---|
| opendevreview | Stephen Finucane proposed openstack/pbr master: Drop Python < 3.11 https://review.opendev.org/c/openstack/pbr/+/988753 | 11:34 |
| stephenfin | fungi: clarkb: I think it's time ☝️ | 11:37 |
| *** haleyb is now known as haleyb|away | 12:38 | |
| mnasiadka | So, observing some weird behaviour with reno jobs in kolla and kolla-ansible - if we merge a patch on stable branch (e.g. 2026.1) - the releasenotes generated on that branch are promoted on https://docs.openstack.org/releasenotes/kolla-ansible/ | 13:38 |
| mnasiadka | So unless the tox releasenotes environment in these project is somewhat different than others or something similar - I have no clue what’s wrong :) | 13:39 |
| mnasiadka | When the build+promote reno jobs run from master - everything is fine | 13:40 |
| fungi | mnasiadka: that was https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/966618 for reference, and yeah it was previously parented to build-reno-releasenotes instead of openstack-tox-docs so i think it's missing the override from https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul.d/python-jobs.yaml#L636 | 13:49 |
| opendevreview | Michal Nasiadka proposed openstack/openstack-zuul-jobs master: Fix releasenotes build for stable branches https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/988768 | 13:50 |
| fungi | stephenfin: ^ was that an oversight or an intentional omission? | 13:50 |
| stephenfin | Oversight. apologies | 13:50 |
| fungi | no worries, just making sure there wasn't a particular reason it was left out | 13:51 |
| mnasiadka | stephenfin: no worries, you made me track the whole reno ,,workflow’’ - so I learned something today ;) | 13:54 |
| mnasiadka | So now that reno translation dance that is/was part of https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/build-releasenotes/tasks/main.yaml needs to be done locally in projects? (Not that Kolla ever had any translations) | 13:55 |
| opendevreview | Mauricio Harley proposed openstack/project-config master: Add #openstack-pqc IRC channel https://review.opendev.org/c/openstack/project-config/+/988770 | 13:58 |
| opendevreview | Merged openstack/openstack-zuul-jobs master: Fix releasenotes build for stable branches https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/988768 | 14:03 |
| *** gibi is now known as gibi_off | 14:46 | |
| clarkb | stephenfin: I think when we drop python2 we shouldn't drop any python3 releases | 15:11 |
| clarkb | we should limit the blast area and move slowly | 15:11 |
| stephenfin | I'm okay to do that, but is it necessary? Can this reasonably break in Python 3? | 15:18 |
| clarkb | stephenfin: it literally did in the PBR test jobs | 15:18 |
| clarkb | I was -1 until I saw that and then chagned to a -2 with some extra explanation | 15:19 |
| stephenfin | Oh 😄 | 15:19 |
| * stephenfin looks | 15:19 | |
| clarkb | historically any time we've tried to make these changes we've also found unexpected corner cases. So in general I think moving slowly to make potential reverts easier and make it more clear what specific part of a change is a problem is a good idea | 15:21 |
| clarkb | I don't think there is a rush here. We've waited for many years as is :) | 15:21 |
| clarkb | I do agree that python2 can probably go at this point | 15:21 |
| stephenfin | Okay, my intention was actually to drop < 3.10 (see the second-last paragraph in the commit message) but the commit summary `python_requires` doesn't reflect that intention | 15:22 |
| stephenfin | More generally though, that's failing "as expected". 1) pip tries install $thing 2) thing indicates it doesn't support the target Python version 3) pip dies. That's what we'd expect to happen | 15:23 |
| fungi | for comparison, i have this note bubbling up closer to the top of my to-do list: "drop python 3.8 from bindep" | 15:23 |
| clarkb | stephenfin: no that isn't the expectation. The expectation is that pip will install an older version of the package with compatibility and nothing will break | 15:23 |
| stephenfin | it was my understanding that the reluctance to drop older version previously was because that python_requires logic wasn't always respected for buildtime (as opposed to runtime) deps | 15:23 |
| clarkb | stephenfin: my inline comment about python_requires and setup_requires not working together is why things break anyway | 15:23 |
| clarkb | stephenfin: correct the buildtime will always install latest. Which doesn't have compatibility then you break | 15:24 |
| clarkb | the expectation from people using python_requires is that you'll install a compatible version in the first place and everything will work | 15:24 |
| stephenfin | is that setup_requires thing still true? | 15:24 |
| clarkb | that doesn't happen here | 15:24 |
| clarkb | stephenfin: I think so. Setuptools does it with an easy_install shim/vendor script iirc and easy_install never added support for it | 15:25 |
| fungi | git-review still supports python 3.8 at the moment too, we could probably switch it to 3.9 and later soon | 15:25 |
| stephenfin | Hmm, I believe all of that is gone from setuptools with some time now, but I'm not sure about older versions tbf | 15:26 |
| clarkb | from my perspective we gain almost nothing dropping python3.8 support for example. But we potentially create massive headaches by removing the support | 15:26 |
| clarkb | I would much rather move slowly given that. If there was a clear win/improvement then I'd probably see it differently but its almost all potential downside | 15:26 |
| stephenfin | Fair. Though personally, I'd like to know it's actually needed rather than assuming that's the case | 15:28 |
| stephenfin | Let me drop the lower bound back down for now while I figure it out | 15:28 |
| fungi | "for some time now" tends to be a very compressed sentiment in the python packaging community, compared to the distro packaged pip ans setuptools versions still in use in supported lts ubuntu, for example | 15:28 |
| clarkb | looks like setuptools/installer.py:_fetch_build_eggs() does the setup_requires fetching | 15:29 |
| clarkb | (so it isn't an easy install vendored code anymore) | 15:30 |
| clarkb | and it does its own parsing of pip links in there. I can't quickly tell if it is filtering python requires | 15:31 |
| clarkb | stephenfin: re knowing its actually needed the problem is that linux distros live for 10 years or so | 15:32 |
| clarkb | we could go and do a ton of work to try and test those platforms and keep track of them all. Or we could do almost zero work and keep the status quo which isn't harming anything | 15:33 |
| opendevreview | Stephen Finucane proposed openstack/pbr master: Drop Python < 3.8 https://review.opendev.org/c/openstack/pbr/+/988753 | 15:34 |
| clarkb | _fetch_build_egg_no_warn does remove env markers | 15:35 |
| clarkb | so you can't control the package version based on python version within the requirement itself | 15:35 |
| stephenfin | fungi: I'll just reiterate that I'm not proposing removing stuff "just because". I was proposing on the assumption that it was simply not necessary anymore... | 15:40 |
| clarkb | stephenfin: fwiw I had secretly hoped that with pyproject.toml being able to control these things much better we'd be able to sort of leave pbr in a more legacy state as people transition to pyproject.toml and potentially use other build tools or maybe take that opportunity to have a PBRnextgen or something as the dep. But there hasn't been a lot of coordination around that | 15:44 |
| clarkb | despite some discussion. It is unfortunate that many of the ideas first encoded in PBR were rejected for years prolonging this transition upstream | 15:44 |
| clarkb | anyway I think that newer patchset looks a lot better. I'll take a second look when there are test results | 15:45 |
| stephenfin | Looking at setuptools/__init__.py, and tracing the path down (_install_setup_requires -> _fetch_build_eggs, (setuptools/dist.py) fetch_build_eggs, (setuptools/installer.py) _fetch_build_eggs -> _fetch_build_egg_no_warn) it seems setuptools HEAD uses pip to install, which presumably means we will get requires_python support implicitly | 15:45 |
| clarkb | stephenfin: it does use pip but it supplies a requierment url to pip that it predetermines without pip | 15:45 |
| stephenfin | Would need to figure out how long that's been the case though, and how the versions correspond to the distro versions | 15:45 |
| stephenfin | ah, indeed | 15:46 |
| clarkb | stephenfin: the part that isn't clear to me is if that url selection honors python_requires. But if you feed pip a specific package url and it sees a mismatch python_requires you'll fail | 15:46 |
| fungi | ubuntu 22.04 lts (still under official support) ships setuptools 59.6.0, as a point of reference | 15:46 |
| clarkb | and sometimes it doesn't use a url it just uses a req name and I'm not sure when itwill do that (maybe it alwys does that?) | 15:47 |
| stephenfin | clarkb: Yeah, I suspect that should likely be an eventual goal. I don't know if there's much of pbr's functionality that we still care about and can't achieve with other installers nowadays | 15:48 |
| clarkb | stephenfin: there are a coupel of things, but they seem solvable (the git sha info and release team relying on the version tags in commit messages come to mind) | 15:48 |
| fungi | the semver trailers, yeah | 15:52 |
| fungi | also pbr is a lot better at making sure everything gets into the package manifest, setuptools is hard to convince to include some things | 16:02 |
| clarkb | fungi: in theory the MANIFEST.in file should win if it comes to that though? | 16:02 |
| clarkb | (I agree pbr makes it easy, but it still solvable) | 16:03 |
| fungi | yes, getting away from hand-managing manifests was one of the reasons pbr came into existence | 16:03 |
| fungi | (nee oslo.packaging, nee bits and bobs in openstack/common) | 16:03 |
| fungi | i just feel like the approach of "well projects can work around lack of pbr by doing this, that and the other" takes us right back to the reason we made pbr to begin with | 16:05 |
| clarkb | to a certain degreee yes. I always felt the primary purpose of pbr was to make pacakging config driven rather than requiring bespoke programming for each package. The other features are nice to haves (and some of them are very nice) | 16:06 |
| clarkb | in theory setuptools-scm gets the file inclusion problem solved like pbr does though | 16:06 |
| clarkb | so we don't have to fully give up on the features either | 16:06 |
| fungi | a primary reason was yes, but at a more fundamental level pbr was "the set of things openstack needs which aren't addressed elsewhere in python packaging" so that it could be applied consistently across all openstack projects/packages rather than scattershot approaches with inconsistencies and variations everywhere | 16:08 |
| fungi | pbr has also historically allowed us to helpfully paper over a ton of backward-incompatible changes in python packaging, for example when setuptools would just arbitrarily decide to rename metadata keys without any clear upgrade path | 16:10 |
| fungi | i do think dropping more and more redundant features from pbr over time makes sense, but having a means to provide consistency and continuity in packaging across all of openstack remains useful, i think | 16:12 |
| fungi | if we don't do that with pbr, we'll need to do it with *something* | 16:13 |
| clarkb | yup I think that dropping pbr is only possible now because pyproject.toml gives you strong control over the package tooling that didn't exist outside of pbr previously | 16:13 |
| clarkb | which in theory addresses backward incompatibility concerns | 16:14 |
| fungi | but also gives you a lot of "flexibility" that different projects are applying in different ways, leading to new sources of inconsistency | 16:14 |
| clarkb | yup now we don't just have setuptools we have setuptools and like 20 other systems | 16:14 |
| fungi | i mean just within openstack... setup.py was extremely flexible and that was a problem, we solved that by making a setuptools plugin and essentially stripping everyone's setup.py down to a few lines. pyproject.toml is the new setup.py and projects are just putting whatever in there | 16:16 |
| clarkb | but it would be easier to address that with linters/checks because its a datastructure not a programming language | 16:17 |
| fungi | yep, today we don't have those linters and checks implemented anywhere i'm aware of though | 16:17 |
| clarkb | correct | 16:18 |
| clarkb | beacuse openstack response to pyproject.toml in general has been "meh" | 16:18 |
| clarkb | which means we're still maintaining PBR and we should be careful with our choices there | 16:18 |
| fungi | but also, just like setuptools used to make backward-incompatible metadata changes with no continuity plan, the metadata expectations in pyproject.toml are seeing a lot of churn and we basically don't have any similar mechanism to apply our own continuity layer | 16:20 |
| fungi | case in point, the license metadata debacle, where if you need to support a certain range of python interpreter versions you literally can't use the license metadata fields | 16:21 |
| fungi | because the versions of setuptools that work with those versions of python have incompatible expectations for how the data in that field should be shaped | 16:22 |
| clarkb | you're still able to use the system built into say setuptools though right? | 16:22 |
| clarkb | rather than relying on pyproject.toml for that info definition | 16:22 |
| fungi | yes, if you don't put your package metadata in pyproject.toml | 16:23 |
| fungi | or don't have a pyproject.toml file at all, maybe (pip and setuptools make certain assumptions based on the presence/absence of that file) | 16:23 |
| -opendevstatus- NOTICE: The Gerrit service on review.opendev.org will be offline momentarily while we restart it onto a new patch release | 18:03 | |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!