| *** tkajinam is now known as Guest2162 | 02:04 | |
| *** tkajinam is now known as Guest2174 | 03:28 | |
| *** pojadhav- is now known as pojadhav | 05:03 | |
| *** tkajinam is now known as Guest2179 | 05:36 | |
| *** pojadhav- is now known as pojadhav | 05:57 | |
| *** dasm|off is now known as dasm | 13:55 | |
| *** frenzy_friday is now known as frenzy_friday|doc | 14:43 | |
| *** blarnath is now known as d34dh0r53 | 14:53 | |
| rosmaita | gmann: i know we cancelled tomorrow's TC meeting, but will you personally be around at that time? | 15:01 |
|---|---|---|
| *** pojadhav is now known as pojadhav|afk | 15:32 | |
| noonedeadpunk | fwiw, we've got issues with pbr and new setuptools that were released on Saturday. Not sure if it was spotted yet, but `python setup.py --version` now returns not only version, but `[pbr] Generating ChangeLog` as the line before version. | 16:18 |
| noonedeadpunk | setuptools used to be constrained in u-c for X and partially for Y but not on Z anymore. | 16:18 |
| fungi | oh fun, you might consider raising that in #openstack-oslo | 16:19 |
| noonedeadpunk | I will ask releases team about what they think about it... | 16:19 |
| noonedeadpunk | oh, should it be oslo? ok | 16:19 |
| fungi | well, pbr is an oslo deliverable. not sure what you wanted to ask the release team though | 16:19 |
| noonedeadpunk | As tbh I'd love to get setuptools constrained... | 16:20 |
| noonedeadpunk | But I think I'm going same road to nowhere with that | 16:20 |
| fungi | that's harder than it sounds, since it gets automatically installed by pip before version constraints can be applied in many cases | 16:20 |
| fungi | and pip isn't designed to automatically upgrade/downgrade existing setuptools versions while installing packages that need setuptools either | 16:21 |
| fungi | but also, calling setup.py directly is entirely deprecated in setuptools since several versions now | 16:21 |
| fungi | (use of setup.py isn't deprecated, but running it as a command is) | 16:22 |
| *** dasm is now known as dasm|off | 16:23 | |
| fungi | also, recent versions of pip install setuptools to a separate isolated "build" environment, which means you may end up with a different version of setuptools used for installing than the version of setuptools which may or may not end up installed in your actual target environment | 16:23 |
| noonedeadpunk | eventually it's true that nobody should run setup.py like that.... | 16:24 |
| fungi | this also gets increasingly complex if you use setuptools as a pyproject.toml build-system.build-backend | 16:24 |
| noonedeadpunk | I think that pip can be leveraged same way | 16:24 |
| fungi | what's running `python setup.py --version` in this case? is it something pbr is doing itself, or some external process running that? | 16:25 |
| fungi | note that if pbr is installed in the environment, it has its own cli entrypoint too | 16:26 |
| noonedeadpunk | ok, I didn't know about "build" environment - so if you run pip install setuptools==65.5.1 it won't result in using these setuptools in venv? | 16:26 |
| fungi | if you directly `pip install setuptools==65.5.1` in a venv, and then you `pip install somethingelse` in that same venv, the version of setuptools used to install somethingelse (if it uses setuptools at all) may be a completely different one ephemerally installed to a temporary isolated build env | 16:27 |
| noonedeadpunk | likely I should just fix way of finding version instead | 16:27 |
| noonedeadpunk | ok, I totally missed that part ^ | 16:28 |
| noonedeadpunk | thanks, I will not bug anyone anymore :) | 16:28 |
| fungi | yeah, it is no longer assumed that all python packages will rely on a common setuptools version, so each package may call a different version of setuptools at installation (or hatch, or flit, or various other pep 517 compliant build backends) | 16:29 |
| noonedeadpunk | good to know, thanks a lot! | 16:30 |
| clarkb | fwiw pyproject.toml is the only reliable way for an individual project to control what setuptools version it will install with | 16:30 |
| fungi | noonedeadpunk: if you can point to where setup.py --version is being called, i might be able to suggest alternative approaches | 16:31 |
| clarkb | if you `pip install setuptools==x.y.x foo` foo isn't installed with setuptools x.y.z they are installed side by side using whatever setuptools already exists | 16:31 |
| fungi | clarkb: but also with pep 517 a totally different version of setuptools may be installed in a temporary isolated build env for each package, so "already exists" is not how i would think of it | 16:32 |
| fungi | you can in fact have venvs without any setuptools installed in them at all even though the packages installed into them used setuptools to install | 16:32 |
| clarkb | fungi: right I mean for projects likle openstack without pyproject.toml | 16:32 |
| clarkb | if you are using "legacy" setuptools then you cannot control setuptools in any way | 16:32 |
| clarkb | this is why I have continuously encouraged openstack to stop trying to constrain setuptools | 16:33 |
| clarkb | its near useless | 16:33 |
| noonedeadpunk | yeah, so I though of it as - I get setuptools installed in venv and then installing like keystone and assuming it will use setuptools from venv | 16:33 |
| clarkb | if instead openstack wishes to have control over setuptools in this way the only reliable method is via prproject.toml | 16:33 |
| noonedeadpunk | But I got that this assumption is wrong as of today | 16:33 |
| noonedeadpunk | Ok, yes, now I get quite clear picture I beleive and it makes sense now | 16:34 |
| fungi | noonedeadpunk: it may work for the exact case you specified, but it depends a lot on how the project is packaged and is likely to change dramatically in the future so you should consider setup.py as an implementation detail and not a stable api | 16:34 |
| noonedeadpunk | yeah, it's better not to do that and more importantly not to rely on that | 16:35 |
| clarkb | I think pbr had a version command ? | 16:35 |
| clarkb | there is pbr freeze and I think a couple of other related utilities to gather that sort of info. That might be mor ereliable? | 16:35 |
| noonedeadpunk | I'm jsut not sure that pbr is installed at the point when it's executed | 16:35 |
| noonedeadpunk | but it's different question I guess | 16:36 |
| noonedeadpunk | well, that used to be reliable for 5 years until this saturday :) | 16:37 |
| noonedeadpunk | anyway | 16:37 |
| fungi | noonedeadpunk: yeah, that's why i asked for a pointer to where it's being used. depending on what information you're trying to get for what from where exactly, the alternative solutions may differ | 16:37 |
| clarkb | noonedeadpunk: while I agree that python packaging needed a major overhaul I'm not sure I agree with upstream in leaving existing tooling behind so aggressively. But ya thats the path they've chosen. And honestly openstack might need to investigate switching to it. The new tooling can do a fair bit of what PBR offers but not all of it. Investigation would be needed to determine | 16:40 |
| clarkb | if that is a problem or not | 16:40 |
| clarkb | if it is a problem more effort in updating PBR to accomodate the new stuff is a good idea. fungi and I have poked at it here and there, but I don't have the time to devote to proprely updating pbr to the new world or pypa expectations | 16:40 |
| fungi | clarkb: a solution i use which doesn't rely on pbr being installed at runtime is to use importlib.metadata (or pkg_resources with python 3.8 and earlier) to get the base version and also to read the pbr.json file as metadata if it exists so i can append git_version info when available | 16:41 |
| fungi | also allows me to simulate the effects of pbr freeze for purposes of listing the versions of all installed packages, and further breaking them down into categories for the calling project, its direct dependencies, and the indirect dependencies they've pulled in | 16:44 |
| noonedeadpunk | fungi: actually yes, I used that method quite a lot to determine path to libraries | 16:44 |
| clarkb | the problem with pkg_resources and importlib is they are the number one performance killer for python command line tools | 16:45 |
| fungi | and i'm doing that in a project which relies on pbr's pyproject.toml build-backend support, so it seems to be a forward-compatible approach | 16:45 |
| clarkb | its fine for long lived services but for example it completely kills osc performance | 16:45 |
| fungi | clarkb: right, i only call those routines *if* the version is requested | 16:45 |
| fungi | so it does make any calls that want to report the version slower (depending on the size of the venv), but as long as you avoid looking up the version every time you invoke a cli for example, it's fine | 16:46 |
| fungi | the place where that's harder to solve is if you're truing to use pkg_resources/importlib for entrypoint discovery, since you likely do need to do that on every invocation (unless you can cache the lookup reliably between runs) | 16:47 |
| clarkb | ya thats the main bit that kills osc | 16:48 |
| clarkb | I'm just completely allergic to the tools now as a result | 16:48 |
| fungi | anyway, this has probably veered waaaay off-topic for the tc channel, mostly my fault, sorry! | 16:48 |
| noonedeadpunk | So basically when I need that - is once during bootstrap to define version we're running in vars file | 16:48 |
| clarkb | they were written with no consideration for performance then hidden in central python package utilities and everyone wonders why python is slow | 16:48 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/bootstrap-ansible.sh#L159 | 16:48 |
| noonedeadpunk | and yeah, pbr is not installed at that point | 16:49 |
| noonedeadpunk | I don't think it installed at all tbh | 16:50 |
| clarkb | noonedeadpunk: it will install as soon as yo uinstall any openstack repos | 16:50 |
| clarkb | since they all depend on it as a setup requires and many as an actual dependency | 16:51 |
| noonedeadpunk | not in that venv | 16:51 |
| clarkb | when you run setup.py --version it does | 16:51 |
| fungi | it will if they include pbr as an install_requires (not if pbr is only listed as a setup_requires) | 16:51 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible/src/branch/master/setup.py#L19 | 16:52 |
| fungi | but also the project isn't being installed in that case, from what i can see | 16:52 |
| fungi | it's running a specific venv's python interpreter to call setup.py from a git workdir for something which has presumably not been pip installed into that venv | 16:53 |
| noonedeadpunk | I guess project is https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/scripts-library.sh#L84 | 16:53 |
| noonedeadpunk | also my sandbox does have project among pip info | 16:54 |
| noonedeadpunk | s/info/freeze/ | 16:54 |
| noonedeadpunk | I think the only reason setup.py --version is used to save ourselves from parcing | 16:54 |
| noonedeadpunk | was super simple kind of... | 16:55 |
| *** frenzy_friday|doc is now known as frenzy_friday | 17:36 | |
| gmann | rosmaita: yes, I will be around tomorrow | 18:00 |
| *** dasm|off is now known as dasm | 20:31 | |
| *** dasm is now known as dasm|off | 22:25 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!