| opendevreview | Merged openstack/tempest master: Switch Neutron RBAC flag in testing https://review.opendev.org/c/openstack/tempest/+/899310 | 09:25 |
|---|---|---|
| elodilles | hi QA team, i have a quick qestion about generating doc from docstrings. Is there an easy way to generate them for *-tempest-plugins along with tempest docstrings (for example in tempest: https://docs.openstack.org/tempest/latest/tests/network/network.admin.html ) | 13:01 |
| elodilles | there might be a description about how to do it, but i haven't found it yet | 13:01 |
| frickler | clarkb: fungi: I think we should be fine with proceeding with 816741 to clear a bit of the path for sean-k-mooney's keystone change? | 14:51 |
| frickler | also, kind of unrelated, but do we want jammy and debian testing for bindep? | 14:52 |
| clarkb | ya I think 816741 is fine as is adding more testing. FWIW I discovered this https://gregoryszorc.com/blog/2023/10/30/my-user-experience-porting-off-setup.py/ and that says that new pip uses a default pyproject.toml definition if you don't supply one | 14:54 |
| clarkb | for this raeson I think we actually do need to set a pyproject.toml to avoid any improper default behavior (and this is likely related to what sean-k-mooney debugged as well since it won't use the pbr shim) | 14:55 |
| sean-k-mooney | i jsut started an email thread on this by the way | 14:59 |
| sean-k-mooney | well the pep517/pip23.1 suuport | 14:59 |
| sean-k-mooney | frickler: clarkb the delta between what i did in the keystoen patch and the bindep patch | 15:00 |
| sean-k-mooney | is that i did not remove setup.py functionality | 15:00 |
| sean-k-mooney | i just added pyproject.toml | 15:00 |
| sean-k-mooney | so that we would not break any exsiting packagers | 15:00 |
| clarkb | sean-k-mooney: ya all that bindep change did was chmod away the +x | 15:00 |
| clarkb | so shouldn't be a huge impact to pacakgers, but agreed making minimal updates is a good idea | 15:00 |
| sean-k-mooney | my proposal would be to deprecate teh setup.py funcitonal in caracal and perhaps remove it in the next release | 15:01 |
| sean-k-mooney | clarkb: ill add that block to my reading list | 15:01 |
| sean-k-mooney | *blog | 15:02 |
| clarkb | sean-k-mooney: there is a lot in the blog but good info that overlaps here. I think fundamentally the pip use of a specific default pyproject.toml if you don't provide one is the underlying source of the issue here (since that forks pip and setuptools behavior off in a different direction) | 15:02 |
| sean-k-mooney | clarkb: ya so it almost works | 15:03 |
| sean-k-mooney | clarkb: the issue is that we use -e in devstack and that breaks the integration | 15:03 |
| sean-k-mooney | if we use there default the wsgi script are generate when we dont use -w | 15:04 |
| sean-k-mooney | *-e | 15:04 |
| sean-k-mooney | but being explcit instead of relying on the fallback and jsut saying "build with pbr" is likely the better approch long term | 15:04 |
| sean-k-mooney | assumign we stay with pbr | 15:04 |
| JayF | clarkb: looks like they are using PEP517 support to install (in this example, diskimage-builder), but are using it via *setuptools* | 15:05 |
| clarkb | yes I think being explicit here is a good idea | 15:05 |
| sean-k-mooney | our use of wsgi_script which is a pbr extenion based on the setuptools console_scripts pattern is what broke | 15:05 |
| JayF | clarkb: so they invoke setuptools legacy support via PEP517, using setup.py and getting into PBR that way | 15:05 |
| clarkb | JayF: ya so they are doing the pip default pyproject.toml path essentially | 15:05 |
| JayF | clarkb: there's a little bit of hand-waviness in there because I didn't want to dig further, but that's essentially the behavior :) | 15:06 |
| sean-k-mooney | in pip 23.1 | 15:06 |
| sean-k-mooney | the invocation of setup.py was removed | 15:06 |
| sean-k-mooney | JayF: so that is what is breaking now | 15:06 |
| JayF | Ah, I bet that's masked in gentoo because that probably breaks the world, not just openstack-y things | 15:06 |
| sean-k-mooney | JayF: now they are building a temp wheel without using setup.py | 15:06 |
| JayF | heh, nope | 15:07 |
| * JayF goes to validate the DIB ebuild is broken in gentoo | 15:07 | |
| JayF | it works; but it's obvious it's ignoring the setup.py based on the large number of QA warnings about files installed not listed in packages | 15:10 |
| JayF | (but those files are still installed) | 15:10 |
| sean-k-mooney | JayF: right it "works" almost good enough provided your not using a pbr extention like wsgi_scripts | 15:14 |
| sean-k-mooney | im not seeting this on the arcive yet but i sent a mail with subject "[all][packaging][release][qa] pep-517 and pip 23" | 15:15 |
| dansmith | sean-k-mooney: so it dropped setuptools but not setup.py entirely right? | 15:15 |
| dansmith | (by default) | 15:15 |
| dansmith | seems like distutils.core.setup still gets called by default | 15:16 |
| sean-k-mooney | other way around. the deprecated using setup.py install 2 years ago in pip 20.2 if pyporject.toml does not exsit | 15:16 |
| dansmith | I know that might not be relevant to us, I'm just trying to make sure I know what all has changed | 15:16 |
| sean-k-mooney | dansmith: they still use setuptool by defautl if its not found i belive just not setup.py install | 15:16 |
| dansmith | sean-k-mooney: okay I'm just testing an older package that doesn't have the toml but it still seems to be working | 15:16 |
| sean-k-mooney | instead of building an egg they now build a wheel | 15:16 |
| dansmith | hmm | 15:17 |
| sean-k-mooney | dansmith: it will still use setuptool in that acase and setup.cfg | 15:17 |
| sean-k-mooney | just not setup.py so it does not enable pbr | 15:17 |
| sean-k-mooney | so anything that is natively uspported in setuptool like console_script still works as those stevador metadata | 15:17 |
| dansmith | okay | 15:18 |
| sean-k-mooney | and some fo pbr generate is working as part of builidng the wheel | 15:18 |
| sean-k-mooney | dansmith: i have a repoducer in https://etherpad.opendev.org/p/pep-517-and-pip-23#L8 | 15:19 |
| sean-k-mooney | dansmith: you can downgrade pip before 23.1 in a seperate env | 15:19 |
| sean-k-mooney | and compare the behavior | 15:19 |
| dansmith | sean-k-mooney: yeah I'm most just trying to grasp what all is changing (this time) | 15:20 |
| sean-k-mooney | the problem this time is we have been ignoring the deprecation and they finally removed the fallback | 15:21 |
| sean-k-mooney | i think the deprecation only printed in verbose mode | 15:21 |
| sean-k-mooney | so we never saw the warning | 15:21 |
| sean-k-mooney | im going to propos patchs for nova/placment and i may do the "core" proejcts quickly | 15:21 |
| dansmith | I understand that, I just don't understand the difference between ignoring setup.py entirely and not, because it seems like it sometimes still "runs" setup.py | 15:21 |
| dansmith | I just need to dig into it after this call | 15:22 |
| dansmith | I've mostly been ignoring it myself | 15:22 |
| sean-k-mooney | ack | 15:22 |
| clarkb | dansmith: sean-k-mooney: I think the distinction is python setup.py vs some tool importing setup.py and running it | 15:22 |
| sean-k-mooney | ill admit i dont fully undersand this eitehr how i figured this out is i started adding logs to pbr | 15:22 |
| sean-k-mooney | and noticed that part of if was nto being called any more | 15:22 |
| clarkb | setup.py is still used when you use setuptools. But it will never be invoked as python setup.py something with modern tools | 15:22 |
| clarkb | and the import path is enough to make pbr work for most stuff but apparently not the wsgi scripts | 15:23 |
| dansmith | clarkb: okay I thought to continue getting that fallback you needed a toml that says that's your build backend | 15:23 |
| sean-k-mooney | clarkb: i think its invoked as "setup.py build" or "setup.py wheel" | 15:23 |
| dansmith | clarkb: but I'm playing with it locally and I don't have that in place and yet I'm still getting the automatic fallback | 15:23 |
| sean-k-mooney | or something like that | 15:23 |
| dansmith | so I'm trying to figure out what the actual change and requirement is | 15:23 |
| dansmith | sean-k-mooney: so only "setup.py install" is removed? | 15:24 |
| clarkb | dansmith: pip specifically has a default pyproject.toml it uses if you don't define one according to the blog linked above | 15:24 |
| dansmith | clarkb: okay I'll read | 15:24 |
| sean-k-mooney | so what is actully breaking us is this code https://github.com/openstack/pbr/blob/d03d617c09e7ba8ddf62d1e53d71685cd708e2da/pbr/packaging.py#L458-L514 is not never invoked | 15:25 |
| sean-k-mooney | because it never does a local install via setup tools now | 15:25 |
| dansmith | okay so as long as a package builds properly into a wheel with build then I guess everything is fine for the moment? | 15:27 |
| sean-k-mooney | so i think "based on skiming that blog" its now doing "python3.12 -m build --wheel ." | 15:27 |
| dansmith | like they build the wheel and then install the wheel with pip instead of letting setup.py do the install? | 15:27 |
| sean-k-mooney | dansmith: yes | 15:27 |
| dansmith | ack | 15:28 |
| sean-k-mooney | and what broken is building an editable wheel | 15:28 |
| sean-k-mooney | does not include the wsgi scripts | 15:28 |
| sean-k-mooney | if you build a non editiabel wheel it does include them | 15:28 |
| sean-k-mooney | im not sure why that is diffent and we might eb able to fix that in pbr | 15:28 |
| sean-k-mooney | that was what i was orginally goign to do | 15:28 |
| sean-k-mooney | but i dont understand how pbr works well enough to do that | 15:29 |
| dansmith | ack | 15:29 |
| sean-k-mooney | by the way im not sure if the issue is in the wheel build or in the pip install | 15:30 |
| sean-k-mooney | i.e. im not sure how the -e flag is applied (in the build phase or in the use of the wheel phase or both) | 15:30 |
| dansmith | I guess I assume no wheel can be installed editable | 15:31 |
| sean-k-mooney | if you read the output you see this https://paste.opendev.org/show/biPIqW8kz1b4aGNRXwPU/ | 15:34 |
| sean-k-mooney | Building editable for keystone (pyproject.toml) ... done | 15:34 |
| sean-k-mooney | that with master (without my change) | 15:34 |
| sean-k-mooney | using the default pyporject.toml form pip i assume | 15:34 |
| sean-k-mooney | dansmith: -e is not a flago for build "python3 -m build --wheel -e ." is invlaid | 15:37 |
| sean-k-mooney | and without -e the wheel has the wsgi files | 15:37 |
| dansmith | sean-k-mooney: yeah I just figured that install wouldn't do an editable for a wheel | 15:38 |
| sean-k-mooney | it is an accepted arg for "python -m pip wheel -e ." | 15:39 |
| sean-k-mooney | that will polute your repo... | 15:40 |
| sean-k-mooney | and the wheel still has the correct content when invoked that way | 15:42 |
| sean-k-mooney | so the issue is in how pip is installing the scripts | 15:43 |
| sean-k-mooney | we could file a bug with pip for this but i dont know if they would fix it | 15:44 |
| clarkb | gmann: what is the plan for flipping python3.11 jobs to voting in caracal? | 18:21 |
| gmann | clarkb: I think we did. elt me check my change is merged or not | 18:25 |
| clarkb | oh yup looks like that did happen at least for openstacksdk | 18:33 |
| gmann | clarkb: it is merged, https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/891238 | 18:34 |
| gmann | and it is running voting https://zuul.openstack.org/builds?job_name=openstack-tox-py311&skip=0&limit=100 | 18:34 |
| gmann | oh did nto check for sdk. let me check if they are not using this template | 18:35 |
| clarkb | thanks. I'm regenerating results on https://review.opendev.org/c/openstack/python-openstackclient/+/898129 and then that can be approved when it ocmes back voting | 18:35 |
| clarkb | gmann: ^ is the actual chnage I'm concerned about and zuul will tell us momentarily | 18:35 |
| clarkb | yup looks like it is voting in the status page | 18:35 |
| gmann | cool. template used is all correct https://github.com/openstack/python-openstackclient/blob/master/.zuul.yaml#L216 | 18:36 |
| frickler | clarkb: do you want to update the commit message accordingly? | 18:41 |
| clarkb | yup can do now that we know its set | 18:41 |
| clarkb | done | 18:44 |
| *** elodilles is now known as elodilles_pto | 20:29 | |
| -opendevstatus- NOTICE: Gerrit on review.opendev.org will be restarted to pick up a configuration change required as part of Gerrit 3.8 upgrade preparations. | 22:02 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!