| opendevreview | Michael Johnson proposed openstack/tempest master: Fix "pause_teardown" help text for docutils https://review.opendev.org/c/openstack/tempest/+/934194 | 00:14 |
|---|---|---|
| opendevreview | Merged openstack/tempest master: Check for glance service for glance tests https://review.opendev.org/c/openstack/tempest/+/934168 | 00:37 |
| *** mhen_ is now known as mhen | 02:42 | |
| mnasiadka | Hello, since https://review.opendev.org/c/openstack/tempest/+/933716 landed - magnum-tempest-plugin 2023.1 jobs are failing, because I'm pretty sure we never used stable branch upper-constraints there (since we depend on kubernetes>=26 for a long time) - and it fails with installing kubernetes pip package - any guidance how to solve that in the best way? | 10:11 |
| mnasiadka | I guess we can delete 2023.1 jobs, since it's going unmaintained... | 10:11 |
| kopecmartin | mnasiadka: hi, yep, that's what we did in master tempest, we dropped 2023.1 jobs - https://review.opendev.org/c/openstack/tempest/+/933771/2 | 14:39 |
| kopecmartin | gmann: i think i found an issue with installing tempest plugins in CI jobs .. devstack currently installs them into venv-tempest | 14:40 |
| kopecmartin | https://opendev.org/openstack/devstack/src/commit/b61b567c2d5c4ebec57ccf870f6c767c292ea847/lib/tempest#L855 | 14:40 |
| kopecmartin | however, that venv doesn't have "tempest run" command there , it runs only unit tests | 14:40 |
| kopecmartin | https://opendev.org/openstack/tempest/src/commit/906b6b85a871288722cf0d670fdda0e7e55ba8c1/tox.ini#L47-L48 | 14:40 |
| kopecmartin | in jobs, we used "all" environment (we set the tox_envlist to "all") which strangely worked until now .. lately i see failures with tempest run stating tempest can't find the tests "the specified regex doens't maatch anything" | 14:42 |
| kopecmartin | which makes sense, because if we use "all" or any other than "venv-tempest" env, we either create a new env (without the installed plugins) or recreate the venv we have and that leads to loosing the installed plugins | 14:42 |
| kopecmartin | i initially thought i could edit the venv-tempest (that inherits from testenv one and thus runs only unit tests) the way it would also run tempest tests (similar to "all" env .. however, that could lead to running tempest unnecessary | 14:44 |
| kopecmartin | plus we're passing additional args like "pip install" to the tox - https://opendev.org/openstack/devstack/src/commit/b61b567c2d5c4ebec57ccf870f6c767c292ea847/lib/tempest#L855 .. and that doesn't work if i make venv-tempest more similar to all environment :D | 14:45 |
| kopecmartin | i'm going in circles .. one thing i don't understand is why it has worked until now | 14:46 |
| ykarel | kopecmartin, do you have failure link what you mentioning above, i have seen similar behavior when working on fixes for py3.8 cleanup | 14:47 |
| kopecmartin | the error can be seen e.g. in watcher - https://review.opendev.org/c/openstack/watcher/+/933909 (patch set 2 , don't mind the latest patchset where i tried to change all to venv-tempest that also failed .. ) | 14:47 |
| kopecmartin | search for "TASK [run-tempest : Run Tempest]" in https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_1ae/933909/2/check/watcher-tempest-functional/1ae8b04/job-output.txt | 14:48 |
| kopecmartin | in that ^^ we can see that the env got recreated which i believe leads to the "The specified regex doesn't match with anything" error | 14:49 |
| ykarel | ok looks related | 14:49 |
| ykarel | 2024-11-01 13:42:46.303565 | controller | all recreate: /opt/stack/tempest/.tox/tempest | 14:49 |
| ykarel | due to constraint change env is recreated | 14:49 |
| ykarel | and installed plugins lost | 14:49 |
| kopecmartin | yep, i'm wondering what's the optimal solution .. define a new tox env in tempest that will be used by devstack (to install the plugins) and by jobs to run the tests? or edit an existing one (that might lead to new failures though) | 14:51 |
| kopecmartin | hmm, a new tox env might not help though, maybe we need to change how we install the plugins in the first place | 14:51 |
| kopecmartin | right now the devstack code relies on the fact that it can pass additional arguments, pip install in this case, .. however, that behavior is not compatible with with a tox env that also runs "tempest run" | 14:52 |
| kopecmartin | at least it seems so | 14:52 |
| ykarel | i see PS2 failure from 1st nov so missing https://github.com/openstack/tempest/commit/b9a8f3b755599feac186c5183226c087332cedbe | 14:55 |
| ykarel | worth to try again PS2 | 14:55 |
| ykarel | i think that would help clearing that issue | 14:57 |
| ykarel | yes it would like it passing in other patches https://zuul.opendev.org/t/openstack/builds?job_name=watcher-tempest-strategies&project=openstack%2Fwatcher&branch=master&skip=0 | 14:59 |
| kopecmartin | au .. so i've been trying to fix something that wasn't actually broken in the first place :D :D | 14:59 |
| frickler | the failures on or around 01 Nov were because https://review.opendev.org/c/openstack/tempest/+/933773 failed in gate | 15:04 |
| frickler | leading to constraints mismatches | 15:05 |
| frickler | which I mentioned in this channel back then | 15:05 |
| kopecmartin | yup, i remember now, i didn't put 2 and 2 together .. :/ | 15:13 |
| mnasiadka | kopecmartin: thanks, did the same in Magnum ;-) | 15:25 |
| johnsom | Any idea if I should just recheck https://review.opendev.org/c/openstack/tempest/+/934194? tempest-full-py3 failed on a very unrelated tempest.api.volume.test_volumes_actions.VolumesActionsTest test. | 16:12 |
| gmann | mnasiadka: yeah due to python 3.8 drop we have to pin Tempest for stable/2023.1 testing and remove its job from master gate | 18:30 |
| gmann | kopecmartin: ykarel: that error is because venv is recreated and all previously installed tempest plugins disappear from the newly tempest venv and so does it throw error if job try to run the tempest plugin tests because those tests are not loaded in the current tester | 18:33 |
| gmann | kopecmartin: ykarel: this (re creation of venv) can happen due to many reason I mean if anything is changed in tox venv command run | 18:34 |
| gmann | because tox create the fresh venv if any changes is detected | 18:34 |
| gmann | kopecmartin: let me look into watcher failure today and if anything we need to fix there. | 18:34 |
| gmann | kopecmartin: is this the change which is passing now? https://review.opendev.org/c/openstack/watcher/+/933909/4 | 18:39 |
| gmann | because we use 'venv-tempest' as venv for tempest to be consistent and if any tox env is run devstack install tempest in venv-tempest only | 18:40 |
| gmann | that is why jobs can run tempest via any tox env (from tox.ini) and devstack make sure everything including tempest (tempest run command) is installed in venv-tempest and tox env use venv-tempest only always | 18:41 |
| kopecmartin | gmann: all good now, the issues were related to the tempest release procedure ... | 20:48 |
| kopecmartin | i didn't realize the jobs ran at the exact time when we were reverting the costraints in tempest back to master | 20:49 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!