Wednesday, 2023-01-04

*** diablo_rojo_phone is now known as Guest17600:26
*** Guest176 is now known as diablo_rojo_phone00:27
*** ysandeep|out is now known as ysandeep06:20
*** soniya is now known as soniya2906:32
*** ysandeep is now known as ysandeep|afk07:42
*** elodilles_pto is now known as elodilles08:42
ykarelHi /me trying to understand why job variants not shown in https://zuul.openstack.org/job/neutron-tempest-plugin-base-nested-switch09:08
ykarelhttps://github.com/openstack/neutron-tempest-plugin/blob/master/zuul.d/base-nested-switch.yaml#L28-L32 one not visible09:09
ykarelsuspecting if it's something to do with regex trying https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/86915109:10
ykarelbut it's also not working if testing in https://review.opendev.org/c/openstack/neutron/+/869146 , job is not being triggered09:11
ykarelcan someone check on what i am missing09:11
*** soniya29 is now known as soniya29|afk09:20
amorinfungi, ack, will check09:41
fricklerykarel: I'm guessing that this may be because neutron-tempest-plugin is branchless. so your parent https://zuul.openstack.org/job/neutron-tempest-plugin-base doesn't have any stable/* variants09:56
*** ysandeep|afk is now known as ysandeep09:59
amorinfungi, should be good, I added some credit to your account and we are cancelling the current bill10:00
ykarelfrickler, but branches: should create those variants, no?10:02
fricklerykarel: iiuc that only matches variants that are created on those branches. but I'm no zuul expert either10:03
frickleramorin: thanks a lot, so we avoided getting shut down this time around, good work10:04
ykarelfrickler, ok Thanks, will wait for others, as other ex. openstack-tox have correct variants visible https://zuul.openstack.org/job/openstack-tox, it's also defined in a branchless project openstack-zuul-jobs10:04
fricklerykarel: I did take a quick look at scheduler logs and this seems to support my theory: No matching parents for job neutron-tempest-plugin-base-nested-switch and change <Change 0x7fd7897aa350 openstack/neutron 869146,1>10:28
ykarelfrickler, but afaiu https://github.com/openstack/neutron-tempest-plugin/blob/master/zuul.d/base-nested-switch.yaml#L32 should have registered a variant10:37
ykarellike the other ex. of openstack-tox job above10:38
fricklermaybe the difference is whether the parent is defined in the same repo or a different one. I would suggest to try to create variants of neutron-tempest-plugin-base too10:39
ykarelfrickler, ohkk will try10:51
*** soniya29|afk is now known as soniya2910:56
*** ysandeep is now known as ysandeep|afk11:07
ykarelfrickler, updated https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/869151/2..3/zuul.d/master_jobs.yaml11:07
ykarelbut still job not running 11:07
*** rlandy|out is now known as rlandy11:14
*** ttx is now known as ttx_11:14
*** ttx[m] is now known as ttx11:15
*** dviroe|out is now known as dviroe11:32
*** dviroe is now known as dviroel11:32
fungithanks amorin!12:15
fricklerykarel: still the same message in the logs. so likely indeed we'll need someone with more knowledge of zuul internals to help12:16
ykarelfrickler, Thanks, fungi can you please help with ^12:23
fungiykarel: maybe once i wake up, though i'll probably end up having to ask on the zuul matrix channel anyway. can you clearly restate: 1. what you did, 2. the behavior you expected, and 3. the behavior that was observed instead?12:25
fungii'm having a bit of trouble following the discussion above, but it's probably just down to insufficient caffeine levels12:26
ykarelfungi, Thanks summary in my questions though https://meetings.opendev.org/irclogs/%23opendev/%23opendev.2023-01-04.log.html#t2023-01-04T09:08:5612:26
*** ysandeep|afk is now known as ysandeep12:27
fungiykarel: sorry i'm really not following which is why i asked you to restate the problem. i'll try to answer my own questions in that case... 1. in the zuul configuration for neutron-tempest-plugin you've created a job named neutron-tempest-plugin-scenario-nested-switch which matches branches stable/queens-ussuri, 2. you're expecting12:50
fungihttps://zuul.openstack.org/job/neutron-tempest-plugin-base-nested-switch (a different job) to show branch variants, 3. i'm lost12:50
fungiseparately, you've created change 869146 to try to run a neutron-tempest-plugin-scenario-ovn-ussuri job (this is a third job?) and it's not running12:51
ykarelfungi, sorry 1. created two job variants one for releases before ussuri and other for later releases https://github.com/openstack/neutron-tempest-plugin/blob/master/zuul.d/base-nested-switch.yaml#L11-L3212:53
ykarel2. i was expecting to see both variants in https://zuul.openstack.org/job/neutron-tempest-plugin-base-nested-switch12:53
ykarel3. i don't see the other job variant in ^ and the jobs which inheriting from this job before ussuri release not running12:53
fungifor branches before ussuri, i guess you mean?12:53
ykarelyes12:53
fungiokay, backing up... you created two variants somewhere else i guess? i see two different job definitions in the file you just linked. one is the definition for the neutron-tempest-plugin-base-nested-switch job, the other is a definition for a neutron-tempest-plugin-scenario-nested-switch job12:56
fungino variants though12:56
ykarelfungi, ahh right :(12:56
fungidid you accidentially mis-name one of them?12:56
ykareltotally my bad12:56
ykarelyes right :)12:57
fungieasy fix ;)12:57
ykarelneed to check why it happened as i recall it used to work12:57
ykarelThanks12:57
fungino worries. just wanted to start at the beginning because i thought i was misunderstanding the premise entirely12:57
ykarelYes thanks found https://github.com/openstack/neutron-tempest-plugin/commit/8988a01a96df301be2ca9a7536256a4cfec0f087 caused it12:58
fungiwhen tends to happen when i roll over in bed and pick up the computer without waking up ;)12:58
ykarelTotally my bad on assuming job configs are correct as they were originally :(12:59
fungiit happens!13:00
opendevreviewJeremy Stanley proposed opendev/system-config master: Upgrade to latest Mailman 3 releases  https://review.opendev.org/c/opendev/system-config/+/86921013:15
*** bhagyashris is now known as bhagyashris|afk13:17
opendevreviewMerged openstack/project-config master: nodepool: remove ubuntu-bionic-arm64-large label  https://review.opendev.org/c/openstack/project-config/+/86844113:38
*** ysandeep is now known as ysandeep|away13:49
opendevreviewMerged opendev/system-config master: Remove old nodepool.yaml testing variables  https://review.opendev.org/c/opendev/system-config/+/86843213:57
*** pojadhav- is now known as pojadhav|ruck13:58
opendevreviewMerged opendev/system-config master: linaro: add nodepool cloud configuration  https://review.opendev.org/c/opendev/system-config/+/86843314:10
opendevreviewMerged opendev/system-config master: Run nodepool on testing credentials change  https://review.opendev.org/c/opendev/system-config/+/86843614:10
fricklerclarkb: I think there is still a patch missing to clean up remainders of us.linaro.cloud or did I just not look close enough https://codesearch.openstack.org/?q=us.linaro.cloud&i=nope&literal=nope&files=&excludeFiles=&repos=14:33
*** dviroel is now known as dviroel|lunch14:54
rosmaitafungi: looks like that python-cinderclient-functional-py38 problem is a real thing15:32
rosmaitadon't know if this is relevant, but the "console" tab for -py38 and -py39 are different15:32
rosmaitathe -py39 one (which is working fine) shows 3 pre playbook, a run playbook, and 3 post playbook15:32
rosmaitathe -py38 one only has pre and post playbook, no run playbook15:32
rosmaita-py38 results:15:32
rosmaitahttps://zuul.opendev.org/t/openstack/build/6997323310894f48855e4b9139b26f1015:32
rosmaita-py39 results: https://zuul.opendev.org/t/openstack/build/9ba34208232b4db49c48ceac72716c9815:32
fungirosmaita: missing a playbook in the console view is usually a sign that it was killed, most often by a timeout15:35
rosmaitaoh, ok, i was hoping i was onto something15:35
rosmaitafungi: eric found something in the cinder tox.ini that might apply here, am trying it now15:38
rosmaitahttps://review.opendev.org/86922615:38
fungirosmaita: in the log, you can see the timeout recorded...15:39
fungi2023-01-04 15:05:20.963106 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/devstack/playbooks/tox/run-both.yaml@master]15:39
fungithat killed the ansible process executing the run playbook, so the json representation of its results was truncated and zuul's console view skips it as unparseable15:40
fungiprobably something that could be improved in the console viewer but would likely require doctoring the json output from the ansible process to turn it into valid json in such cases15:41
rosmaitafungi: yeah, i was just grasping at straws, the run playbook had to be executed because otherwise the tests wouldn't be running!15:42
fungiexactly. something the run playbook executes after stestr probably hung or took way too long. like i said in the meeting, often that's because you ended up with orders of magnitude more logs or subunit than expected15:43
*** dviroel|lunch is now known as dviroel16:06
*** marios is now known as marios|out16:39
Clark[m]frickler: I don't think those changes exist yet as we are still running the builder on it. Next step is to spin up new builder and shut things down?16:40
fungiyes, nb03 is still running there16:42
fungithe openstack tc (based on today's meeting) seems to really really want to pin tox<4 on stable branches. where were the examples of tox auto-upgrading itself when there's a minversion line in tox.ini?16:43
Clark[m]fungi: I think I hit it in zuul-jobs nested tox.ini files for testing roles16:47
Clark[m]And it's not min version but the env requirements iirc16:48
fungioh16:49
Clark[m]When it creates the nested venv to install those deps it always upgrades tox to latest. Maybe it does the same if the running tox is less than min version?16:51
fungiwhere does it create the venv?17:01
rosmaitafungi: i think in .tox/.tox17:02
fungidoes it delete it afterward?17:02
rosmaitai think it does, not sure though17:02
fungii see it has created a .tox/.tmp/ which contains only an empty package/ directory17:02
rosmaitano, it doesn't delete the .tox/.tox17:03
Clark[m]fungi: https://review.opendev.org/c/zuul/zuul-jobs/+/866935/7..8/test-playbooks/python/tox.ini I think that is handling the issue there17:08
Clark[m]I want to say elsewhere I just made v4 work17:08
Clark[m]That dep (tox-bindep) only works with <4 so the pin was the only way to make that work17:09
fricklerClark[m]: ah, o.k., I was stumbling about that when looking for where to turn off the cert check17:12
Clark[m]https://review.opendev.org/c/zuul/zuul-jobs/+/866945/10/test-playbooks/python/tox.yaml is the update made to handle tox v4 running there17:13
Clark[m]Since some newer tox v4 requires --'s before posargs17:13
fricklergthiemonge: tweining: fungi: fyi this seems to be the issue for what octavia saw https://github.com/tox-dev/tox/issues/281117:31
tweiningfrickler: thanks. exactly17:32
fungiClark[m]: rosmaita: thanks, i was able to reproduce the behavior. any tox.requires set will cause tox to upgrade itself to latest regardless of whether tox.min_version is already satisfied or even specified at all18:15
rosmaitafungi: hmmm ... i wonder whether we can use tox<4 in the requires list in the [tox] section to set a ceiling18:17
rosmaitafungi: looks like that could work18:20
rosmaitahere's what i see:18:20
rosmaitain my tox.ini i have:18:20
fungirosmaita: yep, i get the same18:20
rosmaita[tox]18:20
rosmaitadistribute = False18:20
rosmaitaenvlist = py3,pep818:20
rosmaitaminversion = 3.18.018:20
rosmaita# specify virtualenv here to keep local runs consistent with the18:20
rosmaita# gate (it sets the versions of pip, setuptools, and wheel)18:20
rosmaitarequires =18:20
rosmaita  virtualenv>=20.17.118:20
rosmaita  tox<3.2818:20
fungioutput says...18:20
fungi.tox installdeps: requestsexceptions, tox<418:20
fungiand then it proceeds18:21
rosmaitaand then i get:18:21
rosmaita[check-func-jobs] wha'ppen? .tox/.tox/bin/tox --version18:21
rosmaita3.27.1 imported from /home/brosmait/repos/openstack/python-cinderclient/.tox/.tox/lib/python3.10/site-packages/tox/__init__.py18:21
fungibut my original point was that "pinning" tox for stable branches is going to still require merging tox.ini changes for at least some of them18:21
rosmaitaok, so it looks like we can have a minversion as long as we include a cap18:22
rosmaitafungi: right, but at least these changes are minimally invasive18:22
fungiactually it turns out that tox.min_version is irrelevant (as long as it's already satisfied)18:22
fungiit's when tox.requires is set that this becomes a problem18:22
rosmaitai don't know how many people use tox.requires18:23
rosmaitawe use it in cinder so that we could be sure to get the correct version of pip locally to reproduce problems when the pip resolver was rewritten18:23
fungisome projects use it to install tox plugins18:24
rosmaitaok18:25
Clark[m]Running into this was why I decided to focus on nox elsewhere instead of keeping tox alive. I'm ok with bugs and working through them but auto upgrading to latest when not necessary ensuring you hit all the new bugs is not very user friendly18:29
rosmaitaClark[m]: i could not agree more!18:35
fungirunning some stats over gerrit activity across our different namespaces... opendev/.* repositories merged 5.2% more changes in 2022 than in 202119:45
fungier, nevermind, we didn't. my eyes compared the wrong lines19:46
fungithough we did have changes committed by 2.5% more contributors in 2022 than in 202119:47
fungialso we merged changes to 79% more opendev/.* repos in 2022 than in 2021, granted i expect most of that was retiring old puppetry19:48
*** dviroel is now known as dviroel|ourt21:09
opendevreviewJeremy Stanley proposed opendev/system-config master: Feature our cloud donors on opendev.org  https://review.opendev.org/c/opendev/system-config/+/86909121:58
opendevreviewAde Lee proposed openstack/project-config master: Add FIPS job for ubuntu  https://review.opendev.org/c/openstack/project-config/+/86711222:07
*** dasm is now known as dasm|off22:14
*** rlandy is now known as rlandy|out23:19

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!