15:00:14 <kopecmartin> #startmeeting qa
15:00:14 <opendevmeet> Meeting started Tue Nov 22 15:00:14 2022 UTC and is due to finish in 60 minutes.  The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:14 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:14 <opendevmeet> The meeting name has been set to 'qa'
15:00:23 <kopecmartin> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours
15:00:26 <kopecmartin> agenda ^
15:00:28 <lpiwowar> o/
15:00:36 <kopecmartin> \o
15:03:03 <kopecmartin> let's quickly go through the usual
15:03:05 <kopecmartin> #topic Announcement and Action Item (Optional)
15:03:12 <kopecmartin> nothing here
15:03:13 <kopecmartin> #topic Antelope Priority Items progress
15:03:19 <kopecmartin> #link https://etherpad.opendev.org/p/qa-antelope-priority
15:04:32 <kopecmartin> i've seen some progress on "Clean up deprecated lib/neutron code"
15:04:33 <kopecmartin> #link https://bugs.launchpad.net/devstack/+bug/1996748
15:04:47 <kopecmartin> thank you slaweq for working on that
15:05:36 <kopecmartin> regarding s-rbac, there has been some progress too , the s-rbac job seems to be passing now \o/
15:05:39 <kopecmartin> #link https://review.opendev.org/c/openstack/tempest/+/614484
15:05:46 <kopecmartin> thanks gmann for that patch ^^
15:06:09 <kopecmartin> "Use admin clients *only if* admin access needed"
15:06:24 <kopecmartin> anything new here ^^ arxcruz
15:06:42 <arxcruz> kopecmartin: waiting feedback from gmann
15:07:00 <kopecmartin> #link https://review.opendev.org/c/openstack/tempest/+/852377
15:07:36 <kopecmartin> but didn't we concluded that we don't need to add those parameters?
15:07:36 <arxcruz> basically, the code that enable us to use different client is there, once i get a green light, i can start working replacing tests that calls the manager to use non-admin client
15:07:44 <arxcruz> depending on the test/requirements
15:10:46 <kopecmartin> well, we know that situation when one test uses demo and the other one admin can't happen, it's either one or the other and then considering the amount of jobs (which is either voting on tempest patches or in general where tempest is used) we can switch only those methods to demo which will work/perform their action successfully all the time, in any case
15:11:11 * frickler sneaks in late, reads backlog
15:11:37 <arxcruz> yes, that's the idea
15:11:49 <arxcruz> the actual patch doesn't change anything, but enable us to do that
15:12:15 <kopecmartin> so that just brings question whether we need that parameter because at the end, the method is using either admin client or demo client
15:12:37 <kopecmartin> i'm not sure we need to be able to that, that's all i'm saying, but the patch won't hurt
15:13:14 <kopecmartin> it will add an option to change the client, whether it's needed / or will be needed is tough to answer
15:14:20 <kopecmartin> moving on
15:14:22 <kopecmartin> #topic OpenStack Events Updates and Planning
15:14:25 <kopecmartin> skipping for now
15:14:29 <kopecmartin> #topic Gate Status Checks
15:14:40 <kopecmartin> #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade)
15:14:55 <kopecmartin> nothing there, any urgent / gate blocking patch to review?
15:15:51 <kopecmartin> #topic Bare rechecks
15:15:52 <kopecmartin> #link https://etherpad.opendev.org/p/recheck-weekly-summary
15:15:52 <frickler> there might have been https://review.opendev.org/c/openstack/devstack/+/864848 to discuss but I think we've circumvented that now
15:15:59 * kopecmartin checking
15:16:04 <kopecmartin> btw, recheck status looks good
15:17:12 <frickler> it won't look so good next time
15:17:38 <frickler> did an awful lot of rechecks on the devstack-plugin-ceph patch to pin to focal
15:17:50 <frickler> until I gave up and made it non-voting
15:19:16 <kopecmartin> frickler: i saw, thank you for rechecking and getting the patches merged
15:19:29 <kopecmartin> #link
15:19:32 <kopecmartin> #link https://review.opendev.org/c/openstack/devstack/+/864848
15:19:36 <kopecmartin> can it cause any harm?
15:20:13 <kopecmartin> the project is archived under openstack namespace
15:20:19 <frickler> well "harm" I'm not sure, it just doesn't match our policy.
15:21:05 <kopecmartin> i need to check that, i
15:21:09 <frickler> but also a new version of cursive is now on pypi and https://review.opendev.org/c/openstack/barbican/+/864226 + its dep should solve that urgent issue
15:21:10 <kopecmartin> i'm not very familiar with that
15:21:50 <kopecmartin> oh, yeah, released yesterday
15:22:08 <kopecmartin> so it seems, barbican is no longer blocked by that patch then?
15:22:32 <frickler> still waiting for CI but I hope so
15:23:03 <kopecmartin> ok, then it's ok, better to implement it according the policies
15:23:10 <kopecmartin> thanks for pointing that out
15:23:54 <kopecmartin> #topic Periodic jobs Status Checks
15:24:02 <kopecmartin> stable
15:24:02 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=tempest-full-yoga&job_name=tempest-full-xena&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-zed&pipeline=periodic-stable
15:24:08 <kopecmartin> master
15:24:08 <kopecmartin> #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic
15:24:44 <kopecmartin> looks good, although latest tempest-all timeouted
15:24:47 * kopecmartin checking the logs
15:25:56 <kopecmartin> i wonder whether that can be related to jammy, i've seen similar timeouts on cs-9 fips jobs where the reason was that certain operations just took longer on that distro
15:26:02 <kopecmartin> let's keep an eye on that
15:27:08 <kopecmartin> #topic Distros check
15:27:17 <kopecmartin> cs-9
15:27:18 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=tempest-full-centos-9-stream&job_name=devstack-platform-centos-9-stream&skip=0
15:27:24 <kopecmartin> fedora
15:27:25 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-fedora-latest&skip=0
15:27:32 <kopecmartin> debian
15:27:33 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-debian-bullseye&skip=0
15:27:39 <kopecmartin> jammy
15:27:42 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-ubuntu-jammy&job_name=devstack-platform-ubuntu-jammy-ovn-source&job_name=devstack-platform-ubuntu-jammy-ovs&skip=0
15:27:48 <kopecmartin> rocky
15:27:49 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-rocky-blue-onyx
15:28:28 <kopecmartin> there have been some failures, but the latest builds look good
15:28:37 <frickler> cs-9 seems broken on stable branches. do you intend to look into fixing it or should we drop it instead of having it non-voting?
15:29:22 <kopecmartin> i'm not planning to fix that :/
15:29:32 <kopecmartin> last week i proposed a patch to make it n-v on yoga
15:29:55 <kopecmartin> the next logical step will be probably dropping the jobs on stable branches entirely
15:30:48 <frickler> ack
15:30:51 <kopecmartin> there isn't that much of a traffic there (on stable branches) and seeing the failure rates .. it will be better to save some resources instead
15:30:56 <frickler> we also need to check platform-focal job now instead of -jammy
15:31:23 <kopecmartin> good point
15:31:55 <frickler> https://zuul.opendev.org/t/openstack/builds?job_name=devstack-platform-ubuntu-focal&project=openstack/devstack
15:32:01 <frickler> looking good so far
15:32:25 <kopecmartin> perfect, let me update the agenda right away
15:34:29 <kopecmartin> #topic Sub Teams highlights
15:34:33 <kopecmartin> Changes with Review-Priority == +1
15:34:37 <kopecmartin> #link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade)
15:34:50 <kopecmartin> nothing there
15:34:54 <kopecmartin> #topic Open Discussion
15:35:02 <kopecmartin> anything for the open discussion?
15:36:04 <kopecmartin> #topic Bug Triage
15:36:05 <kopecmartin> #link https://etherpad.openstack.org/p/qa-bug-triage-antelope
15:36:35 <kopecmartin> this relatively new bug in devstack has caught my attention
15:36:39 <kopecmartin> #link https://bugs.launchpad.net/devstack/+bug/1997017
15:37:17 <kopecmartin> it looks like the user set a wrong ip in HOST_IP which got me thinking, is it recommended to set that value?
15:37:43 <opendevreview> Elod Illes proposed openstack/tempest master: Set back to use Bionic for Ussuri, Train and Stein  https://review.opendev.org/c/openstack/tempest/+/865300
15:37:57 <kopecmartin> well, the fact i don't set it doesn't mean it's not an option ..
15:38:12 <frickler> I think the issue is with py3.9
15:38:23 <frickler> uwsgi won't use that
15:38:43 <frickler> on 20.04 one should use py3.8
15:38:59 <kopecmartin> oh, i see
15:39:20 <frickler> although also I don't see where in the config the switched to py3.9
15:39:39 <frickler> but the logs show it being used
15:40:07 <kopecmartin> py3.9 is the default python3 on the env maybe/
15:40:08 <kopecmartin> ?
15:40:55 <kopecmartin> they wanted to use that version of python specifically based on the first sentence
15:41:20 <frickler> yes and we don't support that
15:42:03 <kopecmartin> will you comment there please?
15:42:17 <frickler> sure
15:42:59 <kopecmartin> then there's another new bug
15:43:01 <kopecmartin> #link https://bugs.launchpad.net/devstack/+bug/1996465
15:43:31 <kopecmartin> seems like devstack didn't configure nova-cpu.conf correctly
15:43:50 <kopecmartin> i'm not sure i've ever tried devstack on multinode setup
15:43:52 <frickler> I wonder what this devstack-vagrant is meant to be
15:44:09 <frickler> one needs to prepare local.conf specifically for the second node
15:44:18 <frickler> our docs for that are likely outdated
15:44:49 <frickler> so the reference would be the multinode job in our .zuul.yaml
15:45:12 <frickler> I will comment and ask for more data
15:45:32 <kopecmartin> thank you frickler
15:45:40 <kopecmartin> this is all from my side
15:46:01 <kopecmartin> if there isn't anything else
15:46:09 <frickler> so one thing about the ceph plugin
15:46:20 <kopecmartin> sure
15:46:23 <frickler> it is working with jammy now in general
15:46:41 <frickler> but there is a failure during the tempest deployment
15:46:59 <frickler> so that would best need some tempest expert to look at
15:47:22 <kopecmartin> i can take a look
15:47:31 <kopecmartin> did anything change?
15:49:24 <frickler> no, it never ran on jammy before. so something about python-ceph plugin dependencies confliction with tempest venv
15:49:41 <frickler> I can't find the proper zuul result right now
15:50:22 <frickler> maybe create a revert of https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/864948 and that should show the failure now that devstack default has changed
15:51:41 <kopecmartin> ack, i'll have a look
15:51:50 <frickler> https://zuul.opendev.org/t/openstack/build/8c70f6060c3141b890a7f32d6594a62c has the issue
15:52:09 <frickler> note that the "ERROR: cowardly refusing to delete `envdir` (it does not look like a virtualenv): /opt/stack/tempest/.tox/tempest" is not the original failure
15:52:40 <kopecmartin> ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/future'
15:52:47 <frickler> right
15:53:04 <kopecmartin> very weird, i'll have a look
15:53:31 <kopecmartin> anything else?
15:53:35 <frickler> maybe also something for clarkb who has been successful in similar issues
15:53:58 <frickler> that's it from me then
15:54:39 <kopecmartin> thank you frickler
15:54:43 <kopecmartin> see you around
15:54:48 <kopecmartin> #endmeeting