15:00:31 <kopecmartin> #startmeeting qa
15:00:31 <opendevmeet> Meeting started Tue Jan 17 15:00:31 2023 UTC and is due to finish in 60 minutes.  The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:31 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:31 <opendevmeet> The meeting name has been set to 'qa'
15:00:36 <kopecmartin> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours
15:00:39 <kopecmartin> agenda ^^^
15:00:42 <lpiwowar> hi o/
15:00:46 <frickler> \o
15:01:16 <gmann> o/
15:01:57 <kopecmartin> o/
15:03:08 <kopecmartin> let's get started
15:03:09 <kopecmartin> #topic Announcement and Action Item (Optional)
15:03:51 <kopecmartin> hmm, none from my side
15:04:11 <kopecmartin> #topic Antelope Priority Items progress
15:04:12 <kopecmartin> #link https://etherpad.opendev.org/p/qa-antelope-priority
15:04:17 <kopecmartin> any updates on the priority items?
15:04:25 * kopecmartin checking the doc
15:04:58 <frickler> I added the failure mode for ceph plugin on 22.04
15:04:59 <gmann> on RBAC, nova has changed the default which is to enable the scope checks and new defaults by default
15:05:09 <frickler> needs some tempest people to look at it
15:06:44 <kopecmartin> frickler: thank, i'll have a look
15:06:52 <kopecmartin> gmann: that's cool
15:06:54 <gmann> #link https://review.opendev.org/c/openstack/nova/+/866218
15:07:41 <frickler> seems nova has broken kolla with that, but we will pin to disabled scope for now
15:07:41 <gmann> and we have new job running new defaults on tempest as well as on nova gate. in Devstack we are keeping old defaults by default until enabling new defaults by defaults is released
15:08:24 <gmann> frickler: all scope to nova policy are projects which means no change even we enable or disable. enable will only fail if anyone using system token for nova APIs
15:08:38 <gmann> frickler: can you give me the failure link, I will look into that
15:09:58 <frickler> gmann: https://zuul.opendev.org/t/openstack/build/33d2dc93c40a457d83adad4c7720e9be for example
15:10:10 <gmann> and also keeping legacy admin behavior same as before
15:10:23 <gmann> frickler: thanks, I will check those todayu
15:11:37 <frickler> https://72b1f029e56da2cad7f4-0417878d098db6313bb6384c3905736a.ssl.cf2.rackcdn.com/870535/1/gate/kolla-ansible-ubuntu-source/33d2dc9/primary/logs/kolla/nova/nova-api.txt
15:11:47 <frickler> that has the failure
15:12:04 <gmann> k
15:13:15 <kopecmartin> moving on
15:13:16 <kopecmartin> #topic OpenStack Events Updates and Planning
15:13:29 <kopecmartin> the date of the next virtual PTG is March 27-31, 2023
15:13:42 <kopecmartin> #link https://etherpad.opendev.org/p/qa-bobcat-ptg
15:13:53 <kopecmartin> i've just created the etherpad
15:14:11 <kopecmartin> but if anyone has any ideas for a topic to discuss, feel free to draft it there
15:14:33 <kopecmartin> (I'll add more formatting and notes soon)
15:15:10 <kopecmartin> #topic Gate Status Checks
15:15:14 <kopecmartin> #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade)
15:15:25 <kopecmartin> nothing there, anything urgent to review?
15:16:16 <kopecmartin> #topic Bare rechecks
15:16:21 <kopecmartin> #link https://etherpad.opendev.org/p/recheck-weekly-summary
15:16:25 <kopecmartin> this looks good
15:16:39 <kopecmartin> #topic Periodic jobs Status Checks
15:16:44 <kopecmartin> stable
15:16:45 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=tempest-full-yoga&job_name=tempest-full-xena&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-zed&pipeline=periodic-stable
15:16:51 <kopecmartin> master
15:16:51 <kopecmartin> #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic
15:17:00 <kopecmartin> all green \o/
15:17:22 <kopecmartin> #topic Distros check
15:17:28 <kopecmartin> cs-9
15:17:28 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=tempest-full-centos-9-stream&job_name=devstack-platform-centos-9-stream&skip=0
15:17:33 <kopecmartin> fedora
15:17:34 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-fedora-latest&skip=0
15:17:38 <kopecmartin> debian
15:17:39 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-debian-bullseye&skip=0
15:17:44 <kopecmartin> focal
15:17:44 <kopecmartin> #link https://zuul.opendev.org/t/openstack/builds?job_name=devstack-platform-ubuntu-focal&skip=0
15:17:51 <kopecmartin> rocky
15:17:51 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-rocky-blue-onyx
15:18:10 <kopecmartin> isn't missing something?
15:18:30 <frickler> euler?
15:18:36 <kopecmartin> yeah
15:20:07 <kopecmartin> openEuler
15:20:11 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-openEuler-22.03-ovn-source&job_name=devstack-platform-openEuler-22.03-ovs&skip=0
15:20:57 <kopecmartin> all good, all green, nice
15:21:20 <kopecmartin> maybe we want to consider marking the openEuler as voting?
15:22:21 <kopecmartin> #topic Sub Teams highlights
15:22:25 <kopecmartin> Changes with Review-Priority == +1
15:22:30 <kopecmartin> #link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade)
15:22:37 <gmann> it is difficult to see the failure frequency in zuul builds, can we prepare the query in new dashbaord and know what rate it is passing?
15:23:31 <kopecmartin> that should be possible, what do you think lpiwowar?
15:23:32 <lpiwowar> gmann: I think it might be possible. I can check it out.
15:23:37 <gmann> we should start using that dashbaord which can help on voting/non-voting jobs
15:23:44 <gmann> thanks
15:23:46 <kopecmartin> +1
15:24:22 <lpiwowar> gmann: I just need to check it out. Sometimes it is bit tricky to create the query.
15:24:48 <gmann> lpiwowar: ohk, sure. thanks
15:25:33 <kopecmartin> we have 2 reviews with priority
15:25:35 <kopecmartin> one for devstack
15:25:37 <kopecmartin> #link https://review.opendev.org/c/openstack/devstack/+/837207
15:25:43 <kopecmartin> one for tempest
15:25:45 <kopecmartin> #link https://review.opendev.org/c/openstack/tempest/+/869440
15:26:00 <kopecmartin> the tempest's one got already tested by lpiwowar in the stable branches
15:26:17 <lpiwowar> kopecmartin: yes. It should be ok. Thanks for mentioning it.
15:26:18 <kopecmartin> seems good
15:26:18 <gmann> nice, I will check it today then.
15:26:23 <gmann> cool
15:26:36 <frickler> I wonden when ovn will work without 100s of workarounds
15:26:41 <frickler> wonder even
15:26:46 <gmann> :)
15:27:18 <kopecmartin> let's not make it easy :D
15:27:28 <kopecmartin> #topic Open Discussion
15:27:28 <kopecmartin> anything for the open discussion?
15:27:52 <frickler> yes
15:27:54 <lpiwowar> frickler: What fixed the tempest-full-multinode-py3 job?
15:28:00 <lpiwowar> #link https://review.opendev.org/c/openstack/tempest/+/866692
15:28:12 <lpiwowar> I was just curious.
15:28:26 <frickler> lpiwowar: zuul dropped setting private IPs in some situations, that has been reverted for now
15:28:59 <lpiwowar> frickler: Ok, thanks
15:29:03 <frickler> there was a bug opened about opensuse support. and I wonder whether we should rather drop that distro completely
15:29:18 <frickler> https://bugs.launchpad.net/devstack/+bug/2002900
15:29:37 <frickler> would clean up quite a bit of special casing
15:29:47 <frickler> and we don't run any CI on it anymore
15:30:13 <gmann> lpiwowar: just curious, are you testing /getting failure with devstack change of 'cirros version to 0.6.1' ? or in normal run also? seeing depends-on to devstack patch ?
15:30:36 <gmann> or testing if cirrors vesion fix the failure?
15:31:02 <gmann> +1 on dropping opensuse completely as we do not test it
15:33:13 <kopecmartin> makes sense, if it's not tested, let's drop the support
15:33:36 <kopecmartin> gmann: the multinode was failing even with newer cirros image (0.6.1? don't remember) if that's the question
15:33:43 <lpiwowar> gmann: The tempest-multinode-full-py3 job was failing also outside of the patch. So I think that the bump up of the cirros version did not help.
15:34:16 <lpiwowar> kopecmartin: +1
15:34:22 <gmann> ok, yeah i saw failure sometime I thikn the basic scenario test right? but that was not 100% failing
15:34:50 <gmann> is there bug open to know traceback and exact traceback?
15:35:05 <lpiwowar> The scenario tests failed only sometimes. But recently the failure rate went up.
15:35:14 <gmann> and we can check the frequency of failure too. as it is not failure 100% I am sure it is race or timing issue
15:35:37 <gmann> which test?
15:36:22 <lpiwowar> gmann: Give me a moment I will tell you exactly.
15:36:37 <gmann> no hurry, let's add it in bug and we can track there
15:37:45 <lpiwowar> gmann: ack
15:38:18 <gmann> lpiwowar: thanks for debugging gate such failure which will help our CI testing a lot.
15:38:36 <lpiwowar> gmann: thanks!
15:39:19 <lpiwowar> I remember seeing failures of: tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario and tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_instance_hard_reboot_after_vol_snap_deletion
15:39:42 <gmann> k
15:39:45 <lpiwowar> But from time to time there were other tests failing as well.
15:42:31 <lpiwowar> Just adding for documentation. Here is a link for the bug regarding the tempest-multinode-full-py3 job
15:42:34 <lpiwowar> #link https://bugs.launchpad.net/tempest/+bug/1998916
15:42:48 <kopecmartin> thanks
15:42:53 <kopecmartin> cool, anything else?
15:43:00 <gmann> thanks
15:43:41 <lpiwowar> nothing from my side
15:43:46 <kopecmartin> #topic Bug Triage
15:43:50 <kopecmartin> #link https://etherpad.openstack.org/p/qa-bug-triage-antelope
15:44:00 <kopecmartin> etherpad updated
15:44:25 <kopecmartin> i wanted to bring up the suse bug, but we already talked about that
15:44:32 <kopecmartin> that's all from my side
15:45:03 <frickler> gmann: can https://bugs.launchpad.net/devstack/+bug/1999183 be closed?
15:45:11 <frickler> or at least degraded from critical?
15:45:59 <gmann> frickler: not closed as we still have tox4 pin on devstack jobs on master. but I will change priority for this
15:46:32 <frickler> ok, thx
15:47:37 <gmann> done, I have this in my list but did not get time.
15:49:21 <frickler> tox4 has turned out to be a fast moving target, yes
15:50:52 <kopecmartin> yeah, thanks all, if there isn't anything else, let's close today's office hour
15:51:19 <kopecmartin> #endmeeting