15:00:19 <kopecmartin> #startmeeting qa 15:00:19 <opendevmeet> Meeting started Tue May 24 15:00:19 2022 UTC and is due to finish in 60 minutes. The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:19 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:19 <opendevmeet> The meeting name has been set to 'qa' 15:00:27 <kopecmartin> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Weekly_QA_Team_meeting 15:00:29 <kopecmartin> agenda ^^ 15:00:41 <opendevreview> James Parker proposed openstack/whitebox-tempest-plugin master: Take into account shared CPUs per NUMA https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/843157 15:01:27 <gmann> o/ 15:02:47 <kopecmartin> o/ 15:02:54 <frickler> \o 15:02:57 <kopecmartin> #topic Announcement and Action Item (Optional) 15:03:34 <kopecmartin> we have just dropped openeuler support from devstack 15:04:02 <kopecmartin> nothing else from my side here 15:04:05 <frickler> yes, sadly no continued maintainance once the support was added 15:04:43 <frickler> openstacksdk 0.99.0 was released 15:05:08 <frickler> mainly as a 1.0.0pre1 release. that may trigger some issues 15:05:14 <gmann> +1 15:05:35 <kopecmartin> thanks for the heads up 15:06:01 <kopecmartin> moving on 15:06:01 <kopecmartin> #topic Zed Priority Items progress 15:06:07 <kopecmartin> #link https://etherpad.opendev.org/p/qa-zed-priority 15:06:11 <kopecmartin> any updates on this front? 15:06:20 <gmann> nothing special 15:07:37 <kopecmartin> nothing special from my side either 15:07:49 <kopecmartin> #topic OpenStack Events Updates and Planning 15:07:55 <kopecmartin> nothing here for now 15:07:59 <kopecmartin> #topic Gate Status Checks 15:08:08 <kopecmartin> #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade) 15:08:47 <kopecmartin> 2 reviews, both approved, waiting on gates 15:09:13 <gmann> yeah, py36|7 is all stuck 15:09:17 <gmann> and not merged yet 15:10:45 <kopecmartin> yup, mainly because some jobs are quite unstable lately, mainly the ceph one 15:10:56 <kopecmartin> it was a reason of many rechecks 15:11:07 <gmann> yeah, let's see 15:11:52 <kopecmartin> #topic Periodic jobs Status Checks 15:12:00 <kopecmartin> stable 15:12:02 <kopecmartin> #link #link https://zuul.openstack.org/builds?job_name=tempest-full-yoga&job_name=tempest-full-xena&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&pipeline=periodic-stable 15:12:11 <kopecmartin> master 15:12:11 <kopecmartin> #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic 15:12:52 <frickler> the "error" thing was me 15:12:58 <frickler> triggered the wrong project 15:13:11 <frickler> but then manual triggers are broken in zuul somehow 15:13:28 <kopecmartin> :) np 15:13:34 <frickler> everything else still suspicously green 15:13:39 <kopecmartin> apart from those 2 errors it's all green 15:13:43 <kopecmartin> yeah 15:14:17 <kopecmartin> #topic Distros check 15:14:28 <kopecmartin> centos9-stream 15:14:29 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=tempest-full-centos-9-stream&job_name=tempest-full-py3-centos-8-stream&job_name=devstack-platform-centos-8-stream&job_name=devstack-platform-centos-9-stream&skip=0 15:14:37 <kopecmartin> fedora 15:14:37 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-fedora-latest&skip=0 15:14:44 <kopecmartin> debian 15:14:44 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-debian-bullseye&skip=0 15:15:19 <frickler> maybe we should also add a link for the jammy jobs? 15:15:22 <kopecmartin> not that bad, could be better though, especially in case of centos as it failed a few gate jobs 15:15:26 <kopecmartin> yeah 15:15:29 <kopecmartin> good point 15:17:50 <kopecmartin> jammy jobs 15:17:51 <kopecmartin> https://zuul.openstack.org/builds?job_name=devstack-platform-ubuntu-jammy&job_name=devstack-platform-ubuntu-jammy-ovn-source&job_name=devstack-platform-ubuntu-jammy-ovs&skip=0 15:17:53 <kopecmartin> #link https://zuul.openstack.org/builds?job_name=devstack-platform-ubuntu-jammy&job_name=devstack-platform-ubuntu-jammy-ovn-source&job_name=devstack-platform-ubuntu-jammy-ovs&skip=0 15:18:17 <frickler> debian looks a bit flaky, I'll check those later 15:19:01 <kopecmartin> ack, thanks 15:19:03 <kopecmartin> #topic Sub Teams highlights 15:19:07 <frickler> ah, those are mainly the ansible v5 checks 15:19:35 <kopecmartin> Changes with Review-Priority == +1 15:19:36 <kopecmartin> #link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade) 15:20:53 <kopecmartin> regarding the backports in devstack, do we need to merge them in order? from the latest release back? or can i approve all at once? 15:21:44 <gmann> either is fine in devstack bakport 15:21:53 <gmann> approve once also work as we do many times 15:22:38 <kopecmartin> ack, thanks 15:23:00 <kopecmartin> #topic Open Discussion 15:23:05 <kopecmartin> anything for the open discussion? 15:23:21 <frickler> I have one slightly political topic 15:23:24 <gmann> yeah, I have added two items 15:23:33 <gmann> frickler: go ahead first 15:23:38 <frickler> I would like to gather some early planning feedback on the next PTG 15:23:45 <frickler> the foundation seems to think that it is a good idea to have people travelling around the world again 15:23:55 <frickler> to which I 100% do not agree for multiple reasons 15:24:00 <gmann> yeah, its physical event 15:24:02 <frickler> so the question is: can we as a team agree to continue to hold a virtual PTG? 15:24:08 <frickler> maybe outside of the global PTG if some people are still travelling there? 15:24:36 <gmann> good point. even in TC I pointed to ask for community feedback first but that did not happen 15:25:34 <gmann> frickler: once challenge in that is we need to make it either virtual only or physical. hybrid can be more complex 15:25:46 <frickler> I for sure won't travel and I also consider boycotting a mixed one, yes 15:25:54 <gmann> yeah 15:26:28 <kopecmartin> i think we can set a date (different from the PTG one) and hold our own virtual meeting to connect with the community 15:26:49 <gmann> and no slot in physical one? 15:27:43 <kopecmartin> considering the number of attendees during latest PTGs and number of topics as well 15:27:56 <kopecmartin> i would be ok with no slot in the physical event 15:27:58 <gmann> but then it still the issue as frickler pointed 15:28:21 <gmann> we should have in either one. otherwise virtual one will be considered as just a QA call 15:29:29 <gmann> I might travel as it is in USA but not 100% sure, I am totally not fine on travel and completely agree with frickler concern. 15:29:39 <gmann> and I am ok to have virtual one 15:29:49 <kopecmartin> yes, therefore i'm inclined to hold in virtually only, i said i would be ok with no slot in the physical event 15:29:55 <kopecmartin> *hold it 15:29:57 <gmann> +1 15:30:01 <frickler> so maybe not appearing at the in-person PTG as a team would be a good sign towards the foundation 15:30:18 <frickler> I'll also suggest this to other team I'm participating in 15:30:24 <frickler> teams 15:31:15 <frickler> seems we kind of agree on this 15:31:28 <frickler> so gmann you can go ahead with your topics I guess 15:31:31 <kopecmartin> it seems yes 15:31:38 <gmann> yeah 15:31:39 <gmann> sure 15:31:51 <gmann> first is Py36|7 support in Tempest 15:32:27 <gmann> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028622.html 15:32:46 <gmann> as you know py36|7 drop in oslo or projects and we though of keeping support in tempest but that is broken 15:33:23 <gmann> I proposed two option on ML and matthew replied from req perspective option1 and how complex that is 15:33:52 <gmann> and even if we go with that we are kind of saying Tempest master code will work on py36 but with old oslo so no new feature from olso? 15:34:22 <gmann> which give a clear indication that it is better to drop it from Tempest also as its deps are non-py36 now 15:35:06 <gmann> one challenge in that Tempest master will not support py36|7 for stable/victoria to stable/yoga as those branch support py36|7 15:35:41 <gmann> which again has two solution 1. run tempest in py38 in virtual env and test stable/yoga on py36 2. use old tempest to test them 15:36:03 <gmann> I am in favor of dropping in tempest, what you say? 15:36:27 <gmann> may be i am just typing things you all know :) but just in case.. 15:36:45 <kopecmartin> dropping from tempest and testing with py38 in a venv for older releases sounds good 15:36:56 <frickler> what was the reason again for tempest not to have stable branches? 15:37:00 <gmann> yeah, that is what we did for py27 drop too 15:37:58 <opendevreview> Artom Lifshitz proposed openstack/tempest master: DNM: Test encrypted LM with local attach https://review.opendev.org/c/openstack/tempest/+/843163 15:38:00 <gmann> frickler: at least still Tempest master can be used to test stable branch and master openstack. yes not in all python version but we have way 15:38:03 <gmann> via virtual env 15:38:27 <gmann> and give us interop testing benefits, testing stable+master openstack with same set of tests 15:38:42 <opendevreview> Artom Lifshitz proposed openstack/tempest master: DNM: Test encrypted LM with local attach https://review.opendev.org/c/openstack/tempest/+/843163 15:39:16 <kopecmartin> although if it was branched, it would still work for interop i think 15:39:33 <frickler> you could still test yoga openstack against master tempest, even if a yoga tempest branch exists 15:39:34 <kopecmartin> maybe it would be even easier 15:39:41 <gmann> kopecmartin: well, you will test them with different set of tests right? 15:40:02 <kopecmartin> right 15:40:13 <gmann> frickler: but then what is benefit or yoga tempest branch if we continue testing with tempest master 15:40:21 <frickler> you do test with a different set of tests now, if <yoga needs tagged tempest 15:40:33 <kopecmartin> oh, on the other side it would require a lot of changes in the interop tooling 15:40:37 <frickler> you could switch once master stops working 15:40:45 <frickler> like now for py36 jobs 15:40:50 <gmann> frickler: no, we will continue testing with master stable/yoga and older supported branch 15:40:56 <frickler> but still backport changes to tests 15:41:45 <frickler> I'm not convinced a py38 venv will work on c8s 15:41:53 <frickler> did someone test that? 15:42:06 <gmann> stable/victoria - stable/yoga tested on ubuntu bionic and it has py38 so tempest virtual env can run on that its just if any downstream cloud want to test py36 stable/yoga then they need to create tempest master virutal env on py38 15:42:40 <gmann> c8s has py38 which worked for me on fips testing switch 15:42:59 <opendevreview> Merged openstack/devstack stable/yoga: Configure placement section in neutron conf https://review.opendev.org/c/openstack/devstack/+/842747 15:43:30 <frickler> hmm, o.k., I don't want to argue this, I'm not a tempest person really, just tried to understand things 15:43:58 <gmann> with tempest branch, concern is it can end up like 'disable this test in stable as we change API in incompatible way' and it will pass stable branch testing as well as master even API are changed 15:44:16 <gmann> and tempest branchless catches these things 15:46:28 <gmann> it is no different than when we dropped py27 but yes it will be same for every py3 minor version drop in OpenStack 15:46:29 <kopecmartin> yeah, let's workaround the situation with the virtual env and py38 for now 15:46:56 <frickler> o.k., I think you still had a second topic? 15:47:13 <gmann> yeah, sorry. I already started thinking on next meeting of release name 15:47:21 <gmann> (gmann) Stable/ussuri is blocked for Tempest master and master constraints 15:47:38 <gmann> this is more of information as I have not finished the work 15:48:00 <gmann> stable/ussuri is in EM and failing with Tempest master and master constraint 15:48:29 <kopecmartin> thanks for the info 15:48:31 <gmann> I started pining tempest for stable/ussuri but then got distracted, I am doing now and should be ready soon https://review.opendev.org/q/topic:ussuri-pin-tempest 15:48:51 <gmann> and I think once we figure this out then we can make new release of tempest 15:49:04 <gmann> which is this #link https://review.opendev.org/q/topic:ussuri-last 15:49:06 <opendevreview> Rajat Dhasmana proposed openstack/tempest master: Add test to rebuild volume backed instance https://review.opendev.org/c/openstack/tempest/+/831018 15:49:16 <gmann> and we can add poy36|7 drop also in that release or so 15:49:27 <gmann> but we agreed on dropping py36|7 from tempest right? 15:50:28 <kopecmartin> gmann: we did 15:50:44 <gmann> cool. I will work on these two gate blocker and should be ready soon 15:50:50 <gmann> that is all from me 15:51:04 <kopecmartin> thank you gmann 15:51:07 <frickler> o.k., just to complete the ansible thing I mentioned 15:51:22 <frickler> zuul currently runs with ansible 2.9 (I think?) which is eol 15:51:45 <frickler> zuul 6.0.0 now supports to switch to ansible 5, which I think is ansible-core 2.12 15:51:48 <clarkb> yes 2.9 is the default 15:52:01 <frickler> there is a test patch up in devstack, which shows some work to do 15:52:19 <frickler> not urgent, but also not to be delayed forever 15:52:31 <clarkb> https://review.opendev.org/c/openstack/devstack/+/843033 currently it fails when trying to become the stack user. I don't understand it yet but they have a whole document on the problem at https://docs.ansible.com/ansible-core/2.12/user_guide/become.html#risks-of-becoming-an-unprivileged-user 15:52:32 <frickler> #link https://review.opendev.org/c/openstack/devstack/+/843033/ 15:53:44 <frickler> maybe switch to accessing the test nodes as stack user directly instead of zuul sudoing? 15:54:16 <frickler> or dump the stack user and run as zuul. locally I run as ubuntu all the time 15:54:44 <clarkb> ya I'm not sure yet. Probably good to understand why 2.9 worked but 5 doesn't and go from there 15:56:20 <frickler> o.k., that's it from me, thx clarkb for filling in the details 15:56:23 <kopecmartin> thanks for bringing this up frickler 15:56:52 <kopecmartin> i need to run 15:56:56 <kopecmartin> thank you everyone 15:57:00 <kopecmartin> see you around 15:57:01 <kopecmartin> #endmeeting