14:00:37 <gmann> #startmeeting qa 14:00:37 <opendevmeet> Meeting started Tue Aug 3 14:00:37 2021 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:37 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:37 <opendevmeet> The meeting name has been set to 'qa' 14:00:49 <gmann> who all are here today? 14:01:13 <yoctozepto> \o/ 14:01:16 <croelandt> o/ 14:02:25 <opendevreview> Elod Illes proposed openstack/devstack-gate master: Remove neutron-lbaas from PROJECTS list https://review.opendev.org/c/openstack/devstack-gate/+/803346 14:03:15 <gmann> let's start 14:03:40 <gmann> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours 14:03:43 <gmann> today agenda ^^ 14:04:12 <gmann> #topic Announcement and Action Item (Optional) 14:04:26 <gmann> no announcement or AI from previous meeting 14:04:40 <gmann> #topic Xena Priority Items progress 14:04:49 <gmann> #link https://etherpad.opendev.org/p/qa-xena-priority 14:05:40 <gmann> I have patch up for requirement to move tempest from u-c to blacklist file so that tempest master can be used for plugins job 14:05:43 <gmann> #link https://review.opendev.org/c/openstack/requirements/+/787621 14:05:49 <jparoly> o/ 14:06:06 <gmann> no other updates on rest of remaining Xena items 14:06:18 <gmann> #topic OpenStack Events Updates and Planning 14:06:27 <gmann> #link https://etherpad.opendev.org/p/qa-yoga-ptg 14:06:42 <gmann> for PTG, please start adding the topics you would like to discuss 14:08:01 <gmann> #topic Gate Status Checks 14:08:14 <gmann> we have two failure to discuss today 14:08:41 <gmann> one is grenade job failure 14:08:43 <gmann> #link https://review.opendev.org/c/openstack/grenade/+/803317 14:09:07 <gmann> frickler raising the question of backward incompatible change. 14:09:20 <abhishekk> this is blocking glance/cinder/nova and probably others as well 14:09:40 <croelandt> yeah, this is blocking our RBAC work :/ 14:09:46 <gmann> If i understand the original bug clearly, cinder change was done as bug fix instead of microversion 14:10:02 <gmann> #link https://bugs.launchpad.net/cinder/+bug/1926630 14:10:39 <gmann> so the voluem create request fails later so new api checks are just an early failure? 14:10:39 <whoami-rajat__> grenade doesn't check the encryption flow with barbican and also not with the failing drivers so it will never see that failure but it is still wrong from cinder standpoint 14:11:06 <whoami-rajat__> cinder is just moving the async failure to a sync failure by moving the failing code from c-vol to c-api service 14:11:07 <gmann> whoami-rajat__: you mean the volume create request can pass ? 14:11:31 <whoami-rajat__> for LVM, not for rbd nfs and probably other drivers 14:11:39 <gmann> so back to frickler question, how they were passing before cinder change? 14:11:50 <gmann> whoami-rajat__: yeah that is difference 14:12:01 <gmann> I remember we did that change in tempest test too for that reason 14:12:33 <gmann> I am ok for the change as it is changing 4/5xx->400 only 14:12:59 <whoami-rajat__> also how grenade is trying to create an encrypted volume type is not correct 14:13:08 <whoami-rajat__> so it needs to be corrected anyway 14:13:45 <gmann> whoami-rajat__: anything else wrong other than passing cipher ? 14:14:28 <whoami-rajat__> gmann, nothing as i remember, i can go through it again but for now my patch passes glance gate so it seems good 14:14:38 <opendevreview> yatin proposed openstack/devstack master: [DNM] Test c8-stream / fedora-latest with fixes https://review.opendev.org/c/openstack/devstack/+/803144 14:14:40 <gmann> whoami-rajat__: ack thanks 14:14:56 <gmann> frickler: yoctozepto any comment on this how to proceed and unblock gate? 14:16:14 <opendevreview> yatin proposed openstack/devstack master: [DNM] Test c8-stream with reduced tempest concurrency https://review.opendev.org/c/openstack/devstack/+/803144 14:16:26 <yoctozepto> still not understanding why it passed before 14:16:36 <yoctozepto> no testing after request? 14:16:48 <gmann> for lvm which is default in grenade does pass but not other backend like barbican 14:17:03 <gmann> yoctozepto: in grenade yes, it just create it 14:17:13 <yoctozepto> ok, makes sense 14:17:17 * yoctozepto approving 14:18:34 <gmann> thanks, seems like frickler is not online now so let's unblock the gate and if any strong objection from him then we can continue discussion 14:18:46 <gmann> moving to 2nd gate issue 14:18:53 <gmann> centos8 support and fix 14:18:55 <gmann> #link https://review.opendev.org/q/I508eceb00d7501ffcfac73d7bc2272badb241494 14:19:26 * yoctozepto approving rdo fix 14:19:27 <gmann> we have centos8 job fix and it work for victoria, you can see job green in victoria backport 14:19:49 <gmann> yoctozepto: tricky thing here is how to fix centos8 in wallaby 14:20:27 <yoctozepto> you mean stream-only? 14:20:48 <gmann> yeah 14:20:56 <gmann> comment from yatin: "For wallaby need to use c8-strem job devstack-platform-centos-8-stream as RDO wallaby released only for c8-stream so may not work for c8" 14:21:36 <gmann> as there is no wallaby release for centos8 so question is 1. how to fix it or 2. should we change centos8 to centos8-stream in wallaby ? 14:22:00 <yoctozepto> I would as it makes more sense support-wise 14:22:01 <gmann> and wallaby testing runtime defined in TC is centos8 not centos8-stream so 2nd one is more tricky 14:22:10 <tosky> just to clarify: when I've advocated for exceptions on the list thread, I was suggesting to move all centos linux 8 jobs to centos stream 8 whenever an openstack release is still alive after centos linux 8 EOL 14:22:12 <yoctozepto> yeah 14:22:50 <gmann> IMO, we should not change the stable runtime at this stage 14:23:03 <ykarel> centos8-stream is replacement for centos8, so would need to switch atleast when c8 goes EOL at year end 14:23:14 <ykarel> and would be better even before 14:23:15 <gmann> ykarel: yes we did in Xena 14:23:29 <gmann> but we should do in all stable branch or not that is bug question or work 14:23:35 <ykarel> in puppet and tripleo have already switched to c8-stream 14:23:35 <gmann> *big 14:23:43 <gmann> ykarel: all stable also? 14:24:09 <ykarel> gmann, for other stable c8 should work from RDO perspective as we support on both c8 and c8-stream 14:24:19 <ykarel> but wallaby+ only c8-stream 14:24:31 <ykarel> and this would change at year end when c8 goes eol 14:24:36 <gmann> yeah so wallaby also changed right which is in questrion 14:24:51 <tosky> ykarel: what will happen to victoria after c8 EOL? 14:25:07 <gmann> tosky: it still work with old c8 14:25:13 <gmann> as worked in devstack patch 14:25:25 <ykarel> tosky, i think repos will stale for C8 post EOL, i think it's currently being discussed 14:25:31 <ykarel> so not have full context for that 14:25:38 <tosky> gmann: right now, but December is close :) 14:25:44 <ykarel> so atleast there will be no update post December 14:25:53 <ykarel> and repos may exist for some time 14:26:04 <gmann> so just curious, how c8 community migrate the customer using c8 for openstack victoria or wallaby? 14:26:26 <gmann> ykarel: tosky that is risky as you are going to delete the old released/used repo ? 14:26:48 <tosky> that's what I'm trying to understand 14:26:57 <ykarel> c8 users will migrated to either c8-stream or rhel or any other rebuild of rhel 14:27:38 <gmann> ykarel: that seems a contract break if anyone has their production running on c8 with old openstack where it used to work and now might stop ? 14:28:08 <yoctozepto> yeah, I don't like the way it sounds 14:28:13 <gmann> I do not know usage contract of c8 but still 14:28:31 <yoctozepto> I am open to supporting Vicky on centos 8 stream though 14:28:35 <gmann> yoctozepto: yeah, it seems very confusing for me and how migration is planned for old usage 14:28:45 <ykarel> gmann, it should work with both c8 and c8-stream for supported releases 14:28:55 <ykarel> there were some announcement for it 14:28:59 <ykarel> need to fetch link 14:29:10 <gmann> ykarel: but you are saying at some time when repo are deleted then it might not work like currently for stable/wallaby 14:29:12 <tosky> but technically centos 8 stream is not really different from the base one - just gets the updates first instead of later 14:29:46 <ykarel> gmann, for stable/wallaby repos were never published for c8 14:29:54 <ykarel> it was c8-stream from beginning 14:29:58 <yoctozepto> gmann: as tosky and ykarel say, it mostly works by replacing the repos and doing an update 14:30:09 <gmann> ykarel: ohk, then is it miss in our testing runtime? 14:30:16 <ykarel> http://mirror.centos.org/centos/8-stream/cloud/x86_64/ 14:30:27 <ykarel> vs http://mirror.centos.org/centos/8/cloud/x86_64/ 14:30:30 <yoctozepto> gmann: it was announced late 14:30:43 <gmann> ohk 14:31:32 <gmann> how about this plan ?: 1. discuss in TC about updating the wallaby testing runtime to centos8-stream 2. accordingly QA proceed on changes. 14:33:09 <yoctozepto> I agree 14:33:09 <gmann> this is for stable/wallaby only but for victoria it is right fix and make c8 job green #link https://review.opendev.org/c/openstack/devstack/+/803072 14:33:17 <gmann> so I am ok to merge the victoria fix 14:33:31 <gmann> and wallaby one we can continue discussion as per above plan 14:33:34 <yoctozepto> gmann: as need be, I am open to doing the same to Vicky as c8 goes eol before Vicky does 14:34:32 <gmann> yoctozepto: I think we should not change victoria testing as it is changing the history and support. wallaby is fine as there is no way and centos8-stream announced in wallaby time but little later 14:35:10 <ykarel> gmann, that fix can be merged in wallaby too 14:35:13 <yoctozepto> gmann: well, technically it's the same distro, except for the delivery model 14:35:29 <yoctozepto> I'm just saying I'm fine with either going on stream 14:35:54 <yoctozepto> it's a weird situation but it's not that hard for us to adapt now that we support stream in CI 14:36:14 <gmann> ykarel: that does not fix anything in wallaby, there is no c8-stream job there and c8 still failing. I would like to see some job running with that fix 14:36:51 <gmann> yoctozepto: humm, not sure still. anyways let's discuss in TC how much backport we can do for c8->c8-stream 14:36:54 <ykarel> gmann, actually the fix will be needed for any c8 release 14:37:15 <gmann> ykarel: confused. for wallaby c8 release? 14:37:31 <yoctozepto> gmann: yeah, I would prefer we did not have to at all but life ain't easy this time of the century 14:38:52 <gmann> sure. we can proceed as what TC decide on this. as QA we need to comply with defined runtime. 14:38:54 <ykarel> gmann, so the patch is specific to how rpm name is computed for c8, it's doing it's work in computing the correct name for wallaby and installing the rpm 14:39:10 <ykarel> but since the job there is c8, it's looking for wrong repo and fails 14:39:30 <ykarel> but okk if it's squashed with c8-stream job addition 14:39:57 <gmann> ykarel: yeah, let's wait for Thursday TC meet and then we can proceed on this. 14:40:04 <ykarel> ok Thanks 14:40:31 <gmann> ykarel: you would like to merge victoria one now? I am +2 on that and to merge before wallaby 14:40:56 <ykarel> gmann, fine by me, it's unblocking some cinder jobs 14:41:02 <ykarel> so should be ok to go with it 14:41:10 <gmann> yeah please remove your -W 14:41:19 <gmann> yoctozepto: what you say? https://review.opendev.org/c/openstack/devstack/+/803072 14:41:26 <ykarel> removing 14:41:46 <gmann> though we are skipping wallaby backport and merging vicrotia first but I think it is fine in this siutation 14:41:46 <yoctozepto> gmann: cherrypick id perhaps? 14:41:56 <gmann> yoctozepto: ah yeah that can be updated. +1 14:41:58 <yoctozepto> I am ok with the skip 14:41:58 <gmann> ykarel: ^^ 14:42:02 <gmann> k 14:42:04 <yoctozepto> just add the cherrypick metadata 14:42:04 <ykarel> sure will update 14:42:09 <gmann> thanks 14:42:10 <yoctozepto> ok 14:42:28 <gmann> any other gate blocker? 14:42:51 <gmann> #topic Periodic jobs Status Checks 14:42:55 <gmann> #link https://zuul.openstack.org/builds?job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-train-py3&pipeline=periodic-stable 14:43:06 <gmann> #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic 14:43:11 <gmann> all green here 14:43:25 <gmann> #topic Sub Teams highlights 14:43:39 <gmann> any updates, highlights we need to discuss for today ? 14:43:48 <gmann> from subteam 14:44:13 <opendevreview> yatin proposed openstack/devstack stable/victoria: Fix Usage of rdo-release rpm https://review.opendev.org/c/openstack/devstack/+/803072 14:44:23 <gmann> #topic Gate Blocker Fix / Urgent Change 14:44:26 <gmann> #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking) 14:44:29 <gmann> nothing here 14:44:39 <gmann> I think we can merge this to about topic of gate status 14:44:53 <gmann> as both are relevant and overlap 14:45:25 <gmann> #action gmann merge the two gate check topic into one 14:45:30 <gmann> #topic Open Discussion 14:45:45 <gmann> anything else to discuss by anyone which is not in agenda ? 14:46:17 <rosmaita> just want to say thanks for the work on getting centos-8 devstack build working in victoria 14:46:29 <eliadcohen> +1 14:46:38 <gmann> rosmaita: +1, thanks to ykarel 14:46:50 <rosmaita> ykarel: ++ 14:47:18 <gmann> moving next 14:47:21 <gmann> #topic Bug Triage 14:47:24 <ykarel> also in noticed tempest tests random failures in master job, so trying out tempest concurrency reduce https://review.opendev.org/c/openstack/devstack/+/803144 14:48:30 <gmann> ykarel: ok, can you file a bug and how frequent it is? and which test 14:48:57 <ykarel> gmann, sure will file 14:49:02 <gmann> thanks 14:49:06 <gmann> #link https://etherpad.opendev.org/p/qa-bug-triage-xena 14:49:10 <gmann> bug report ^^ 14:49:25 <gmann> no critical bug or other that what we already discussed 14:49:43 <gmann> that is all from me today. any other bug we need to discuss? 14:49:49 <yoctozepto> not from me 14:50:17 <gmann> k 14:50:19 <gmann> thanks all for joining. 14:50:22 <gmann> #endmeeting