14:00:37 <gmann> #startmeeting qa
14:00:37 <opendevmeet> Meeting started Tue Aug  3 14:00:37 2021 UTC and is due to finish in 60 minutes.  The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:37 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:37 <opendevmeet> The meeting name has been set to 'qa'
14:00:49 <gmann> who all are here today?
14:01:13 <yoctozepto> \o/
14:01:16 <croelandt> o/
14:02:25 <opendevreview> Elod Illes proposed openstack/devstack-gate master: Remove neutron-lbaas from PROJECTS list  https://review.opendev.org/c/openstack/devstack-gate/+/803346
14:03:15 <gmann> let's start
14:03:40 <gmann> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours
14:03:43 <gmann> today agenda ^^
14:04:12 <gmann> #topic Announcement and Action Item (Optional)
14:04:26 <gmann> no announcement or AI from previous meeting
14:04:40 <gmann> #topic Xena Priority Items progress
14:04:49 <gmann> #link https://etherpad.opendev.org/p/qa-xena-priority
14:05:40 <gmann> I have patch up for requirement to move tempest from u-c to blacklist file so that tempest master can be used for plugins job
14:05:43 <gmann> #link https://review.opendev.org/c/openstack/requirements/+/787621
14:05:49 <jparoly> o/
14:06:06 <gmann> no other updates on rest of remaining Xena items
14:06:18 <gmann> #topic OpenStack Events Updates and Planning
14:06:27 <gmann> #link https://etherpad.opendev.org/p/qa-yoga-ptg
14:06:42 <gmann> for PTG, please start adding the topics you would like to discuss
14:08:01 <gmann> #topic Gate Status Checks
14:08:14 <gmann> we have two failure to discuss today
14:08:41 <gmann> one is grenade job failure
14:08:43 <gmann> #link https://review.opendev.org/c/openstack/grenade/+/803317
14:09:07 <gmann> frickler raising the question of backward incompatible change.
14:09:20 <abhishekk> this is blocking glance/cinder/nova and probably others as well
14:09:40 <croelandt> yeah, this is blocking our RBAC work :/
14:09:46 <gmann> If i understand the original bug clearly, cinder change was done as bug fix instead of microversion
14:10:02 <gmann> #link https://bugs.launchpad.net/cinder/+bug/1926630
14:10:39 <gmann> so the voluem create request fails later so new api checks are just an early failure?
14:10:39 <whoami-rajat__> grenade doesn't check the encryption flow with barbican and also not with the failing drivers so it will never see that failure but it is still wrong from cinder standpoint
14:11:06 <whoami-rajat__> cinder is just moving the async failure to a sync failure by moving the failing code from c-vol to c-api service
14:11:07 <gmann> whoami-rajat__: you mean the volume create request can pass ?
14:11:31 <whoami-rajat__> for LVM, not for rbd nfs and probably other drivers
14:11:39 <gmann> so back to frickler question, how they were passing before cinder change?
14:11:50 <gmann> whoami-rajat__: yeah that is difference
14:12:01 <gmann> I remember we did that change in tempest test too for that reason
14:12:33 <gmann> I am ok for the change as it is changing 4/5xx->400 only
14:12:59 <whoami-rajat__> also how grenade is trying to create an encrypted volume type is not correct
14:13:08 <whoami-rajat__> so it needs to be corrected anyway
14:13:45 <gmann> whoami-rajat__: anything else wrong other than passing cipher ?
14:14:28 <whoami-rajat__> gmann, nothing as i remember, i can go through it again but for now my patch passes glance gate so it seems good
14:14:38 <opendevreview> yatin proposed openstack/devstack master: [DNM] Test c8-stream / fedora-latest with fixes  https://review.opendev.org/c/openstack/devstack/+/803144
14:14:40 <gmann> whoami-rajat__: ack thanks
14:14:56 <gmann> frickler: yoctozepto any comment on this how to proceed and unblock gate?
14:16:14 <opendevreview> yatin proposed openstack/devstack master: [DNM] Test c8-stream with reduced tempest concurrency  https://review.opendev.org/c/openstack/devstack/+/803144
14:16:26 <yoctozepto> still not understanding why it passed before
14:16:36 <yoctozepto> no testing after request?
14:16:48 <gmann> for lvm which is default in grenade does pass but not other backend like barbican
14:17:03 <gmann> yoctozepto: in grenade yes, it just create it
14:17:13 <yoctozepto> ok, makes sense
14:17:17 * yoctozepto approving
14:18:34 <gmann> thanks, seems like frickler is not online now so let's unblock the gate and if any strong objection from him then we can continue discussion
14:18:46 <gmann> moving to 2nd gate issue
14:18:53 <gmann> centos8 support and fix
14:18:55 <gmann> #link https://review.opendev.org/q/I508eceb00d7501ffcfac73d7bc2272badb241494
14:19:26 * yoctozepto approving rdo fix
14:19:27 <gmann> we have centos8 job fix and it work for victoria, you can see job green in victoria backport
14:19:49 <gmann> yoctozepto: tricky thing here is how to fix centos8 in wallaby
14:20:27 <yoctozepto> you mean stream-only?
14:20:48 <gmann> yeah
14:20:56 <gmann> comment from yatin: "For wallaby need to use c8-strem job devstack-platform-centos-8-stream as RDO wallaby released only for c8-stream so may not work for c8"
14:21:36 <gmann> as there is no wallaby release for centos8 so question is 1. how to fix it or 2. should we change centos8 to centos8-stream in wallaby ?
14:22:00 <yoctozepto> I would as it makes more sense support-wise
14:22:01 <gmann> and wallaby testing runtime defined in TC is centos8 not centos8-stream so 2nd one is more tricky
14:22:10 <tosky> just to clarify: when I've advocated for exceptions on the list thread, I was suggesting to move all centos linux 8 jobs to centos stream 8 whenever an openstack release is still alive after centos linux 8 EOL
14:22:12 <yoctozepto> yeah
14:22:50 <gmann> IMO, we should not change the stable runtime at this stage
14:23:03 <ykarel> centos8-stream is replacement for centos8, so would need to switch atleast when c8 goes EOL at year end
14:23:14 <ykarel> and would be better even before
14:23:15 <gmann> ykarel: yes we did in Xena
14:23:29 <gmann> but we should do in all stable branch or not that is bug question or work
14:23:35 <ykarel> in puppet and tripleo have already switched to c8-stream
14:23:35 <gmann> *big
14:23:43 <gmann> ykarel: all stable also?
14:24:09 <ykarel> gmann, for other stable c8 should work from RDO perspective as we support on both c8 and c8-stream
14:24:19 <ykarel> but wallaby+ only c8-stream
14:24:31 <ykarel> and this would change at year end when c8 goes eol
14:24:36 <gmann> yeah so wallaby also changed right which is in questrion
14:24:51 <tosky> ykarel: what will happen to victoria after c8 EOL?
14:25:07 <gmann> tosky: it still work with old c8
14:25:13 <gmann> as worked in devstack patch
14:25:25 <ykarel> tosky, i think repos will stale for C8 post EOL, i think it's currently being discussed
14:25:31 <ykarel> so not have full context for that
14:25:38 <tosky> gmann: right now, but December is close :)
14:25:44 <ykarel> so atleast there will be no update post December
14:25:53 <ykarel> and repos may exist for some time
14:26:04 <gmann> so just curious, how c8 community migrate the customer using c8 for openstack victoria or wallaby?
14:26:26 <gmann> ykarel: tosky that is risky as you are going to delete the old released/used repo ?
14:26:48 <tosky> that's what I'm trying to understand
14:26:57 <ykarel> c8 users will migrated to either c8-stream or rhel or any other rebuild of rhel
14:27:38 <gmann> ykarel: that seems a contract break if anyone has their production running on c8 with old openstack where it used to work and now might stop ?
14:28:08 <yoctozepto> yeah, I don't like the way it sounds
14:28:13 <gmann> I do not know usage contract of c8 but still
14:28:31 <yoctozepto> I am open to supporting Vicky on centos 8 stream though
14:28:35 <gmann> yoctozepto: yeah, it seems very confusing for me and how migration is planned for old usage
14:28:45 <ykarel> gmann, it should work with both c8 and c8-stream for supported releases
14:28:55 <ykarel> there were some announcement for it
14:28:59 <ykarel> need to fetch link
14:29:10 <gmann> ykarel: but you are saying at some time when repo are deleted then it might not work like currently for stable/wallaby
14:29:12 <tosky> but technically centos 8 stream is not really different from the base one - just gets the updates first instead of later
14:29:46 <ykarel> gmann, for stable/wallaby repos were never published for c8
14:29:54 <ykarel> it was c8-stream from beginning
14:29:58 <yoctozepto> gmann: as tosky and ykarel say, it mostly works by replacing the repos and doing an update
14:30:09 <gmann> ykarel: ohk, then is it miss in our testing runtime?
14:30:16 <ykarel> http://mirror.centos.org/centos/8-stream/cloud/x86_64/
14:30:27 <ykarel> vs http://mirror.centos.org/centos/8/cloud/x86_64/
14:30:30 <yoctozepto> gmann: it was announced late
14:30:43 <gmann> ohk
14:31:32 <gmann> how about this plan ?: 1. discuss in TC about updating the wallaby testing runtime to centos8-stream 2. accordingly QA proceed on changes.
14:33:09 <yoctozepto> I agree
14:33:09 <gmann> this is for stable/wallaby only but for victoria it is right fix and make c8 job green #link https://review.opendev.org/c/openstack/devstack/+/803072
14:33:17 <gmann> so I am ok to merge the victoria fix
14:33:31 <gmann> and wallaby one we can continue discussion as per above plan
14:33:34 <yoctozepto> gmann: as need be, I am open to doing the same to Vicky as c8 goes eol before Vicky does
14:34:32 <gmann> yoctozepto: I think we should not change victoria testing as it is changing the history and support. wallaby is fine as there is no way and centos8-stream announced in wallaby time but little later
14:35:10 <ykarel> gmann, that fix can be merged in wallaby too
14:35:13 <yoctozepto> gmann: well, technically it's the same distro, except for the delivery model
14:35:29 <yoctozepto> I'm just saying I'm fine with either going on stream
14:35:54 <yoctozepto> it's a weird situation but it's not that hard for us to adapt now that we support stream in CI
14:36:14 <gmann> ykarel:  that does not fix anything in wallaby, there is no c8-stream job there and c8 still failing. I would like to see some job running with that fix
14:36:51 <gmann> yoctozepto: humm, not sure still. anyways let's discuss in TC how much backport we can do for c8->c8-stream
14:36:54 <ykarel> gmann, actually the fix will be needed for any c8 release
14:37:15 <gmann> ykarel: confused. for wallaby c8 release?
14:37:31 <yoctozepto> gmann: yeah, I would prefer we did not have to at all but life ain't easy this time of the century
14:38:52 <gmann> sure. we can proceed as what TC decide on this. as QA we need to comply with defined runtime.
14:38:54 <ykarel> gmann, so the patch is specific to how rpm name is computed for c8, it's doing it's work in computing the correct name for wallaby and installing the rpm
14:39:10 <ykarel> but since the job there is c8, it's looking for wrong repo and fails
14:39:30 <ykarel> but okk if it's squashed with c8-stream job addition
14:39:57 <gmann> ykarel: yeah, let's wait for Thursday TC meet and then we can proceed on this.
14:40:04 <ykarel> ok Thanks
14:40:31 <gmann> ykarel: you would like to merge victoria one now? I am +2 on that and to merge before wallaby
14:40:56 <ykarel> gmann, fine by me, it's unblocking some cinder jobs
14:41:02 <ykarel> so should be ok to go with it
14:41:10 <gmann> yeah please remove your -W
14:41:19 <gmann> yoctozepto: what you say? https://review.opendev.org/c/openstack/devstack/+/803072
14:41:26 <ykarel> removing
14:41:46 <gmann> though we are skipping wallaby backport and merging vicrotia first but I think it is fine in this siutation
14:41:46 <yoctozepto> gmann: cherrypick id perhaps?
14:41:56 <gmann> yoctozepto: ah yeah that can be updated. +1
14:41:58 <yoctozepto> I am ok with the skip
14:41:58 <gmann> ykarel: ^^
14:42:02 <gmann> k
14:42:04 <yoctozepto> just add the cherrypick metadata
14:42:04 <ykarel> sure will update
14:42:09 <gmann> thanks
14:42:10 <yoctozepto> ok
14:42:28 <gmann> any other gate blocker?
14:42:51 <gmann> #topic Periodic jobs Status Checks
14:42:55 <gmann> #link https://zuul.openstack.org/builds?job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-train-py3&pipeline=periodic-stable
14:43:06 <gmann> #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic
14:43:11 <gmann> all green here
14:43:25 <gmann> #topic Sub Teams highlights
14:43:39 <gmann> any updates, highlights we need to discuss for today ?
14:43:48 <gmann> from subteam
14:44:13 <opendevreview> yatin proposed openstack/devstack stable/victoria: Fix Usage of rdo-release rpm  https://review.opendev.org/c/openstack/devstack/+/803072
14:44:23 <gmann> #topic Gate Blocker Fix / Urgent Change
14:44:26 <gmann> #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking)
14:44:29 <gmann> nothing here
14:44:39 <gmann> I think we can merge this to about topic of gate status
14:44:53 <gmann> as both are relevant and overlap
14:45:25 <gmann> #action gmann merge the two gate check topic into one
14:45:30 <gmann> #topic Open Discussion
14:45:45 <gmann> anything else to discuss by anyone which is not in agenda ?
14:46:17 <rosmaita> just want to say thanks for the work on getting centos-8 devstack build working in victoria
14:46:29 <eliadcohen> +1
14:46:38 <gmann> rosmaita: +1, thanks to ykarel
14:46:50 <rosmaita> ykarel: ++
14:47:18 <gmann> moving next
14:47:21 <gmann> #topic Bug Triage
14:47:24 <ykarel> also in noticed tempest tests random failures in master job, so trying out tempest concurrency reduce https://review.opendev.org/c/openstack/devstack/+/803144
14:48:30 <gmann> ykarel: ok, can you file a bug and how frequent it is? and which test
14:48:57 <ykarel> gmann, sure will file
14:49:02 <gmann> thanks
14:49:06 <gmann> #link https://etherpad.opendev.org/p/qa-bug-triage-xena
14:49:10 <gmann> bug report  ^^
14:49:25 <gmann> no critical bug or other that what we already discussed
14:49:43 <gmann> that is all from me today. any other bug we need to discuss?
14:49:49 <yoctozepto> not from me
14:50:17 <gmann> k
14:50:19 <gmann> thanks all for joining.
14:50:22 <gmann> #endmeeting