| *** pojadhav|afk is now known as pojadhav | 02:36 | |
| *** jpena|off is now known as jpena | 08:03 | |
| jm1 | Hello :) Atm devstack has issues with reboots, e.g. swift and cinder will fail (kind of) because their file based storage wont be recreated on boot. Is this by intention/design? | 08:21 |
|---|---|---|
| jm1 | If not, maybe we should fix this? E.g. https://review.opendev.org/c/openstack/devstack/+/827918 | 08:22 |
| *** akekane_ is now known as abhishekk | 08:31 | |
| frickler | jm1: I'd say we don't intentionally break on reboot, but we also don't test that scenario, so it will be difficult to detect regressions, in particular for non-default scenarios. but I think patches to improve things should be welcome | 09:08 |
| *** lifeless_ is now known as lifeless | 09:24 | |
| opendevreview | LiZekun proposed openstack/hacking master: Use TOX_CONSTRAINTS_FILE https://review.opendev.org/c/openstack/hacking/+/828256 | 09:34 |
| opendevreview | LiZekun proposed openstack/hacking master: Use TOX_CONSTRAINTS_FILE https://review.opendev.org/c/openstack/hacking/+/828256 | 09:35 |
| *** chaconpiza__ is now known as chaconpiza | 09:42 | |
| jm1 | frickler: hmmm... to catch regressions, couldn't we/I add a reboot to the zuul playbooks just at the end of the devstack setup? | 10:23 |
| frickler | jm1: I don't think we would want to do this for every job, but maybe a dedicated job that does this and runs tempest afterwards would be feasible. you may want to look at the fips jobs for how to reboot and reconnect with zuul again | 10:38 |
| jm1 | frickler: ok, will first fix cinder/lvm else it is broken from the start ^^ | 10:39 |
| opendevreview | Jakob Meng proposed openstack/devstack master: Made LVM backing disk persistent https://review.opendev.org/c/openstack/devstack/+/828280 | 11:07 |
| jm1 | submitted a patch against devstack which fixes reboot issues with LVM backing files. https://review.opendev.org/c/openstack/devstack/+/828280 | 11:10 |
| opendevreview | Jakob Meng proposed openstack/devstack master: Added Zuul job devstack-reboot https://review.opendev.org/c/openstack/devstack/+/828289 | 12:06 |
| *** pojadhav is now known as pojadhav|afk | 12:33 | |
| *** dmellado_ is now known as dmellado | 13:20 | |
| *** pojadhav|afk is now known as pojadhav | 13:31 | |
| *** Guest2040 is now known as spotz | 14:37 | |
| opendevreview | Jakob Meng proposed openstack/devstack master: Added Zuul job devstack-reboot https://review.opendev.org/c/openstack/devstack/+/828289 | 14:47 |
| opendevreview | Attila Fazekas proposed opendev/elastic-recheck master: Revert "Add query for WARN: failed: route add -net bug 1857151" https://review.opendev.org/c/opendev/elastic-recheck/+/828355 | 14:52 |
| *** akaha|rover is now known as akahat|dinner | 14:55 | |
| kopecmartin | #startmeeting qa | 15:00 |
| opendevmeet | Meeting started Tue Feb 8 15:00:19 2022 UTC and is due to finish in 60 minutes. The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'qa' | 15:00 |
| kopecmartin | #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Weekly_QA_Team_meeting | 15:00 |
| kopecmartin | agenda ^^ | 15:00 |
| gmann | o/ | 15:00 |
| kopecmartin | hi gmann, let's start | 15:02 |
| kopecmartin | #topic Announcement and Action Item (Optional) | 15:02 |
| kopecmartin | we have PTL elections coming up | 15:02 |
| kopecmartin | #link https://governance.openstack.org/election/ | 15:03 |
| gmann | yeah, nomination is open now until 15th Feb | 15:03 |
| frickler | kopecmartin: do you want to run again? | 15:03 |
| frickler | I would support that | 15:03 |
| gmann | I hope so :) | 15:03 |
| kopecmartin | i wanted to ask first whether there is anyone who would like to run for qa PTL | 15:04 |
| gmann | I too support for sure | 15:04 |
| gmann | not me | 15:04 |
| alexYefimov_ | kopecmartin ++ | 15:04 |
| kopecmartin | if not, then yes, i'd like to run again | 15:04 |
| gmann | thanks kopecmartin | 15:05 |
| frickler | +1 | 15:05 |
| kopecmartin | thanks :) | 15:05 |
| kopecmartin | #topic Yoga Priority Items progress | 15:06 |
| kopecmartin | #link https://etherpad.opendev.org/p/qa-yoga-priority | 15:06 |
| kopecmartin | any updates on the priority items? | 15:06 |
| gmann | on RBAC, I am trying to finish the new policy change in Nova first and then will go through the devstack required changes based on what new default nova end up with | 15:06 |
| kopecmartin | regarding FIPS, we've merged some patches , i need to check what else is needed | 15:07 |
| gmann | two nodeset for centos stream is also merged | 15:07 |
| gmann | there are multinode job adding patch but that is failing somewhere. I commented on that | 15:07 |
| kopecmartin | gmann: thanks | 15:08 |
| kopecmartin | regarding rbac, let me know if i can do anything | 15:08 |
| kopecmartin | i'd like to get to the point where we can see where tempest stands on a s-rbac env | 15:08 |
| gmann | yeah, policy defaults are again changing on service side so tempest needs to hold it until they are ready | 15:09 |
| frickler | there's also this fix for neutron system scope, which needs reviews https://review.opendev.org/c/openstack/devstack/+/826851 | 15:09 |
| gmann | keystone is already ready, nova will be in Yoga release along with other like neutron or so | 15:10 |
| gmann | frickler: ack, will check | 15:10 |
| kopecmartin | that one is on my list too | 15:11 |
| frickler | for keystone the changes in devstack are also still pending, I just rechecked two, the next needs rebasing | 15:11 |
| gmann | yeah, i saw those. waiting for the gate result | 15:12 |
| gmann | also we need to move those setting from keystone if it work before test phase. need to check my old patch which was failing earlier | 15:13 |
| kopecmartin | Cleanup of duplicated scenario.manager - one of our interns is working on this, a good task for initial onboarding | 15:14 |
| gmann | +1, nice | 15:14 |
| kopecmartin | and i have no updates for the rest of the items | 15:14 |
| kopecmartin | moving on | 15:15 |
| kopecmartin | #topic OpenStack Events Updates and Planning | 15:15 |
| kopecmartin | we'll start gathering ptg topics once the Z release is named | 15:15 |
| kopecmartin | should be done anytime soon ,right? | 15:15 |
| frickler | IIUC it depends on trademark checking | 15:16 |
| kopecmartin | good, thanks | 15:16 |
| gmann | kopecmartin: yeah, 1 more week and we will be ready. | 15:17 |
| kopecmartin | btw, team registration is open, i have in my TODO, i'll fill the registration form and will follow up with a doodle to find the best time slots for our sessions | 15:17 |
| kopecmartin | gmann: perfect | 15:17 |
| gmann | +1 | 15:17 |
| kopecmartin | #topic Gate Status Checks | 15:17 |
| kopecmartin | #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking) | 15:17 |
| gmann | kopecmartin: I will also schedule TC+PTL sessions early PTG week, just to note for scheduling QA sessions. | 15:17 |
| frickler | F35 I approved before the meeting | 15:18 |
| frickler | and notified neutron+nova | 15:18 |
| kopecmartin | gmann: ack | 15:18 |
| kopecmartin | frickler: thanks | 15:18 |
| gmann | frickler: do you know centos-stream8 is stable now, so that we can again make the jobs voting ? | 15:19 |
| frickler | I saw a couple of failures in the F35, so IMO it is not stable | 15:19 |
| gmann | they are not stable seems https://zuul.openstack.org/builds?job_name=tempest-integrated-compute-centos-8-stream&job_name=tempest+tempest-full-py3-centos-8-stream&skip=0 | 15:19 |
| frickler | in the F35 change | 15:19 |
| gmann | frickler: yeah | 15:20 |
| frickler | and centos-9-stream is hard broken with some attachment failures | 15:20 |
| frickler | very likely some libvirt/qemu version issue with nova | 15:21 |
| frickler | someone was looking at that, but I lost track whether there was any progress | 15:21 |
| frickler | gmann: maybe you can pick that up in the nova meeting? | 15:22 |
| gmann | k | 15:22 |
| gmann | sure | 15:22 |
| kopecmartin | btw, it seems we broke neutron-tempest-with-uwsgi job | 15:22 |
| kopecmartin | #link https://bugs.launchpad.net/neutron/+bug/1960022 | 15:22 |
| kopecmartin | with | 15:22 |
| kopecmartin | #link https://review.opendev.org/c/openstack/tempest/+/814085 | 15:22 |
| kopecmartin | i'm looking into it , i'm failing to install the env locally | 15:23 |
| kopecmartin | will try again , after that I'll leave it up to zuul and move the experiments on how to fix that to the WIP reviews | 15:24 |
| frickler | kopecmartin: let me know if you want a help node for debugging | 15:24 |
| frickler | held | 15:24 |
| *** pojadhav is now known as pojadhav|dinner | 15:24 | |
| kopecmartin | frickler: i have an internal openstack where i can spawn one .. the problem i have is with the installation, it fails for various reasons | 15:25 |
| kopecmartin | at least it gives me more insights into devstack | 15:25 |
| gmann | kopecmartin: so it is failing because we are waiting for it ? | 15:25 |
| kopecmartin | it looks like that, although i doesn't make sense to me right now, however, if would be too big coincidence if that review is not related | 15:26 |
| kopecmartin | the job started falling after the review got merged | 15:26 |
| gmann | humm | 15:27 |
| kopecmartin | #topic Periodic jobs Status Checks | 15:28 |
| frickler | if it is 100% failure, testing against a revert should be pretty easy to confirm that? | 15:28 |
| kopecmartin | frickler: yup, i can propose a dnm patch just to confirm that that's the real culprit | 15:28 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=tempest-full-xena-py3&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-train-py3&pipeline=periodic-stable | 15:29 |
| kopecmartin | #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic | 15:29 |
| frickler | for the periodic jobs, I noticed there isn't a xena one | 15:29 |
| gmann | but it seems failing only on victoria and ussuri ? | 15:29 |
| frickler | and train also no longer runs, so it can be removed from that query | 15:29 |
| kopecmartin | oh | 15:29 |
| gmann | frickler: ohk, did I miss that | 15:29 |
| kopecmartin | gmann: it seems the job is running only on victoria and ussuri | 15:30 |
| gmann | kopecmartin: ohk | 15:30 |
| gmann | stable jobs are up to date https://github.com/openstack/tempest/blob/master/zuul.d/project.yaml#L166 | 15:31 |
| frickler | oh, full-xena, not -py3 | 15:31 |
| frickler | then just the query above is wrong | 15:31 |
| kopecmartin | yeah, my bad | 15:32 |
| gmann | #link https://zuul.openstack.org/builds?job_name=tempest-full-xena&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&pipeline=periodic-stable | 15:32 |
| gmann | ^^ this one we can use | 15:32 |
| gmann | yeah I removed the py3 from xena onwards as everything is py3 | 15:32 |
| kopecmartin | i've fixed the link in the agenda | 15:33 |
| gmann | thanks | 15:33 |
| kopecmartin | frickler: good catch! | 15:33 |
| kopecmartin | anyway, periodic jobs seem fine | 15:34 |
| kopecmartin | #topic Sub Teams highlights | 15:34 |
| kopecmartin | Changes with Review-Priority == +1 | 15:34 |
| kopecmartin | #link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking) | 15:34 |
| kopecmartin | i see 2 reviews, will try to check them soon | 15:34 |
| kopecmartin | #topic Open Discussion | 15:35 |
| kopecmartin | anything for the open discussion? | 15:35 |
| frickler | there was this idea proposed of doing reboot checks for devstack | 15:35 |
| frickler | which might be useful it we get it to work in a stable way | 15:36 |
| frickler | but let me know if you disagree, then we can stop investing work | 15:37 |
| kopecmartin | interesting, what would it be about exactly? restarting the env after deployment and checking whether is up and running? | 15:37 |
| frickler | yes, my idea would be "run devstack", "reboot", "run tempest" | 15:37 |
| frickler | the reboot part is happing in the fips jobs anyway, but before devstack | 15:38 |
| frickler | but this way rebooting during a CI run isn't a new thing anymore | 15:39 |
| gmann | what is it about to catch/test ? | 15:39 |
| kopecmartin | or maybe , devstack, tempest, reboot, some subset of tempest? the subset can grow after it's stable | 15:39 |
| frickler | currently it fails because swift and cinder loopback setup isn't rebootsafe | 15:39 |
| gmann | and as separate job right not by default in all devstack jobs? | 15:39 |
| frickler | see https://review.opendev.org/c/openstack/devstack/+/828280 | 15:40 |
| frickler | yes, I would want a dedicated job | 15:40 |
| frickler | jm1 may even be still around who proposed this | 15:41 |
| frickler | anyway, we don't need a long discussion right now, just wanted to raise awareness for this idea | 15:42 |
| jm1 | https://review.opendev.org/c/openstack/devstack/+/828289 | 15:42 |
| kopecmartin | that's interesting, i don't see a reason why not to try that in one separate job and let's see where it gets us | 15:42 |
| gmann | ok, looks good idea. +1 | 15:43 |
| kopecmartin | jm1: thanks, added to my watchlist | 15:43 |
| kopecmartin | anything else for the open discussion? | 15:43 |
| frickler | just one other headsup | 15:44 |
| frickler | I started integrating cirros CI in opendev | 15:44 |
| frickler | since the old travis setup no longer exists | 15:45 |
| frickler | anyone wanting to help and learn about zuul is welcome to join | 15:45 |
| frickler | https://review.opendev.org/c/cirros/cirros/+/827916 | 15:45 |
| frickler | my current painpoint is how to collect artifacts and where to store them | 15:46 |
| frickler | and that's it from me | 15:46 |
| kopecmartin | frickler: thanks for the headsup | 15:47 |
| kopecmartin | one last thing to do | 15:47 |
| kopecmartin | #topic Bug Triage | 15:47 |
| kopecmartin | #link https://etherpad.opendev.org/p/qa-bug-triage-yoga | 15:48 |
| kopecmartin | bug numbers are recorded at ^ | 15:48 |
| kopecmartin | #link https://bugs.launchpad.net/tempest/+bug/1911044 | 15:48 |
| kopecmartin | gmann: ^ | 15:48 |
| kopecmartin | we can proceed with unittest2 removal, right? | 15:48 |
| kopecmartin | there is already a patch proposed | 15:48 |
| kopecmartin | #link https://review.opendev.org/c/openstack/tempest/+/826191 | 15:48 |
| gmann | kopecmartin: about to ask the link. I think yes as stable/train does not use tempest master. but let me check again in case | 15:50 |
| gmann | I will review it today | 15:50 |
| kopecmartin | it doesn't, we pinned tempest for train | 15:51 |
| kopecmartin | #link https://opendev.org/openstack/devstack/src/commit/8a22f7380c7029e931fe9103320f24a223b619d1/stackrc#L319 | 15:51 |
| kopecmartin | gmann: ok, thanks | 15:51 |
| kopecmartin | that's all from my side | 15:51 |
| kopecmartin | thank you everyone | 15:51 |
| kopecmartin | see you around | 15:51 |
| kopecmartin | #endmeeting | 15:52 |
| opendevmeet | Meeting ended Tue Feb 8 15:52:15 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:52 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/qa/2022/qa.2022-02-08-15.00.html | 15:52 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/qa/2022/qa.2022-02-08-15.00.txt | 15:52 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/qa/2022/qa.2022-02-08-15.00.log.html | 15:52 |
| frickler | oh, wait | 15:52 |
| frickler | meh | 15:52 |
| kopecmartin | oh, sorry | 15:52 |
| kopecmartin | what's up? | 15:52 |
| frickler | my bad, I was too slow | 15:52 |
| frickler | that was an idea in the patch that made chmod -x for stackrc | 15:52 |
| gmann | kopecmartin: i mean in case anyone else need unittest2 than stable/train. train is all fine. | 15:53 |
| frickler | which is to drop the shebang headers for all files that are only included and not meant to be executed on their own | 15:53 |
| frickler | which I think is sensible | 15:53 |
| frickler | but currently our bashate configuration explicitly requires this | 15:53 |
| frickler | so I was wondering what was the historic reasoning for that and whether it still holds | 15:54 |
| frickler | see https://review.opendev.org/c/openstack/devstack/+/827815/1 | 15:55 |
| frickler | the solution would be to split the list in https://opendev.org/openstack/devstack/src/branch/master/tox.ini#L17-L34 and have one part running without the shebang test | 15:56 |
| frickler | if nobody has a good argument not to do that, I will prepare a patch with that | 15:57 |
| gmann | not sure about reason. But i agree that it make sense to remove it. | 15:57 |
| gmann | it will be more clear in term they are not meant to be executed as alone | 15:58 |
| *** pojadhav|dinner is now known as pojadhav | 15:58 | |
| frickler | o.k., great | 15:59 |
| kopecmartin | i have no idea why it was made executable , it's good as it is now, without x permission | 15:59 |
| opendevreview | Andre Aranha proposed openstack/devstack stable/xena: Add openstack-two-node-centos-8-stream https://review.opendev.org/c/openstack/devstack/+/828244 | 15:59 |
| gmann | I think we removed it few file from tempest too very long ago. by masayukig I think. | 16:00 |
| abhishekk | can anyone from the team help to understand this failure? | 16:14 |
| abhishekk | https://review.opendev.org/c/openstack/glance/+/825115 | 16:14 |
| gmann | abhishekk: because of this https://review.opendev.org/c/openstack/tempest/+/825112/6/tempest/config.py#879 | 16:36 |
| gmann | that config option is removed in depends-on tempest patch | 16:37 |
| abhishekk | ohh | 16:38 |
| abhishekk | thank you, will have a look | 16:38 |
| gibi | gmann: about the rescue issue in the centos 9 stream job. In that job I see libvirt 8.0.0 and qemu 6.2.0 that is way newer than what we use in the ubuntu jobs (libvirt 6.0.0 and qemu 4.2.0) | 16:46 |
| gibi | gmann: and nova fails to detach the volume from the guest via libvirt even after retries | 16:47 |
| gibi | gmann: so if I have to guess then something is changed in the libvirt between 6.0.0 and 8.0.0 that makes this faulty | 16:48 |
| gmann | gibi: ohk, thanks for debugging. yeah, let me open bug in devatack then in case anyone from centos9-stream side check it and fix. | 16:50 |
| gibi | gmann: OK. thanks! | 16:50 |
| opendevreview | Abhishek Kekane proposed openstack/tempest master: Add image cache client https://review.opendev.org/c/openstack/tempest/+/825111 | 16:51 |
| opendevreview | Abhishek Kekane proposed openstack/tempest master: Add tempest coverage for new cache API https://review.opendev.org/c/openstack/tempest/+/825112 | 16:51 |
| *** jpena is now known as jpena|off | 17:04 | |
| *** akahat|dinner is now known as akahat|rover | 17:05 | |
| opendevreview | Martin Kopec proposed openstack/tempest master: Revert "Fix test_rebuild_server test by waiting for floating ip" https://review.opendev.org/c/openstack/tempest/+/828245 | 17:18 |
| gmann | kopecmartin: frickler gibi FYI, logged devstack bug for centos9-stream job failure https://bugs.launchpad.net/devstack/+bug/1960346 | 17:57 |
| opendevreview | Jakob Meng proposed openstack/devstack master: Added Zuul job devstack-reboot https://review.opendev.org/c/openstack/devstack/+/828289 | 18:09 |
| opendevreview | Jakob Meng proposed openstack/devstack master: Added Zuul job devstack-reboot-tempest https://review.opendev.org/c/openstack/devstack/+/828289 | 19:14 |
| opendevreview | Jakob Meng proposed openstack/devstack master: Added Zuul job devstack-reboot-tempest https://review.opendev.org/c/openstack/devstack/+/828289 | 19:27 |
| *** fpantano is now known as fmount | 19:29 | |
| *** osmanlicilegi is now known as Guest1 | 20:56 | |
| opendevreview | Brian Haley proposed openstack/devstack master: Add support for IPv6 tunnel endpoints https://review.opendev.org/c/openstack/devstack/+/710519 | 22:17 |
| opendevreview | Brian Haley proposed openstack/tempest master: Add an IPv6 tunnel endpoint job definition https://review.opendev.org/c/openstack/tempest/+/828431 | 22:20 |
| opendevreview | Merged openstack/devstack master: Bump fedora-latest to F35 https://review.opendev.org/c/openstack/devstack/+/827576 | 22:34 |
| opendevreview | Merged openstack/devstack master: Fix deployment of Neutron with enforced scopes https://review.opendev.org/c/openstack/devstack/+/826851 | 22:39 |
| opendevreview | Merged openstack/devstack master: Revert "Disable enforcing scopes in Neutron temporary" https://review.opendev.org/c/openstack/devstack/+/826852 | 22:41 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!