| opendevreview | Balazs Gibizer proposed openstack/tempest master: Skip test_list_all_networks if no known network configured https://review.opendev.org/c/openstack/tempest/+/883892 | 07:43 |
|---|---|---|
| opendevreview | Rajat Dhasmana proposed openstack/tempest master: Add test to rebuild volume backed instance https://review.opendev.org/c/openstack/tempest/+/831018 | 11:30 |
| opendevreview | Lukas Piwowarski proposed openstack/tempest master: [WIP] Catching broken tests in tempest-full-test-account-* jobs https://review.opendev.org/c/openstack/tempest/+/880630 | 11:33 |
| dansmith | clarkb: if we have a multinode job and we need to synchronize between two nodes (i.e. one waits for the other to generate some config) what's the best way to do that? | 14:23 |
| dansmith | is sshing from one to the other to poll the best way? | 14:23 |
| dansmith | I assume the real best way is to be able to pre-generate everything such that you're not really exchanging everything | 14:24 |
| dansmith | I guess I'm thinking of something finer-grained like you would do in ansible, but I guess if devstack is just one ansible step, there's probably not much of an option | 14:25 |
| frickler | dansmith: iiuc devstack runs first on primary, generating tls certs for example, which are then copied to the other nodes before devstack runs there | 14:39 |
| dansmith | frickler: oh yea? that seems like it would take a long time, but if so.. do we have a way to say "include this file in what gets copied" ? | 14:39 |
| frickler | dansmith: currently it seems to be specific to ca data and ceph conf, but you could add something here I guess https://opendev.org/openstack/devstack/src/branch/master/roles/orchestrate-devstack/tasks/main.yaml | 14:41 |
| dansmith | frickler: I'm actually asking about this for ceph | 14:42 |
| dansmith | because there's some work going on that ssh polls from one node to the other and then scp's the conf file | 14:42 |
| dansmith | frickler: so yeah, thanks for the pointer to that, because (something like) that's what I was thinking was going on | 14:43 |
| frickler | that's for the cephadm case? | 14:44 |
| dansmith | yeah | 14:44 |
| * frickler still likes ceph-ansible much better | 14:45 | |
| dansmith | idk, cephadm has been easy so far. I mostly want whatever solution someone else will keep working for me :) | 14:46 |
| kopecmartin | #startmeeting qa | 15:00 |
| opendevmeet | Meeting started Tue May 23 15:00:23 2023 UTC and is due to finish in 60 minutes. The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'qa' | 15:00 |
| kopecmartin | #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours | 15:00 |
| kopecmartin | agenda ^^ | 15:00 |
| lpiwowar | o/ | 15:01 |
| kopecmartin | \o | 15:01 |
| frickler | o/ | 15:05 |
| kopecmartin | let's get started, the usual | 15:05 |
| kopecmartin | #topic Announcement and Action Item (Optional) | 15:05 |
| kopecmartin | \o\ | 15:06 |
| kopecmartin | no announcements this week | 15:06 |
| kopecmartin | #topic Bobcat Priority Items progress | 15:06 |
| kopecmartin | #link https://etherpad.opendev.org/p/qa-bobcat-priority | 15:06 |
| * kopecmartin checking the status | 15:07 | |
| kopecmartin | 'New cirros image update' is almost done, the last patch is being merged now | 15:08 |
| lpiwowar | \o/ | 15:08 |
| kopecmartin | regarding 'Tempest tests prepare the server with SSHABLE by default' | 15:09 |
| kopecmartin | it seems that the following patch is the last one | 15:09 |
| kopecmartin | #link https://review.opendev.org/c/openstack/tempest/+/842240 | 15:09 |
| kopecmartin | in merge conflict atm, gmann^ | 15:10 |
| kopecmartin | I don't see any other updates | 15:10 |
| kopecmartin | #topic Gate Status Checks | 15:10 |
| kopecmartin | #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade) | 15:11 |
| kopecmartin | one patch, already approved | 15:11 |
| kopecmartin | anything urgent to review? | 15:12 |
| frickler | https://review.opendev.org/c/openstack/grenade/+/883742 would be good to get fixed | 15:12 |
| frickler | this is to resolve a zuul config error because the heat job was deleted | 15:13 |
| frickler | failing with neutron-fwaas missing, not sure why | 15:14 |
| frickler | we could make the job n-v to merge the above fix or I could just force-merge | 15:15 |
| frickler | at least I don't care too much about those old branches | 15:15 |
| frickler | just want to get rid of config errors | 15:15 |
| kopecmartin | well, clearly the patch is not related to the error in any way, the logs confuse me , no apparent failure | 15:17 |
| kopecmartin | i'm ok with force merging it | 15:18 |
| kopecmartin | although the issue with the job will remain | 15:18 |
| kopecmartin | but i don't see a reason why suffer from config issues till it's resolved | 15:18 |
| frickler | ok, will do | 15:19 |
| clarkb | dansmith: frickler: problems like that are why we added ansible to devstack to help with multinode. It is just easier to express that sort of thing in a tool like ansible | 15:19 |
| dansmith | clarkb: ++ | 15:19 |
| opendevreview | Merged openstack/grenade stable/stein: [stable-only] Remove grenade-heat job https://review.opendev.org/c/openstack/grenade/+/883742 | 15:22 |
| kopecmartin | great, moving on | 15:22 |
| kopecmartin | #topic Bare rechecks | 15:22 |
| kopecmartin | #link https://etherpad.opendev.org/p/recheck-weekly-summary | 15:22 |
| kopecmartin | 8 rechecks over the last 7 days, none of them bare \o/ | 15:23 |
| kopecmartin | #topic Periodic jobs Status Checks | 15:23 |
| kopecmartin | periodic stable full | 15:23 |
| kopecmartin | #link https://zuul.openstack.org/builds?pipeline=periodic-stable&job_name=tempest-full-yoga&job_name=tempest-full-xena&job_name=tempest-full-zed&job_name=tempest-full-2023-1 | 15:23 |
| kopecmartin | periodic stable slow | 15:23 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=tempest-slow-2023-1&job_name=tempest-slow-zed&job_name=tempest-slow-yoga&job_name=tempest-slow-xena | 15:23 |
| kopecmartin | periodic extra tests | 15:23 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=tempest-full-2023-1-extra-tests&job_name=tempest-full-zed-extra-tests&job_name=tempest-full-yoga-extra-tests&job_name=tempest-full-xena-extra-tests | 15:23 |
| kopecmartin | periodic master | 15:23 |
| kopecmartin | #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic | 15:23 |
| kopecmartin | seems like the centos9-stream-fips is broken | 15:24 |
| kopecmartin | and tempest-all occasionally timeouts | 15:25 |
| kopecmartin | *-no-admin-py3 job is a known issue, no progress on the LP yet | 15:25 |
| lpiwowar | kopecmartin: Not yet. I'm sorry. I will try to focus on it during this week. | 15:26 |
| kopecmartin | no worries, it's been broken forever | 15:27 |
| kopecmartin | i don't understand why the fips job failed o.O | 15:28 |
| kopecmartin | "/usr/bin/python3.9: No module named pip" ? | 15:30 |
| frickler | is there an earlier yum failure? iirc we still ignore those | 15:31 |
| frickler | maybe c9s mirroring is broken once again | 15:31 |
| kopecmartin | don't see any, i see it reinstalled python3-setuptools, seems that successfully and then it failed on still not having pip | 15:32 |
| kopecmartin | but even before that | 15:32 |
| kopecmartin | "No match for argument: liberasurecode-devel" | 15:32 |
| kopecmartin | "Error: Unable to find a match: liberasurecode-devel" | 15:33 |
| kopecmartin | anyway, moving on | 15:34 |
| kopecmartin | #topic Distros check | 15:34 |
| kopecmartin | cs-9 | 15:34 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=tempest-full-centos-9-stream&job_name=devstack-platform-centos-9-stream&skip=0 | 15:34 |
| kopecmartin | fedora | 15:34 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=devstack-platform-fedora-latest&skip=0 | 15:34 |
| kopecmartin | debian | 15:34 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=devstack-platform-debian-bullseye&skip=0 | 15:34 |
| kopecmartin | focal | 15:34 |
| kopecmartin | #link https://zuul.opendev.org/t/openstack/builds?job_name=devstack-platform-ubuntu-focal&skip=0 | 15:34 |
| kopecmartin | rocky | 15:34 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=devstack-platform-rocky-blue-onyx | 15:34 |
| kopecmartin | openEuler | 15:34 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=devstack-platform-openEuler-22.03-ovn-source&job_name=devstack-platform-openEuler-22.03-ovs&skip=0 | 15:34 |
| kopecmartin | all green \o/, what a nice surprise | 15:34 |
| frickler | do we want to drop fedora if nobody makes f38 support? | 15:35 |
| frickler | ianw: ^^? | 15:35 |
| frickler | our "fedora-latest" job is actually "fedora-eol" now | 15:36 |
| kopecmartin | right, fedora-36 is EOL since last week | 15:37 |
| kopecmartin | #link https://docs.fedoraproject.org/en-US/releases/eol/ | 15:37 |
| kopecmartin | well, if there won't be fedora-38 label we could use in the job, we'll have to drop the support as it won't be tested in the CI | 15:37 |
| clarkb | right I sent email asking if anyone needed fedora since the lack of updates indicated that the answer was probably no | 15:38 |
| clarkb | we didn't get any feedback from anyone that needed or wanted fedora | 15:38 |
| kopecmartin | that makes it easier | 15:39 |
| clarkb | I've already started some of the safer fedora cleanups (for unused bits) but I expect that to progress through to complete removal | 15:39 |
| clarkb | this is all in opendev's nodepool stuff not devstack but devstack cleanup would be a necessary step if we do that | 15:39 |
| frickler | o.k., I'll prepare some patches | 15:41 |
| kopecmartin | sure, let me know it starts, i'll help with patches and reviews too | 15:41 |
| frickler | that'll probably go backwards through some cycles | 15:41 |
| clarkb | the main thing seems to be that for forward looking CI centos stream is a better fit for what we do anyway | 15:42 |
| clarkb | because it is a platform with multiple yaers of support unlike fedora | 15:42 |
| clarkb | reduces churn and all that | 15:42 |
| kopecmartin | +1 | 15:42 |
| frickler | stream is continously breaking, so I don't think it is too well suited, too | 15:43 |
| clarkb | well that too, but I think it breaks in smaller chunks than the major uplifts fedora does | 15:43 |
| clarkb | which is a big reason why fedora updates have fallen behind recently | 15:43 |
| kopecmartin | right | 15:46 |
| kopecmartin | moving on | 15:46 |
| kopecmartin | #topic Sub Teams highlights | 15:46 |
| kopecmartin | Changes with Review-Priority == +1 | 15:46 |
| kopecmartin | #link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade) | 15:46 |
| kopecmartin | no reviews there | 15:46 |
| kopecmartin | #topic Open Discussion | 15:46 |
| kopecmartin | anything for the open discussion? | 15:46 |
| clarkb | the mysql tune down has been enabled by default | 15:47 |
| clarkb | not sure if that was mentioned earlier, but thoughti t was worth calling out | 15:47 |
| frickler | good point, on thing to watch out for is whether there are timeouts because of that | 15:48 |
| frickler | in case mysql things get too slow | 15:48 |
| dansmith | yep, although I've seen nothing but improvement from the jobs we enabled it on early :) | 15:48 |
| kopecmartin | oh, so maybe that's why tempest-all started timeouting occasionally? | 15:49 |
| dansmith | kopecmartin: it merged middle of the day yesterday, so depends on when you're talking about | 15:49 |
| kopecmartin | not related then :) | 15:49 |
| dansmith | :) | 15:49 |
| kopecmartin | if there isn't anything else | 15:52 |
| kopecmartin | #topic Bug Triage | 15:52 |
| kopecmartin | #link https://etherpad.openstack.org/p/qa-bug-triage-bobcat | 15:52 |
| kopecmartin | one bug down in tempest - as the cirros update is going forward | 15:52 |
| kopecmartin | other than that, it's been quite quiet on the bug front | 15:53 |
| kopecmartin | and that's all from my side | 15:53 |
| kopecmartin | thanks everyone | 15:54 |
| kopecmartin | #endmeeting | 15:54 |
| opendevmeet | Meeting ended Tue May 23 15:54:25 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:54 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/qa/2023/qa.2023-05-23-15.00.html | 15:54 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/qa/2023/qa.2023-05-23-15.00.txt | 15:54 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/qa/2023/qa.2023-05-23-15.00.log.html | 15:54 |
| frickler | o/ | 15:54 |
| lpiwowar | thanks kopecmartin | 15:54 |
| opendevreview | Merged openstack/devstack master: Revert "Revert "Bump cirros version to 0.6.1"" https://review.opendev.org/c/openstack/devstack/+/881437 | 16:08 |
| frickler | kopecmartin: I would propose to close https://bugs.launchpad.net/devstack/+bug/2015638 as "won't fix" since no patch was submitted? | 16:24 |
| opendevreview | Ashley Rodriguez proposed openstack/devstack-plugin-ceph master: [WIP][DNM] Remote Ceph with cephadm https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/876747 | 16:52 |
| opendevreview | Ashley Rodriguez proposed openstack/devstack-plugin-ceph master: [WIP][DNM] Remote Ceph with cephadm https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/876747 | 17:00 |
| opendevreview | Ashley Rodriguez proposed openstack/devstack-plugin-ceph master: [WIP][DNM] Remote Ceph with cephadm https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/876747 | 17:03 |
| gmann | kopecmartin: can you please review these https://review.opendev.org/c/openstack/tempest/+/883701 https://review.opendev.org/c/openstack/devstack/+/883556/1 https://review.opendev.org/c/openstack/devstack/+/883601 | 17:25 |
| dansmith | gmann: frickler: clarkb: might be good to get some more skilled eyes on this approach in case we want to redirect before it gets too far: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/876747 | 17:33 |
| dansmith | feels fragile tome | 17:33 |
| clarkb | I think I would personally try to replace as much of the explicit ssh user@host stuff with ansible | 17:39 |
| clarkb | that is why multinode devstack is ansible first | 17:39 |
| dansmith | yeah, agree | 17:39 |
| dansmith | clarkb: can you add your thoughts in there? | 17:40 |
| clarkb | dansmith: I did my best :) | 17:46 |
| dansmith | clarkb: cool, thanks | 17:47 |
| opendevreview | Dan Smith proposed openstack/devstack stable/yoga: Try to reduce mysql memory usage https://review.opendev.org/c/openstack/devstack/+/883970 | 18:22 |
| kopecmartin | frickler: i agree, marked as won't fix just now | 19:00 |
| kopecmartin | gmann: done | 19:00 |
| opendevreview | Merged openstack/devstack-plugin-ceph master: Enable validation and disable block-migration https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/882987 | 23:46 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!