15:00:46 <ykarel> #startmeeting neutron_ci 15:00:46 <opendevmeet> Meeting started Tue May 21 15:00:46 2024 UTC and is due to finish in 60 minutes. The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:46 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:46 <opendevmeet> The meeting name has been set to 'neutron_ci' 15:00:52 <ykarel> Ping list: bcafarel, lajoskatona, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira 15:00:53 <mlavalle> o/ 15:02:11 <slaweq> o/ 15:02:48 <lajoskatona> o/ 15:03:21 <haleyb> o/ but am double-booked 15:03:56 <ykarel> k i think we cans start with topics 15:04:02 <ykarel> #topic Actions from previous meetings 15:04:10 <ykarel> mlavalle to check failures for test_subport_delete functional test 15:04:31 <mlavalle> I did check it. There are several cases over the past week 15:04:44 <mlavalle> it seems to me a race condition in the test 15:05:09 <mlavalle> where we delete the subport using the trunks plugin 15:05:44 <mlavalle> and there is a event notification that is processed byt the ovn driver 15:06:15 <mlavalle> sometimes we go and check if the subport has been removed from the OVN NBDB 15:06:29 <mlavalle> and the event hasn't been processed yet 15:06:43 <mlavalle> so later today I'll add a delay before the checking 15:06:53 <mlavalle> and see if that takes care of it 15:07:31 <ykarel> thx mlavalle for looking into this, can you also take care of reporting lp and tracking fixes as part of that? 15:07:42 <mlavalle> will do 15:07:45 <mlavalle> for sure 15:07:47 <ykarel> thx much 15:07:59 <ykarel> next one was on me 15:08:00 <ykarel> ykarel to check failure in test_virtual_port_host_update_upon_failover functional test 15:08:15 <ykarel> i didn't get chance to look into this, so will check this week 15:08:23 <ykarel> #action ykarel to check failure in test_virtual_port_host_update_upon_failover functional test 15:08:29 <ykarel> #topic Stable branches 15:08:46 <ykarel> bcafarel, around? 15:08:57 <ykarel> Not much activity in stable, but overall looks good 15:09:15 <ykarel> considering periodic runs 15:09:30 <ykarel> k let's move to next 15:09:32 <ykarel> #topic Stadium projects 15:09:51 <ykarel> there were couple of failures 15:09:52 <ykarel> functional jobs broken for vpnaas, bgpvpn, sfc, fwaas with sqlalchemy 2.0, need fixes similar to https://review.opendev.org/c/openstack/neutron/+/919594 15:10:24 <ykarel> there are fixes proposed already by tkajinam https://review.opendev.org/q/topic:%22fix_str_url%22 15:10:56 <lajoskatona> yes these are from sqlalchemy 15:11:00 <ykarel> also https://review.opendev.org/q/topic:%22sqlalchemy-20%22 15:11:18 <ihrachys> no neutron patches that are not merged 15:11:23 <ykarel> for networking i think we already have all needed merged 15:11:49 <ykarel> so we should be good now 15:11:56 <lajoskatona> As I see sfc and vpnaas alsa has this isseu I will check them after the meeting 15:12:26 <ykarel> thx lajoskatona , but i saw patches in sfc and vpnaas atleast for functional failures 15:13:01 <lajoskatona> ykarel: true, I checked only open patches, cool 15:13:32 <ykarel> k then all good :), let's move to next 15:13:39 <ykarel> #topic Rechecks 15:14:11 <ykarel> we had comparitively more rechecs due to various issues 15:14:40 <ykarel> sqlalchemy/pylint/tox-cover ones, and some random failures 15:15:06 <ykarel> bare rechecks wise not bad, had 3/23 bare rechecks, let's keep doing it 15:15:17 <lajoskatona> +1 15:15:28 <ykarel> #topic Unit tests/pep8 15:15:47 <ykarel> there was timeout in pep8, and pep8 timeouts fixed by https://review.opendev.org/c/openstack/neutron/+/919594 15:16:08 <ykarel> cover jobs are intermittently failing, ihrachys reported https://bugs.launchpad.net/neutron/+bug/2065821 15:16:42 <ykarel> it's not assigned yet, but i think haleyb mentioned he will take a look 15:17:09 <haleyb> yes, i will take a look 15:17:14 <ykarel> thx haleyb 15:17:27 <ykarel> #topic fullstack/functional 15:17:51 <opendevreview> Fernando Royo proposed openstack/ovn-octavia-provider stable/2023.2: Adding isolation to functional tests https://review.opendev.org/c/openstack/ovn-octavia-provider/+/920109 15:17:55 <opendevreview> Merged openstack/neutron master: Update grenade_from_branch for skip level jobs https://review.opendev.org/c/openstack/neutron/+/920075 15:17:58 <ykarel> fullstack jobs was broken with sqlalchemy 2.0, is now fixed by ixed by https://review.opendev.org/c/openstack/neutron/+/919594 15:18:10 <ykarel> functional tests broken with ovsdbapp 2.7.1 https://bugs.launchpad.net/neutron/+bug/2066263 15:18:34 <ykarel> ^ not impacting gate, was detected by pro active tests on requirements patch 15:18:49 <ykarel> pushed https://review.opendev.org/c/openstack/neutron/+/920101 15:19:15 <ykarel> please review, this will unblock ovsdbapp release 15:19:45 <ykarel> #topic grenade 15:20:08 <ykarel> Skip level jobs in master periodic were broken 15:20:15 <ykarel> #topic Periodic 15:20:22 <ykarel> https://zuul.openstack.org/builds?job_name=neutron-ovn-grenade-multinode-skip-level&job_name=neutron-ovs-grenade-multinode-skip-level&project=openstack%2Fneutron&branch=master&skip=0 15:20:36 <ykarel> reported https://bugs.launchpad.net/neutron/+bug/2066221 15:20:50 <ykarel> fix https://review.opendev.org/c/openstack/neutron/+/920075 just merged, so should be good now 15:21:33 <ykarel> k that's it on failures 15:21:40 <ykarel> #topic Grafana 15:21:46 <ykarel> https://grafana.opendev.org/d/f913631585/neutron-failure-rate 15:21:52 <ykarel> let's have a quick look here 15:24:15 <ykarel> gate queue looks good, there are some spikes in check for pep8/fullstack/cover and those are known issues 15:24:28 <ykarel> anything to add here? 15:24:34 <mlavalle> not from me 15:24:41 <mlavalle> looks good 15:25:17 <lajoskatona> nothing from me 15:26:20 <ykarel> ack 15:26:21 <ykarel> #topic On Demand 15:26:29 <ykarel> anything else you would like to raise? 15:26:47 <lajoskatona> I have few patches fro stadiums 15:27:06 <lajoskatona> Testing runtime update: https://review.opendev.org/q/topic:%22update-testing-runtime%22 15:27:44 <lajoskatona> and for the pyproject.toml patches: https://review.opendev.org/q/topic:%22pip-23.1-support%22 15:27:56 <ykarel> thx lajoskatona 15:27:58 <lajoskatona> this one is not filtered for networking 15:29:01 <ykarel> lajoskatona, wrt pyproject.toml, that do not work with rhel9/centos 9 stream with GLOBAL_VENV=False, right? 15:29:13 <ykarel> i recently saw an issue with it and tap-as-a-service 15:29:30 <ykarel> as there it was already merged 15:29:47 <lajoskatona> interesting 15:30:12 <lajoskatona> this global_venv thing caused me nightmares anyway on ubuntu as well 15:30:24 <mlavalle> lol 15:31:26 <lajoskatona> ok, so that can be a topic how to do that, but perhaps not just for networking ? 15:32:03 <ykarel> for tap-as-a-service i think as we not testing in centos-9-stream it was not caught 15:32:17 <ykarel> not sure about other project where the patch is open 15:32:59 <ykarel> for reference it failed as:- 15:33:00 <ykarel> ++functions-common:_safe_permission_operation:2170 sudo chown -R stack '/opt/stack/tap-as-a-service/*.egg-info' 15:33:00 <ykarel> chown: cannot access '/opt/stack/tap-as-a-service/*.egg-info': No such file or directory 15:34:10 <ykarel> and dropping pyproject.toml from tap-as-a-service installation went fine with centos 9-stream + GLOBAL_VENV=false 15:34:13 <lajoskatona> as a lot of these were merged openstack wide it can be a globa problem 15:35:49 <lajoskatona> thanks ykarel for pointing to this issue with those patches 15:36:10 <ykarel> yes and if those merges together with https://review.opendev.org/c/openstack/devstack/+/898845 i think that should catch some issues 15:36:32 <lajoskatona> armageddon? 15:36:33 <ykarel> i haven't checked where are we with full support of that, but something to check for 15:37:45 <lajoskatona> that's it from me 15:37:50 <opendevreview> Fernando Royo proposed openstack/ovn-octavia-provider stable/2023.1: Adding isolation to functional tests https://review.opendev.org/c/openstack/ovn-octavia-provider/+/920112 15:37:54 <ykarel> ok can check this offline, thx everyone for joining 15:38:04 <ykarel> #endmeeting