15:01:13 <ykarel> #startmeeting neutron_ci
15:01:13 <opendevmeet> Meeting started Mon Mar 24 15:01:13 2025 UTC and is due to finish in 60 minutes.  The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:13 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:13 <opendevmeet> The meeting name has been set to 'neutron_ci'
15:01:19 <ykarel> Ping list: bcafarel, lajoskatona, slawek, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira
15:01:20 <mlavalle> \o
15:01:23 <lajoskatona> o/
15:01:27 <ralonsoh> hello
15:01:28 <mtomaska> o/
15:01:28 <slaweq> o/
15:01:38 <lajoskatona> IRC only this time?
15:01:50 <opendevreview> Merged openstack/neutron master: [FT] Refactor the ``TestNeutronServer._check_active`` method.  https://review.opendev.org/c/openstack/neutron/+/942798
15:01:59 <ykarel> yeap IRC only today
15:02:24 <bcafarel> o/
15:02:57 <ykarel> Hello everyone, lets start with topics
15:03:03 <ykarel> #topic Actions from previous meetings
15:03:17 <ykarel> ralonsoh to send fixes for TestNBDbMonitorOverTcp failures
15:03:30 <ralonsoh> I pushed a patch for this, if I'm not wrong
15:03:39 <ykarel> yes i saw those
15:03:49 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/944969
15:03:52 <ralonsoh> both
15:04:06 <ykarel> https://review.opendev.org/c/openstack/neutron/+/944968
15:04:13 <ykarel> thx ralonsoh for these
15:04:37 <ykarel> can you send backports too for these
15:04:42 <ralonsoh> sure
15:04:49 <ykarel> thx moving to next
15:04:59 <ykarel> ykarel to check https://bugs.launchpad.net/neutron/+bug/2101857
15:05:09 <ykarel> proposed https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/944965
15:05:13 <ykarel> it's already merged
15:05:22 <ralonsoh> +1 for this finding
15:05:28 <ykarel> #topic Stable branches
15:05:37 <ykarel> bcafarel, any update ^
15:06:38 <bcafarel> not that much I am lagging behind a bit
15:06:51 <bcafarel> but it looked all good from backports that I went through
15:07:22 <ykarel> thx for the update, for periodic runs also it was all good, except some random failures in fullstack and functional
15:07:38 <ykarel> #topic Stadium projects
15:07:44 <ykarel> weekly run was green
15:07:47 <ykarel> lajoskatona, anything to add
15:08:05 <lajoskatona> bagpipe failed but it was not related to bagpipe but some image test as I remember
15:08:23 <ykarel> but ^ was last to last week
15:08:34 <ykarel> last week i see all green
15:08:58 <lajoskatona> oooh, tru, it again failed to refresh for me
15:09:58 <slaweq> I have qq about neutron-fwaas and its scenario job: do we know why we don't have any ovn based job for that stadium project? Is there any reason except "nobody proposed it so far"?
15:10:33 <slaweq> and if that is maybe the only reason, do we want to add neutron-tempest-plugin-fwaas-ovn job maybe?
15:10:45 <lajoskatona> salweq: no idea, I remember it was asked in the past, but suppose I forgot to follow up
15:11:04 <ykarel> if that works with ovn yes sure we should have that job
15:11:12 <lajoskatona> +1 for OVN job
15:11:17 <slaweq> ok, thx for confirmation
15:11:34 <slaweq> I am currently evaluating state of this project so maybe I will propose job like that
15:11:42 <slaweq> that's all from me
15:11:58 <ykarel> seems https://review.opendev.org/q/topic:%22bug/1971958%22 added the support
15:12:17 <ykarel> and there is an abondon patch to add job
15:12:22 <lajoskatona> https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/905421
15:12:26 <slaweq> yeah, I see now
15:12:27 <lajoskatona> yes exactly
15:12:42 <ykarel> so let's restore that
15:12:46 <slaweq> I will reopen this patch and rerun it to see the results
15:12:46 <ykarel> to start with
15:12:58 <ykarel> thx slaweq for following over this
15:13:14 <opendevreview> Lajos Katona proposed openstack/neutron-tempest-plugin master: Add test job for firewall_v2 backend OVN  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/905421
15:14:23 <ykarel> let's move to next topic
15:14:24 <ykarel> #topic Rechecks
15:14:53 <ykarel> we still had couple of rechecks due to known random issues
15:15:13 <ykarel> bare rechecks wise it was better this week, 3/41
15:15:47 <ykarel> let's keep avoiding bare rechecks
15:16:06 <ykarel> Now let's check failures
15:16:12 <ykarel> #topic fullstack/functional
15:16:18 <ykarel> https://bugs.launchpad.net/neutron/+bug/2103424
15:16:35 <ykarel> It was opened by ralonsoh , i found one another occurrence so added in comment
15:16:47 <ykarel> 2 similar failures in last 15 days
15:16:50 <ralonsoh> I'll check it tomorrow
15:16:56 <ralonsoh> I'll assign it to myself
15:17:09 <ykarel> k thx
15:17:17 <ykarel> #action ralonsoh to check https://bugs.launchpad.net/neutron/+bug/2103424
15:17:40 <ykarel> also some failures in l3 failover fullstack tests
15:17:47 <ykarel> - test_ha_router_failover_graceful
15:17:47 <ykarel> - https://7f1261c1f7c867223017-371138c00916065f498de90723b300c8.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-fullstack-fips/af92329/testr_results.html
15:17:47 <ykarel> - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0e2/944979/2/check/neutron-fullstack/0e2e9a0/testr_results.html
15:17:47 <ykarel> - https://962baa9ccd0a43825d0b-0a894e3582385412e867858a97ee7e54.ssl.cf2.rackcdn.com/943404/10/check/neutron-fullstack/f89edf5/testr_results.html
15:17:49 <ykarel> - https://4656218b6b9f64be6a53-b8f5baac1536449624f55cad0c5791e8.ssl.cf1.rackcdn.com/944803/1/gate/neutron-fullstack/dfdcc58/testr_results.html
15:17:52 <ykarel> - test_ha_router_failover_host_failure
15:17:54 <ykarel> - https://59cf76542d9f261862f7-56852c2a39b5b5caf6602061c30b0cc0.ssl.cf2.rackcdn.com/944979/2/check/neutron-fullstack/a9d8a2d/testr_results.html
15:17:59 <ykarel> - https://d99bd7feea5bac8bf2c7-371f69b7cb442b1ed4a26cfca1c7e4db.ssl.cf2.rackcdn.com/944879/1/gate/neutron-fullstack-with-uwsgi/9f7dbcb/testr_results.html
15:18:02 <ykarel> - https://0c63ab9652170854bf26-a09d1a3317eb4b9b558e42ad19c25861.ssl.cf2.rackcdn.com/944013/2/check/neutron-fullstack-7/518678e/testr_results.html
15:18:07 <ykarel> Reported generic bug for these l3 ha failover fullstack tests failures https://bugs.launchpad.net/neutron/+bug/2104019
15:18:43 <ykarel> Since these tests got added we have one or other random failures, some of those got fixed in past but we still see these
15:19:43 <ykarel> can monitor these in coming weeks for frequency
15:20:16 <ykarel> anyone up to check these failures?
15:20:28 <mlavalle> i'll take a look
15:20:36 <ykarel> thx mlavalle
15:20:47 <ykarel> #action mlavalle to check https://bugs.launchpad.net/neutron/+bug/2104019
15:20:55 <ykarel> #topic Tempest/Scenario
15:21:07 <ykarel> Still seeing random failures(frequency is less now with previous reverts related to eventlet) in ovs job https://launchpad.net/bugs/2101840
15:21:19 <ykarel> Trying some more reverts for quite old patches to confirm the cause
15:21:25 <ykarel> - https://review.opendev.org/c/openstack/neutron/+/945314
15:21:25 <ykarel> - https://review.opendev.org/c/openstack/neutron/+/945315
15:21:39 <ralonsoh> I saw the reverts, maybe 945315 could be related
15:21:47 <ralonsoh> but the other one I don't think so
15:22:01 <opendevreview> Tobias Urdin proposed openstack/neutron master: Allow service role more RBAC access for Octavia  https://review.opendev.org/c/openstack/neutron/+/945329
15:22:30 <ykarel> ralonsoh, yes could be, just pushed both independently to gather stats
15:22:42 <ralonsoh> I'll keep an eye on them
15:23:08 <ykarel> sure, thx much
15:23:59 <ykarel> #topic Periodic
15:24:35 <ykarel> we still seeing https://bugs.launchpad.net/openstacksdk/+bug/2087978 in stable 2023.2 and 2024.1 periodics randomly
15:24:43 <ykarel> pinged #openstack-dns to take a look
15:24:54 <ralonsoh> thanks
15:25:51 <ykarel> i tried to look myself but couldn't find myself too the issue that explains the random behavior
15:26:07 <ykarel> default quota configured for zones = 10
15:26:42 <ykarel> there are in total 18 zones created during job run and deleted too serially, still not sure why quota issue being hit
15:27:09 <ykarel> designate people may have more idea, so let's see
15:27:17 <ykarel> #topic Grafana
15:27:26 <ykarel> https://grafana.opendev.org/d/f913631585/neutron-failure-rate
15:27:34 <ykarel> let's also have quick look at dashboard
15:28:46 <ykarel> can see some spikes in ovs, fullstack and functional jobs and for those we have some known issues
15:29:52 <ykarel> anything to add?
15:30:01 <ralonsoh> no, thanks
15:30:08 <slaweq> nope
15:30:17 <lajoskatona> nothing from me
15:30:25 <ykarel> k thx
15:30:27 <ykarel> #topic On Demand
15:30:34 <ykarel> anything else you would like to raise?
15:31:27 <ykarel> i have one small one
15:31:41 <ykarel> we have Holiday Next Monday in India, so will be out
15:32:26 <lajoskatona> Have a good time :-)
15:32:30 <ralonsoh> we can skip next week unless there is a fire, right?
15:32:37 <ykarel> i was thinking to cancel that, and if have anything can just raise it on Neutron meeting on Tuesday
15:32:44 <bcafarel> +1
15:32:49 <slaweq> +1 for canceling next week's meeting then
15:33:27 <ykarel> ok then will send a cancellation email for it
15:33:48 <lajoskatona> +1 for cancel
15:34:01 <mlavalle> \o+1
15:34:08 <ykarel> anything else to raise?
15:37:14 <ykarel> k if nothing else let's close early and have everyone 23 minutes back
15:37:20 <ykarel> Thx everyone for joining
15:37:23 <ykarel> #endmeeting