15:01:13 <ykarel> #startmeeting neutron_ci 15:01:13 <opendevmeet> Meeting started Mon Mar 24 15:01:13 2025 UTC and is due to finish in 60 minutes. The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:13 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:13 <opendevmeet> The meeting name has been set to 'neutron_ci' 15:01:19 <ykarel> Ping list: bcafarel, lajoskatona, slawek, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira 15:01:20 <mlavalle> \o 15:01:23 <lajoskatona> o/ 15:01:27 <ralonsoh> hello 15:01:28 <mtomaska> o/ 15:01:28 <slaweq> o/ 15:01:38 <lajoskatona> IRC only this time? 15:01:50 <opendevreview> Merged openstack/neutron master: [FT] Refactor the ``TestNeutronServer._check_active`` method. https://review.opendev.org/c/openstack/neutron/+/942798 15:01:59 <ykarel> yeap IRC only today 15:02:24 <bcafarel> o/ 15:02:57 <ykarel> Hello everyone, lets start with topics 15:03:03 <ykarel> #topic Actions from previous meetings 15:03:17 <ykarel> ralonsoh to send fixes for TestNBDbMonitorOverTcp failures 15:03:30 <ralonsoh> I pushed a patch for this, if I'm not wrong 15:03:39 <ykarel> yes i saw those 15:03:49 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/944969 15:03:52 <ralonsoh> both 15:04:06 <ykarel> https://review.opendev.org/c/openstack/neutron/+/944968 15:04:13 <ykarel> thx ralonsoh for these 15:04:37 <ykarel> can you send backports too for these 15:04:42 <ralonsoh> sure 15:04:49 <ykarel> thx moving to next 15:04:59 <ykarel> ykarel to check https://bugs.launchpad.net/neutron/+bug/2101857 15:05:09 <ykarel> proposed https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/944965 15:05:13 <ykarel> it's already merged 15:05:22 <ralonsoh> +1 for this finding 15:05:28 <ykarel> #topic Stable branches 15:05:37 <ykarel> bcafarel, any update ^ 15:06:38 <bcafarel> not that much I am lagging behind a bit 15:06:51 <bcafarel> but it looked all good from backports that I went through 15:07:22 <ykarel> thx for the update, for periodic runs also it was all good, except some random failures in fullstack and functional 15:07:38 <ykarel> #topic Stadium projects 15:07:44 <ykarel> weekly run was green 15:07:47 <ykarel> lajoskatona, anything to add 15:08:05 <lajoskatona> bagpipe failed but it was not related to bagpipe but some image test as I remember 15:08:23 <ykarel> but ^ was last to last week 15:08:34 <ykarel> last week i see all green 15:08:58 <lajoskatona> oooh, tru, it again failed to refresh for me 15:09:58 <slaweq> I have qq about neutron-fwaas and its scenario job: do we know why we don't have any ovn based job for that stadium project? Is there any reason except "nobody proposed it so far"? 15:10:33 <slaweq> and if that is maybe the only reason, do we want to add neutron-tempest-plugin-fwaas-ovn job maybe? 15:10:45 <lajoskatona> salweq: no idea, I remember it was asked in the past, but suppose I forgot to follow up 15:11:04 <ykarel> if that works with ovn yes sure we should have that job 15:11:12 <lajoskatona> +1 for OVN job 15:11:17 <slaweq> ok, thx for confirmation 15:11:34 <slaweq> I am currently evaluating state of this project so maybe I will propose job like that 15:11:42 <slaweq> that's all from me 15:11:58 <ykarel> seems https://review.opendev.org/q/topic:%22bug/1971958%22 added the support 15:12:17 <ykarel> and there is an abondon patch to add job 15:12:22 <lajoskatona> https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/905421 15:12:26 <slaweq> yeah, I see now 15:12:27 <lajoskatona> yes exactly 15:12:42 <ykarel> so let's restore that 15:12:46 <slaweq> I will reopen this patch and rerun it to see the results 15:12:46 <ykarel> to start with 15:12:58 <ykarel> thx slaweq for following over this 15:13:14 <opendevreview> Lajos Katona proposed openstack/neutron-tempest-plugin master: Add test job for firewall_v2 backend OVN https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/905421 15:14:23 <ykarel> let's move to next topic 15:14:24 <ykarel> #topic Rechecks 15:14:53 <ykarel> we still had couple of rechecks due to known random issues 15:15:13 <ykarel> bare rechecks wise it was better this week, 3/41 15:15:47 <ykarel> let's keep avoiding bare rechecks 15:16:06 <ykarel> Now let's check failures 15:16:12 <ykarel> #topic fullstack/functional 15:16:18 <ykarel> https://bugs.launchpad.net/neutron/+bug/2103424 15:16:35 <ykarel> It was opened by ralonsoh , i found one another occurrence so added in comment 15:16:47 <ykarel> 2 similar failures in last 15 days 15:16:50 <ralonsoh> I'll check it tomorrow 15:16:56 <ralonsoh> I'll assign it to myself 15:17:09 <ykarel> k thx 15:17:17 <ykarel> #action ralonsoh to check https://bugs.launchpad.net/neutron/+bug/2103424 15:17:40 <ykarel> also some failures in l3 failover fullstack tests 15:17:47 <ykarel> - test_ha_router_failover_graceful 15:17:47 <ykarel> - https://7f1261c1f7c867223017-371138c00916065f498de90723b300c8.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-fullstack-fips/af92329/testr_results.html 15:17:47 <ykarel> - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0e2/944979/2/check/neutron-fullstack/0e2e9a0/testr_results.html 15:17:47 <ykarel> - https://962baa9ccd0a43825d0b-0a894e3582385412e867858a97ee7e54.ssl.cf2.rackcdn.com/943404/10/check/neutron-fullstack/f89edf5/testr_results.html 15:17:49 <ykarel> - https://4656218b6b9f64be6a53-b8f5baac1536449624f55cad0c5791e8.ssl.cf1.rackcdn.com/944803/1/gate/neutron-fullstack/dfdcc58/testr_results.html 15:17:52 <ykarel> - test_ha_router_failover_host_failure 15:17:54 <ykarel> - https://59cf76542d9f261862f7-56852c2a39b5b5caf6602061c30b0cc0.ssl.cf2.rackcdn.com/944979/2/check/neutron-fullstack/a9d8a2d/testr_results.html 15:17:59 <ykarel> - https://d99bd7feea5bac8bf2c7-371f69b7cb442b1ed4a26cfca1c7e4db.ssl.cf2.rackcdn.com/944879/1/gate/neutron-fullstack-with-uwsgi/9f7dbcb/testr_results.html 15:18:02 <ykarel> - https://0c63ab9652170854bf26-a09d1a3317eb4b9b558e42ad19c25861.ssl.cf2.rackcdn.com/944013/2/check/neutron-fullstack-7/518678e/testr_results.html 15:18:07 <ykarel> Reported generic bug for these l3 ha failover fullstack tests failures https://bugs.launchpad.net/neutron/+bug/2104019 15:18:43 <ykarel> Since these tests got added we have one or other random failures, some of those got fixed in past but we still see these 15:19:43 <ykarel> can monitor these in coming weeks for frequency 15:20:16 <ykarel> anyone up to check these failures? 15:20:28 <mlavalle> i'll take a look 15:20:36 <ykarel> thx mlavalle 15:20:47 <ykarel> #action mlavalle to check https://bugs.launchpad.net/neutron/+bug/2104019 15:20:55 <ykarel> #topic Tempest/Scenario 15:21:07 <ykarel> Still seeing random failures(frequency is less now with previous reverts related to eventlet) in ovs job https://launchpad.net/bugs/2101840 15:21:19 <ykarel> Trying some more reverts for quite old patches to confirm the cause 15:21:25 <ykarel> - https://review.opendev.org/c/openstack/neutron/+/945314 15:21:25 <ykarel> - https://review.opendev.org/c/openstack/neutron/+/945315 15:21:39 <ralonsoh> I saw the reverts, maybe 945315 could be related 15:21:47 <ralonsoh> but the other one I don't think so 15:22:01 <opendevreview> Tobias Urdin proposed openstack/neutron master: Allow service role more RBAC access for Octavia https://review.opendev.org/c/openstack/neutron/+/945329 15:22:30 <ykarel> ralonsoh, yes could be, just pushed both independently to gather stats 15:22:42 <ralonsoh> I'll keep an eye on them 15:23:08 <ykarel> sure, thx much 15:23:59 <ykarel> #topic Periodic 15:24:35 <ykarel> we still seeing https://bugs.launchpad.net/openstacksdk/+bug/2087978 in stable 2023.2 and 2024.1 periodics randomly 15:24:43 <ykarel> pinged #openstack-dns to take a look 15:24:54 <ralonsoh> thanks 15:25:51 <ykarel> i tried to look myself but couldn't find myself too the issue that explains the random behavior 15:26:07 <ykarel> default quota configured for zones = 10 15:26:42 <ykarel> there are in total 18 zones created during job run and deleted too serially, still not sure why quota issue being hit 15:27:09 <ykarel> designate people may have more idea, so let's see 15:27:17 <ykarel> #topic Grafana 15:27:26 <ykarel> https://grafana.opendev.org/d/f913631585/neutron-failure-rate 15:27:34 <ykarel> let's also have quick look at dashboard 15:28:46 <ykarel> can see some spikes in ovs, fullstack and functional jobs and for those we have some known issues 15:29:52 <ykarel> anything to add? 15:30:01 <ralonsoh> no, thanks 15:30:08 <slaweq> nope 15:30:17 <lajoskatona> nothing from me 15:30:25 <ykarel> k thx 15:30:27 <ykarel> #topic On Demand 15:30:34 <ykarel> anything else you would like to raise? 15:31:27 <ykarel> i have one small one 15:31:41 <ykarel> we have Holiday Next Monday in India, so will be out 15:32:26 <lajoskatona> Have a good time :-) 15:32:30 <ralonsoh> we can skip next week unless there is a fire, right? 15:32:37 <ykarel> i was thinking to cancel that, and if have anything can just raise it on Neutron meeting on Tuesday 15:32:44 <bcafarel> +1 15:32:49 <slaweq> +1 for canceling next week's meeting then 15:33:27 <ykarel> ok then will send a cancellation email for it 15:33:48 <lajoskatona> +1 for cancel 15:34:01 <mlavalle> \o+1 15:34:08 <ykarel> anything else to raise? 15:37:14 <ykarel> k if nothing else let's close early and have everyone 23 minutes back 15:37:20 <ykarel> Thx everyone for joining 15:37:23 <ykarel> #endmeeting