15:01:31 <ykarel> #startmeeting neutron_ci
15:01:31 <opendevmeet> Meeting started Mon Nov 17 15:01:31 2025 UTC and is due to finish in 60 minutes.  The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:31 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:31 <opendevmeet> The meeting name has been set to 'neutron_ci'
15:01:37 <ykarel> Ping list: bcafarel, lajoskatona, slawek, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira
15:01:44 <ykarel> Hi everyone, it's IRC today
15:01:46 <ralonsoh> hello
15:01:48 <mtomaska> o/
15:02:01 <slaweq> o/
15:04:37 <opendevreview> Sergey Kraynev proposed openstack/neutron master: Replace response body in after hook for 404 error  https://review.opendev.org/c/openstack/neutron/+/966498
15:05:50 <ykarel> Let's start with actions from previous week
15:06:14 <ykarel> lajoskatona to open bug for bagpipe failures
15:08:37 <ralonsoh> I'll do it this afternoon, or tomorrow morning
15:09:00 <ykarel> looks like lajos not around
15:09:03 <ykarel> thx ralonsoh
15:09:36 <ykarel> ichen to report bug for test_add_manager functional failures
15:09:51 <ykarel> #link https://bugs.launchpad.net/neutron/+bug/2131024
15:10:11 <ralonsoh> patches: https://review.opendev.org/q/I64a82952a0895d7ffd8d3e886c4615d62bbe4b20
15:10:35 <ykarel> thx ralonsoh ichen
15:10:48 <ykarel> ichen to report issue with test_sync_ha functional tests
15:11:18 <ralonsoh> reported and patch: https://review.opendev.org/c/openstack/neutron/+/966789
15:11:20 <ykarel> looks like already tracked in https://bugs.launchpad.net/neutron/+bug/2130622
15:11:50 <ykarel> thx ralonsoh
15:12:19 <ykarel> #topic Stable branches
15:12:35 <ykarel> bcafarel, around?
15:13:05 <ichen> Regarding bug 2131024, I don’t know if it’s related, but I rechecked after the patch was in place and ran into another timeout error.  This time at test-create_bridges.
15:13:11 <ykarel> Not much activity on stable, wrt failures in periodics and unmaintained which we will check in next sections
15:15:06 <ykarel> ichen, could be different issue as well
15:15:26 <ykarel> but you can post the results on bug comment itself to follow up on this
15:16:06 <cardoe> ralonsoh, haleyb: Would you like a demo of the l2vni bits at this Friday's Neutron Drivers meeting? TheJulia would likely be interested in attending.
15:16:14 <ichen> Will do.  So far it seems like I have the only failure.
15:16:20 <ralonsoh> cardoe, in a meeting now, we should wait
15:17:04 <ykarel> ralonsoh, wrong channel :)
15:17:40 <ykarel> sorry missed above message :(
15:18:28 <ykarel> #topic Stadium projects
15:18:37 <ykarel> failures in bagpipe and dynamic-routing
15:18:53 <ralonsoh> I'm working on the dynamic-routing ones
15:18:59 <ralonsoh> but I didn't find the problem yet
15:19:01 <ykarel> for dynamic-routing it's already being tracked
15:19:12 <ralonsoh> I marked as non-voting 2025.2 and master n-d-r
15:19:15 <ykarel> for bagpipe we already discussed above
15:19:22 <ykarel> ralonsoh, ack
15:19:41 <ykarel> #topic Rechecks
15:19:52 <ykarel> we still have some rechecks due to known intermittent issues
15:20:02 <ykarel> bare recheck wise it was good 0/23 bare rechecks
15:20:33 <ykarel> #topic fullstack/functional
15:20:38 <ykarel> test_sync_unify_ha_chassis_group_network_existing_hcg
15:20:46 <ralonsoh> fix pushed
15:20:48 <ykarel> https://b023373d7c444abb2078-8e6e263cc7d5bd2680bb2429d6dd2093.ssl.cf2.rackcdn.com/openstack/2eb56db528924de497e15bf106bdb208/testr_results.html
15:20:48 <ykarel> https://b34cc94439764c444f06-da6d66fdec213ff875550bc735254b25.ssl.cf2.rackcdn.com/openstack/7e75dc3f316049daae11252e35b52bec/testr_results.html
15:20:48 <ykarel> https://3bef97670d6a1ae092d5-54a30f9e4df167ae2747c6c3ff2739ca.ssl.cf1.rackcdn.com/openstack/81c325de624543b1a3e780e5fce3c4cb/testr_results.html
15:20:57 <ykarel> Tracked in https://bugs.launchpad.net/neutron/+bug/2130622
15:21:03 <ykarel> test_add_manager_appends
15:21:09 <ykarel> https://c355270b22583c2d2af0-42801c5a43c64ea303a559bec7f7cdd7.ssl.cf5.rackcdn.com/openstack/79c496668d834a1795ecdb3cb8398708/testr_results.html
15:21:14 <ykarel> Tracked in https://bugs.launchpad.net/neutron/+bug/2131024
15:21:17 <ykarel> thx ralonsoh
15:22:13 <ykarel> #topic Tempest/Scenario
15:22:22 <ykarel> test_rebuild_server
15:22:30 <ykarel> https://7c3ab39fb588a71f5255-f4d168cdb20f40841821e4b213645c0f.ssl.cf1.rackcdn.com/openstack/9a6ee94228e94cd88f05a2d166982bce/testr_results.html
15:22:31 <ykarel> https://f5099681efe8f43a08c5-01a74348816d1fa0c2c779804dc21128.ssl.cf2.rackcdn.com/openstack/e549066a8611451dbafa44bef6498b8a/testr_results.html
15:22:57 <ykarel> Seen this multiple times and from traces it's kernel panic in guest VM
15:23:06 <ykarel> will do check more and report an issue for this
15:23:18 <ykarel> #action ykarel to check/report issue for test_rebuild_server
15:23:31 <ykarel> #topic grenade
15:23:40 <ykarel> stable/2024.2 and 2025.1 triggered by 2024.1 unmaintained https://bugs.launchpad.net/neutron/+bug/2131215
15:24:09 <ykarel> for this patches are up in grenade project but are blocked on some octavia job failures
15:25:59 <ykarel> may be related to same issue seen in designate months back due to oslo.utils release https://review.opendev.org/c/openstack/designate/+/955684
15:26:02 <ykarel> cc elodilles_pto
15:26:28 <ykarel> #topic Periodic
15:26:42 <ykarel> - broken since 14th https://zuul.openstack.org/builds?job_name=openstack-tox-py312-with-ovsdbapp-master&project=openstack/neutron
15:26:51 <ykarel> 9 tests failing in TestOvnNbIdlNotifyHandler and TestOvnSbIdlNotifyHandler https://e489c7a3a7985c7eab99-163c7d45d239b0b9fa754872d6b3d883.ssl.cf5.rackcdn.com/openstack/6e4dd6601bed4950a743e72de7c54ad5/testr_results.html
15:27:02 <ralonsoh> I'll check them right now
15:27:07 <ykarel> there were couple of patches merged in ovsdbapp so may be related to that?
15:27:14 <ralonsoh> we had a release recently
15:27:28 <ralonsoh> yes, most most probably (one of them mine... so we have a winner)
15:28:05 <ralonsoh> failing since last friday
15:28:08 <ykarel> ralonsoh, but not released https://review.opendev.org/c/openstack/releases/+/966559
15:28:25 <ralonsoh> yeah but I realised this is ovsdbapp master CI job
15:28:40 <ykarel> so once that merges we may have failure in neutron gate
15:28:48 <ralonsoh> so, bytw, I'm blocking the release patch
15:29:06 <ykarel> just release will not impact, but u-c update for that
15:29:16 <ykarel> but yes good to block and confirm this is related
15:29:19 <ykarel> thx ralonsoh
15:29:33 <ykarel> #action ralonsoh to check issue with neutron ovsdbapp master job
15:29:52 <ykarel> unmaintained/2024.1 jobs broken https://zuul.openstack.org/buildset/688c86e23239473b82ebba743fae237b
15:30:10 <ykarel> with neutron-tempest-plugin 3.0.1.dev10 depends on neutron-lib>=3.21.1
15:30:20 <ykarel> looks need jobs adjustment
15:30:27 <ykarel> i can check this
15:30:40 <ralonsoh> thanks
15:30:44 <ykarel> #action ykarel to check/clear issue with unmaintained/2024.1 jobs
15:31:06 <ykarel> #topic Grafana
15:31:14 <ykarel> https://grafana.opendev.org/d/f913631585/neutron-failure-rate
15:31:20 <ykarel> let's have quick look here too
15:33:06 <ykarel> looks normal, failures already discussed in gate
15:33:14 <ykarel> anything to add?
15:33:41 <ralonsoh> nothing
15:34:23 <ykarel> #topic On Demand
15:34:32 <ykarel> anything else you would like to raise?
15:34:43 <ichen> Not from me
15:34:52 <ralonsoh> nothing, thanks
15:36:22 <ykarel> ok then
15:36:26 <ykarel> thx for joining
15:36:37 <ykarel> let's close early and have all ~24 minutes back
15:36:41 <ykarel> #endmeeting