15:01:21 <slaweq> #startmeeting neutron_ci
15:01:22 <openstack> Meeting started Tue May 11 15:01:21 2021 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:24 <slaweq> hi
15:01:26 <openstack> The meeting name has been set to 'neutron_ci'
15:01:33 <ralonsoh> hi
15:01:56 <lajoskatona> Hi
15:02:14 <slaweq> Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate
15:02:46 <slaweq> I think we can start now
15:02:51 <slaweq> #topic Actions from previous meetings
15:02:58 <slaweq> ralonsoh to mark test_keepalived_spawns_conflicting_pid_vrrp_subprocess functional test as unstable
15:03:28 <ralonsoh> ok, that won't fix the CI
15:03:38 <ralonsoh> because this test is timing out
15:03:59 <ralonsoh> and that cannot be marked as unstable
15:04:10 <slaweq> how often it happens? do You remember?
15:04:28 <ralonsoh> not too often, but still too much
15:04:39 <ralonsoh> this is because the sigterm signal does not reply
15:04:51 <ralonsoh> even with shell "kill" command (same as with os.kill)
15:05:11 <ralonsoh> I was working in a patch for privsep
15:05:16 <ralonsoh> but I never finished it
15:05:29 <ralonsoh> I'll continue with this one, but I'll need time
15:06:28 <slaweq> sure
15:06:43 <slaweq> maybe we should simply skip those tests for now?
15:07:01 <ralonsoh> I think so, I'll send a patch
15:07:06 <slaweq> ++
15:07:20 <slaweq> #action ralonsoh to skip test_keepalived_spawns_conflicting_pid_vrrp_subprocess functional tests
15:07:30 <slaweq> ok, next one
15:07:32 <slaweq> lajoskatona to check stadium job's and what needs to be switched to ovs explicity
15:08:19 <lajoskatona> Yeah, I have to run through it again, but as I remember we are ok
15:08:34 <slaweq> that's good news, thanks lajoskatona
15:08:55 <slaweq> and what about networking-sfc? this one has to be switched right?
15:10:25 <lajoskatona> yes
15:10:51 <slaweq> k
15:11:04 <slaweq> can You send patches to switch that one?
15:11:10 <slaweq> or maybe bcafarel already did that?
15:11:27 <lajoskatona> yes, I can,
15:11:32 <slaweq> thx a lot
15:11:53 <slaweq> #action lajoskatona to switch networking-sfc jobs to ovs explicitly
15:12:07 <slaweq> next one     ralonsoh or lajoskatona will add logging of the console log, related to the https://bugs.launchpad.net/neutron/+bug/1926109
15:12:08 <openstack> Launchpad bug 1926109 in neutron "SSH timeout (wait timeout) due to potential paramiko issue" [Critical,New]
15:13:10 <lajoskatona> I checked this one and as I see we log console log everywhere, but for some reason it is not always logged to subunit html report
15:13:19 <lajoskatona> only in tempest.log
15:15:49 <slaweq> ahh, ok
15:15:51 <slaweq> good to know
15:16:02 <slaweq> so we need to check in tempest.log in such case
15:16:04 <slaweq> thx
15:16:13 <slaweq> ok, next one
15:16:15 <slaweq> slaweq to check frequency of the multicast issue in the ovn job and report a LP bug for that
15:16:28 <slaweq> I checked and opened bug https://bugs.launchpad.net/neutron/+bug/1926780
15:16:28 <openstack> Launchpad bug 1926780 in neutron "Multicast traffic scenario test is failing sometimes on OVN job" [High,Confirmed]
15:16:42 <slaweq> but I didn't had time to look at it more
15:17:50 <slaweq> ok, next one
15:17:52 <slaweq> ralonsoh: to check periodic nftables jobs
15:18:04 <ralonsoh> done and merged
15:18:08 <ralonsoh> (I'll find the patch)
15:18:10 <slaweq> thank You :)
15:18:22 <slaweq> no need, we trust You ;)
15:18:27 <ralonsoh> hehehe thanks
15:18:32 <slaweq> ok, let's move on
15:18:34 <slaweq> #topic Stadium projects
15:18:48 <slaweq> lajoskatona: anything regarding stadium projects ci?
15:20:18 <lajoskatona> nothing, as I remmeber from the week
15:20:25 <slaweq> great :)
15:20:36 <slaweq> that's news which I like in this meeting ;)
15:20:51 <lajoskatona> for bgpvpn (or bagpipe?) I pushed patches for stable brancehs to make l-c job non-voting
15:20:52 <slaweq> #topic Stable branches
15:21:05 <lajoskatona> yeah so this overlaps the next topic :-)
15:21:11 <slaweq> thx, I will review that patch
15:21:26 <slaweq> regarding stable branches, I have just one thing
15:21:43 <lajoskatona> this one is active, and I hope can be backported: https://review.opendev.org/c/openstack/networking-bgpvpn/+/743487
15:21:47 <slaweq> just few minutes ago there was proposed patch to go to EM phase in train
15:22:00 <slaweq> https://review.opendev.org/c/openstack/releases/+/790760
15:22:37 <slaweq> when this will be done we should pin neutron-tempest-plugin in stable/train jobs and remove them from the check/gate queues in neutron-tempest-plugin repo
15:22:43 <slaweq> I will do it this week
15:22:53 <lajoskatona> +1
15:23:05 <slaweq> #action slaweq to pin neutron-tempest-plugin for train after it will be EM
15:23:37 <slaweq> other than that I didn't saw any issues with stable branches and ci
15:23:46 <slaweq> things are merging pretty ok AFAICT
15:25:53 <slaweq> I think we can move on
15:25:53 <slaweq> #topic Grafana
15:25:53 <slaweq> #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate
15:26:17 <slaweq> I don't see any serious issues there recently
15:26:31 <slaweq> seems like things are running pretty ok
15:26:38 <slaweq> except failures which we already know
15:27:41 <slaweq> we should maybe go with https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/788412 to make our neutron-tempest-plugin jobs to be more stable
15:27:41 <lajoskatona> Nearly offtopic: have you read the mail about stopping logstash?
15:27:57 <slaweq> lajoskatona: yes, I read it today and I'm sad about that :/
15:28:16 <lajoskatona> it was not perfect but better than nothing :-(
15:28:27 <slaweq> yeah
15:28:38 <lajoskatona> sorry for disturbing...
15:29:23 <slaweq> no problem at all
15:29:39 <ralonsoh> +w to this patch
15:29:46 <slaweq> if You know about any alternative, I will be very happy to listen about it :)
15:29:51 <slaweq> thx ralonsoh
15:31:41 <slaweq> ok, let's move on
15:32:02 <slaweq> tbh I didn't found any new serious issues worth to discuss here today
15:32:31 <slaweq> I did found few functional tests failures, but each of them were just one time failure so far
15:32:41 <slaweq> I listed them in the https://etherpad.opendev.org/p/neutron-ci-meetings
15:33:10 <slaweq> other than that there are only known issues which I saw in our ci
15:33:43 <slaweq> do You maybe have any issues which You would like to discuss?
15:33:49 <ralonsoh> well, FT CI job is not very stable
15:34:03 <ralonsoh> i've seen a lot of timeouts recentlyt
15:34:09 <ralonsoh> but I din't find the cause
15:34:21 <slaweq> hmm
15:34:31 <slaweq> I didn't saw many timeouts when I was checking that
15:34:40 <slaweq> but I will check once again, maybe I missed something
15:34:48 <slaweq> #action slaweq to check functional job timeouts
15:37:16 <slaweq> and that's basically all from me for today
15:37:27 <slaweq> if You don't have anything else, we can finish meeting earlier
15:37:43 <ralonsoh> no, thanks
15:37:45 <lajoskatona> \o/
15:38:01 <slaweq> ok
15:38:07 <slaweq> thx for attending the meeting
15:38:12 <slaweq> and have a great evening :)
15:38:15 <slaweq> o/
15:38:18 <slaweq> #endmeeting