15:00:12 <slaweq> #startmeeting neutron_ci
15:00:13 <opendevmeet> Meeting started Tue Jan 11 15:00:12 2022 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:13 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:13 <opendevmeet> The meeting name has been set to 'neutron_ci'
15:00:13 <ralonsoh> as many times as needed
15:00:18 <ralonsoh> hello again
15:00:18 <slaweq> hi
15:00:21 <lajoskatona> Hi
15:00:23 <mlavalle> o/
15:00:28 <slaweq> welcome on the CI meeting :)
15:01:09 * mlavalle doesn't feel very well. Had a shingles vaccine shot yesterday. Having a noticeable reaction today
15:01:41 <obondarev> hi
15:02:10 <lajoskatona> mlavalle: get well
15:02:42 <mlavalle> it caught me bt surpirse. I didn't expect to be this noticebale
15:03:01 <slaweq> mlavalle: I hope You will get better soon
15:03:11 <slaweq> lets start with the meeting now
15:03:18 <slaweq> Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate
15:03:24 <slaweq> #topic Actions from previous meetings
15:03:29 <damiandabrowski[m]> I'm not sure if i should continue GARP topic when You have another meeting but just to summarize: we manually link up qg- interface after keepalived is started. So when keepalived starts, qg- interface is down, so no GARPs will be send and vrrp_garp_interval doesn't matter.
15:03:29 <damiandabrowski[m]> Trust me guys, I've spend a lot of hours on this issue but now I'm stuck, I would really appreciate someone from L3 team to help me with this.
15:03:29 <slaweq> ralonsoh to check bug https://bugs.launchpad.net/neutron/+bug/1956344
15:03:37 <ralonsoh> merged
15:03:45 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/823498
15:04:08 <slaweq> thx ralonsoh
15:04:12 <bcafarel> o/
15:04:21 <slaweq> in fact this was most often issue which I noticed in the jobs from last week
15:04:28 <slaweq> (before Your patch was merged)
15:04:41 <mlavalle> that's a good clarification :-)
15:05:06 <slaweq> ok, next one
15:05:08 <slaweq> slaweq to check failure https://71d2302875cffcacbcb7-bd54a9781d6bc663ca8af93b25749dfd.ssl.cf5.rackcdn.com/823300/1/gate/neutron-functional-with-uwsgi/1938908/testr_results.html
15:05:16 <slaweq> I did some investigation last week
15:05:34 <slaweq> and so far I only found that router from the failed test was in FAULT state:
15:05:41 <slaweq> sty 04 12:04:58 ubuntu-focal-ovh-bhs1-0027878805 Keepalived_vrrp[113555]: Netlink reports ha-597350ae-19 down... (full message at https://matrix.org/_matrix/media/r0/download/matrix.org/rGInaqncbBcOoUuIFmbnZKNs)
15:05:57 <slaweq> but I had no idea why it happened like that
15:06:10 <slaweq> I opened bug https://bugs.launchpad.net/neutron/+bug/1956958
15:06:28 <slaweq> and proposed patch https://review.opendev.org/c/openstack/neutron/+/824098 to add some more logs in those functional tests
15:06:56 <slaweq> I hope I will be able to move forward with that issue when it will happen again and I will have those extra logs added
15:07:11 <slaweq> thx for fast review of the patch obondarev and ralonsoh
15:07:22 <mlavalle> I just merged it
15:07:49 <slaweq> thx mlavalle
15:08:09 <slaweq> and that are all action items from last week on my list
15:08:13 <slaweq> anything else I forgot maybe?
15:08:41 <mlavalle> I just want to mention that the fedora jib has been well behaved since last week
15:08:47 <mlavalle> job^^^
15:09:07 <slaweq> yes, I saw it in periodic jobs that it's green - thx a lot mlavalle
15:09:11 <mlavalle> https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovn-tempest-ovs-master-fedora
15:10:04 <slaweq> with that I think we can move on to the next topic
15:10:06 <slaweq> #topic Stable branches
15:10:14 <slaweq> bcafarel: any updates/new issues there?
15:11:37 <bcafarel> we got quite a few patches merged, overall good
15:11:52 <bcafarel> there are a few timeouts in ussuri/train hopefully that patch of yours will help (looking for link)
15:12:11 <ralonsoh> right, good catch
15:12:29 <bcafarel> https://review.opendev.org/c/openstack/neutron/+/823963 - get your glasses to spot the change :)
15:13:30 <lajoskatona> and new stable releases for neutron on the way: https://review.opendev.org/c/openstack/releases/+/823779
15:13:53 <bcafarel> right thanks lajoskatona for them
15:14:03 <slaweq> ++
15:14:20 <lajoskatona> elod commented to have minor version bump, I will check it after the meeting
15:14:55 <slaweq> ok, that's good
15:15:20 <slaweq> if there's nothing else, we can probably move to the next topic now
15:15:27 <slaweq> #topic Stadium projects
15:15:39 <slaweq> any updates/issues here?
15:16:25 <lajoskatona> I am struggling with networking-odl to have proper requirements: https://review.opendev.org/c/openstack/networking-odl/+/823428
15:17:16 <lajoskatona> and bgpvpn has some failures, and as I remember there was a patch in neutron and we have to have at least an RC of Neutron to make bgpvpn green
15:17:23 <lajoskatona> but this one I have to double check again
15:18:13 <lajoskatona> and that's it for stadium
15:19:13 <slaweq> ok, thx lajoskatona
15:19:29 <slaweq> if You would need help with that requirements issue, please let me know, I can take a look
15:19:53 <lajoskatona> slaweq: thanks
15:20:54 <slaweq> ok, next topic
15:20:55 <slaweq> #topic Grafana
15:21:06 <slaweq> #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate
15:21:16 <slaweq> in overall grafana looks good for me
15:21:26 <slaweq> and we are merging pretty many patches recently :)
15:21:45 <mlavalle> +1
15:22:44 <lajoskatona> +1
15:23:11 <slaweq> avg number of rechecks to get patch merged is also not that big recently
15:23:28 <slaweq> around 1 - not perfect but not that bad as it was
15:23:44 <lajoskatona> the graph seems to be quite short and empty (like Neutron-tempest-plugin Failure Rates (Gate queue)) is this just due to beginning of year or we have to change something is dashboards?
15:23:45 <bcafarel> that is much better indeed
15:24:06 <ralonsoh> FT was a bit unstable lately
15:24:46 <slaweq> lajoskatona: I don't think we need to change anything as in the number of job runs graphs there are non-zero values
15:25:00 <slaweq> I would wait and see in next days how it will be
15:25:07 <lajoskatona> slaweq: ok
15:25:44 <slaweq> ralonsoh: regarding FT, the main issue I saw last week was the one which You fixed this week
15:26:08 <slaweq> and your patch was merged today, so lets see how it will work in next days too
15:27:46 <slaweq> and TBH when I was looking at failed jobs from this week, I didn't really noticed any new issues there
15:27:54 <slaweq> only one in the periodic job
15:28:00 <slaweq> https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fab/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-postgres-full/fab8f09/testr_results.html
15:28:08 <slaweq> did You saw something like that already?
15:28:36 <slaweq> I'm not sure even if it's neutron or maybe nova issue
15:28:39 <ralonsoh> no. Could be related to Nova? during the detaching
15:28:42 <ralonsoh> right
15:28:47 <ralonsoh> i was writing this
15:28:52 <slaweq> :)
15:29:04 <slaweq> so maybe there is someone who would like to check it
15:29:15 <ralonsoh> I'll check it
15:29:19 <slaweq> and report bug for the correct team
15:29:20 <slaweq> thx ralonsoh
15:29:30 <slaweq> #action ralonsoh to check https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fab/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-postgres-full/fab8f09/testr_results.html
15:29:49 <slaweq> other than that I don't really have anything else to discuss today
15:30:06 * slaweq wishes to have such few topics on CI meeting every week :)
15:30:18 <slaweq> do You have anything else You would like to talk about today?
15:30:27 <slaweq> or if not, I will give You some time back
15:30:54 <lajoskatona> \o/
15:31:13 <mlavalle> \o/
15:31:16 <ralonsoh> bye
15:31:20 <slaweq> ok, so lets finish the meeting earlier today
15:31:26 <mlavalle> o/
15:31:28 <slaweq> thx for attending it and have a great week
15:31:32 <opendevreview> Merged openstack/os-ken stable/victoria: Add requirements.txt to tox environment  https://review.opendev.org/c/openstack/os-ken/+/824091
15:31:35 <bcafarel> o/
15:31:37 <slaweq> #endmeeting