15:00:12 <slaweq> #startmeeting neutron_ci 15:00:13 <opendevmeet> Meeting started Tue Jan 11 15:00:12 2022 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:13 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:13 <opendevmeet> The meeting name has been set to 'neutron_ci' 15:00:13 <ralonsoh> as many times as needed 15:00:18 <ralonsoh> hello again 15:00:18 <slaweq> hi 15:00:21 <lajoskatona> Hi 15:00:23 <mlavalle> o/ 15:00:28 <slaweq> welcome on the CI meeting :) 15:01:09 * mlavalle doesn't feel very well. Had a shingles vaccine shot yesterday. Having a noticeable reaction today 15:01:41 <obondarev> hi 15:02:10 <lajoskatona> mlavalle: get well 15:02:42 <mlavalle> it caught me bt surpirse. I didn't expect to be this noticebale 15:03:01 <slaweq> mlavalle: I hope You will get better soon 15:03:11 <slaweq> lets start with the meeting now 15:03:18 <slaweq> Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate 15:03:24 <slaweq> #topic Actions from previous meetings 15:03:29 <damiandabrowski[m]> I'm not sure if i should continue GARP topic when You have another meeting but just to summarize: we manually link up qg- interface after keepalived is started. So when keepalived starts, qg- interface is down, so no GARPs will be send and vrrp_garp_interval doesn't matter. 15:03:29 <damiandabrowski[m]> Trust me guys, I've spend a lot of hours on this issue but now I'm stuck, I would really appreciate someone from L3 team to help me with this. 15:03:29 <slaweq> ralonsoh to check bug https://bugs.launchpad.net/neutron/+bug/1956344 15:03:37 <ralonsoh> merged 15:03:45 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/823498 15:04:08 <slaweq> thx ralonsoh 15:04:12 <bcafarel> o/ 15:04:21 <slaweq> in fact this was most often issue which I noticed in the jobs from last week 15:04:28 <slaweq> (before Your patch was merged) 15:04:41 <mlavalle> that's a good clarification :-) 15:05:06 <slaweq> ok, next one 15:05:08 <slaweq> slaweq to check failure https://71d2302875cffcacbcb7-bd54a9781d6bc663ca8af93b25749dfd.ssl.cf5.rackcdn.com/823300/1/gate/neutron-functional-with-uwsgi/1938908/testr_results.html 15:05:16 <slaweq> I did some investigation last week 15:05:34 <slaweq> and so far I only found that router from the failed test was in FAULT state: 15:05:41 <slaweq> sty 04 12:04:58 ubuntu-focal-ovh-bhs1-0027878805 Keepalived_vrrp[113555]: Netlink reports ha-597350ae-19 down... (full message at https://matrix.org/_matrix/media/r0/download/matrix.org/rGInaqncbBcOoUuIFmbnZKNs) 15:05:57 <slaweq> but I had no idea why it happened like that 15:06:10 <slaweq> I opened bug https://bugs.launchpad.net/neutron/+bug/1956958 15:06:28 <slaweq> and proposed patch https://review.opendev.org/c/openstack/neutron/+/824098 to add some more logs in those functional tests 15:06:56 <slaweq> I hope I will be able to move forward with that issue when it will happen again and I will have those extra logs added 15:07:11 <slaweq> thx for fast review of the patch obondarev and ralonsoh 15:07:22 <mlavalle> I just merged it 15:07:49 <slaweq> thx mlavalle 15:08:09 <slaweq> and that are all action items from last week on my list 15:08:13 <slaweq> anything else I forgot maybe? 15:08:41 <mlavalle> I just want to mention that the fedora jib has been well behaved since last week 15:08:47 <mlavalle> job^^^ 15:09:07 <slaweq> yes, I saw it in periodic jobs that it's green - thx a lot mlavalle 15:09:11 <mlavalle> https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovn-tempest-ovs-master-fedora 15:10:04 <slaweq> with that I think we can move on to the next topic 15:10:06 <slaweq> #topic Stable branches 15:10:14 <slaweq> bcafarel: any updates/new issues there? 15:11:37 <bcafarel> we got quite a few patches merged, overall good 15:11:52 <bcafarel> there are a few timeouts in ussuri/train hopefully that patch of yours will help (looking for link) 15:12:11 <ralonsoh> right, good catch 15:12:29 <bcafarel> https://review.opendev.org/c/openstack/neutron/+/823963 - get your glasses to spot the change :) 15:13:30 <lajoskatona> and new stable releases for neutron on the way: https://review.opendev.org/c/openstack/releases/+/823779 15:13:53 <bcafarel> right thanks lajoskatona for them 15:14:03 <slaweq> ++ 15:14:20 <lajoskatona> elod commented to have minor version bump, I will check it after the meeting 15:14:55 <slaweq> ok, that's good 15:15:20 <slaweq> if there's nothing else, we can probably move to the next topic now 15:15:27 <slaweq> #topic Stadium projects 15:15:39 <slaweq> any updates/issues here? 15:16:25 <lajoskatona> I am struggling with networking-odl to have proper requirements: https://review.opendev.org/c/openstack/networking-odl/+/823428 15:17:16 <lajoskatona> and bgpvpn has some failures, and as I remember there was a patch in neutron and we have to have at least an RC of Neutron to make bgpvpn green 15:17:23 <lajoskatona> but this one I have to double check again 15:18:13 <lajoskatona> and that's it for stadium 15:19:13 <slaweq> ok, thx lajoskatona 15:19:29 <slaweq> if You would need help with that requirements issue, please let me know, I can take a look 15:19:53 <lajoskatona> slaweq: thanks 15:20:54 <slaweq> ok, next topic 15:20:55 <slaweq> #topic Grafana 15:21:06 <slaweq> #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate 15:21:16 <slaweq> in overall grafana looks good for me 15:21:26 <slaweq> and we are merging pretty many patches recently :) 15:21:45 <mlavalle> +1 15:22:44 <lajoskatona> +1 15:23:11 <slaweq> avg number of rechecks to get patch merged is also not that big recently 15:23:28 <slaweq> around 1 - not perfect but not that bad as it was 15:23:44 <lajoskatona> the graph seems to be quite short and empty (like Neutron-tempest-plugin Failure Rates (Gate queue)) is this just due to beginning of year or we have to change something is dashboards? 15:23:45 <bcafarel> that is much better indeed 15:24:06 <ralonsoh> FT was a bit unstable lately 15:24:46 <slaweq> lajoskatona: I don't think we need to change anything as in the number of job runs graphs there are non-zero values 15:25:00 <slaweq> I would wait and see in next days how it will be 15:25:07 <lajoskatona> slaweq: ok 15:25:44 <slaweq> ralonsoh: regarding FT, the main issue I saw last week was the one which You fixed this week 15:26:08 <slaweq> and your patch was merged today, so lets see how it will work in next days too 15:27:46 <slaweq> and TBH when I was looking at failed jobs from this week, I didn't really noticed any new issues there 15:27:54 <slaweq> only one in the periodic job 15:28:00 <slaweq> https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fab/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-postgres-full/fab8f09/testr_results.html 15:28:08 <slaweq> did You saw something like that already? 15:28:36 <slaweq> I'm not sure even if it's neutron or maybe nova issue 15:28:39 <ralonsoh> no. Could be related to Nova? during the detaching 15:28:42 <ralonsoh> right 15:28:47 <ralonsoh> i was writing this 15:28:52 <slaweq> :) 15:29:04 <slaweq> so maybe there is someone who would like to check it 15:29:15 <ralonsoh> I'll check it 15:29:19 <slaweq> and report bug for the correct team 15:29:20 <slaweq> thx ralonsoh 15:29:30 <slaweq> #action ralonsoh to check https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fab/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-postgres-full/fab8f09/testr_results.html 15:29:49 <slaweq> other than that I don't really have anything else to discuss today 15:30:06 * slaweq wishes to have such few topics on CI meeting every week :) 15:30:18 <slaweq> do You have anything else You would like to talk about today? 15:30:27 <slaweq> or if not, I will give You some time back 15:30:54 <lajoskatona> \o/ 15:31:13 <mlavalle> \o/ 15:31:16 <ralonsoh> bye 15:31:20 <slaweq> ok, so lets finish the meeting earlier today 15:31:26 <mlavalle> o/ 15:31:28 <slaweq> thx for attending it and have a great week 15:31:32 <opendevreview> Merged openstack/os-ken stable/victoria: Add requirements.txt to tox environment https://review.opendev.org/c/openstack/os-ken/+/824091 15:31:35 <bcafarel> o/ 15:31:37 <slaweq> #endmeeting