15:00:20 #startmeeting neutron_ci 15:00:20 Meeting started Tue Jun 15 15:00:20 2021 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:20 The meeting name has been set to 'neutron_ci' 15:00:33 hi 15:00:39 Hi 15:00:43 hi 15:01:20 o/ 15:02:41 Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate 15:02:41 Please open now :) 15:02:42 #topic Actions from previous meetings 15:02:42 ralonsoh to update our ci job's definitions 15:02:42 and I think we can start 15:02:54 yes, I have some questions about this 15:03:05 are we going to duplicate all OVS jobs? 15:03:15 are we going to have OVS and OVN jobs? 15:03:38 personally I would be happy if we could end up with the same number of jobs which we had before switch default backend 15:04:15 so IMO we should have now "default" jobs which I those without backend in the name to be running ovn 15:04:25 and change *-ovn" jobs to be "-ovs" jobs 15:04:31 hi 15:04:33 but then, as in https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/791255, we are mixing OVS and OVN 15:04:45 I mean, we don't have a consistent CI 15:04:55 we have mixed OVS and OVN 15:05:29 "and change *-ovn" jobs to be "-ovs" jobs" --> didn't we decided to explicitly name all jobs? 15:05:33 with the name of the backend? 15:05:58 yes, we talked about that 15:06:17 ok, initially I'll explicitly rename all jobs 15:06:35 then we'll see if we need to have a duplicated CI running in "periodic" 15:06:37 we should always remember that jobs which e.g. inherits from some tempest jobs should use default devstack's config 15:07:27 ok, I'll push a patch 15:07:38 so e.g. job neutron-tempest-slow-py3 which inherits from tempest-slow-py3 directly but only limits number of tests run should use ovn as it's what is used in tempest-slow-py3 15:08:03 and similar thing is also for some other jobs like e.g. neutron-tempest-ipv6-only 15:08:19 and should be named with -ovn suffix 15:08:20 ralonsoh: so with "-ovn" for default jobs? 15:08:26 we can name those jobs "-ovn" but we should use default settings, no set explicitly backend in the job's definition 15:08:36 yes 15:08:40 does it makes sense? 15:08:43 perfect 15:08:48 or -default-backend (or shorter one if possible)? 15:09:10 changing the default backend is not usual 15:09:14 I prefer -ovn 15:09:54 I'm ok with "-ovn" 15:10:04 hmm true next default change is not for tomorrow :) 15:10:19 bcafarel I hope so :P 15:11:24 so ralonsoh You will continue that next week, right? 15:11:29 yes 15:11:35 great, thx 15:11:43 #action ralonsoh to update our ci job's definitions 15:11:54 ok, that was only AI from last week 15:12:00 we can move on to the next topic 15:13:26 #topic Stadium projects 15:13:38 anything to discuss regarding stadium projects' CI today? 15:13:51 nothing for today 15:14:01 I haven't seen any issue 15:14:09 good news, thx :) 15:14:19 #topic Stable branches 15:14:24 same question for stable branches 15:14:52 and similar answer, all good from what I saw 15:14:59 great 15:15:11 I just today sent some patches for EM branches: 15:15:16 Train: https://review.opendev.org/q/topic:%22clean-non-voting-jobs%22+(status:open%20OR%20status:merged) 15:15:16 Stein: https://review.opendev.org/q/topic:%22clean-non-voting-jobs-stable%252Fstein%22+(status:open%20OR%20status:merged) 15:15:23 please review them 15:15:48 sure 15:15:56 I noticed last week that we are still running non-voting jobs on those branches and we agreed some time ago that we should move them to the experimental queue 15:16:03 so those patches are doing exactly that 15:16:26 for stadium projects it will also be a nice spot check on CI status in those branches 15:16:46 bcafarel yes, I saw already some pep8/py27 failures 15:16:59 but I didn't had time to check them closely 15:18:30 ok, let's move on 15:18:34 #topic Grafana 15:19:30 http://grafana.openstack.org/dashboard/db/neutron-failure-rate#link 15:19:59 I don't see anything critical on the grafana this week 15:20:11 we need to update it as not all jobs are there 15:20:28 but I want to update it after ovs->ovn migration cleaning 15:22:13 except that, I think that our biggest problem still is fullstack job 15:22:20 and those failing L3 related tests 15:22:45 is it same issue? 15:23:29 I saw 2nd dhcp agent mostly workarounded that 15:23:39 obondarev let me check 15:24:07 https://614c13582752de139914-134fee3611f99557a1466ec5867d2d52.ssl.cf5.rackcdn.com/795781/3/check/neutron-fullstack-with-uwsgi/bf03ae2/testr_results.html 15:24:12 first example 15:24:20 it's from today morning 15:24:30 I think it's the same issue, or at least looks similar 15:25:35 yeah, looks the same.. 15:25:47 https://zuul.opendev.org/t/openstack/builds?job_name=neutron-fullstack-with-uwsgi 15:26:02 obondarev here You have more exmples 15:26:16 I think that I saw similar failures in most of those cases 15:27:57 hmm, but I see dhcp agents not hanging 15:28:03 same for ovs agents 15:28:14 so that might be not related to privsep 15:28:39 I'll look into this when have some time (tomorrow I hope) 15:29:00 obondarev thx 15:29:08 For the hanging issue privsep change is on its way: https://review.opendev.org/c/openstack/oslo.privsep/+/794993 15:29:16 #action obondarev to check fullstack timeouts in L3 agent tests 15:29:46 ^^ please review this patch 15:29:46 lajoskatona++ thx 15:30:56 I added it to my review list for tomorrow morning 15:31:04 thx 15:32:04 ok, let's move on 15:32:19 I just have one question about periodic jobs left for today 15:32:38 ralonsoh how is Your patch for fix neutron-lib-master periodic job? 15:32:42 done 15:32:44 one sec 15:32:46 is it merged? 15:32:49 no 15:32:57 ahh, ok 15:33:11 and this is blocking requirements patch for n-lib 2.12.0 15:33:16 https://review.opendev.org/c/openstack/neutron/+/794236 15:33:36 ok, I have a +2 from liuyulong 15:33:54 it's in the gate now 15:33:56 thx 15:34:01 thanks 15:35:12 I didn't found today any new issues worth to discuss here 15:35:28 so if You don't have anything else, we can finish meeting earlier today 15:35:43 nothing from me 15:35:50 nothing here 15:37:34 ok, so thx for attending the meeting 15:37:42 bye! 15:37:42 have a great evening and see You online 15:37:43 o/ 15:37:46 #endmeeting