Wednesday, 2020-07-15

*** yamamoto has quit IRC00:03
*** rmcall has quit IRC00:06
*** jamesmcarthur has quit IRC00:11
*** jamesmcarthur has joined #openstack-meeting-300:22
*** ricolin_ has joined #openstack-meeting-300:59
*** jamesmcarthur has quit IRC01:03
*** jamesmcarthur has joined #openstack-meeting-301:05
*** yamamoto has joined #openstack-meeting-301:26
*** jamesmcarthur has quit IRC01:30
*** jamesmcarthur has joined #openstack-meeting-301:51
*** yamamoto has quit IRC02:00
*** apetrich has quit IRC02:14
*** rmcall has joined #openstack-meeting-302:31
*** jamesmcarthur has quit IRC02:43
*** jamesmcarthur has joined #openstack-meeting-302:43
*** jamesmcarthur has quit IRC02:47
*** psachin has joined #openstack-meeting-303:39
*** jamesmcarthur has joined #openstack-meeting-303:43
*** jamesmcarthur has quit IRC03:45
*** jamesmcarthur_ has joined #openstack-meeting-303:45
*** yamamoto has joined #openstack-meeting-303:59
*** yamamoto has quit IRC03:59
*** yamamoto has joined #openstack-meeting-304:00
*** markvoelker has joined #openstack-meeting-304:11
*** markvoelker has quit IRC04:15
*** markvoelker has joined #openstack-meeting-305:01
*** markvoelker has quit IRC05:05
*** links has joined #openstack-meeting-305:11
*** psahoo has joined #openstack-meeting-305:55
*** jamesmcarthur_ has quit IRC05:57
*** psahoo has quit IRC06:22
*** psahoo has joined #openstack-meeting-306:40
*** ralonsoh has joined #openstack-meeting-306:48
*** e0ne has joined #openstack-meeting-308:06
*** markvoelker has joined #openstack-meeting-308:12
*** markvoelker has quit IRC08:17
*** yamamoto has quit IRC08:55
*** yamamoto has joined #openstack-meeting-309:00
*** yamamoto has quit IRC09:02
*** yamamoto has joined #openstack-meeting-309:08
*** apetrich has joined #openstack-meeting-309:12
*** psahoo has quit IRC09:14
*** psahoo has joined #openstack-meeting-309:30
*** ricolin_ has quit IRC09:49
*** jraju__ has joined #openstack-meeting-310:29
*** links has quit IRC10:30
*** yamamoto has quit IRC11:01
*** yamamoto has joined #openstack-meeting-311:40
*** yamamoto has quit IRC12:06
*** baojg has quit IRC12:17
*** baojg has joined #openstack-meeting-312:17
*** yamamoto has joined #openstack-meeting-312:23
*** yamamoto has quit IRC12:42
*** yamamoto has joined #openstack-meeting-313:24
*** artom has joined #openstack-meeting-313:31
*** baojg has quit IRC13:33
*** yamamoto has quit IRC13:47
*** tosky has joined #openstack-meeting-314:29
*** maciejjozefczyk has joined #openstack-meeting-314:45
*** lajoskatona has joined #openstack-meeting-315:00
ralonsoh#startmeeting neutron_ci15:00
openstackMeeting started Wed Jul 15 15:00:25 2020 UTC and is due to finish in 60 minutes.  The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"15:00
openstackThe meeting name has been set to 'neutron_ci'15:00
lajoskatonao/15:00
ralonsohtoday I'll need a bit of help from you15:01
bcafarelhalf o/ (listening to an internal meeting in // )15:01
ralonsohI know...15:01
ralonsohok, let's go15:02
ralonsohGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:02
ralonsohPlease open now :)15:02
ralonsoh(as usual)15:02
ralonsoh#topic Actions from previous meetings15:02
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"15:02
ralonsohmaciejjozefczyk to check neutron_tempest_plugin.scenario.test_connectivity.NetworkConnectivityTest.test_connectivity_through_2_routers in ovn jobs15:02
maciejjozefczykheya15:02
maciejjozefczykyes im looking at this one15:02
ralonsohcool15:03
maciejjozefczykI believe I tracked something https://review.opendev.org/#/c/740491/15:03
ralonsohI see a job failing15:03
maciejjozefczykit started failing somewhere around those refactors15:03
maciejjozefczykhttps://github.com/ovn-org/ovn/commits/master15:03
ralonsohso this is not always happening15:04
maciejjozefczykovn-northd: Document OVS register usage in logical flows. +15:04
maciejjozefczykyes, thats strange... for stable release it works fine15:04
maciejjozefczykim gonna continue investigating whats this15:04
ralonsohsure thanks!15:04
maciejjozefczykI was unable to reproduce it locally so I'm using gate to do so15:04
ralonsohand did you have time for the other action?15:05
ralonsohmaciejjozefczyk to check failing neutron-ovn-tempest-full-multinode-ovs-master job15:05
maciejjozefczykthats the same15:06
maciejjozefczykths scenario started failing in this particular job15:06
ralonsohok, I'll remove it from the logs15:06
maciejjozefczykkk15:06
ralonsohmy bad, as I said, I'll need help today!15:06
ralonsohand the last one15:06
ralonsohslaweq to change condition in the TestNeutronServer to have better logging15:06
ralonsohI think we can wait two weeks for this one15:07
ralonsohsomething to add here?15:07
ralonsohok15:07
ralonsoh#topic Stadium projects15:08
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"15:08
lajoskatonaralonsoh: I think this is for logging: https://review.opendev.org/74028315:08
lajoskatonaso ongoing15:08
ralonsohyou are right15:08
ralonsohI didn't see that15:08
ralonsohI'll add it to the logs15:08
ralonsohok, I'll check that after the meeting15:09
ralonsohabout the grenade jobs15:09
ralonsohneutron-ovn has an ongoing patch15:09
ralonsohhttps://review.opendev.org/#/c/729591/15:09
ralonsohbut is failing due to an unknown variable TEMPEST_CONFIG15:10
ralonsohif I have time, I'll review this patch this week15:10
ralonsohDo we have another ongoing patch related to https://etherpad.openstack.org/p/neutron-train-zuulv3-py27drop?15:11
lajoskatonaI have similar for odl, I have to go back15:11
lajoskatonaas I had some reviews from QA team15:11
ralonsohlink?15:11
lajoskatonahttps://review.opendev.org/72564715:11
lajoskatonaI found it finally.15:11
lajoskatonaThe job itself is failing, but it was failing previously as well15:12
lajoskatonaI had to ping again the odl community, but not that responsive recently15:12
*** jraju__ has quit IRC15:12
ralonsohyeah, I can't help you there, but I can review it later today15:13
lajoskatonaralonsoh: thanks15:13
ralonsohnext item is IPv6-only CI15:14
ralonsohhttps://etherpad.opendev.org/p/neutron-stadium-ipv6-testing15:14
ralonsohbut I see most of the patches still in WIP15:14
ralonsohand not very active15:14
bcafarelindeed it has been some time since I last checked that one15:15
lajoskatonaas I remember gmann was/is  the driver and he is really busy nowadays15:15
ralonsohI know15:15
ralonsoh(we need to focus on one single task and finish it)15:16
lajoskatona+1 (happy ideal world)15:16
ralonsohok, next topic15:17
ralonsoh#topic Switch to Ubuntu Focal15:17
*** openstack changes topic to "Switch to Ubuntu Focal (Meeting topic: neutron_ci)"15:17
ralonsoh#link https://etherpad.opendev.org/p/neutron-victoria-switch_to_focal15:17
ralonsohI rebased https://review.opendev.org/#/c/734304/15:17
ralonsohwe merged the patch installing ovs from PyPI15:17
ralonsohthat means the problem we had in this patch should be solved15:17
ralonsoh(the CI is too busy and 734304 doesn't start)15:18
ralonsoh#link https://review.opendev.org/#/c/737370/15:19
ralonsohis blocked due to https://review.opendev.org/#/c/734700/15:19
ralonsohuntil we don't have the jobs migrated in tempest, we can't continue in neutron15:19
ralonsohand the same for ODL15:20
ralonsoh#link https://review.opendev.org/#/c/736703/15:20
bcafarelfor tempest jobs depends-on 734700 should work (at least that was the recommended way to test in last progress report)15:20
bcafareleither that or I got confused in my review ids15:21
ralonsohbcafarel, and in Neutron the patch is working https://review.opendev.org/#/c/737370/15:22
ralonsoh"neutron-tempest-dvr-ha-multinode-full" is passing15:22
ralonsohI think we should just wait for the patch in tempest to land15:23
ralonsohdo we have something else in this topic?15:24
ralonsohok, let's move then15:25
ralonsoh#topic Stable branches15:25
*** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)"15:25
ralonsohUssuri dashboard: http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=115:25
ralonsohbcafarel, I'll need your expertise here15:25
ralonsohand in Train dashboard: http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=115:25
bcafarelok so the good news part is, gates are back in shape in all (neutron) stable branches!15:26
ralonsohif I'm not wrong, the pep8 patches are merged15:26
ralonsohcool15:26
bcafarelwhich may be a part of the gates backlog, I sent a few rechecks this morning on top of them15:27
ralonsohwell, I see the recheck queue is almost at 100%15:28
bcafarelon the not-so-nice side, any other project wanting green stable gates needs to backport that isort fix until they release a new version15:28
ralonsohbcafarel, I'm tracking the pylint project, once they release a version supporting the new isort release, I'll update requirements15:28
ralonsohin all needed branches15:28
bcafarelralonsoh++ quite a few folks will really appreciate that one I think :)15:29
ralonsohapart from this, something remarkable this week in the stable branches?15:29
bcafarelnot much, we should have more interesting data next week15:29
bcafarel(gates fixes are too recent here)15:30
ralonsohsure15:30
ralonsohthanks a lot15:30
ralonsohok, let's move on15:30
ralonsoh#topic Grafana15:30
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"15:30
ralonsoh#link http://grafana.openstack.org/d/Hj5IHcSmz/neutron-failure-rate?orgId=115:30
ralonsohI don't see anything to be commented here15:31
ralonsohwell yes,m neutron-ovn-tempest-ovs-master-fedora15:31
ralonsohstill broken15:32
* ralonsoh review neutron-ovn-tempest-ovs-master-fedora periodic job15:32
ralonsohI'll check it later15:32
ralonsohI don't have the cool script slawek uses to see the number of rechecks15:33
ralonsohanyway, something else to add here?15:34
ralonsohok, next topic15:34
ralonsoh#topic fullstack/functional15:34
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"15:34
ralonsohI filled a bug in oslo.privsep15:35
ralonsoh#link https://bugs.launchpad.net/oslo.privsep/+bug/188750615:35
openstackLaunchpad bug 1887506 in oslo.privsep "Privileged daemon should not monkey patch "os", "threading" or "socket" libraries" [Undecided,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)15:35
ralonsohthe goal is to un-monkey-patch the privileged daemon to avoid the timeouts we have during the FTs execution15:35
ralonsoh#link https://review.opendev.org/#/c/740970/15:35
bcafarelwow that looks nice15:36
ralonsohwell, I know it works locally15:37
ralonsohbut that means nothing, you know15:37
ralonsohI have https://review.opendev.org/#/c/741017/ to check this patch in the Neutron CI15:37
ralonsohI have another log from the previous meeting15:38
ralonsohneutron.tests.functional.test_server.TestWsgiServer.test_restart_wsgi_on_sighup_multiple_workers15:38
ralonsohhttps://f7a63aeb9edd557a2176-4740624f0848c8c3257f704064a4516f.ssl.cf2.rackcdn.com/736026/4/gate/neutron-functional/d7d5c47/testr_results.html15:38
ralonsohI don't see this error assigned, so I'll take it (if I have some time this week)15:38
bcafarelmaybe have something like https://review.opendev.org/#/c/741017/ with all tests disabled (except functional/FT) and run a good number of rechecks?15:38
ralonsohhmmm right!15:39
bcafarelthat would give more confidence in the "un-monkey-ing fixes timeouts"15:39
ralonsohyou are right, I'll comment the rest of the jobs there15:39
ralonsohI have nothing in the fullstack plate15:40
ralonsohdo you have something?15:40
ralonsohok, next topic15:41
ralonsoh#topic Tempest/Scenario15:41
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"15:41
ralonsoh#link https://review.opendev.org/#/c/736186/15:41
ralonsohslawek left this patch to be reviwed15:41
ralonsohaaaand I see that was merged!15:41
ralonsohsorry, I'll remove it now15:41
bcafarelthe best review, W+115:41
ralonsohand the next one15:42
ralonsoh#link https://review.opendev.org/#/c/739955/15:42
ralonsohit makes sense, I'll review it now15:42
ralonsohand we have also the problem with the ovn tempest job15:43
ralonsohbut this is already addressed15:43
ralonsohsomething else here?15:43
ralonsohok, and last topic15:44
bcafarelnot from me15:44
ralonsoh#topic Periodic15:44
*** openstack changes topic to "Periodic (Meeting topic: neutron_ci)"15:44
ralonsohhttp://zuul.openstack.org/buildsets?project=openstack%2Fneutron&pipeline=periodic&branch=master15:44
ralonsohas commented, the Fedora job15:44
ralonsohI'll try to review it this week15:44
ralonsoh(any help will be welcomed)15:45
maciejjozefczykovn fedora periodic?15:45
ralonsoh neutron-ovn-tempest-ovs-master-fedora15:45
ralonsohhttp://grafana.openstack.org/d/Hj5IHcSmz/neutron-failure-rate?orgId=115:45
ralonsohIn the "periodic jobs" window15:46
*** yamamoto has joined #openstack-meeting-315:46
bcafarelopt/stack/ovn/controller/ovn-controller.c:505: undefined reference to `ovsdb_idl_reset_min_index'15:46
maciejjozefczykhttps://zuul.openstack.org/builds?job_name=neutron-ovn-tempest-ovs-master-fedora15:47
maciejjozefczykaight. ok15:47
maciejjozefczykI'll fix that tomorrow morning15:47
maciejjozefczyksorry for late fix15:47
ralonsohhahahaha15:47
ralonsohnp!15:47
maciejjozefczykyeah i saw this problem before15:47
ralonsohwhat is this?15:47
maciejjozefczykits a problem in OVN_BRANCH OVS_BRANCH combination15:47
ralonsohahhhh ok15:47
ralonsohI saw we are using ovn 20.06 now15:48
ralonsohright?15:48
maciejjozefczykits not yet merged15:48
maciejjozefczykok15:48
maciejjozefczykso in this job it is OVN_BRANCH=master and  OVS_BRANCH=51e9479da62edb04a5be47a7655de75c299b9fa115:48
maciejjozefczykso its broken because of it, OVN relates to things not be a part of that OVS hash15:49
maciejjozefczykI'll fix that tomorrow, thanks!15:49
ralonsohmaciejjozefczyk, thanks to you15:49
ralonsohthat's all for today15:50
maciejjozefczyk\o15:50
ralonsohdo you have something else to add?15:50
ralonsohthank you all15:51
ralonsoh#endmeeting15:51
bcafarelno, I agree with maciejjozefczyk's conclusion "\o" :)15:51
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:51
openstackMeeting ended Wed Jul 15 15:51:21 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:51
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-07-15-15.00.html15:51
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-07-15-15.00.txt15:51
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-07-15-15.00.log.html15:51
ralonsohhehhee bye!15:51
maciejjozefczykbb15:51
lajoskatonaBye15:51
*** lajoskatona has quit IRC15:53
*** maciejjozefczyk has quit IRC16:18
*** yamamoto has quit IRC16:24
*** baojg has joined #openstack-meeting-316:28
*** e0ne has quit IRC16:46
*** e0ne has joined #openstack-meeting-316:47
*** baojg has quit IRC16:49
*** baojg has joined #openstack-meeting-316:50
*** psachin has quit IRC16:58
*** psahoo has quit IRC17:19
*** jamesmcarthur has joined #openstack-meeting-317:36
*** e0ne has quit IRC18:00
*** jamesmcarthur has quit IRC18:02
*** jamesmcarthur has joined #openstack-meeting-318:03
*** jamesmcarthur has quit IRC18:07
*** jamesmcarthur has joined #openstack-meeting-318:17
*** yamamoto has joined #openstack-meeting-318:21
*** slaweq has joined #openstack-meeting-318:21
*** tosky has quit IRC18:25
*** tosky has joined #openstack-meeting-318:25
*** baojg has quit IRC18:38
*** baojg has joined #openstack-meeting-318:39
*** yamamoto has quit IRC19:01
*** rmcall has quit IRC19:22
*** rmcall has joined #openstack-meeting-319:23
*** rmcallis has joined #openstack-meeting-319:25
*** rmcall has quit IRC19:28
*** rmcallis__ has joined #openstack-meeting-319:28
*** rmcallis has quit IRC19:31
*** ralonsoh has quit IRC19:49
*** raildo has quit IRC20:16
*** raildo has joined #openstack-meeting-320:16
*** raildo has quit IRC20:22
*** raildo has joined #openstack-meeting-320:24
*** slaweq has quit IRC20:29
*** raildo has quit IRC20:30
*** jamesmcarthur_ has joined #openstack-meeting-320:45
*** jamesmcarthur has quit IRC20:48
*** yamamoto has joined #openstack-meeting-321:00
*** rmcall has joined #openstack-meeting-321:27
*** rmcallis__ has quit IRC21:28
*** rmcall has quit IRC21:30
*** rmcall has joined #openstack-meeting-321:31
*** rmcall has quit IRC21:35
*** yamamoto has quit IRC21:35
*** rmcallis has joined #openstack-meeting-321:36
*** yamamoto has joined #openstack-meeting-321:36
*** rmcallis has quit IRC21:42
*** logan- has quit IRC22:17
*** logan- has joined #openstack-meeting-322:19
*** baojg has quit IRC22:43
*** baojg has joined #openstack-meeting-322:44
*** tosky has quit IRC22:48
*** yamamoto has quit IRC22:59
*** jamesmcarthur has joined #openstack-meeting-323:00
*** jamesmcarthur_ has quit IRC23:04
*** markvoelker has joined #openstack-meeting-323:11
*** markvoelker has quit IRC23:15
*** yamamoto has joined #openstack-meeting-323:30
*** yamamoto has quit IRC23:32
*** yamamoto has joined #openstack-meeting-323:32
*** baojg has quit IRC23:58
*** jamesmcarthur has quit IRC23:58
*** baojg has joined #openstack-meeting-323:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!