Wednesday, 2020-02-26

*** yamamoto has joined #openstack-meeting-301:47
*** yamamoto has quit IRC02:07
*** yamamoto has joined #openstack-meeting-302:10
*** hongbin has joined #openstack-meeting-303:49
*** hongbin has quit IRC04:39
*** links has joined #openstack-meeting-305:25
*** e0ne has joined #openstack-meeting-305:59
*** kozhukalov has joined #openstack-meeting-306:08
*** e0ne has quit IRC06:15
*** e0ne has joined #openstack-meeting-306:16
*** e0ne has quit IRC06:30
*** slaweq_ has joined #openstack-meeting-307:42
*** slaweq_ is now known as slaweq07:45
*** ralonsoh has joined #openstack-meeting-308:53
*** psachin has joined #openstack-meeting-309:25
*** tsmith2 has quit IRC10:09
*** tsmith2 has joined #openstack-meeting-310:10
*** e0ne has joined #openstack-meeting-310:37
*** e0ne has quit IRC10:38
*** e0ne has joined #openstack-meeting-310:54
*** e0ne has quit IRC11:27
*** ianychoi_ is now known as ianychoi11:32
*** yamamoto has quit IRC11:49
*** yamamoto has joined #openstack-meeting-312:09
*** psachin has quit IRC12:25
*** yamamoto has quit IRC12:51
*** e0ne has joined #openstack-meeting-313:04
*** psachin has joined #openstack-meeting-313:05
*** e0ne has quit IRC13:43
*** jamesmcarthur has joined #openstack-meeting-313:53
*** e0ne has joined #openstack-meeting-314:01
*** slaweq has quit IRC14:05
*** slaweq has joined #openstack-meeting-314:09
*** maciejjozefczyk has joined #openstack-meeting-314:18
*** yamamoto has joined #openstack-meeting-314:22
*** jamesmcarthur has quit IRC14:27
*** jamesmcarthur has joined #openstack-meeting-314:30
*** jamesmcarthur has quit IRC14:32
*** links has quit IRC14:36
*** yamamoto has quit IRC14:48
*** yamamoto has joined #openstack-meeting-314:50
*** yamamoto has quit IRC14:55
slaweq#startmeeting neutron_ci15:00
openstackMeeting started Wed Feb 26 15:00:25 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"15:00
openstackThe meeting name has been set to 'neutron_ci'15:00
slaweqping ralonsoh: ci meeting15:02
slaweqok, lets start15:02
slaweqGrafana dashboard:
slaweqPlease open now :)15:02
slaweq#topic Actions from previous meetings15:03
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"15:03
slaweqfirst one15:03
slaweqralonsoh to talk with oslo people about our functional tests rootwrap issue15:03
ralonsohone secx15:03
ralonsohpatch merged and released 6.0.115:03
ralonsohoslo rootwrap15:03
slaweqralonsoh++ thx a lot15:04
slaweqand indeed I didn't saw such issues in last few days15:04
slaweqok, next one15:04
slaweqslaweq to try to reproduce and debug neutron-tempest-dvr ssh issue15:04
slaweqI didn't have time for that yet15:05
*** psachin has quit IRC15:05
slaweqbut I have some idea about this dvr job15:05
slaweqit is single node dvr job which IMO don't makes too much sense15:05
slaweqso my proposal is to15:05
slaweqa) remove this job from our ci15:05
slaweqb) change a bit neutron-tempest-dvr-ha-multinode-full to run networking related tests only15:05
slaweqc) make neutron-tempest-dvr-ha-multinode-full voting in few weeks if it will be stable enough15:06
slaweqso finally we will have one less job in our CI15:06
bcafarelhmm did we not plan something like that in Shanghai PTG? (dropping single node dvr)15:06
slaweqwhat do You think about it?15:06
bcafarel(and +1 on the plan of course)15:06
slaweqbcafarel: nope, we agreed to drop some grenade single node jobs15:06
bcafarelaah ok15:07
haleybi'm fine with it as long as multinode job is stable15:07
bcafarelanyway yes that sounds good, it will be add more visibility (and stability) to multinode dvr job15:07
slaweqhaleyb: multinode job seems to be at least as stable as single node one15:08
slaweqand in fact this single node job is testing almost only API15:08
haleybit wasn't always that way, but you and miguel have fixed some hard bugs :)15:08
slaweqhaleyb: but I'm not talking about neutron-tempest-plugin-dvr-multinode-scenario job which is still pretty not stable15:09
slaweqI'm talking about promoting neutron-tempest-dvr-ha-multinode-full to be voting and replace neutron-tempest-dvr15:09
slaweqthe differences between neutron-tempest-plugin-dvr-multinode-scenario and neutron-tempest-dvr-ha-multinode-full are:15:10
slaweq1. one is DVR and the other is DVR HA15:10
*** yamamoto has joined #openstack-meeting-315:10
slaweq2. one runs neutron-tempest-plugin tests and the other runs tempest API + some basic scenario tests15:10
slaweqso, can I assume that I have "green light" from the team to do that? :)15:12
slaweqthx, so I will propose patch(es) for that asap :)15:12
slaweq#action slaweq to drop neutron-tempest-dvr job and finally replace it with neutron-tempest-dvr-ha-multinode-full15:13
slaweqok, next one15:13
slaweqralonsoh to check periodic neutron-ovn-tempest-ovs-master-fedora job's failures15:13
ralonsohI spent 20 mins on this15:14
ralonsohand then I had other high priority tasks15:14
ralonsohsorry again15:14
slaweqralonsoh: no problem :)15:14
ralonsoh(this is the 2nd time I miss this task(15:14
slaweqdo You want to assign it to yourself for next week?15:14
slaweqor should we simply report a bug and maybe someone will check it?15:15
ralonsohnext week15:15
slaweqralonsoh: ok :)15:15
slaweq#action ralonsoh to check periodic neutron-ovn-tempest-ovs-master-fedora job's failures15:16
slaweqok, next one15:16
slaweqslaweq to report tempest bug with routers cleanup15:16
slaweqI checked this error and it wasn't problem with cleaning in tempest in fact15:16
slaweqthe root cause of this was that API call to add_router_interface was processed for more than 30 seconds by Neutron15:16
slaweqso urllib got timeout waiting for response and it retried15:17
slaweqbut in the retry this subnet was already plugged to the router as first call was already finished15:17
slaweqso it failed on adding interface to the router and because of that it wasn't added to cleanup list and failed to be cleaned too15:18
slaweqso I didn't report tempest bug for that :)15:18
slaweqand that's all actions from last week15:18
slaweqanything else You want to ask/add in this section?15:19
slaweqif no, lets move on15:19
slaweq#topic Stadium projects15:19
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"15:19
slaweqstandardize on zuul v315:19
slaweqwe are almost done there15:19
slaweqI need to fix my patches for neutron-vpnaas15:20
*** mugsie has quit IRC15:20
slaweqnetworking-bgpvpn is almost there - patch waits for some other fix for tempest job15:20
slaweqwe merged neutron-dynamic-routing patch recently15:20
njohnstonI need to check to see if we have any changes in the field yet for networking-odl or networking-midonet15:20
slaweqnjohnston: yes, those 2 projects are probably not touched yet15:21
bcafarelhopefully current recheck will be the good one this time (bagpipe is back up and running thanks to amotoki lajoskatona15:21
*** mugsie has joined #openstack-meeting-315:22
slaweqand according to py2 leftovers, we need to merge then release last version with py2 support and then merge
slaweqso please review
slaweqthx bcafarel and ralonsoh for review of it :)15:23
bcafarel:) njohnston looks like you are next on it then15:23
slaweqor haleyb :)15:23
slaweqas he is here too :)15:23
bcafareloh true15:24
njohnston +2+W15:24
haleybnate beat me to it15:24
slaweqthx :)15:24
bcafarelhaleyb: today you are in grey in IRC colors, so quite stealthy15:24
slaweqno, in my weechat haleyb is red :)15:25
* haleyb can't hide15:25
slaweqanything else You want to ask regarding stadium projects?15:25
* njohnston uses limechat today so you are all golden15:25
bcafarelfinal rocky releases will be out for neutron et all (so including stadium projects)15:26
slaweqbcafarel: yes, it should be done this week, today or tomorrow15:27
slaweqok, lets move on15:28
slaweqnext topic15:28
slaweq#topic Grafana15:28
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"15:28
slaweqwe don't have almost anything in gate recently15:28
slaweqand it's mostly because of ovn tempest jobs failing almost all the time15:29
ralonsohwe are on it now15:29
slaweqthis patch should fix it probably
ralonsohthis one for tempest-plugin15:30
ralonsohand then we need to recheck the FTs15:30
ralonsohyesterday we detected an error in OVN core15:30
ralonsoh(the bug is opened)15:30
slaweqralonsoh: ok, thx for that too15:30
ralonsohand we have this
slaweqbut for now most important is this tempest-plugin issue15:30
ralonsohyes, I now15:30
maciejjozefczykyes, the bugs is about missing cherry-pick of one patch to ovn 2.1215:31
maciejjozefczykits already fixed but not cherry-picked15:31
maciejjozefczykI asked ovn folks to do it15:31
maciejjozefczykbut anyways blacklisting it for now makes perfect sense15:32
slaweqthx maciejjozefczyk and ralonsoh for taking care of it15:32
slaweqnjohnston: haleyb please check if You have a minute15:32
slaweqthat should allow us to make zuul happy at least from time to time ;)15:32
haleybslaweq: i did a second ago, +2+W15:32
njohnstonhaleyb beat me to it15:32
slaweqyou have many races today ;P15:33
haleyband meetings15:33
njohnstonWe need to have one more - tiebreaker!15:33
slaweqnjohnston: :D15:33
slaweqif I will have one more I will send it to You as a tiebreaker :)15:34
slaweqfrom other things in grafana, still functional tests are failing a lot15:34
slaweqand I have one new "star" there15:34
slaweqanythig else regarding grafana You want to add?15:35
slaweqif no, lets talk about functional tests now15:36
slaweq#topic Fullstack/Functional15:36
*** openstack changes topic to "Fullstack/Functional (Meeting topic: neutron_ci)"15:36
slaweqI saw that during last week test failed many times15:36
slaweqI have now opened LP to report this bug there15:36
ralonsohbut this is because the problem in rootwrap15:37
slaweqralonsoh: yes, now I see15:37
ralonsohok ok15:37
slaweqI just saw some "FileNotFound" error and I though that it's something different :)15:37
slaweqand this looks like issue with Your rootwrap patch already15:38
slaweqralonsoh: do You think we should report new bug for that?15:39
ralonsohbut I have this one15:40
openstackLaunchpad bug 1864090 in oslo.rootwrap ""PathFilter" can raise exception when executing "os.path.realpath()"" [Undecided,Fix released] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)15:40
slaweqit's not with Your fix15:43
slaweqralonsoh: so should Your fix solve this problem with functional tests?15:43
ralonsohyes, I think so15:44
slaweqok, so lets keep fingers crossed for that :)15:44
slaweqand that was my main concern for today in fact15:45
slaweqother than that I saw some single failures so lets just keep an eye on it for now15:46
*** psachin has joined #openstack-meeting-315:46
slaweqregarding scenario jobs we already discussed about ovn jobs and neutron-tempest-dvr so we are also good there15:46
slaweqso that's basically all from my for today15:46
slaweqdo You have anything else You want to discuss today?15:46
slaweq#topic On demand agenda15:47
*** openstack changes topic to "On demand agenda (Meeting topic: neutron_ci)"15:47
maciejjozefczykI thinnk in near future we would be able to enable ovn multinode job15:48
maciejjozefczykwith fixes:15:48
maciejjozefczykit will be green (was already, but failed on test_two_sec_groups - quota issue)15:49
maciejjozefczykthats all from me15:49
slaweqok, thx maciejjozefczyk :)15:50
slaweqif there is nothing else for today, I think I can You back few minutes15:50
slaweqthx for attending15:50
slaweqand see You next week15:50
*** openstack changes topic to "OpenStack Meetings ||"15:51
openstackMeeting ended Wed Feb 26 15:51:00 2020 UTC.  Information about MeetBot at . (v 0.1.4)15:51
openstackMinutes (text):
*** psachin has quit IRC15:57
*** e0ne has quit IRC15:59
*** e0ne has joined #openstack-meeting-315:59
*** e0ne has quit IRC16:10
*** maciejjozefczyk has quit IRC16:21
*** jamesmcarthur has joined #openstack-meeting-316:36
*** psachin has joined #openstack-meeting-316:36
*** yamamoto has quit IRC16:39
*** rubasov has quit IRC16:47
*** yamamoto has joined #openstack-meeting-317:11
*** psachin has quit IRC17:12
*** yamamoto has quit IRC17:17
*** e0ne has joined #openstack-meeting-317:48
*** jamesmcarthur has quit IRC17:58
*** jamesmcarthur has joined #openstack-meeting-318:00
*** jamesmcarthur has quit IRC18:05
*** jamesmcarthur has joined #openstack-meeting-318:21
*** yamamoto has joined #openstack-meeting-318:22
*** igordc has joined #openstack-meeting-318:37
*** maciejjozefczyk has joined #openstack-meeting-318:42
*** igordc has quit IRC18:43
*** maciejjozefczyk has quit IRC18:57
*** jamesmcarthur has quit IRC19:31
*** yamamoto has quit IRC19:31
*** jamesmcarthur has joined #openstack-meeting-319:40
*** jamesmcarthur has quit IRC19:58
*** yamamoto has joined #openstack-meeting-320:02
*** e0ne has quit IRC20:06
*** yamamoto has quit IRC20:08
*** jamesmcarthur has joined #openstack-meeting-320:21
*** kozhukalov has quit IRC20:33
*** kozhukalov has joined #openstack-meeting-320:36
*** ralonsoh has quit IRC21:22
*** bnemec has quit IRC21:23
*** kozhukalov has quit IRC21:29
*** slaweq has quit IRC21:32
*** jamesmcarthur has quit IRC21:36
*** jamesmcarthur has joined #openstack-meeting-321:36
*** jamesmcarthur has quit IRC21:44
*** jamesmcarthur has joined #openstack-meeting-321:45
*** jamesmcarthur has quit IRC21:59
*** jamesmcarthur has joined #openstack-meeting-322:00
*** jamesmcarthur has quit IRC22:01
*** jamesmcarthur has joined #openstack-meeting-322:05
*** jamesmcarthur has quit IRC22:29
*** jamesmcarthur has joined #openstack-meeting-322:30
*** jamesmcarthur has quit IRC22:35
*** jamesmcarthur has joined #openstack-meeting-322:35
*** jamesmcarthur has quit IRC22:49
*** jamesmcarthur has joined #openstack-meeting-322:54
*** jamesmcarthur has quit IRC23:15
*** bnemec has joined #openstack-meeting-323:18

Generated by 2.15.3 by Marius Gedminas - find it at!