| *** yamamoto has joined #openstack-meeting-3 | 01:47 | |
| *** yamamoto has quit IRC | 02:07 | |
| *** yamamoto has joined #openstack-meeting-3 | 02:10 | |
| *** hongbin has joined #openstack-meeting-3 | 03:49 | |
| *** hongbin has quit IRC | 04:39 | |
| *** links has joined #openstack-meeting-3 | 05:25 | |
| *** e0ne has joined #openstack-meeting-3 | 05:59 | |
| *** kozhukalov has joined #openstack-meeting-3 | 06:08 | |
| *** e0ne has quit IRC | 06:15 | |
| *** e0ne has joined #openstack-meeting-3 | 06:16 | |
| *** e0ne has quit IRC | 06:30 | |
| *** slaweq_ has joined #openstack-meeting-3 | 07:42 | |
| *** slaweq_ is now known as slaweq | 07:45 | |
| *** ralonsoh has joined #openstack-meeting-3 | 08:53 | |
| *** psachin has joined #openstack-meeting-3 | 09:25 | |
| *** tsmith2 has quit IRC | 10:09 | |
| *** tsmith2 has joined #openstack-meeting-3 | 10:10 | |
| *** e0ne has joined #openstack-meeting-3 | 10:37 | |
| *** e0ne has quit IRC | 10:38 | |
| *** e0ne has joined #openstack-meeting-3 | 10:54 | |
| *** e0ne has quit IRC | 11:27 | |
| *** ianychoi_ is now known as ianychoi | 11:32 | |
| *** yamamoto has quit IRC | 11:49 | |
| *** yamamoto has joined #openstack-meeting-3 | 12:09 | |
| *** psachin has quit IRC | 12:25 | |
| *** yamamoto has quit IRC | 12:51 | |
| *** e0ne has joined #openstack-meeting-3 | 13:04 | |
| *** psachin has joined #openstack-meeting-3 | 13:05 | |
| *** e0ne has quit IRC | 13:43 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 13:53 | |
| *** e0ne has joined #openstack-meeting-3 | 14:01 | |
| *** slaweq has quit IRC | 14:05 | |
| *** slaweq has joined #openstack-meeting-3 | 14:09 | |
| *** maciejjozefczyk has joined #openstack-meeting-3 | 14:18 | |
| *** yamamoto has joined #openstack-meeting-3 | 14:22 | |
| *** jamesmcarthur has quit IRC | 14:27 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 14:30 | |
| *** jamesmcarthur has quit IRC | 14:32 | |
| *** links has quit IRC | 14:36 | |
| *** yamamoto has quit IRC | 14:48 | |
| *** yamamoto has joined #openstack-meeting-3 | 14:50 | |
| *** yamamoto has quit IRC | 14:55 | |
| slaweq | #startmeeting neutron_ci | 15:00 |
|---|---|---|
| openstack | Meeting started Wed Feb 26 15:00:25 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| slaweq | hi | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 15:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 15:00 |
| njohnston | o/ | 15:00 |
| bcafarel | o/ | 15:01 |
| slaweq | ping ralonsoh: ci meeting | 15:02 |
| ralonsoh | hi | 15:02 |
| slaweq | ok, lets start | 15:02 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:02 |
| slaweq | Please open now :) | 15:02 |
| slaweq | #topic Actions from previous meetings | 15:03 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 15:03 | |
| slaweq | first one | 15:03 |
| slaweq | ralonsoh to talk with oslo people about our functional tests rootwrap issue | 15:03 |
| ralonsoh | one secx | 15:03 |
| ralonsoh | https://review.opendev.org/#/c/708936/ | 15:03 |
| ralonsoh | patch merged and released 6.0.1 | 15:03 |
| ralonsoh | oslo rootwrap | 15:03 |
| slaweq | ralonsoh++ thx a lot | 15:04 |
| slaweq | and indeed I didn't saw such issues in last few days | 15:04 |
| slaweq | ok, next one | 15:04 |
| slaweq | slaweq to try to reproduce and debug neutron-tempest-dvr ssh issue | 15:04 |
| slaweq | I didn't have time for that yet | 15:05 |
| *** psachin has quit IRC | 15:05 | |
| slaweq | but I have some idea about this dvr job | 15:05 |
| slaweq | it is single node dvr job which IMO don't makes too much sense | 15:05 |
| slaweq | so my proposal is to | 15:05 |
| slaweq | a) remove this job from our ci | 15:05 |
| slaweq | b) change a bit neutron-tempest-dvr-ha-multinode-full to run networking related tests only | 15:05 |
| slaweq | c) make neutron-tempest-dvr-ha-multinode-full voting in few weeks if it will be stable enough | 15:06 |
| slaweq | so finally we will have one less job in our CI | 15:06 |
| bcafarel | hmm did we not plan something like that in Shanghai PTG? (dropping single node dvr) | 15:06 |
| slaweq | what do You think about it? | 15:06 |
| bcafarel | (and +1 on the plan of course) | 15:06 |
| slaweq | bcafarel: nope, we agreed to drop some grenade single node jobs | 15:06 |
| bcafarel | aah ok | 15:07 |
| haleyb | i'm fine with it as long as multinode job is stable | 15:07 |
| bcafarel | anyway yes that sounds good, it will be add more visibility (and stability) to multinode dvr job | 15:07 |
| slaweq | haleyb: multinode job seems to be at least as stable as single node one | 15:08 |
| slaweq | and in fact this single node job is testing almost only API | 15:08 |
| haleyb | it wasn't always that way, but you and miguel have fixed some hard bugs :) | 15:08 |
| slaweq | haleyb: but I'm not talking about neutron-tempest-plugin-dvr-multinode-scenario job which is still pretty not stable | 15:09 |
| slaweq | I'm talking about promoting neutron-tempest-dvr-ha-multinode-full to be voting and replace neutron-tempest-dvr | 15:09 |
| slaweq | the differences between neutron-tempest-plugin-dvr-multinode-scenario and neutron-tempest-dvr-ha-multinode-full are: | 15:10 |
| slaweq | 1. one is DVR and the other is DVR HA | 15:10 |
| *** yamamoto has joined #openstack-meeting-3 | 15:10 | |
| slaweq | 2. one runs neutron-tempest-plugin tests and the other runs tempest API + some basic scenario tests | 15:10 |
| haleyb | ack | 15:11 |
| slaweq | so, can I assume that I have "green light" from the team to do that? :) | 15:12 |
| ralonsoh | +1 | 15:12 |
| njohnston | +1 | 15:12 |
| slaweq | thx, so I will propose patch(es) for that asap :) | 15:12 |
| haleyb | +1 | 15:12 |
| slaweq | #action slaweq to drop neutron-tempest-dvr job and finally replace it with neutron-tempest-dvr-ha-multinode-full | 15:13 |
| slaweq | ok, next one | 15:13 |
| slaweq | ralonsoh to check periodic neutron-ovn-tempest-ovs-master-fedora job's failures | 15:13 |
| ralonsoh | I spent 20 mins on this | 15:14 |
| ralonsoh | and then I had other high priority tasks | 15:14 |
| ralonsoh | sorry again | 15:14 |
| slaweq | ralonsoh: no problem :) | 15:14 |
| ralonsoh | (this is the 2nd time I miss this task( | 15:14 |
| slaweq | do You want to assign it to yourself for next week? | 15:14 |
| slaweq | or should we simply report a bug and maybe someone will check it? | 15:15 |
| ralonsoh | next week | 15:15 |
| slaweq | ralonsoh: ok :) | 15:15 |
| slaweq | thx | 15:15 |
| slaweq | #action ralonsoh to check periodic neutron-ovn-tempest-ovs-master-fedora job's failures | 15:16 |
| slaweq | ok, next one | 15:16 |
| slaweq | slaweq to report tempest bug with routers cleanup | 15:16 |
| slaweq | I checked this error and it wasn't problem with cleaning in tempest in fact | 15:16 |
| slaweq | the root cause of this was that API call to add_router_interface was processed for more than 30 seconds by Neutron | 15:16 |
| slaweq | so urllib got timeout waiting for response and it retried | 15:17 |
| slaweq | but in the retry this subnet was already plugged to the router as first call was already finished | 15:17 |
| slaweq | so it failed on adding interface to the router and because of that it wasn't added to cleanup list and failed to be cleaned too | 15:18 |
| slaweq | so I didn't report tempest bug for that :) | 15:18 |
| slaweq | and that's all actions from last week | 15:18 |
| slaweq | anything else You want to ask/add in this section? | 15:19 |
| ralonsoh | no | 15:19 |
| slaweq | if no, lets move on | 15:19 |
| slaweq | #topic Stadium projects | 15:19 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 15:19 | |
| slaweq | standardize on zuul v3 | 15:19 |
| slaweq | Etherpad: https://etherpad.openstack.org/p/neutron-train-zuulv3-py27drop | 15:19 |
| slaweq | we are almost done there | 15:19 |
| slaweq | I need to fix my patches for neutron-vpnaas | 15:20 |
| *** mugsie has quit IRC | 15:20 | |
| bcafarel | nice | 15:20 |
| slaweq | networking-bgpvpn is almost there - patch waits for some other fix for tempest job | 15:20 |
| slaweq | we merged neutron-dynamic-routing patch recently | 15:20 |
| njohnston | I need to check to see if we have any changes in the field yet for networking-odl or networking-midonet | 15:20 |
| slaweq | njohnston: yes, those 2 projects are probably not touched yet | 15:21 |
| bcafarel | hopefully current recheck will be the good one this time (bagpipe is back up and running thanks to amotoki lajoskatona | 15:21 |
| *** mugsie has joined #openstack-meeting-3 | 15:22 | |
| slaweq | and according to py2 leftovers, we need to merge https://review.opendev.org/#/c/699213/ then release last version with py2 support and then merge https://review.opendev.org/#/c/704257/ | 15:22 |
| slaweq | so please review https://review.opendev.org/#/c/699213/ | 15:23 |
| slaweq | :) | 15:23 |
| slaweq | thx bcafarel and ralonsoh for review of it :) | 15:23 |
| ralonsoh | sure | 15:23 |
| bcafarel | :) njohnston looks like you are next on it then | 15:23 |
| slaweq | or haleyb :) | 15:23 |
| slaweq | as he is here too :) | 15:23 |
| bcafarel | oh true | 15:24 |
| njohnston | https://review.opendev.org/699213 +2+W | 15:24 |
| haleyb | nate beat me to it | 15:24 |
| slaweq | thx :) | 15:24 |
| bcafarel | haleyb: today you are in grey in IRC colors, so quite stealthy | 15:24 |
| haleyb | oh | 15:24 |
| slaweq | no, in my weechat haleyb is red :) | 15:25 |
| * haleyb can't hide | 15:25 | |
| slaweq | anything else You want to ask regarding stadium projects? | 15:25 |
| * njohnston uses limechat today so you are all golden | 15:25 | |
| bcafarel | :) | 15:26 |
| bcafarel | final rocky releases will be out for neutron et all (so including stadium projects) | 15:26 |
| slaweq | bcafarel: yes, it should be done this week, today or tomorrow | 15:27 |
| njohnston | \o/ | 15:27 |
| slaweq | ok, lets move on | 15:28 |
| slaweq | next topic | 15:28 |
| slaweq | #topic Grafana | 15:28 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 15:28 | |
| slaweq | #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:28 |
| slaweq | we don't have almost anything in gate recently | 15:28 |
| slaweq | and it's mostly because of ovn tempest jobs failing almost all the time | 15:29 |
| ralonsoh | we are on it now | 15:29 |
| ralonsoh | https://review.opendev.org/#/c/710019/ | 15:29 |
| slaweq | this patch should fix it probably https://review.opendev.org/#/c/709691/ | 15:29 |
| ralonsoh | this one for tempest-plugin | 15:30 |
| ralonsoh | and then we need to recheck the FTs | 15:30 |
| ralonsoh | yesterday we detected an error in OVN core | 15:30 |
| ralonsoh | (the bug is opened) | 15:30 |
| slaweq | ralonsoh: ok, thx for that too | 15:30 |
| ralonsoh | and we have this https://review.opendev.org/#/c/709762/ | 15:30 |
| slaweq | but for now most important is this tempest-plugin issue | 15:30 |
| ralonsoh | yes, I now | 15:30 |
| maciejjozefczyk | yes, the bugs is about missing cherry-pick of one patch to ovn 2.12 | 15:31 |
| maciejjozefczyk | its already fixed but not cherry-picked | 15:31 |
| maciejjozefczyk | I asked ovn folks to do it | 15:31 |
| maciejjozefczyk | but anyways blacklisting it for now makes perfect sense | 15:32 |
| slaweq | thx maciejjozefczyk and ralonsoh for taking care of it | 15:32 |
| slaweq | njohnston: haleyb please check https://review.opendev.org/#/c/709691/ if You have a minute | 15:32 |
| slaweq | that should allow us to make zuul happy at least from time to time ;) | 15:32 |
| haleyb | slaweq: i did a second ago, +2+W | 15:32 |
| njohnston | haleyb beat me to it | 15:32 |
| slaweq | LOL | 15:32 |
| maciejjozefczyk | :) | 15:32 |
| slaweq | you have many races today ;P | 15:33 |
| haleyb | and meetings | 15:33 |
| njohnston | We need to have one more - tiebreaker! | 15:33 |
| slaweq | njohnston: :D | 15:33 |
| slaweq | if I will have one more I will send it to You as a tiebreaker :) | 15:34 |
| slaweq | from other things in grafana, still functional tests are failing a lot | 15:34 |
| slaweq | and I have one new "star" there | 15:34 |
| slaweq | anythig else regarding grafana You want to add? | 15:35 |
| ralonsoh | no | 15:36 |
| slaweq | if no, lets talk about functional tests now | 15:36 |
| slaweq | #topic Fullstack/Functional | 15:36 |
| *** openstack changes topic to "Fullstack/Functional (Meeting topic: neutron_ci)" | 15:36 | |
| slaweq | I saw that during last week test neutron.tests.functional.services.trunk.drivers.openvswitch.agent.test_trunk_manager.TrunkManagerTestCase.test_connectivity failed many times | 15:36 |
| slaweq | I have now opened LP to report this bug there | 15:36 |
| ralonsoh | but this is because the problem in rootwrap | 15:37 |
| slaweq | ralonsoh: yes, now I see | 15:37 |
| ralonsoh | ok ok | 15:37 |
| slaweq | I just saw some "FileNotFound" error and I though that it's something different :) | 15:37 |
| slaweq | and this looks like issue with Your rootwrap patch already | 15:38 |
| slaweq | ralonsoh: do You think we should report new bug for that? | 15:39 |
| ralonsoh | but I have this one | 15:40 |
| ralonsoh | https://bugs.launchpad.net/oslo.rootwrap/+bug/1864090 | 15:40 |
| openstack | Launchpad bug 1864090 in oslo.rootwrap ""PathFilter" can raise exception when executing "os.path.realpath()"" [Undecided,Fix released] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 15:40 |
| slaweq | it's not with Your fix | 15:43 |
| ralonsoh | ? | 15:43 |
| slaweq | ralonsoh: so should Your fix https://review.opendev.org/#/c/708936/ solve this problem with functional tests? | 15:43 |
| ralonsoh | yes, I think so | 15:44 |
| slaweq | ok, so lets keep fingers crossed for that :) | 15:44 |
| slaweq | and that was my main concern for today in fact | 15:45 |
| slaweq | other than that I saw some single failures so lets just keep an eye on it for now | 15:46 |
| *** psachin has joined #openstack-meeting-3 | 15:46 | |
| slaweq | regarding scenario jobs we already discussed about ovn jobs and neutron-tempest-dvr so we are also good there | 15:46 |
| slaweq | so that's basically all from my for today | 15:46 |
| slaweq | do You have anything else You want to discuss today? | 15:46 |
| slaweq | #topic On demand agenda | 15:47 |
| *** openstack changes topic to "On demand agenda (Meeting topic: neutron_ci)" | 15:47 | |
| maciejjozefczyk | I thinnk in near future we would be able to enable ovn multinode job | 15:48 |
| maciejjozefczyk | with fixes: | 15:48 |
| maciejjozefczyk | #link https://review.opendev.org/#/c/709111/ | 15:48 |
| maciejjozefczyk | #link https://review.opendev.org/#/c/709110/ | 15:48 |
| maciejjozefczyk | it will be green (was already, but failed on test_two_sec_groups - quota issue) | 15:49 |
| maciejjozefczyk | thats all from me | 15:49 |
| slaweq | ok, thx maciejjozefczyk :) | 15:50 |
| slaweq | if there is nothing else for today, I think I can You back few minutes | 15:50 |
| slaweq | thx for attending | 15:50 |
| njohnston | o/ | 15:50 |
| slaweq | and see You next week | 15:50 |
| ralonsoh | bye! | 15:50 |
| slaweq | o/ | 15:50 |
| slaweq | #endmeeting | 15:51 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:51 | |
| openstack | Meeting ended Wed Feb 26 15:51:00 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:51 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-02-26-15.00.html | 15:51 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-02-26-15.00.txt | 15:51 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-02-26-15.00.log.html | 15:51 |
| maciejjozefczyk | bb! | 15:51 |
| *** psachin has quit IRC | 15:57 | |
| *** e0ne has quit IRC | 15:59 | |
| *** e0ne has joined #openstack-meeting-3 | 15:59 | |
| *** e0ne has quit IRC | 16:10 | |
| *** maciejjozefczyk has quit IRC | 16:21 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 16:36 | |
| *** psachin has joined #openstack-meeting-3 | 16:36 | |
| *** yamamoto has quit IRC | 16:39 | |
| *** rubasov has quit IRC | 16:47 | |
| *** yamamoto has joined #openstack-meeting-3 | 17:11 | |
| *** psachin has quit IRC | 17:12 | |
| *** yamamoto has quit IRC | 17:17 | |
| *** e0ne has joined #openstack-meeting-3 | 17:48 | |
| *** jamesmcarthur has quit IRC | 17:58 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 18:00 | |
| *** jamesmcarthur has quit IRC | 18:05 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 18:21 | |
| *** yamamoto has joined #openstack-meeting-3 | 18:22 | |
| *** igordc has joined #openstack-meeting-3 | 18:37 | |
| *** maciejjozefczyk has joined #openstack-meeting-3 | 18:42 | |
| *** igordc has quit IRC | 18:43 | |
| *** maciejjozefczyk has quit IRC | 18:57 | |
| *** jamesmcarthur has quit IRC | 19:31 | |
| *** yamamoto has quit IRC | 19:31 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 19:40 | |
| *** jamesmcarthur has quit IRC | 19:58 | |
| *** yamamoto has joined #openstack-meeting-3 | 20:02 | |
| *** e0ne has quit IRC | 20:06 | |
| *** yamamoto has quit IRC | 20:08 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 20:21 | |
| *** kozhukalov has quit IRC | 20:33 | |
| *** kozhukalov has joined #openstack-meeting-3 | 20:36 | |
| *** ralonsoh has quit IRC | 21:22 | |
| *** bnemec has quit IRC | 21:23 | |
| *** kozhukalov has quit IRC | 21:29 | |
| *** slaweq has quit IRC | 21:32 | |
| *** jamesmcarthur has quit IRC | 21:36 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 21:36 | |
| *** jamesmcarthur has quit IRC | 21:44 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 21:45 | |
| *** jamesmcarthur has quit IRC | 21:59 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 22:00 | |
| *** jamesmcarthur has quit IRC | 22:01 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 22:05 | |
| *** jamesmcarthur has quit IRC | 22:29 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 22:30 | |
| *** jamesmcarthur has quit IRC | 22:35 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 22:35 | |
| *** jamesmcarthur has quit IRC | 22:49 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 22:54 | |
| *** jamesmcarthur has quit IRC | 23:15 | |
| *** bnemec has joined #openstack-meeting-3 | 23:18 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!