| *** ricolin has joined #openstack-meeting-3 | 01:13 | |
| *** ricolin_ has joined #openstack-meeting-3 | 01:18 | |
| *** ricolin has quit IRC | 01:21 | |
| *** apetrich has quit IRC | 02:09 | |
| *** artom has quit IRC | 02:32 | |
| *** psachin has joined #openstack-meeting-3 | 03:33 | |
| *** tobberydberg has joined #openstack-meeting-3 | 05:45 | |
| *** slaweq_ has joined #openstack-meeting-3 | 06:37 | |
| *** maciejjozefczyk has joined #openstack-meeting-3 | 07:06 | |
| *** lpetrut has joined #openstack-meeting-3 | 07:07 | |
| *** ralonsoh has joined #openstack-meeting-3 | 07:14 | |
| *** links has joined #openstack-meeting-3 | 07:56 | |
| *** slaweq_ is now known as slaweq | 08:10 | |
| *** e0ne has joined #openstack-meeting-3 | 08:12 | |
| *** ricolin_ is now known as ricolin | 08:30 | |
| *** apetrich has joined #openstack-meeting-3 | 09:16 | |
| *** artom has joined #openstack-meeting-3 | 09:51 | |
| *** e0ne has quit IRC | 10:52 | |
| *** e0ne has joined #openstack-meeting-3 | 10:53 | |
| *** raildo has joined #openstack-meeting-3 | 12:28 | |
| *** belmoreira has joined #openstack-meeting-3 | 12:39 | |
| *** belmoreira has quit IRC | 12:39 | |
| *** artom has quit IRC | 12:39 | |
| *** artom has joined #openstack-meeting-3 | 13:25 | |
| *** liuyulong has joined #openstack-meeting-3 | 13:33 | |
| slaweq | #startmeeting neutron_ci | 15:00 |
|---|---|---|
| openstack | Meeting started Wed Apr 15 15:00:14 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| slaweq | hi | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 15:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 15:00 |
| njohnston | o/ | 15:00 |
| slaweq | hi njohnston | 15:01 |
| ralonsoh | hi | 15:01 |
| bcafarel | o/ | 15:01 |
| njohnston | hello, how are you? Did you have a good dyngus day? | 15:01 |
| slaweq | hi | 15:01 |
| slaweq | njohnston: yes, thx | 15:01 |
| slaweq | I had | 15:01 |
| slaweq | do You have it also in US? | 15:01 |
| *** lajoskatona has joined #openstack-meeting-3 | 15:01 | |
| njohnston | my father's family is from Buffalo, New York which has one of the most active dyngus day celebrations in the US | 15:02 |
| slaweq | nice :) | 15:02 |
| lajoskatona | Hi | 15:02 |
| slaweq | I was splashing water on my kids from neighborhood from the window on first floor | 15:03 |
| slaweq | it was fun :) | 15:03 |
| slaweq | hi lajoskatona :) | 15:03 |
| slaweq | ok, lets start meeting | 15:03 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:03 |
| slaweq | #topic Actions from previous meetings | 15:03 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 15:03 | |
| slaweq | first action | 15:04 |
| slaweq | slaweq to continue investigation of fullstack SG test broken pipe failures | 15:04 |
| slaweq | I found out that in some cases it may be race condition between starting client and reading from it | 15:04 |
| slaweq | so solution IMHO is to handle BrokenPipeException in same way as e.g RuntimeError | 15:05 |
| njohnston | makes sense | 15:05 |
| slaweq | patch is here https://review.opendev.org/#/c/718781 | 15:05 |
| *** lpetrut has quit IRC | 15:05 | |
| slaweq | please review if You will have some time | 15:05 |
| slaweq | next one | 15:06 |
| slaweq | slaweq to check server termination on multicast test | 15:06 |
| slaweq | I finally dig into it | 15:06 |
| slaweq | and I found out that in tempest "all-plugin" env there is timeout 1200 seconds for each test set | 15:07 |
| slaweq | and in case of our tests which require advanced image it may be not enough simply | 15:07 |
| slaweq | so we are hitting this timeout e.g. during cleanup phase | 15:07 |
| bcafarel | where do we ask again for more powerful hardware? ;) | 15:08 |
| slaweq | I proposed patch https://review.opendev.org/#/c/719927/ to increase this timeout for some tests only | 15:08 |
| slaweq | but as I talked with gmann today, it seems that better way would be to do it like e.g. ironic did: https://opendev.org/openstack/ironic/src/branch/master/zuul.d/ironic-jobs.yaml#L414 | 15:08 |
| slaweq | and set higher timeout globally "per job" | 15:08 |
| slaweq | so I will update my patch and do it that way | 15:09 |
| slaweq | bcafarel: I think that it's more matter of nested virtualisation | 15:09 |
| slaweq | it would be much better if that would work fine finally | 15:09 |
| bcafarel | oh yes | 15:09 |
| slaweq | anyway, that's all about that one from me | 15:10 |
| slaweq | questions/comments? | 15:10 |
| lajoskatona | bcafarel: from fortnebula there is option to fetch bigger VMs | 15:10 |
| lajoskatona | but that is just temporary for testing things | 15:10 |
| bcafarel | slaweq: so you will update to use longer overall timeout? (which may help for later "long" tests) | 15:10 |
| lajoskatona | at least it was when I last needed mor MEM | 15:11 |
| bcafarel | nice I did not know that | 15:11 |
| bcafarel | though hopefully just larger timeouts will be enough | 15:11 |
| slaweq | bcafarel: yes, I will update to set longer timeout per test in job definition | 15:11 |
| slaweq | it's not "job's timeout" but "test's timeout" | 15:12 |
| bcafarel | ok sounds good | 15:12 |
| slaweq | ok, next one | 15:12 |
| slaweq | slaweq to ping yamamoto about midonet gate problems | 15:12 |
| *** ianychoi has quit IRC | 15:13 | |
| slaweq | I sent him an email last week | 15:13 |
| slaweq | but he didn't reply | 15:13 |
| slaweq | I will try to catch him during drivers meeting because for now networking-midonet's gate is still broken | 15:13 |
| slaweq | #action slaweq to ping yamamoto about midonet gate problems | 15:14 |
| lajoskatona | I looked for him as well to ask about taas, as the reviews are stopped there | 15:14 |
| slaweq | he's not very active in community today | 15:14 |
| slaweq | and I think he is the only "active" maintainer of networking-midonet | 15:15 |
| slaweq | ok, last one from previous week | 15:16 |
| slaweq | bcafarel to check and update stable branches grafana dashboards | 15:16 |
| bcafarel | done in https://review.opendev.org/#/c/718676/ in the end that was almost full rewrite | 15:17 |
| slaweq | thx bcafarel | 15:17 |
| bcafarel | stable dashboards are quite close to neutron master ones now: | 15:17 |
| bcafarel | http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=1 | 15:17 |
| bcafarel | http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=1 | 15:18 |
| njohnston | thanks bcafarel! | 15:18 |
| slaweq | I added links to meeting agenda | 15:19 |
| slaweq | ok, I think that this is all regarding actions from last week | 15:19 |
| slaweq | #topic Stadium projects | 15:19 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 15:19 | |
| slaweq | any updates about zuulv3 ? | 15:19 |
| slaweq | ok, I guess this silence means "no updates" :) | 15:21 |
| njohnston | nothing from me | 15:21 |
| lajoskatona | not from me at least | 15:21 |
| njohnston | networking-odl still has https://review.opendev.org/#/c/672925/ pending | 15:21 |
| slaweq | ok | 15:21 |
| njohnston | and networking-midonet still has nothing IIUC | 15:22 |
| slaweq | I also don't have any updates about IPv6-only testing | 15:22 |
| slaweq | regarding stadium issues, I know only about this one with networking-midonet which we already talked about | 15:22 |
| slaweq | any other issues/questions regarding stadium and ci? | 15:22 |
| njohnston | nope | 15:24 |
| bcafarel | not from me either | 15:24 |
| ralonsoh | no | 15:24 |
| slaweq | ok, lets move on | 15:24 |
| slaweq | #topic Stable branches | 15:24 |
| *** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)" | 15:24 | |
| slaweq | Train dashboard: http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=1 | 15:25 |
| slaweq | Stein dashboard: http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=1 | 15:25 |
| slaweq | I don't see too much data on those dashboards so far | 15:25 |
| njohnston | there rarely is | 15:26 |
| ralonsoh | I need to review neutron-ovn-tempest-ovs-release, it's failing in all jobs | 15:27 |
| slaweq | ralonsoh: but are You talking about stable branches or master? | 15:27 |
| ralonsoh | sorry, master | 15:28 |
| ralonsoh | my bad | 15:28 |
| bcafarel | fullstack hopefully will be better with your fixes backports slaweq | 15:28 |
| slaweq | bcafarel: we will see :) | 15:28 |
| slaweq | there is also this rally issue https://bugs.launchpad.net/neutron/+bug/1871596 | 15:28 |
| openstack | Launchpad bug 1871596 in neutron "Rally job on stable branches is failing" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq) | 15:28 |
| bcafarel | grenade yes is one of the common recheck causes in stable (though luckily not that bad) | 15:28 |
| slaweq | but we should be good with it now, right? | 15:29 |
| bcafarel | yep andreykurilin has merged the change to fix rocky | 15:29 |
| slaweq | great | 15:29 |
| bcafarel | (and networking-ovn stein/rocky also) | 15:29 |
| slaweq | thx lajoskatona and bcafarel for taking care of this | 15:29 |
| bcafarel | just waiting for some rechecks to complete to mark it as "back in working order"! | 15:29 |
| slaweq | bcafarel: so we can close this LP, right? | 15:29 |
| lajoskatona | good to hear :-) | 15:31 |
| slaweq | today I got couple of +1 from zuul for various stable branches, so IMO it's fine now | 15:31 |
| bcafarel | ok checking last results it does indeed look good | 15:32 |
| bcafarel | one LP down | 15:32 |
| slaweq | \o/ | 15:32 |
| lajoskatona | ok I abandon then my rally devstack plugin capping patch | 15:32 |
| slaweq | thx lajoskatona | 15:32 |
| slaweq | anything else related to stable brances for today? | 15:33 |
| slaweq | ok, lets move on | 15:34 |
| slaweq | #topic Grafana | 15:34 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 15:34 | |
| slaweq | I still need to update my patch https://review.opendev.org/#/c/718392/ - thx bcafarel for review | 15:34 |
| bcafarel | getting at correct position in the file will probably take longer than fixing my nit :) | 15:35 |
| slaweq | ralonsoh: speaking about ovn jobs, it seems that all of them, except "slow" are going up with failure rates today | 15:35 |
| slaweq | so probably there is some problem with those jobs | 15:35 |
| slaweq | https://e93bf74b2537cfa96a59-e4f20cff14b59b3a1c5b0d28b2b173f9.ssl.cf5.rackcdn.com/719765/3/check/neutron-ovn-tempest-ovs-release/0982c80/testr_results.html | 15:36 |
| slaweq | seems that this subnetpool test is a culprit | 15:37 |
| ralonsoh | I'll take a look today | 15:37 |
| slaweq | ralonsoh: thx | 15:37 |
| slaweq | #action ralonsoh to check ovn jobs failure | 15:38 |
| slaweq | except that I think that things looks pretty good this week | 15:38 |
| slaweq | even functional jobs are finally working better thanks to maciejjozefczyk and ralonsoh's work | 15:39 |
| slaweq | anything else regarding grafana? do You see anything which worries You? | 15:40 |
| maciejjozefczyk | ralonsoh, I can help if you'll need any help :) | 15:41 |
| slaweq | thx maciejjozefczyk for volunteering :) | 15:42 |
| ralonsoh | thanks a lot! | 15:43 |
| slaweq | ok, regarding other issues, I don't have anything really new for today | 15:43 |
| slaweq | so if You don't have anything else to talk about today, I will give You some time back :) | 15:44 |
| bcafarel | I won't complain that we have less to talk about in this meeting :) | 15:44 |
| maciejjozefczyk | :) | 15:44 |
| slaweq | bcafarel: :) | 15:44 |
| njohnston | +100 | 15:45 |
| slaweq | ok, so thx for attending and see You online :) | 15:45 |
| slaweq | o/ | 15:45 |
| slaweq | #endmeeting | 15:45 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:45 | |
| ralonsoh | bye | 15:45 |
| openstack | Meeting ended Wed Apr 15 15:45:45 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:45 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-04-15-15.00.html | 15:45 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-04-15-15.00.txt | 15:45 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-04-15-15.00.log.html | 15:45 |
| bcafarel | o/ | 15:45 |
| lajoskatona | o/ | 15:46 |
| *** lajoskatona has left #openstack-meeting-3 | 15:46 | |
| maciejjozefczyk | bb! | 15:47 |
| *** ianychoi has joined #openstack-meeting-3 | 15:47 | |
| *** maciejjozefczyk has quit IRC | 16:09 | |
| *** psachin has quit IRC | 16:17 | |
| *** maciejjozefczyk has joined #openstack-meeting-3 | 16:35 | |
| *** maciejjozefczyk has quit IRC | 16:41 | |
| *** ricolin has quit IRC | 17:10 | |
| *** ralonsoh has quit IRC | 17:55 | |
| *** links has quit IRC | 18:01 | |
| *** bbowen has quit IRC | 18:03 | |
| *** bbowen has joined #openstack-meeting-3 | 18:12 | |
| *** e0ne has quit IRC | 18:31 | |
| *** hongbin has joined #openstack-meeting-3 | 19:01 | |
| *** slaweq has quit IRC | 20:08 | |
| *** slaweq has joined #openstack-meeting-3 | 20:08 | |
| *** e0ne has joined #openstack-meeting-3 | 20:24 | |
| *** slaweq has quit IRC | 20:28 | |
| *** e0ne has quit IRC | 21:02 | |
| *** e0ne has joined #openstack-meeting-3 | 21:03 | |
| *** e0ne has quit IRC | 21:42 | |
| *** raildo has quit IRC | 21:53 | |
| *** negronjl has quit IRC | 22:27 | |
| *** negronjl has joined #openstack-meeting-3 | 22:31 | |
| *** hongbin has quit IRC | 22:53 | |
| *** hongbin has joined #openstack-meeting-3 | 23:01 | |
| *** negronjl has quit IRC | 23:10 | |
| *** negronjl has joined #openstack-meeting-3 | 23:15 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!