Wednesday, 2020-05-20

*** jamesmcarthur has quit IRC00:01
*** jamesmcarthur has joined #openstack-meeting-300:13
*** jamesmcarthur has quit IRC00:16
*** jamesmcarthur has joined #openstack-meeting-300:29
*** jamesmcarthur has quit IRC00:30
*** Adri2000 has quit IRC00:37
*** Adri2000 has joined #openstack-meeting-300:45
*** jamesmcarthur has joined #openstack-meeting-301:59
*** jamesmcarthur has quit IRC02:10
*** apetrich has quit IRC02:10
*** jamesmcarthur has joined #openstack-meeting-302:51
*** psachin has joined #openstack-meeting-303:36
*** jamesmcarthur has quit IRC03:54
*** jamesmcarthur has joined #openstack-meeting-303:55
*** jamesmcarthur has quit IRC04:12
*** jamesmcarthur has joined #openstack-meeting-304:12
*** jamesmcarthur has quit IRC04:16
*** jamesmcarthur has joined #openstack-meeting-304:16
*** lifeless has quit IRC04:22
*** diablo_rojo has quit IRC04:30
*** lifeless has joined #openstack-meeting-304:52
*** psachin has quit IRC05:05
*** links has joined #openstack-meeting-305:31
*** jamesmcarthur has quit IRC05:32
*** jamesmcarthur has joined #openstack-meeting-305:33
*** jamesmcarthur has quit IRC05:38
*** lifeless has quit IRC05:41
*** links has quit IRC05:43
*** links has joined #openstack-meeting-305:45
*** lifeless has joined #openstack-meeting-305:50
*** jamesmcarthur has joined #openstack-meeting-306:09
*** jamesmcarthur has quit IRC06:19
*** slaweq has joined #openstack-meeting-306:45
*** belmoreira has joined #openstack-meeting-306:50
*** ralonsoh has joined #openstack-meeting-306:50
*** jamesmcarthur has joined #openstack-meeting-306:54
*** apetrich has joined #openstack-meeting-306:55
*** jamesmcarthur has quit IRC07:05
*** maciejjozefczyk has joined #openstack-meeting-307:07
*** masahito has joined #openstack-meeting-307:58
ttx#startmeeting large_scale_sig08:00
openstackMeeting started Wed May 20 08:00:19 2020 UTC and is due to finish in 60 minutes.  The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot.08:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:00
*** openstack changes topic to " (Meeting topic: large_scale_sig)"08:00
openstackThe meeting name has been set to 'large_scale_sig'08:00
ttx#topic Rollcall08:00
*** openstack changes topic to "Rollcall (Meeting topic: large_scale_sig)"08:00
ttxAnyone here for the Large Scale SIG meeting ?08:00
amorinhello!08:00
*** mdelavergne has joined #openstack-meeting-308:00
masahitoo/08:00
ttxamorin: hey! How are things? Back at the office or still remote?08:01
mdelavergneHi!08:01
ttxmasahito: hi!08:01
amorinstill remote here08:01
amorinwill be remote whole summer I thinik08:01
ttxbelmoreira: ping08:01
ttxmdelavergne: what about you?08:01
ttxmasahito: how are things in Japan today ?08:01
mdelavergnestill remote :)08:02
masahitoGetting better now :-)08:02
ttxOK! Our agenda for today is at:08:02
ttx#link https://etherpad.openstack.org/p/large-scale-sig-meeting08:02
ttx#topic Progress on "Documenting large scale operations" goal08:02
*** openstack changes topic to "Progress on "Documenting large scale operations" goal (Meeting topic: large_scale_sig)"08:02
ttx#link https://etherpad.openstack.org/p/large-scale-sig-documentation08:02
ttxamorin: any recent progress ?08:02
amorinyes, I finally took some time to create this:08:03
amorinhttps://review.opendev.org/#/c/729190/08:03
amorinvery small doc change to add a link to wiki page08:03
*** felixhuettner has joined #openstack-meeting-308:03
amorinplease review and let me know if this is a good strategy08:03
ttxWill do!08:04
amorinand that's all on my side :P08:04
ttxHopefully that will encourage others to share more config things in the future08:05
amorinyes08:05
ttxAnything else on this goal?08:05
amorinI can do more than this one08:05
amorinbut I'd like to find a common aggrement before08:05
ttxYeah, but let's wait to see if they like it08:05
amorinyes08:05
ttxIf the review is ignored let me know and I'll try to pull some strings08:06
ttx#topic Progress on "Scaling within one cluster" goal08:06
amorinok, I can wait a little bit08:06
*** openstack changes topic to "Progress on "Scaling within one cluster" goal (Meeting topic: large_scale_sig)"08:06
ttx#link https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling08:06
ttxI had a TODO around creating an empty oslo-metrics repository08:06
ttxIt's now up on opendev and Gerrit at:08:06
ttx#link https://opendev.org/openstack/oslo.metrics08:06
ttxReady for masahito to push initial code!08:07
ttxmasahito: how is that effort going ?08:07
masahitoOh thanks! I'll do that.08:07
amorinnice08:07
masahitoI didn't notice the new repo08:07
ttxIt's not mirrored to GitHub yet, will set that up once there is code in it08:08
masahitogot it.08:08
ttx#action masahito to prepare oslo.metrics initial POC code release08:08
ttxIt will be great to have something in, so that we can discuss it during the Opendev on Large Scale ops08:08
ttxAnything else on this goal, before we move on to discuss opendev?08:09
ttxok, moving on then08:10
ttx#topic Opendev event on Large scale operations, June 29 - July 108:10
*** openstack changes topic to "Opendev event on Large scale operations, June 29 - July 1 (Meeting topic: large_scale_sig)"08:10
ttx#link https://etherpad.openstack.org/p/LargeScaleOps_OpenDev08:10
ttxbelmoreira, masahito: I think you finalized the agenda ?08:10
masahitoyup. there is. https://www.openstack.org/events/opendev-2020/opendev-schedule08:11
ttxah thanks, that's what I was looking for08:11
ttxThe 'User stories' session will be critical for us to get new data points, and hopefully new members08:12
ttxSo it would be great to have good SIG presence during that segment08:12
ttxAlso the "Software Stack" session on the second day08:13
amorin+108:13
ttxBaremetal cluster scaling on day 3 might open up a new goal for us, if people are excited by the idea08:13
ttxBut the first two are well aligned with our current goals.08:14
ttxSo.. mark your calendars! Also register, it's free iirc08:14
amorinyes it's free08:14
amorinI am registering right now08:15
ttxOK, any question on that?08:15
ttx#topic PTG sessions of interest (June 1-5)08:16
*** openstack changes topic to "PTG sessions of interest (June 1-5) (Meeting topic: large_scale_sig)"08:16
ttxIn two weeks we have the PTG event, which is basically team meetings to plan the upcoming work08:16
ttxSince we have Opendev shortly after, last meeting we decided not to have specific time during the event for Large Scale SIG08:16
ttxThat frees up for potentially attending other teams meetings and doing some advocacy08:17
ttxLike... The Oslo team is meeting on the Monday, between 15 and 17 UTC.08:17
ttxmasahito: That's very late for you, but I can stand in for you and report your progress.08:17
ttxWe'll just need to sync on what I should say beforehand08:17
ttxhaving the code up will be of great help08:18
*** felixhuettner has quit IRC08:18
ttxThe Scientific SIG is also meeting on the Monday, between 13 and 17 UTC.08:18
masahitoyeah, that't too late to me ....08:18
masahitothank you.08:18
ttxThey usually have use cases that require large scale, so it could be good for one of us to join that. Maybe belmoreira08:18
ttx#link https://www.openstack.org/ptg#tab_schedule08:19
ttxHere is the schedule ^08:19
ttx(click on the PDF download link)08:19
ttxOps meetup is Wednesday, between 13 and 15 UTC.08:19
ttxmasahito: most teams ended up choosing a time that is not very APAC-friendly08:20
ttxUnfortunately there was no miracle solution for that08:20
ttxAutomation SIG is on Thursday, between 13 and 16 UTC08:20
ttxThat is all those that stood out for me as potentially interesting for SIG attendees08:20
ttxAnyone interested in attending any of those?08:21
ttx(I'll do Oslo)08:21
amorinI can do ops08:22
ttxgood, I have a conflict on that one and won't be able to join08:23
ttxI suspect belmoreira or oneswig will be on the scientific SIG one08:23
mdelavergneI'll probably go to automation too if I can08:23
ttxAutomation I might be able to attend some, but it's not that critical for us08:23
ttxmdelavergne: great!08:24
ttxOK, any other question on that?08:24
ttx(you should also register for this one -- it's possible we'll communicate meeting passwords by email to avoid zoom-bombing08:25
ttx#topic Next meeting08:25
*** openstack changes topic to "Next meeting (Meeting topic: large_scale_sig)"08:25
ttxOur next meeting falls in the middle of the PTG, so I suggest we move to the week after08:25
ttxJune 10. Does that work for everyone ?08:25
amorinok for me08:26
masahitoworks for me08:26
ttx#info next meeting: Jun 10, 8:00UTC08:26
ttxAlright... I think we are done for this meeting08:27
ttxThanks everyone!08:27
amorinthanks08:27
mdelavergnethanks!08:27
masahitoThanks08:27
ttx#endmeeting08:27
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:27
openstackMeeting ended Wed May 20 08:27:31 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:27
openstackMinutes:        http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-05-20-08.00.html08:27
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-05-20-08.00.txt08:27
openstackLog:            http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-05-20-08.00.log.html08:27
*** mdelavergne has quit IRC08:30
*** jamesmcarthur has joined #openstack-meeting-309:02
*** jamesmcarthur has quit IRC09:06
*** links has quit IRC09:22
*** links has joined #openstack-meeting-309:24
*** jraju__ has joined #openstack-meeting-309:29
*** links has quit IRC09:30
*** masahito has quit IRC09:32
*** jraju__ has quit IRC10:55
*** raildo has joined #openstack-meeting-311:02
*** jamesmcarthur has joined #openstack-meeting-311:03
*** jraju__ has joined #openstack-meeting-311:18
*** jamesmcarthur has quit IRC11:22
*** Luzi has joined #openstack-meeting-311:44
*** raildo has quit IRC12:03
*** raildo has joined #openstack-meeting-312:06
*** lpetrut has joined #openstack-meeting-312:32
*** njohnston is now known as njohnston|pto13:07
*** jamesmcarthur has joined #openstack-meeting-313:19
*** jamesmcarthur has quit IRC13:26
*** jamesmcarthur has joined #openstack-meeting-313:35
*** liuyulong has joined #openstack-meeting-313:52
*** Luzi has quit IRC13:58
*** lpetrut has quit IRC14:46
*** belmoreira has quit IRC14:54
slaweq#startmeeting neutron_ci15:01
openstackMeeting started Wed May 20 15:01:15 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
slaweqhi15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
*** openstack changes topic to " (Meeting topic: neutron_ci)"15:01
openstackThe meeting name has been set to 'neutron_ci'15:01
slaweqhi15:01
slaweq(sorry, too many meetings at once)15:01
ralonsohhi15:02
bcafarelo/15:02
maciejjozefczyk\o15:02
slaweqok, lets do that fast :)15:03
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:03
slaweqPlease open now :)15:03
slaweq#topic Actions from previous meetings15:04
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"15:04
slaweqralonsoh to continue checking ovn jobs timeouts15:04
ralonsohI'm on it15:04
ralonsohfocusing in ovsdbapp and python-ovn15:05
slaweqok, thx ralonsoh15:05
ralonsohbut no conclusions yet15:05
slaweqso I will keep it for next week, just to track that15:05
ralonsohsure15:05
slaweq#action ralonsoh to continue checking ovn jobs timeouts15:05
slaweqthx a lot15:05
slaweqnext one15:05
slaweqbcafarel to update stable branches grafana dashboards15:05
bcafarel#link https://review.opendev.org/#/c/729291/15:07
bcafarelas we had updated these recently, this time it was easier15:07
slaweqthx bcafarel15:08
slaweqand the last one from previous week15:08
slaweqslaweq to switch functional uwsgi job to be voting15:08
slaweqPatch proposed https://review.opendev.org/72958815:08
slaweqI will also update grafana when that will be merged15:08
slaweqand that's all from last week15:09
bcafarelironic that this specific job failed on this patch :)15:09
slaweqouch15:10
slaweqlets check it couple of times before we will merge it15:10
bcafarelhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9b8/729588/1/check/neutron-functional-with-uwsgi/9b8e96a/testr_results.html looks unrelated15:11
*** jamesmcarthur has quit IRC15:11
slaweqyes, doesn't look like related to uwsgi15:12
*** jamesmcarthur has joined #openstack-meeting-315:12
slaweqok, lets move on15:13
slaweq#topic Stadium projects15:13
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"15:13
slaweqstandardize on zuul v315:13
slaweqI started looking today at neutron-ovn-grenade job15:13
slaweqand I saw that it's not run in neutron gate at all15:14
slaweqso I proposed to add it as non-voting job for now15:14
slaweqhttps://review.opendev.org/#/c/729591/15:14
slaweqand then I will also work on migration to zuulv315:14
slaweqI just want to have some legacy job's results to compare during migration15:14
slaweqI don't think there is any other update about that15:16
slaweqbut if I am wrong, please tell now :)15:16
bcafarellooks like the silent crowd agrees15:17
slaweqyeah15:17
slaweqso lets move on15:17
slaweq#topic Stable branches15:17
*** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)"15:17
slaweqTrain dashboard: http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=115:18
slaweqStein dashboard: http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=115:18
bcafarel^ that will change soon :)15:18
slaweq:)15:19
bcafarelI did not see many failures last week I think it was good on stable front15:19
slaweqyes, that's also my impression15:20
slaweqmore recheckes are on master branch15:20
slaweqanything else regarding stable branches for today?15:21
bcafarelnot from me15:22
slaweqok, so lets move on15:22
slaweq#topic Grafana15:22
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"15:22
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:23
slaweqovn related jobs are a bit high in check queue - around 40%15:24
slaweqhttp://grafana.openstack.org/d/Hj5IHcSmz/neutron-failure-rate?viewPanel=16&orgId=115:24
slaweqnow they are going down a bit but still are top on this graph15:24
slaweqbut I don't have now any specific failure to check15:27
slaweqmaybe it's just because there was recently pretty many WIP or DNM patches related to ovn driver15:27
slaweqlets check that in next days and we will see what to do with it15:27
bcafarel+115:28
ralonsohprobably15:28
slaweqother than that all looks ok'ish IMO15:28
slaweqok15:29
slaweqanything else regading grafana?15:29
ralonsohno15:30
slaweqok, lets move on15:30
slaweq#topic fullstack/functional15:30
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"15:30
slaweqI saw few issues in functional tests this week, like e.g.15:30
slaweqhttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d0/726168/2/check/neutron-functional/6d0b174/testr_results.html15:30
slaweqbut it was only once when I saw it15:31
maciejjozefczykYeah I noticed that too.15:31
slaweqI don't see anything obvious in log from this test: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d0/726168/2/check/neutron-functional/6d0b174/controller/logs/dsvm-functional-logs/neutron.tests.functional.agent.l3.test_ha_router.L3HATestFailover.test_ha_router_failover.txt15:33
slaweqbut I will check it more deeply this week15:34
slaweq#action slaweq to check failure in test_ha_router_failover: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d0/726168/2/check/neutron-functional/6d0b174/testr_results.html15:34
slaweqregarding fullstack tests, I still see failing firewall test, like https://93b88ea1fe64fba121c6-f42d955827477dc68a274454ee4340d5.ssl.cf2.rackcdn.com/726168/2/check/neutron-fullstack/cb8503e/testr_results.html15:34
slaweqI think we need to reopen bug related to this and mark this test as unstable again15:35
slaweqwhat do You think?15:36
ralonsohok15:36
ralonsohshouldn't we have more debug info there?15:36
slaweqyes, I will try to add some additional logging in this test15:37
ralonsohI remember we added the routes, the devices, etc.15:37
ralonsoh(maybe I'm wrong)15:37
slaweqralonsoh: not to fullstack tests AFAIR15:37
slaweqbut I will check that15:37
ralonsohups15:37
slaweq#action slaweq to add additional logging for fullstack's firewall tests15:38
slaweq#action slaweq to reopen bug related to failing fuillstack firewall tests15:38
slaweqand that's all what I have regarding functional and fullstack tests for today15:38
slaweqlets move on15:39
slaweq#topic Tempest/Scenario15:39
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"15:39
slaweqhere I found one new issue, Address already allocated in subnet15:39
slaweqhttps://2f302d35d7c2b9201857-14c47b0c762b46266aadd7f2c624d382.ssl.cf5.rackcdn.com/665467/70/check/neutron-tempest-plugin-scenario-openvswitch/9514b9a/testr_results.html15:39
slaweqmaybe it's just http request timeout on client's side and after recheck it was already done in neutron15:40
slaweqbut IMO worth to check that15:40
slaweqany volunteer for that?15:40
ralonsohme15:40
ralonsohbut on friday15:40
slaweqralonsoh: great, thx a lot15:40
slaweq#action ralonsoh to check  Address already allocated in subnet issue in tempest job15:41
slaweqok, and the last thing for today15:41
slaweq#topic Periodic15:41
*** openstack changes topic to "Periodic (Meeting topic: neutron_ci)"15:41
slaweqthx maciejjozefczyk for fixing fedora ovn job, it's now fine15:41
maciejjozefczyk\o/15:41
slaweqtoday I noticed that openstack-tox-py36-with-ovsdbapp-master is failing15:41
slaweqBug is reported https://bugs.launchpad.net/ovsdbapp/+bug/187971715:42
openstackLaunchpad bug 1879717 in ovsdbapp "Neutron's unit tests are failing with ovsdbapp from master branch" [Undecided,New]15:42
slaweqand otherwiseguy already proposed patch for that15:42
maciejjozefczykYes, looks like its related to autoindex feature recently merged...15:42
bcafarelhttps://review.opendev.org/728306 ?15:42
slaweqwe cached it in last minute because there is proposed new release of ovsdbapp and if we would release it, neutron gate would be doomed15:42
slaweqbcafarel: https://review.opendev.org/72962715:43
slaweqand basically that's all from me for today15:44
slaweqanything else You want to discuss?15:44
bcafarelthanks (and nice to see the -with-xx-master jobs catching this kind of issues!)15:44
slaweqbcafarel: yes, good that we are checking them at least once a week :)15:45
bcafarelthat too :)15:45
bcafareland nothing else from me15:45
slaweqbtw. I started reorganisation of zuul jobs definitions in neutron-tempest-plugin https://review.opendev.org/#/c/729567/15:45
slaweqthere is some issue there now, so I have to check it but please be ready for review when it will pass zuul :)15:45
bcafarelnice15:47
slaweqif You don't have anything else, I will give You 13 minutes back15:47
slaweqthx for attending and have a great rest of the week :)15:47
slaweqo/15:47
bcafarelo/15:47
slaweq#endmeeting15:47
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:47
openstackMeeting ended Wed May 20 15:47:56 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:47
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-05-20-15.01.html15:48
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-05-20-15.01.txt15:48
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-05-20-15.01.log.html15:48
ralonsohbye15:48
maciejjozefczykbb!15:48
*** jraju__ has quit IRC16:20
*** jamesmcarthur has quit IRC16:36
*** jamesmcarthur has joined #openstack-meeting-316:39
*** liuyulong has quit IRC17:00
*** jamesmcarthur has quit IRC17:24
*** raildo has quit IRC17:25
*** raildo has joined #openstack-meeting-317:25
*** jamesmcarthur has joined #openstack-meeting-317:45
*** jamesmcarthur has quit IRC17:51
*** ralonsoh has quit IRC17:54
*** jamesmcarthur has joined #openstack-meeting-318:00
*** e0ne has quit IRC18:06
*** jamesmcarthur has quit IRC18:40
*** jamesmcarthur has joined #openstack-meeting-318:41
*** e0ne has joined #openstack-meeting-319:17
*** e0ne has quit IRC19:44
*** jamesmcarthur has quit IRC20:19
*** jamesmcarthur has joined #openstack-meeting-320:19
*** maciejjozefczyk has quit IRC20:30
*** jamesmcarthur has quit IRC21:12
*** jamesmcarthur has joined #openstack-meeting-321:12
*** jamesmcarthur has quit IRC21:15
*** jamesmcarthur has joined #openstack-meeting-321:15
*** jamesmcarthur has quit IRC21:32
*** jamesmcarthur has joined #openstack-meeting-321:33
*** jamesmcarthur has quit IRC21:35
*** jamesmcarthur has joined #openstack-meeting-321:42
*** raildo has quit IRC21:56
*** raildo_ has joined #openstack-meeting-321:56
*** slaweq has quit IRC22:13
*** slaweq has joined #openstack-meeting-322:21
*** slaweq has quit IRC22:25
*** jamesmcarthur has quit IRC22:29
*** jamesmcarthur has joined #openstack-meeting-322:30
*** slaweq has joined #openstack-meeting-322:39
*** slaweq has quit IRC22:44
*** jamesmcarthur has quit IRC22:52
*** jamesmcarthur has joined #openstack-meeting-322:53
*** jamesmcarthur has quit IRC22:56
*** jamesmcarthur has joined #openstack-meeting-322:56
*** raildo_ has quit IRC23:03
*** jamesmcarthur has quit IRC23:07
*** jamesmcarthur has joined #openstack-meeting-323:14
*** jamesmcarthur has quit IRC23:20
*** jamesmcarthur has joined #openstack-meeting-323:20
*** jamesmcarthur has quit IRC23:25
*** jamesmcarthur has joined #openstack-meeting-323:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!