| *** macz_ has joined #openstack-meeting-3 | 02:34 | |
| *** macz_ has quit IRC | 02:38 | |
| *** psachin has joined #openstack-meeting-3 | 03:41 | |
| *** macz_ has joined #openstack-meeting-3 | 04:22 | |
| *** macz_ has quit IRC | 04:26 | |
| *** ricolin_ has joined #openstack-meeting-3 | 05:15 | |
| *** ricolin_ has quit IRC | 05:26 | |
| *** ralonsoh has joined #openstack-meeting-3 | 06:54 | |
| *** slaweq has joined #openstack-meeting-3 | 07:05 | |
| *** e0ne has joined #openstack-meeting-3 | 07:07 | |
| *** slaweq has quit IRC | 07:09 | |
| *** haleyb has quit IRC | 07:10 | |
| *** haleyb has joined #openstack-meeting-3 | 07:11 | |
| *** slaweq has joined #openstack-meeting-3 | 07:14 | |
| *** njohnston has quit IRC | 07:33 | |
| *** tosky has joined #openstack-meeting-3 | 07:41 | |
| *** tosky_ has joined #openstack-meeting-3 | 08:12 | |
| *** tosky is now known as Guest98160 | 08:13 | |
| *** tosky_ is now known as tosky | 08:13 | |
| *** Guest98160 has quit IRC | 08:15 | |
| *** lpetrut has joined #openstack-meeting-3 | 09:13 | |
| *** raildo has joined #openstack-meeting-3 | 11:04 | |
| *** belmoreira has joined #openstack-meeting-3 | 11:22 | |
| *** njohnston_ has joined #openstack-meeting-3 | 11:26 | |
| *** psahoo has joined #openstack-meeting-3 | 11:52 | |
| *** raildo_ has joined #openstack-meeting-3 | 13:03 | |
| *** raildo has quit IRC | 13:03 | |
| *** ttx has quit IRC | 13:04 | |
| *** ttx has joined #openstack-meeting-3 | 13:09 | |
| *** psahoo_ has joined #openstack-meeting-3 | 13:09 | |
| *** psahoo has quit IRC | 13:12 | |
| *** ttx has quit IRC | 13:16 | |
| *** ttx has joined #openstack-meeting-3 | 13:17 | |
| *** Adri2000 has joined #openstack-meeting-3 | 13:25 | |
| *** genekuo_ has joined #openstack-meeting-3 | 14:29 | |
| *** macz_ has joined #openstack-meeting-3 | 14:35 | |
| *** macz_ has quit IRC | 14:39 | |
| *** psahoo_ has quit IRC | 14:43 | |
| *** lpetrut has quit IRC | 14:45 | |
| *** njohnston_ is now known as njohnston | 14:52 | |
| *** priteau has joined #openstack-meeting-3 | 14:57 | |
| *** slaweq_ has joined #openstack-meeting-3 | 14:58 | |
| *** macz_ has joined #openstack-meeting-3 | 14:58 | |
| *** slaweq has quit IRC | 14:58 | |
| *** lajoskatona has joined #openstack-meeting-3 | 15:01 | |
| slaweq_ | #startmeeting neutron_ci | 15:01 |
|---|---|---|
| openstack | Meeting started Wed Oct 7 15:01:15 2020 UTC and is due to finish in 60 minutes. The chair is slaweq_. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| slaweq_ | hi | 15:01 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 15:01 | |
| openstack | The meeting name has been set to 'neutron_ci' | 15:01 |
| lajoskatona | Hi | 15:01 |
| bcafarel | o/ | 15:01 |
| ralonsoh | hi | 15:02 |
| slaweq_ | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:02 |
| slaweq_ | Please open now :) | 15:02 |
| *** mdelavergne has joined #openstack-meeting-3 | 15:03 | |
| slaweq_ | #topic Actions from previous meetings | 15:03 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 15:03 | |
| slaweq_ | bcafarel to update our grafana dashboards for stable branches | 15:03 |
| bcafarel | in progress, not sent yet (I wanted to check jobs listed there) | 15:04 |
| slaweq_ | ok, thx bcafarel | 15:04 |
| slaweq_ | I will assign it to You for next week | 15:04 |
| slaweq_ | just to remember about it | 15:04 |
| slaweq_ | ok? | 15:04 |
| bcafarel | sounds good, also to have reviewers if it gets forgotten | 15:04 |
| slaweq_ | #action bcafarel to update our grafana dashboards for stable branches | 15:04 |
| slaweq_ | thx a lot | 15:04 |
| slaweq_ | ok, next one | 15:05 |
| slaweq_ | ralonsoh to report a bug and check failing openstack-tox-py36-with-ovsdbapp-master periodic job | 15:05 |
| *** mlavalle has joined #openstack-meeting-3 | 15:05 | |
| ralonsoh | I sent a patch to try to solve it | 15:05 |
| ralonsoh | one sec | 15:05 |
| ralonsoh | (should be on the etherpad) | 15:05 |
| slaweq_ | I don't see it on etherpad | 15:06 |
| ralonsoh | https://review.opendev.org/#/c/755256/ | 15:06 |
| ralonsoh | avoid to monkey patch processutils | 15:06 |
| ralonsoh | well, use the original current_thread _active | 15:07 |
| ralonsoh | but we'll need a new version of oslo.concurrency | 15:07 |
| slaweq_ | and it seems that it helped | 15:07 |
| ralonsoh | at least locally | 15:08 |
| ralonsoh | but I can't say that in the CI | 15:08 |
| slaweq_ | https://zuul.openstack.org/buildset/aa6cb9d44d1a49368494071338c7415e | 15:08 |
| slaweq_ | :) | 15:08 |
| slaweq_ | it helped | 15:08 |
| ralonsoh | ah8hh ok, this is2 another problem | 15:08 |
| ralonsoh | sorry | 15:08 |
| ralonsoh | #link https://review.opendev.org/#/c/749537/ | 15:09 |
| ralonsoh | this is the patch | 15:09 |
| ralonsoh | sorry again | 15:09 |
| slaweq_ | :) | 15:09 |
| slaweq_ | don't need to sorry | 15:09 |
| slaweq_ | good that it's fixed :) | 15:09 |
| slaweq_ | thx ralonsoh | 15:09 |
| slaweq_ | and thx otherwiseguy | 15:09 |
| slaweq_ | ok, so I think we can move on to the next topics | 15:10 |
| slaweq_ | #topic Switch to Ubuntu Focal | 15:10 |
| *** openstack changes topic to "Switch to Ubuntu Focal (Meeting topic: neutron_ci)" | 15:10 | |
| slaweq_ | Etherpad: https://etherpad.opendev.org/p/neutron-victoria-switch_to_focal | 15:10 |
| slaweq_ | we still have some stadium projects to check/change | 15:10 |
| slaweq_ | but I didn't had time this week | 15:10 |
| slaweq_ | do You have any other updates on that? | 15:10 |
| ralonsoh | no | 15:11 |
| lajoskatona | no | 15:11 |
| bcafarel | https://review.opendev.org/#/c/754068/ longing for second +2 for sfc :) | 15:12 |
| bcafarel | else topic:migrate-to-focal list looks good for us | 15:13 |
| slaweq_ | bcafarel: I already gave +2 :) | 15:13 |
| slaweq_ | so I can't help with that one now | 15:13 |
| slaweq_ | ralonsoh: lajoskatona but You can ;) | 15:13 |
| ralonsoh | sure | 15:13 |
| lajoskatona | done :-) | 15:13 |
| slaweq_ | thx | 15:14 |
| bcafarel | thanks :) | 15:14 |
| lajoskatona | Shall I have a slighly related question, do we need this any more: https://review.opendev.org/755721 ? | 15:15 |
| slaweq_ | lajoskatona: nope | 15:15 |
| slaweq_ | it was an issue with pypi mirror | 15:15 |
| lajoskatona | slaweq_: yeah that's why I asked :-) I abandone it then | 15:16 |
| slaweq_ | and I think ralonsoh fixed it on devstack by capping setuptools version | 15:16 |
| ralonsoh | but that was rejected | 15:16 |
| ralonsoh | the problem was in the pypi server | 15:16 |
| slaweq_ | ralonsoh: ahh, ok | 15:16 |
| ralonsoh | admins talked to pypi folks to solve that | 15:16 |
| slaweq_ | most important is that problem is fixed now :) | 15:16 |
| ralonsoh | yes | 15:16 |
| slaweq_ | thx ralonsoh and lajoskatona for taking care of it :) | 15:16 |
| slaweq_ | ok | 15:18 |
| lajoskatona | no problem | 15:18 |
| slaweq_ | regrading standardize on zuul v3 | 15:18 |
| slaweq_ | we merged networking-odl patch https://review.opendev.org/#/c/725647/ | 15:18 |
| slaweq_ | so the last one missing is https://review.opendev.org/#/c/729591/ for neutron | 15:18 |
| slaweq_ | and it just failed again, at least functional tests job: https://40f71fdb4a17c8b8e33a-40a7733116b3138073a0fe5a58665a17.ssl.cf5.rackcdn.com/729591/21/check/neutron-functional-with-uwsgi/aace04f/testr_results.html | 15:19 |
| tosky | which received its fair share of rechecks | 15:19 |
| slaweq_ | :/ | 15:19 |
| ralonsoh | slaweq_, that's the other related problem I was talking this morning | 15:20 |
| ralonsoh | now we don't fail in the OVN method | 15:21 |
| ralonsoh | but in the "old_method" --> L3 plugin | 15:21 |
| ralonsoh | I need to check if this is related | 15:21 |
| ralonsoh | I'll talk to otherwiseguy | 15:21 |
| slaweq_ | ralonsoh: ok | 15:21 |
| tosky | please remember to vote also on the networking-odl backport for stable/victoria: https://review.opendev.org/#/c/756324/ | 15:22 |
| slaweq_ | tosky: I already did | 15:22 |
| slaweq_ | I think we need bcafarel's vote also | 15:22 |
| tosky | yeah, another stable core | 15:23 |
| tosky | or neutron stable core | 15:23 |
| bcafarel | reviewed and W+1 :) | 15:23 |
| slaweq_ | thx | 15:24 |
| slaweq_ | so I think we can move on to the next topic now | 15:24 |
| slaweq_ | #topic Stable branches | 15:24 |
| *** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)" | 15:24 | |
| slaweq_ | Ussuri dashboard: http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=1 | 15:25 |
| slaweq_ | Train dashboard: http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=1 | 15:25 |
| bcafarel | one thing I remember now on stable dashboards, we will also need a victoria template for neuton-tempest-plugin | 15:25 |
| bcafarel | and switch neutron stable/victoria to it | 15:25 |
| slaweq_ | bcafarel: yes, true | 15:26 |
| slaweq_ | I will do this template | 15:26 |
| slaweq_ | thx for reminder | 15:26 |
| slaweq_ | #action slaweq to make neutron-tempest-plugin victoria template | 15:26 |
| bcafarel | np, I remembered when my test dashboard came up empty for them | 15:26 |
| slaweq_ | btw. I have one new issue in stable/train | 15:29 |
| slaweq_ | https://bugs.launchpad.net/neutron/+bug/1898748 | 15:29 |
| openstack | Launchpad bug 1898748 in neutron "[stable/train] Creation of the QoS policy takes ages" [Critical,New] | 15:29 |
| slaweq_ | did You saw it already maybe? | 15:30 |
| ralonsoh | no | 15:30 |
| slaweq_ | it seems that it breaks devstack gate for stable/train :/ | 15:30 |
| bcafarel | I don't think I saw it either | 15:31 |
| slaweq_ | is there anyone who wants to check that maybe? | 15:31 |
| slaweq_ | if not, I will try to check that | 15:32 |
| ralonsoh | I'll try to take a look at this error tomorrow | 15:32 |
| slaweq_ | thx ralonsoh :) | 15:32 |
| slaweq_ | ok, lets move on | 15:33 |
| slaweq_ | #topic Grafana | 15:33 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 15:33 | |
| slaweq_ | http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:33 |
| slaweq_ | IMO worst thing from voting jobs is neutron-functional-with-uwsgi now | 15:34 |
| slaweq_ | and we have couple of issues there | 15:34 |
| slaweq_ | and also most of the ovn based jobs are failing 100% of times | 15:35 |
| slaweq_ | anything else You have regarding grafana in general? | 15:36 |
| slaweq_ | or should we move on to the specific job types? | 15:36 |
| bcafarel | nothing from me | 15:37 |
| slaweq_ | ok, so lets move on | 15:37 |
| slaweq_ | #topic functional/fullstack | 15:37 |
| *** openstack changes topic to "functional/fullstack (Meeting topic: neutron_ci)" | 15:37 | |
| slaweq_ | I reported today https://bugs.launchpad.net/neutron/+bug/1898859 | 15:38 |
| openstack | Launchpad bug 1898859 in neutron "Functional test neutron.tests.functional.agent.linux.test_keepalived.KeepalivedManagerTestCase.test_keepalived_spawns_conflicting_pid_vrrp_subprocess is failing" [High,Confirmed] | 15:38 |
| slaweq_ | as I saw it at least twice recently | 15:38 |
| slaweq_ | IIRC we already saw it in the past too but I wasn't sure if we have bug reported for that already | 15:38 |
| ralonsoh | related to the ns deletion | 15:38 |
| ralonsoh | https://review.opendev.org/#/c/754938/ | 15:39 |
| ralonsoh | please, review ^^ | 15:39 |
| slaweq_ | ahh, right | 15:40 |
| slaweq_ | now I remember :) | 15:40 |
| slaweq_ | so I will mark https://bugs.launchpad.net/neutron/+bug/1898859 as duplicate of https://bugs.launchpad.net/neutron/+bug/1838793 | 15:40 |
| openstack | Launchpad bug 1898859 in neutron "Functional test neutron.tests.functional.agent.linux.test_keepalived.KeepalivedManagerTestCase.test_keepalived_spawns_conflicting_pid_vrrp_subprocess is failing" [High,Confirmed] | 15:40 |
| ralonsoh | I think you can join both LP bugs | 15:40 |
| openstack | Launchpad bug 1838793 in neutron ""KeepalivedManagerTestCase" tests failing during namespace deletion" [High,Confirmed] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 15:40 |
| ralonsoh | yes | 15:40 |
| slaweq_ | lajoskatona: can You check that patch from ralonsoh? | 15:41 |
| slaweq_ | I hope it will help us a bit with this functional tests job :) | 15:41 |
| lajoskatona | slaweq_: sure, I cheked it in the past, so has some background :-) | 15:41 |
| slaweq_ | lajoskatona: thx a lot | 15:42 |
| slaweq_ | and for other issues with functional tests I know that ralonsoh told me that he will open LPs | 15:42 |
| ralonsoh | the one related to the agents | 15:42 |
| ralonsoh | test_agent_show | 15:42 |
| *** belmoreira has quit IRC | 15:44 | |
| slaweq_ | yes, did You report it already? | 15:45 |
| ralonsoh | not yet | 15:45 |
| ralonsoh | I'm still investigating the error | 15:45 |
| slaweq_ | k | 15:45 |
| slaweq_ | ok, lets move on then | 15:47 |
| slaweq_ | #topic Tempest/Scenario | 15:47 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 15:47 | |
| slaweq_ | first, I reported today bug: https://bugs.launchpad.net/neutron/+bug/1898862 | 15:47 |
| openstack | Launchpad bug 1898862 in neutron "Job neutron-ovn-tempest-ovs-release-ipv6-only is failing 100% of times" [High,Confirmed] | 15:47 |
| slaweq_ | becuase neutron-ovn-tempest-ovs-release-ipv6-only is failing 100% of times and usually (or always even) there is 9 tests failing there | 15:48 |
| slaweq_ | so it's very reproducible | 15:48 |
| slaweq_ | I will try to ping lucasgomes or jlibosva to take a look at that one | 15:48 |
| slaweq_ | there is also ovn related issue https://bugs.launchpad.net/neutron/+bug/1885900 | 15:49 |
| openstack | Launchpad bug 1885900 in neutron "test_trunk_subport_lifecycle is failing in ovn based jobs" [Critical,Confirmed] - Assigned to Lucas Alvares Gomes (lucasagomes) | 15:49 |
| slaweq_ | which I saw today again | 15:49 |
| slaweq_ | and we still have some random ssh authentication failures | 15:50 |
| slaweq_ | like e.g. https://3b00945aa0cfe70597e9-73e59f2d88a36c349deccf374592c99f.ssl.cf5.rackcdn.com/755752/3/gate/neutron-tempest-linuxbridge/4bbc7f9/testr_results.html | 15:50 |
| slaweq_ | or https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_807/750166/5/gate/neutron-tempest-plugin-scenario-linuxbridge/8073d8a/testr_results.html | 15:50 |
| slaweq_ | and in those cases there is no any "pattern", like always same tests or always same backend | 15:51 |
| slaweq_ | it happens everywhere | 15:51 |
| slaweq_ | and I tend to think that this is issue which ralonsoh found some time ago in our d/s ci | 15:51 |
| slaweq_ | with paramiko and some race condition | 15:51 |
| ralonsoh | with paramiko | 15:51 |
| ralonsoh | yes | 15:51 |
| slaweq_ | I couldn't reproduce that locally | 15:51 |
| slaweq_ | but some race is there IMO | 15:52 |
| ralonsoh | once paramiko tries to log into a VM without the keys, even when the keys are installed, the SSH connection is not possible | 15:52 |
| slaweq_ | maybe we can try to check console log first to see if ssh key was confiugred already | 15:52 |
| slaweq_ | before ssh to the instance | 15:52 |
| slaweq_ | if that will fail for any reason (e.g. custom guest os which don't log things like cirros), we can always try ssh at the end | 15:53 |
| slaweq_ | as "fallback" option | 15:53 |
| slaweq_ | wdyt? | 15:53 |
| ralonsoh | it worths to try it | 15:54 |
| slaweq_ | we can maybe propose that first in neutron-tempest-plugin | 15:54 |
| bcafarel | worth a try | 15:54 |
| slaweq_ | and if that will work, then propose to tempest too | 15:54 |
| slaweq_ | ok, I will give it a try | 15:54 |
| ralonsoh | (I was doing the opposite: reviewing the paramiko code) | 15:55 |
| slaweq_ | #action slaweq to propose patch to check console log before ssh to instance | 15:55 |
| slaweq_ | ralonsoh: if You will find issue on paramiko's side, we can always revert workaround from neutron-tempest-plugin :) | 15:55 |
| ralonsoh | of course | 15:55 |
| slaweq_ | ok, I have one more issue related to ovn jobs: https://bugs.launchpad.net/neutron/+bug/1898863 | 15:56 |
| openstack | Launchpad bug 1898863 in neutron "OVN based scenario jobs failing 100% of times" [Critical,Confirmed] | 15:56 |
| slaweq_ | did You saw that before? | 15:56 |
| bcafarel | on dstat?? | 15:56 |
| slaweq_ | yes | 15:57 |
| slaweq_ | but I saw it only on ovn based jobs | 15:57 |
| slaweq_ | :/ | 15:57 |
| ralonsoh | no sorry, that's new to me | 15:57 |
| slaweq_ | ok, anyone wants to take a look at that? | 15:57 |
| slaweq_ | if not than it's also fine for now as it affects "only" non-voting jobs | 15:58 |
| ralonsoh | https://bugs.launchpad.net/ubuntu/+source/dstat/+bug/1866619 | 15:58 |
| openstack | Launchpad bug 1866619 in dstat (Ubuntu) "OverflowError when machine suspends and resumes after a longer while" [Undecided,Confirmed] | 15:58 |
| ralonsoh | DistroRelease: Ubuntu 20.04 | 15:58 |
| slaweq_ | so we will probably need to disable dstat as temporary workaround | 15:59 |
| slaweq_ | thx ralonsoh | 15:59 |
| ralonsoh | yes | 15:59 |
| slaweq_ | ok | 16:00 |
| slaweq_ | we are out of time today | 16:00 |
| slaweq_ | thx for attending the meeting | 16:00 |
| slaweq_ | o/ | 16:00 |
| ralonsoh | bye! | 16:00 |
| slaweq_ | #endmeeting | 16:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 16:00 | |
| openstack | Meeting ended Wed Oct 7 16:00:19 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-10-07-15.01.html | 16:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-10-07-15.01.txt | 16:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-10-07-15.01.log.html | 16:00 |
| bcafarel | lengthy one :/ | 16:00 |
| lajoskatona | o/ | 16:00 |
| *** lajoskatona has left #openstack-meeting-3 | 16:00 | |
| ttx | #startmeeting large_scale_sig | 16:01 |
| openstack | Meeting started Wed Oct 7 16:01:37 2020 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| *** openstack changes topic to " (Meeting topic: large_scale_sig)" | 16:01 | |
| openstack | The meeting name has been set to 'large_scale_sig' | 16:01 |
| ttx | #topic Rollcall | 16:01 |
| *** openstack changes topic to "Rollcall (Meeting topic: large_scale_sig)" | 16:01 | |
| mdelavergne | Hi! | 16:01 |
| ttx | Who is here for the Large Scale SIG meeting ? | 16:01 |
| ttx | mdelavergne: hi! | 16:01 |
| genekuo_ | Hi | 16:01 |
| ttx | genekuo_: Hi! | 16:02 |
| genekuo_ | I'm first time here | 16:02 |
| genekuo_ | I'm a Infrastructure Engineer at LINE | 16:02 |
| genekuo_ | masahito is my colleague | 16:02 |
| mdelavergne | Welcome! | 16:03 |
| ttx | amorin: around? | 16:03 |
| ttx | It might be just us 3 today | 16:04 |
| ttx | Our agenda for today is at: | 16:04 |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-meeting | 16:04 |
| ttx | #topic PTG/Summit plans update | 16:04 |
| *** openstack changes topic to "PTG/Summit plans update (Meeting topic: large_scale_sig)" | 16:04 | |
| ttx | A reminder on the Large Scale SIG activities around Summit and PTG | 16:04 |
| ttx | Our Forum session is Tuesday, October 20, 7:30am-8:15am CT | 16:04 |
| ttx | #link https://www.openstack.org/summit/2020/summit-schedule/events/24746/share-your-openstack-scaling-story | 16:04 |
| ttx | That makes it super early for our US friends and a bit late for our APAC friends | 16:05 |
| ttx | genekuo_: it must be super-late for you now | 16:05 |
| genekuo_ | I'm usually sleep late | 16:05 |
| genekuo_ | So it's fine | 16:05 |
| ttx | I'll moderate the discussion, but we'll also have active participants to help seed the discussion and encourage others to share | 16:06 |
| ttx | amorin and belmoreira said they would help | 16:06 |
| ttx | In preparation for this session, please add to the etherpad at: | 16:06 |
| ttx | #link https://etherpad.opendev.org/p/w-forum-scaling-stories | 16:06 |
| *** slaweq has joined #openstack-meeting-3 | 16:06 | |
| *** slaweq_ has quit IRC | 16:07 | |
| ttx | especially if you have things you'd like to see covered | 16:07 |
| ttx | The week after that during PTG week we will have two one-hour sessions: | 16:07 |
| ttx | #info PTG meeting Wednesday Oct 28 7UTC-8UTC and 16UTC-17UTC | 16:07 |
| ttx | Those will be more traditional meetings, the idea being to onboard any new recruit from that forum session | 16:07 |
| ttx | Questions on that topic? | 16:07 |
| mdelavergne | Not from myself | 16:08 |
| ttx | alright, moving on | 16:08 |
| ttx | #topic Meaningful monitoring | 16:08 |
| *** openstack changes topic to "Meaningful monitoring (Meeting topic: large_scale_sig)" | 16:08 | |
| ttx | Last month we discussed forming a new workstream around "meaningful monitoring" | 16:09 |
| ttx | I tried to bootstrap it in the following etherpad: | 16:09 |
| ttx | #link https://etherpad.opendev.org/p/large-scale-sig-meaningful-monitoring | 16:09 |
| ttx | genekuo_: is that something that is of interest for you? | 16:09 |
| genekuo_ | I'll probably will be upstreaming oslo.metrics code that we current have | 16:10 |
| ttx | genekuo_: ok, we will cover that in a minute | 16:10 |
| ttx | Obviously we need to discuss what we mean by "meaningful monitoring" | 16:10 |
| mdelavergne | It would be nice to have some feedback from those who launched this topic :( | 16:10 |
| ttx | Is it actionable monitoring, like opinionated/focused monitoring... | 16:10 |
| ttx | mdelavergne: yeah, tI was hoping they would be here today | 16:11 |
| ttx | since it's "their" time | 16:11 |
| genekuo_ | This topic is interesting as we have a lot of notifications | 16:11 |
| genekuo_ | Most of them are not that useful | 16:11 |
| ttx | right, so I could see a need for a more targeted monitoring that instead of showing everything, tracks golden signals | 16:11 |
| ttx | (as described in that etherpad) | 16:11 |
| ttx | But yes I agree with mdelavergne it would be good to hear from those who raised that topic first and hear of their definition | 16:12 |
| ttx | moving on to the next workstream | 16:12 |
| ttx | #topic Progress on "Scaling within one cluster" goal | 16:12 |
| *** openstack changes topic to "Progress on "Scaling within one cluster" goal (Meeting topic: large_scale_sig)" | 16:12 | |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling | 16:12 |
| ttx | Regarding oslo.metrics, I did push a basic functional test so that we are reasonably sure that it actually works: | 16:13 |
| ttx | #link https://review.opendev.org/#/c/755069/ | 16:13 |
| ttx | genekuo_: would be good to get your review on it (or masahito's) | 16:13 |
| genekuo_ | Got it | 16:13 |
| ttx | Do you know when you'll be able to push the latest version? | 16:13 |
| genekuo_ | I'll also start writing test once I upstream most of our codes | 16:13 |
| genekuo_ | There not much left | 16:13 |
| genekuo_ | I can probably finish it by next week | 16:13 |
| ttx | great! | 16:14 |
| ttx | Note that according to my testing it seems to be missing the other side of the code -- the change in oslo.messaging to actually emit those metrics | 16:14 |
| ttx | genekuo_: do you have the code for that too? | 16:14 |
| genekuo_ | We currently haven't have any test yet I think | 16:14 |
| genekuo_ | Have to double check | 16:15 |
| ttx | ok, because as far as I can tell, the oslo.metric code only handles the reception of the message on the socket and it's storage in a Prometheus metric | 16:15 |
| ttx | The other side of this workstream is the collection of scaling stories | 16:16 |
| ttx | #link https://etherpad.openstack.org/p/scaling-stories | 16:16 |
| genekuo_ | Yes | 16:16 |
| ttx | Nothing new posted there... our next action is the forum session in two weeks | 16:16 |
| ttx | Anything else on this "Scaling within one cluster" goal? Questions? Comments? | 16:16 |
| genekuo_ | I think I can add something to the scaling stories part | 16:17 |
| mdelavergne | nice | 16:17 |
| genekuo_ | We did hit some issue scaling, I'll discuss with masahito tomorrow | 16:17 |
| ttx | genekuo_: perfect! Any story, even short, helps! | 16:17 |
| ttx | It's basically about "what happens when we add nodes to a cluster, what failed first" | 16:17 |
| ttx | (and bonus points for telling how you solved it) | 16:18 |
| genekuo_ | Got it | 16:18 |
| ttx | Moving on to next goal | 16:18 |
| ttx | #topic Progress on "Documenting large scale operations" goal | 16:18 |
| *** openstack changes topic to "Progress on "Documenting large scale operations" goal (Meeting topic: large_scale_sig)" | 16:18 | |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-documentation | 16:18 |
| ttx | amorin was working on pushing OSarchiver to the OSops repository | 16:18 |
| ttx | I guess we'll have to wait for an update on that | 16:19 |
| *** psachin has quit IRC | 16:19 | |
| ttx | So for now, just let me know if you have questions on that goal, and if you can help with anything in it | 16:19 |
| ttx | #topic Next meeting | 16:20 |
| *** openstack changes topic to "Next meeting (Meeting topic: large_scale_sig)" | 16:20 | |
| genekuo_ | Sounds clear to me for now | 16:20 |
| ttx | In two weeks we'll have the Forum session and the week after the live meetings | 16:20 |
| ttx | So I propose we get back to our regular rotation two weeks after that | 16:20 |
| ttx | Next IRC meeting will be EU+APAC Nov 10, 8utc, then US+EU Nov 24, 16utc. | 16:20 |
| ttx | Does that work? | 16:20 |
| mdelavergne | ok | 16:20 |
| mdelavergne | yep | 16:20 |
| genekuo_ | ok | 16:20 |
| ttx | I'll probably have to send the personal reminder to jpenick and Erik next time | 16:21 |
| ttx | since they seem to miss the one I send to teh ML | 16:21 |
| ttx | #info next meetings: Nov 10, 8utc; Nov 24, 16utc | 16:21 |
| mdelavergne | probably, yes! | 16:21 |
| ttx | #topic Open discussion | 16:21 |
| *** openstack changes topic to "Open discussion (Meeting topic: large_scale_sig)" | 16:21 | |
| ttx | Anything else you'd like to discuss? | 16:21 |
| *** tosky has quit IRC | 16:21 | |
| ttx | genekuo_: anything you think this group should do, that is not covered in those 3 goals? | 16:21 |
| genekuo_ | Haven't think about it yet | 16:22 |
| genekuo_ | Looks good for me for now | 16:22 |
| genekuo_ | I'll think about it and provide more feedback if there is | 16:22 |
| ttx | feel free to think about it and let us know next time! This is really about what the participants want to do, and try to use the group to help them achieve those objectives | 16:23 |
| ttx | Like amorin is leading the doc effort, and you're leading the oslo.metric effort | 16:23 |
| ttx | and the rest of the group facilitates | 16:23 |
| genekuo_ | Sounds good | 16:24 |
| ttx | Alright, if you have nothing else... I propose we close early and let genekuo_ go to bed :) | 16:24 |
| genekuo_ | Thanks! | 16:24 |
| mdelavergne | ahah | 16:24 |
| ttx | Thanks everyone! | 16:24 |
| mdelavergne | thanks to you! | 16:24 |
| ttx | Hopefully will see you at the PTG meeting in 3 weeks! | 16:24 |
| ttx | (and maybe at the Forum session in two weeks if you can make it!) | 16:24 |
| genekuo_ | I will join if possible | 16:25 |
| ttx | #endmeeting | 16:25 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 16:25 | |
| openstack | Meeting ended Wed Oct 7 16:25:06 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:25 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-10-07-16.01.html | 16:25 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-10-07-16.01.txt | 16:25 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-10-07-16.01.log.html | 16:25 |
| mdelavergne | see you! | 16:25 |
| *** mdelavergne has quit IRC | 16:25 | |
| *** genekuo_ has left #openstack-meeting-3 | 16:33 | |
| *** ralonsoh has quit IRC | 17:12 | |
| *** ralonsoh has joined #openstack-meeting-3 | 17:12 | |
| *** macz_ has quit IRC | 17:14 | |
| *** macz_ has joined #openstack-meeting-3 | 17:15 | |
| *** Adri2000 has quit IRC | 17:20 | |
| *** e0ne has quit IRC | 17:23 | |
| *** mlavalle has quit IRC | 17:56 | |
| *** mlavalle has joined #openstack-meeting-3 | 18:14 | |
| *** lpetrut has joined #openstack-meeting-3 | 18:18 | |
| *** lpetrut has quit IRC | 18:36 | |
| *** Adri2000 has joined #openstack-meeting-3 | 18:43 | |
| *** priteau has quit IRC | 19:12 | |
| *** tosky has joined #openstack-meeting-3 | 19:26 | |
| *** raildo_ is now known as raildo | 19:43 | |
| *** slaweq has quit IRC | 20:14 | |
| *** gmann is now known as gmann_lunch | 20:22 | |
| *** slaweq has joined #openstack-meeting-3 | 20:23 | |
| *** ralonsoh_ has joined #openstack-meeting-3 | 20:30 | |
| *** purplerbot has quit IRC | 20:31 | |
| *** ralonsoh has quit IRC | 20:33 | |
| *** purplerbot has joined #openstack-meeting-3 | 20:33 | |
| *** ralonsoh_ has quit IRC | 20:51 | |
| *** slaweq has quit IRC | 20:58 | |
| *** gmann_lunch is now known as gmann | 21:04 | |
| *** raildo has quit IRC | 21:11 | |
| *** njohnston has quit IRC | 22:04 | |
| *** tosky has quit IRC | 22:22 | |
| *** macz_ has quit IRC | 23:02 | |
| *** mlavalle has quit IRC | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!