Monday, 2025-03-17

« Sunday, 2025-03-16 Index Tuesday, 2025-03-18 »
opendevreviewCandido Campos Rivas proposed x/whitebox-neutron-tempest-plugin master: Increase burst window size to avoid unstabilities in slow envs  https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/94461607:31
opendevreviewOpenStack Release Bot proposed openstack/neutron stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/neutron/+/94462908:04
opendevreviewOpenStack Release Bot proposed openstack/neutron stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/neutron/+/94463008:04
opendevreviewOpenStack Release Bot proposed openstack/neutron master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/neutron/+/94463108:04
fricklerralonsoh: seems like https://bugs.launchpad.net/neutron/+bug/2101840 is severely affecting kolla CI, too. given how close we are to the release (and that neutron was just branched), maybe reverting those evenlet changes for 2025.1 might be in order?08:09
ralonsohfrickler, what changes in particular?08:22
ralonsohfrickler, because the last 2 changes were just removing last bits08:23
ralonsohI did the biggest part some months ago with no problem08:23
ralonsohfrickler, what is the issue in kolla?08:24
mnasiadkaSeems that instances are not getting dhcp addresses in ml2/ovs - just like in the bug08:25
ralonsohmnasiadka, yes but we need more details, some logs08:25
ralonsohsomething to investigate08:25
mnasiadkahttps://896708bdc2b98ec08f8b-d8efb2c8dc9c4e35e02c540c6f97e75e.ssl.cf2.rackcdn.com/938649/8/check/kolla-ansible-ubuntu/38b365c/primary/logs/ansible/test-core-openstack and https://896708bdc2b98ec08f8b-d8efb2c8dc9c4e35e02c540c6f97e75e.ssl.cf2.rackcdn.com/938649/8/check/kolla-ansible-ubuntu/38b365c/primary/logs/kolla/neutron/index.html08:26
mnasiadkaActually the dnsmasq log is interesting: https://896708bdc2b98ec08f8b-d8efb2c8dc9c4e35e02c540c6f97e75e.ssl.cf2.rackcdn.com/938649/8/check/kolla-ansible-ubuntu/38b365c/primary/logs/kolla/neutron/dnsmasq.txt08:28
mnasiadkafrickler: Maybe it makes sense to rekick that job and hold a node?08:31
ralonsohthere could be a problem with oslo.service08:33
ralonsohthat is still using eventlet (the only implemented backend for now)08:33
ralonsohso if we are removing the monkey patch in DHCP agent and oslo.service is using this bakend, we can have problems08:33
ralonsohI'll talk this afternoon with lajoskatona, haleyb slaweq ykarel about this08:34
ralonsohwe would revert the last patch08:34
ralonsohhttps://review.opendev.org/c/openstack/neutron/+/942530/08:34
ralonsohand retake it once eventlet implements kernel backend08:34
ralonsoheventlet no, oslo.service08:35
mnasiadkaIs it ok if I create a revert as WIP and with Workflow -1 to not merge it, just for the case of testing a DNM patch in Kolla with Depends-on on this revert to make sure the revert helps? (hope I didn't overcomplicate the writeup :-) )08:37
ralonsohI'm pushing this now08:38
mnasiadkathanks08:38
opendevreviewRodolfo Alonso proposed openstack/neutron master: Revert "[eventlet-removal] Remove the usage of eventlet in the DHCP agent"  https://review.opendev.org/c/openstack/neutron/+/94463408:40
lajoskatonaralonsoh, mnasiadka: let's go with revert for dhcp agent08:45
mnasiadkaTesting the revert in Kolla - https://review.opendev.org/c/openstack/kolla-ansible/+/94463508:46
opendevreviewRodolfo Alonso proposed openstack/neutron master: DNM == Test "neutron-tempest-plugin-openvswitch"  https://review.opendev.org/c/openstack/neutron/+/94261508:47
opendevreviewMerged openstack/neutron master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/neutron/+/94463108:55
opendevreviewRodolfo Alonso proposed openstack/neutron master: [OVN] Add ``AgentCache`` deletion test  https://review.opendev.org/c/openstack/neutron/+/94463609:05
fricklerralonsoh: mnasiadka: sorry, I was in a meeting. do you still want a held job for investigating? if yes, which one?09:19
mnasiadkaI don't for now - testing the revert in 94463509:20
ralonsohfrickler, I'm using the neutron ones09:21
bcafarelslaweq: I'm reviewing https://review.opendev.org/c/openstack/neutron/+/944022 backports, side comment do we need a release note there? even if fixing a bug this changes existing behavior that users may have been (incorrectly) relying on09:38
opendevreviewMerged openstack/neutron stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/neutron/+/94462909:39
opendevreviewOpenStack Release Bot proposed openstack/ovn-bgp-agent stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/ovn-bgp-agent/+/94464409:50
opendevreviewOpenStack Release Bot proposed openstack/ovn-bgp-agent stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/ovn-bgp-agent/+/94464509:50
opendevreviewOpenStack Release Bot proposed openstack/ovn-bgp-agent master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/ovn-bgp-agent/+/94464609:50
opendevreviewJimin Shin proposed openstack/neutron master: Fix to check empty list  https://review.opendev.org/c/openstack/neutron/+/94465009:55
mnasiadkaralonsoh: On the other note, I think it started failing miserably on 15th of March (after the nightly container image rebuild) - so there had to be some other change that caused it09:58
opendevreviewMerged openstack/neutron stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/neutron/+/94463010:01
opendevreviewMerged openstack/neutron master: Update CIDR notation for loopback and multicast address  https://review.opendev.org/c/openstack/neutron/+/94276310:01
opendevreviewMerged openstack/neutron master: Update OVN installation guide with tunings for VLAN + DVR  https://review.opendev.org/c/openstack/neutron/+/93996110:01
opendevreviewCandido Campos Rivas proposed x/whitebox-neutron-tempest-plugin master: Increase burst window size to avoid unstabilities in slow envs  https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/94461610:01
opendevreviewOpenStack Release Bot proposed openstack/neutron-vpnaas stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/neutron-vpnaas/+/94466310:02
opendevreviewOpenStack Release Bot proposed openstack/neutron-vpnaas stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/neutron-vpnaas/+/94466410:02
opendevreviewOpenStack Release Bot proposed openstack/neutron-vpnaas master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/neutron-vpnaas/+/94466510:02
opendevreviewMerged openstack/ovn-bgp-agent master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/ovn-bgp-agent/+/94464610:03
opendevreviewOpenStack Release Bot proposed openstack/neutron-fwaas stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/neutron-fwaas/+/94467910:08
opendevreviewOpenStack Release Bot proposed openstack/neutron-fwaas stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/neutron-fwaas/+/94468010:08
opendevreviewOpenStack Release Bot proposed openstack/neutron-fwaas master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/neutron-fwaas/+/94468110:08
fricklerralonsoh: so the revert doesn't seem to help in kolla, but we have found https://review.opendev.org/c/openstack/neutron/+/926497 which does touch dhcp code. can you make look at it once more and see if the change looks legit?10:16
fricklerand the timing of the merge would match when kolla jobs started failing10:16
ralonsohfrickler, try proposing a DNM revert for this patch and test with your CI10:17
opendevreviewLajos Katona proposed openstack/neutron master: DHCP Agent: Log resource update events  https://review.opendev.org/c/openstack/neutron/+/94402510:18
ralonsohare you using IPv6?10:18
mnasiadkawell, we're not defining ipv6 networks, but there is ipv6 stack on the hosts10:18
ralonsohif not, I don't see anything relevant from this patch (but just looking at the code)10:19
opendevreviewDr. Jens Harbott proposed openstack/neutron master: Revert "Optionally configure IPv6 metadata address"  https://review.opendev.org/c/openstack/neutron/+/94471410:20
opendevreviewMerged openstack/ovn-bgp-agent stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/ovn-bgp-agent/+/94464410:23
opendevreviewMerged openstack/ovn-bgp-agent stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/ovn-bgp-agent/+/94464510:23
opendevreviewMerged openstack/neutron-fwaas stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/neutron-fwaas/+/94467910:36
opendevreviewMerged openstack/neutron-vpnaas stable/2025.1: Update .gitreview for stable/2025.1  https://review.opendev.org/c/openstack/neutron-vpnaas/+/94466310:36
opendevreviewMerged openstack/neutron-vpnaas stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/neutron-vpnaas/+/94466410:40
opendevreviewMerged openstack/neutron-fwaas master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/neutron-fwaas/+/94468110:48
fricklerralonsoh: mnasiadka: seems the dhcp container is failing because (likely after the above patch) it now actually tries to use the IPv6 metadata address: Failure waiting for address fe80::a9fe:a9fe to become ready: Address not present on interface 10:48
fricklerhttps://3612101d6c142bf9c77a-c96c299047b55dcdeaefef8e344ceab6.ssl.cf1.rackcdn.com/944635/1/check/kolla-ansible-debian/8ca4360/primary/logs/container_logs/neutron_dhcp_agent.txt10:48
fricklernot sure yet whether that is actually a bug in kolla or a regression in neutron, possibly the former10:49
ralonsohfrickler, so yes, this patch seems to break the dhcp agent10:50
opendevreviewMerged openstack/neutron-vpnaas master: Update master for stable/2025.1  https://review.opendev.org/c/openstack/neutron-vpnaas/+/94466510:54
opendevreviewMerged openstack/neutron-fwaas stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1  https://review.opendev.org/c/openstack/neutron-fwaas/+/94468011:08
opendevreviewJimin Shin proposed openstack/neutron master: Check resource_request is required  https://review.opendev.org/c/openstack/neutron/+/94475611:13
opendevreviewJimin Shin proposed openstack/neutron master: Duplicated table lookup in extending port resource request  https://review.opendev.org/c/openstack/neutron/+/94465011:18
opendevreviewJimin Shin proposed openstack/neutron master: Extend port resource request when not using qos minimum rules  https://review.opendev.org/c/openstack/neutron/+/94475611:19
fricklerpassing with the revert https://zuul.opendev.org/t/openstack/build/b5f1e6675db049e784df4d2dec74aaa211:32
opendevreviewyatin proposed openstack/neutron unmaintained/2023.1: Use the declarative attribute ``standard_attr_id``  https://review.opendev.org/c/openstack/neutron/+/94475911:39
ralonsohfrickler, ok so I'll open a LP bug with this info11:43
ralonsohfrickler, https://bugs.launchpad.net/neutron/+bug/210341711:45
ralonsohhaleyb, ^11:45
priteauralonsoh: do you need the same check for need_ipv6_metadata in enable_isolated_metadata_proxy()12:07
priteauhere: https://opendev.org/openstack/neutron/src/commit/60c572acb18e3181d93cf6cc60920362f264aa6b/neutron/agent/dhcp/agent.py#L78512:09
opendevreviewRodolfo Alonso proposed openstack/neutron stable/2025.1: Fast exit if "segments" plugin is not loaded  https://review.opendev.org/c/openstack/neutron/+/94476212:13
ralonsohpriteau, sorry, what is the context?12:15
priteauThat IPv6 bug in kolla12:15
ralonsohto be honest, I don't know. Please report this bug in LP to be investigated. But the patch reverted is doing the opposite (or tried to): not to set IPv6 when there was not ipv6 subnet12:18
priteauI have a patch to test12:19
ralonsohso if we revert the patch and ipv6 is enabled, we are always setting the ipv6 address in the interface12:19
opendevreviewPierre Riteau proposed openstack/neutron master: Spawn IPv6 metadata proxy only when required  https://review.opendev.org/c/openstack/neutron/+/94476312:19
priteauLet's see if this works ^12:19
opendevreviewyatin proposed openstack/neutron stable/2024.2: [OVN] Isolate test_ovn_db_sync.TestOvnNbSync.* FTs  https://review.opendev.org/c/openstack/neutron/+/94476412:32
opendevreviewyatin proposed openstack/neutron stable/2024.1: [OVN] Isolate test_ovn_db_sync.TestOvnNbSync.* FTs  https://review.opendev.org/c/openstack/neutron/+/94476512:32
opendevreviewyatin proposed openstack/neutron stable/2023.2: [OVN] Isolate test_ovn_db_sync.TestOvnNbSync.* FTs  https://review.opendev.org/c/openstack/neutron/+/94476612:33
opendevreviewMerged openstack/neutron stable/2024.2: [S-RBAC] Fix policies for the SG rules API  https://review.opendev.org/c/openstack/neutron/+/94419812:39
priteauOf course my patch needs fixes to unit tests12:44
haleybralonsoh: did i break ipv6 (again)?13:18
haleybor miss one case?13:18
priteauhaleyb: Does this fix makes sense to you? I will update it with unit test fixes once it has gone through full Neutron and Kolla checks. https://review.opendev.org/c/openstack/neutron/+/94476313:22
ralonsohhaleyb, that was, most probably, https://review.opendev.org/q/7c32d43c4b37ec7629b1ff191e54c389c03b6f8d13:23
ralonsohI approved it 2 days ago and at the same time kolla CI stop working13:23
ralonsohhaleyb, check the logs13:23
haleybpriteau: yes, it looks correct, similar to the other changes i did. don't know how i missed it13:24
ralonsohhaleyb, sorry,  wrong link13:24
ralonsohhttps://review.opendev.org/c/openstack/neutron/+/94471413:24
ralonsohlogs: https://3612101d6c142bf9c77a-c96c299047b55dcdeaefef8e344ceab6.ssl.cf1.rackcdn.com/944635/1/check/kolla-ansible-debian/8ca4360/primary/logs/container_logs/neutron_dhcp_agent.txt13:25
ralonsohhaleyb, so maybe we don't need the revert, right? With the upper patch13:25
ralonsoh-> https://review.opendev.org/c/openstack/neutron/+/94476313:25
haleybralonsoh: right, with priteau we might not13:25
haleybhis patch that is13:26
ralonsohcool, then let's test this in the kolla CI frickler ^13:26
ralonsohif that works, then we can skip the revert and merge this patch in master and 2025.113:26
haleybright, and i'll add that to my 2024.2 backport if it all works13:27
ralonsohyeah, I commented on it too13:28
ralonsohthanks!13:28
ralonsohpriteau, are you testing kolla CI with your patch??13:28
priteauralonsoh: yes, here: https://review.opendev.org/c/openstack/kolla-ansible/+/94477613:31
ralonsohperfect13:31
opendevreviewMerged openstack/neutron stable/2024.1: [S-RBAC] Fix policies for the SG rules API  https://review.opendev.org/c/openstack/neutron/+/94419913:37
opendevreviewMerged openstack/neutron stable/2023.2: [S-RBAC] Fix policies for the SG rules API  https://review.opendev.org/c/openstack/neutron/+/94420013:37
mnasiadkaSeems to fix it, thanks priteau 14:17
priteauI was just going to post this14:26
priteauNow I need to fix the unit tests14:26
opendevreviewPierre Riteau proposed openstack/neutron master: Spawn IPv6 metadata proxy only when required  https://review.opendev.org/c/openstack/neutron/+/94476314:43
ralonsohykarel, hello15:02
ykarelralonsoh, sorry got late, starting now :)15:02
ralonsohno problem hehehehe15:02
ykarel#startmeeting neutron_ci15:03
opendevmeetMeeting started Mon Mar 17 15:03:31 2025 UTC and is due to finish in 60 minutes.  The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot.15:03
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:03
opendevmeetThe meeting name has been set to 'neutron_ci'15:03
ykarelPing list: bcafarel, lajoskatona, slawek, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira15:03
ykarelThis will be video meeting this time: https://meetpad.opendev.org/neutron-ci-meetings15:03
mlavalle\o15:03
lajoskatonao/15:05
bcafarelbrowsr crash, rejoining15:06
ykarel#topic Actions from previous meetings15:07
ykarelralonsoh to check and open bug for privsep related failure in functional job15:07
ralonsohhttps://review.opendev.org/c/openstack/oslo.privsep/+/94323715:07
ykarellajoskatona to check https://bugs.launchpad.net/neutron/+bug/210184015:09
ralonsohhttps://review.opendev.org/c/openstack/neutron/+/942615/415:10
ykarelhttps://review.opendev.org/c/openstack/neutron/+/94239315:10
ralonsohhttps://review.opendev.org/c/openstack/neutron/+/94253015:11
ykarelykarel to check next runs for fullstack fips and report issue with fullstack15:14
ykarel    lajoskatona to check fullstack test failure test_configurations_are_synced_towards_placement(NIC Switch agent)15:14
lajoskatonahttps://review.opendev.org/c/openstack/neutron/+/94401315:15
ykarelralonsoh to open an issue for random issues in ovn ubuntu jammy job15:16
ralonsohhttps://review.opendev.org/c/openstack/openstack-zuul-jobs/+/93545915:17
ykarel#action ykarel to check https://bugs.launchpad.net/neutron/+bug/210185715:19
ykarel#topic Stable branches15:19
ykarelall good wrt patches merge and periodic CI15:19
ykarel#topic Stadium projects15:20
ykarelbagpipe https://zuul.openstack.org/build/1c5618fa46114b659d60045a28250714 - glance test failed, likely random issue?15:21
ykarel#topic Rechecks15:21
ykarel5/26 bare rechecks15:22
ykarelIvan Anfimov15:23
lajoskatonapre-release checklist: https://docs.openstack.org/neutron/latest/contributor/policies/release-checklist.html15:23
ykarel#topic fullstack/functional15:24
ykarelstable ovnnbsync failures https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_935/944170/1/check/neutron-functional-with-uwsgi/9359500/testr_results.html15:24
ykarelproposed backport https://review.opendev.org/q/Ia0309e9129df992498820b239d8a145c1400aa6a15:24
ykarel- neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_ovsdb_monitor.TestNBDbMonitorOverTcp.test_floatingip_mac_bindings15:24
ykarel- oslo_db.exception.DBReferenceError: (sqlite3.IntegrityError) FOREIGN KEY constraint failed15:24
ykarel    - https://ac967958d7db9925bf11-8258bbe2bfc0c508066a865a71f46e18.ssl.cf1.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-functional/ed0c42a/testr_results.html15:24
ykarel    - similar to https://bugs.launchpad.net/neutron/+bug/208842315:24
ykarel#action ralonsoh to send fixes for TestNBDbMonitorOverTcp failures15:26
ykarel#topic Tempest/Scenario15:26
ykarelovs dhcp issue https://bugs.launchpad.net/neutron/+bug/210184015:26
ykarel#topic Periodic15:26
ykarelhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0bb/periodic/opendev.org/openstack/neutron/stable/2023.2/openstacksdk-functional-devstack-networking/0bb5384/testr_results.html15:26
ykarel    - openstack.exceptions.HttpException: HttpException: 413: Client Error for url: https://10.0.16.67/dns/v2/zones, Quota exceeded for zones.15:26
ykarelhttps://review.opendev.org/c/openstack/neutron/+/94475915:27
ykarelknown random issue https://bugs.launchpad.net/openstacksdk/+bug/208797815:27
ralonsoh-> https://review.opendev.org/c/openstack/openstacksdk/+/93476215:27
ykarelpostgres https://zuul.openstack.org/build/5f5d397dc66945ed85c11722e362ba8a15:28
ykarelhttps://review.opendev.org/c/openstack/neutron/+/94475915:28
ykarel#topic Grafana15:28
ykarelhttps://grafana.opendev.org/d/f913631585/neutron-failure-rate15:28
ykarel#topic On Demand15:32
ykarel#endmeeting15:33
opendevmeetMeeting ended Mon Mar 17 15:33:06 2025 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:33
opendevmeetMinutes:        https://meetings.opendev.org/meetings/neutron_ci/2025/neutron_ci.2025-03-17-15.03.html15:33
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/neutron_ci/2025/neutron_ci.2025-03-17-15.03.txt15:33
opendevmeetLog:            https://meetings.opendev.org/meetings/neutron_ci/2025/neutron_ci.2025-03-17-15.03.log.html15:33
opendevreviewLajos Katona proposed openstack/neutron master: If OVS Manager creation failes retry to set values  https://review.opendev.org/c/openstack/neutron/+/93911716:31
opendevreviewMerged openstack/neutron stable/2024.2: [OVN] Isolate test_ovn_db_sync.TestOvnNbSync.* FTs  https://review.opendev.org/c/openstack/neutron/+/94476418:48
opendevreviewMerged openstack/neutron stable/2024.1: [OVN] Isolate test_ovn_db_sync.TestOvnNbSync.* FTs  https://review.opendev.org/c/openstack/neutron/+/94476518:48
opendevreviewMerged openstack/neutron stable/2023.2: [OVN] Isolate test_ovn_db_sync.TestOvnNbSync.* FTs  https://review.opendev.org/c/openstack/neutron/+/94476618:48
opendevreviewMerged openstack/neutron stable/2024.2: [OVN] Do not delete twice the agent from the cache  https://review.opendev.org/c/openstack/neutron/+/94439318:48
opendevreviewMerged openstack/neutron stable/2025.1: Fast exit if "segments" plugin is not loaded  https://review.opendev.org/c/openstack/neutron/+/94476218:48
opendevreviewRodolfo Alonso proposed openstack/neutron master: Add missing Neutron milestones  https://review.opendev.org/c/openstack/neutron/+/94480320:44
opendevreviewRodolfo Alonso proposed openstack/neutron master: Open the 2025.2 (Flamingo) DB branch  https://review.opendev.org/c/openstack/neutron/+/94480420:47
opendevreviewBrian Haley proposed openstack/neutron master: Disable skip-level jobs in check queue  https://review.opendev.org/c/openstack/neutron/+/94480821:22
opendevreviewBrian Haley proposed openstack/neutron master: Update jobs based on testing runtime for 2025.2  https://review.opendev.org/c/openstack/neutron/+/94480921:22
opendevreviewBrian Haley proposed openstack/neutron master: Update jobs based on testing runtime for 2025.2  https://review.opendev.org/c/openstack/neutron/+/94480921:34
« Sunday, 2025-03-16 Index Tuesday, 2025-03-18 »

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!