| opendevreview | Winicius Allan Bezerra da Silva proposed openstack/watcher master: Add new WorkloadPreemption feature https://review.opendev.org/c/openstack/watcher/+/999202 | 03:26 |
|---|---|---|
| opendevreview | Winicius Allan Bezerra da Silva proposed openstack/watcher master: Add new WorkloadPreemption feature https://review.opendev.org/c/openstack/watcher/+/999202 | 03:28 |
| opendevreview | David proposed openstack/watcher-dashboard master: Fix audit list filter to use goal and strategy https://review.opendev.org/c/openstack/watcher-dashboard/+/999043 | 08:25 |
| opendevreview | David proposed openstack/watcher-dashboard master: Switch to admin project before Playwright tests https://review.opendev.org/c/openstack/watcher-dashboard/+/997960 | 08:30 |
| opendevreview | David proposed openstack/watcher-dashboard master: Fix audit list filter to use goal and strategy https://review.opendev.org/c/openstack/watcher-dashboard/+/999043 | 08:32 |
| opendevreview | David proposed openstack/watcher-dashboard master: Switch to admin project before Playwright tests https://review.opendev.org/c/openstack/watcher-dashboard/+/997960 | 08:34 |
| opendevreview | Joan Gilabert proposed openstack/watcher master: Change default database storage engine to InnoDB https://review.opendev.org/c/openstack/watcher/+/999241 | 08:50 |
| opendevreview | David proposed openstack/watcher-tempest-plugin master: Update get_host_other_than to select hosts on the same host aggregate https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/999254 | 10:07 |
| *** M00SE3 is now known as M00SE | 10:14 | |
| jgilaber | #startmeeting watcher | 13:01 |
| opendevmeet | Meeting started Thu Jul 30 13:01:44 2026 UTC and is due to finish in 60 minutes. The chair is jgilaber. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:01 |
| opendevmeet | The meeting name has been set to 'watcher' | 13:01 |
| dviroel | o/ | 13:01 |
| sean-k-mooney | o/ | 13:01 |
| jgilaber | hi! feel free to add topics to the agenda while we wait https://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L31 | 13:02 |
| rlandy | o/ | 13:02 |
| jgilaber | courtesy ping list: amoralej chandankumar morenod | 13:02 |
| morenod | o/ | 13:02 |
| jgilaber | ok, let's start with the first topic from dviroel | 13:03 |
| jgilaber | #topic Event Audit tests in watcher-tempest-plugin | 13:04 |
| dviroel | ack | 13:04 |
| dviroel | yeah, I added this one | 13:04 |
| dviroel | just to raise that I was looking at that | 13:04 |
| dviroel | and as we know, there isn't event type audit testing coverage in our plugin | 13:04 |
| dviroel | i was looking how feasible is to have it tested in our ci | 13:05 |
| chandankumar | o/ | 13:05 |
| dviroel | i shouldn't be hard to get an aodh alarm to trigger an event audit, with a dummy strategy | 13:05 |
| dviroel | that would give us a minimal test coverage, which we can expand afterwrds | 13:06 |
| opendevreview | David proposed openstack/watcher master: Add strategy as filter on /v1/audits/detail endpoint https://review.opendev.org/c/openstack/watcher/+/999094 | 13:06 |
| dviroel | sounds interesting for you folks? | 13:06 |
| dviroel | I may work on next week or so... | 13:07 |
| amoralej | o/ | 13:07 |
| dviroel | when I have some time | 13:07 |
| dviroel | but in order to create the alarm with aodh | 13:07 |
| dviroel | we would need to set up a client | 13:07 |
| dviroel | or create a dependency on telemetry plugin | 13:07 |
| amoralej | but we have an api | 13:07 |
| amoralej | i thing the trigger based on alarms should not be in watcher | 13:08 |
| amoralej | but the other way around | 13:08 |
| amoralej | it'd aodh or whatever external system who should call the watcher api to trigger the audit | 13:09 |
| amoralej | imo | 13:09 |
| dviroel | correct | 13:09 |
| dviroel | the idea of the scenario test is to have the aodh triggering the alarm | 13:09 |
| amoralej | i think for the scenario test we should just trigger the alarm from api | 13:10 |
| sean-k-mooney | dviroel: we dont actully need a depency on the telemetry plugin | 13:10 |
| sean-k-mooney | we can jsut use the tempest client ot create it ourselve | 13:11 |
| sean-k-mooney | so i think that proably the better aprpoch | 13:11 |
| dviroel | amoralej: you mean to call our webhook POST api? | 13:11 |
| amoralej | yes | 13:11 |
| sean-k-mooney | we can do that too | 13:11 |
| sean-k-mooney | we shoudl test that as well | 13:11 |
| dviroel | sean-k-mooney: we would need to create ourselves then, since we don't have the client in tempest only in the plugin | 13:11 |
| sean-k-mooney | yes | 13:12 |
| sean-k-mooney | so i woudl start with just callign the webhook directly | 13:12 |
| sean-k-mooney | we can alwasy add end to end testing with adoh after | 13:12 |
| opendevreview | Merged openstack/watcher-dashboard master: Switch to admin project before Playwright tests https://review.opendev.org/c/openstack/watcher-dashboard/+/997960 | 13:12 |
| dviroel | yeah, calling directly the api would allow us to expand out tests | 13:12 |
| dviroel | but in a scenario test i would also expect a service calling it, for integration purpose | 13:13 |
| amoralej | yep, for me that's second step, as we should start with the more open case and we can then run more specific and complex cases | 13:13 |
| sean-k-mooney | as i said we can do both | 13:13 |
| sean-k-mooney | event bases audits are not inteded to only be called by openstack services | 13:14 |
| sean-k-mooney | they can be called by external monitoring systems | 13:14 |
| amoralej | btw, i think in aodh you can register webhooks urls directly in the alarms? | 13:14 |
| sean-k-mooney | wiht or without keystoen auth | 13:14 |
| dviroel | amoralej: yes | 13:14 |
| sean-k-mooney | i.e. using alert manager in promethous or nagios alarms | 13:14 |
| amoralej | i don't know much about that tbh | 13:14 |
| amoralej | i mean we may not need a client, just to configure the alarm | 13:15 |
| sean-k-mooney | amoralej: yep but you can also just have a cron jobs that dues a curl request | 13:15 |
| amoralej | yep | 13:15 |
| sean-k-mooney | the event based triger is not exipclty for aodh | 13:15 |
| sean-k-mooney | so we shoudl test it both ways | 13:15 |
| amoralej | exactly, that's why i think we should start just by calling the webhook directly | 13:16 |
| sean-k-mooney | +1 | 13:16 |
| dviroel | right, but I think that in scenarion test we should have a service validating it | 13:16 |
| sean-k-mooney | no senairo test does not mean only via prject | 13:16 |
| sean-k-mooney | senairo test just mean any test with multiple stages | 13:16 |
| sean-k-mooney | i.e. a non trival api test | 13:17 |
| amoralej | watcher-tempest-plugin should focus on validating watcher behavior | 13:17 |
| sean-k-mooney | if you want to condier the drict call a api test that fine | 13:17 |
| dviroel | that why i said we should | 13:17 |
| dviroel | not must | 13:17 |
| sean-k-mooney | so tempest plugins shoudl be generally frestandign | 13:17 |
| amoralej | adding the aodh layer is nice as integration between aodh and watcher, but as i said next step | 13:17 |
| sean-k-mooney | i.e. not depend on each other | 13:17 |
| opendevreview | David proposed openstack/watcher-dashboard master: Add Playwright tests to validate goals and strategies https://review.opendev.org/c/openstack/watcher-dashboard/+/996067 | 13:17 |
| dviroel | so we know that at least one other service could be integrated with watch | 13:17 |
| sean-k-mooney | so we can add aodh related tests to ours | 13:18 |
| sean-k-mooney | but we need to mage them check the service enabeld config option in tempst | 13:18 |
| sean-k-mooney | and only run if aodh is deployed and enabeld there | 13:18 |
| amoralej | actually, adding it to aodh tempest plugin may be more appropiate :) | 13:18 |
| sean-k-mooney | maybe although im not sure | 13:18 |
| sean-k-mooney | they woudl still have to build a watcher clinet | 13:18 |
| dviroel | so my point is: we can validate that our API works, but would be good to know which other services could be used with watcher | 13:19 |
| dviroel | i prefer to land in our repo, where we control things | 13:19 |
| amoralej | but that may expand to whatever customers may decide. aodh may be a good and valid case, but not the only one | 13:19 |
| dviroel | can be aodh/prometheus | 13:19 |
| dviroel | aodh i know that works :) | 13:20 |
| dviroel | it was the only one that I tested | 13:20 |
| dviroel | i may try with prometheus too as sean-k-mooney mentioned | 13:20 |
| dviroel | that would work too | 13:20 |
| dviroel | i think that we agree then | 13:21 |
| amoralej | but my point is, what if prometheus, i.e. doesn't work for any reason? if the api works as expected, that's our scope | 13:21 |
| sean-k-mooney | for promethious currently you woudl have to disabel keystone auth since i dont think its alarm magner thign can pass a keyton token | 13:21 |
| amoralej | it may be a problem in prometheus or whatever | 13:21 |
| sean-k-mooney | so we woudl have to test that in a diffent job | 13:21 |
| amoralej | i doubt we should validate alertmanater in prometheus tbh | 13:21 |
| dviroel | amoralej: if doesn't work or if is broken doesn't mean that is an issue with watcher | 13:23 |
| dviroel | correct | 13:23 |
| sean-k-mooney | i just mention it because the orgianl intent of event based audit | 13:24 |
| dviroel | yeah | 13:24 |
| sean-k-mooney | was for an external syste to deciend a pre defiend audit shoudl be triggered | 13:24 |
| sean-k-mooney | rhat then using simple cron triggrees | 13:24 |
| sean-k-mooney | in tempst we can emulate that with a direct api call | 13:24 |
| sean-k-mooney | so i htink that enouch for "external triggered" we can special case aodh | 13:25 |
| sean-k-mooney | because that is the native openstack consumer | 13:25 |
| sean-k-mooney | one intersitng usecase long term for this | 13:25 |
| amoralej | what it'd be good is to clearly document how those can be called and requirements (i.e. how to manage auth). And yeah, as aodh is openstack, it deserves special treatment | 13:25 |
| sean-k-mooney | is if we compelt the premetable intnsce work, i belive aodh can triger on a no valid host shcudling failr form nova | 13:26 |
| dviroel | yeah, i've agree with your points | 13:26 |
| sean-k-mooney | so when nova raises no valid host, we coudl use aodh to tell watcher to reap premtable instnaces via an event trigger | 13:26 |
| dviroel | what bothers me is: do we know if that works with aodh? we are not sure... | 13:26 |
| dviroel | maybe a ci test could periodically check that | 13:27 |
| sean-k-mooney | im not agaisnt the aodh senario test just to be clear | 13:27 |
| dviroel | sean-k-mooney: yep | 13:27 |
| sean-k-mooney | i think you can and shoudl write that | 13:27 |
| sean-k-mooney | but if that was broekn it woudl likely be an aodh bug not a watcher one | 13:27 |
| sean-k-mooney | having the direct call test tells use which it is | 13:28 |
| dviroel | yeah, and we would need to skip the test in that case in our ci | 13:28 |
| dviroel | but still worth keeping it I think | 13:28 |
| dviroel | in case that happens | 13:28 |
| sean-k-mooney | i.e. if direct call passes and aodh does not then it an aodh bug, if but break it our bug | 13:28 |
| sean-k-mooney | we can readd aodh to one of our jobs | 13:28 |
| dviroel | and depending on the aodh issue, we may remove the job and call that aodh can't integrate with our api | 13:28 |
| sean-k-mooney | it used to be in the job but we removed it because we were not using it | 13:28 |
| sean-k-mooney | i think you shoudl just wrtie the patch and we can review them | 13:29 |
| dviroel | ack, we can also evaluate that too | 13:29 |
| dviroel | ack | 13:29 |
| dviroel | i think that we agree | 13:29 |
| sean-k-mooney | +1 | 13:30 |
| dviroel | tests directly calling the api | 13:30 |
| dviroel | and an additional test with aodh | 13:30 |
| amoralej | yep | 13:30 |
| dviroel | i may work on that next week, depending on how audit-pipeline goes | 13:30 |
| dviroel | thanks for the feedbacks | 13:30 |
| jgilaber | thanks for bringing this up dviroel, other questions/comments on this topic? | 13:31 |
| dviroel | ++ | 13:32 |
| jgilaber | ok moving to reviews | 13:32 |
| jgilaber | #topic Reviews | 13:32 |
| jgilaber | #link https://etherpad.opendev.org/p/watcher-2026.2-status | 13:33 |
| jgilaber | we've made some progress on the cidnerclient to sdk work, thanks dviroel amoralej ++ for the reviews | 13:33 |
| jgilaber | only two patches left for that series | 13:33 |
| jgilaber | anyone wants to highligth anything else? | 13:33 |
| dviroel | i would ask another reviewer to look at https://review.opendev.org/c/openstack/python-watcherclient/+/993661 | 13:34 |
| dviroel | mayve jgilaber since you reviewed the watcher patch, which merged | 13:34 |
| dviroel | thanks amoralej and jgilaber for reviewing the audit template default_parameters patch | 13:34 |
| jgilaber | sure I'll review it | 13:35 |
| dviroel | other than that, i will submit new patches for audit-pipeline | 13:35 |
| dviroel | there were some changes while testing the audit pipeline handler... | 13:35 |
| dviroel | i should bring them next week | 13:35 |
| amoralej | I'll review the patch adding the db and objects | 13:36 |
| dviroel | thanks amoralej++ | 13:36 |
| dviroel | I saw that Winicius proposed a patch too, i will add to the review etherpad if not there yet | 13:36 |
| dviroel | #link https://review.opendev.org/c/openstack/watcher/+/999202 | 13:36 |
| dviroel | I didn't check that patch yet too | 13:37 |
| dviroel | related to the preemtible instances feature | 13:37 |
| amoralej | i did a high level review and recommended to split in several patches | 13:37 |
| sean-k-mooney | oh cool | 13:37 |
| sean-k-mooney | yes that woudl help with review if nothign else | 13:37 |
| dviroel | thanks amoralej - yeah, we should assist since it is his first feature proposal in watcher | 13:38 |
| amoralej | i have doubts about splitting goal and strategy, but for actions they should be first independen reviews imo | 13:38 |
| sean-k-mooney | i woudl preoably do the appler chagne first to add the actions | 13:38 |
| sean-k-mooney | maybe the planner change next and finally the addtion of the stragy and goal | 13:38 |
| amoralej | which applier change? | 13:38 |
| sean-k-mooney | the applier chagne are adding delete and shelve actions | 13:38 |
| sean-k-mooney | the goal/stragey defintion need to be last because that makes it viabale at the api | 13:39 |
| amoralej | i think that should be after actions are added imo, given that the refer the action name as registered | 13:39 |
| amoralej | yep | 13:39 |
| sean-k-mooney | yep so i think we agree on the split | 13:39 |
| sean-k-mooney | im just looking at the file ist for now but ill try an loop back to this properly later | 13:39 |
| dviroel | news actions could also be used/tested with actuator too, in case we don't merge the new strategy, new action are still valid | 13:40 |
| dviroel | but yeah, we can split the review | 13:40 |
| sean-k-mooney | yes that is true | 13:40 |
| dviroel | also easier to review and update | 13:40 |
| sean-k-mooney | although that would a sperte tempest change | 13:40 |
| dviroel | yeah | 13:40 |
| sean-k-mooney | speakign of we shoudl also wait for the tempest test before merging the sereis | 13:41 |
| sean-k-mooney | but it s a good start | 13:41 |
| amoralej | yep | 13:41 |
| sean-k-mooney | amoralej: i agree with your comment by the way and the docs | 13:41 |
| sean-k-mooney | it also will need a release not in the last commit | 13:41 |
| dviroel | +1 | 13:41 |
| amoralej | yes | 13:41 |
| sean-k-mooney | anyway ill note that in the review | 13:42 |
| jgilaber | anything else from the status etherpad? | 13:42 |
| amoralej | I'd like to mention https://review.opendev.org/c/openstack/watcher/+/993088/ , i finally implemented the cache in the model as suggested by dviroel | 13:43 |
| amoralej | so, i also splitted in two reviews, one for the change in the model and one for the strategy | 13:43 |
| dviroel | yeah, I saw that in yesterday, but I still need to review in details | 13:43 |
| amoralej | sure, np | 13:44 |
| dviroel | thanks amoralej | 13:44 |
| amoralej | in the one for the cache, note https://review.opendev.org/c/openstack/watcher/+/998883/2/watcher/decision_engine/strategy/strategies/base.py | 13:44 |
| amoralej | i'm restarting the cache when creating the strategy. tbh, it should be always be empty at that point as the cache is only populated by calls that are done by the strategies | 13:45 |
| amoralej | but just as a defensive approach | 13:46 |
| dviroel | yeah, iirc the model is copied at every strategy execution, but i had to check that in the code again | 13:46 |
| amoralej | yes, exactly | 13:46 |
| dviroel | but it is good to clear things that we know that should't be reused | 13:46 |
| sean-k-mooney | im not sure that reseting it for every execution fo a stagy will be correct | 13:47 |
| sean-k-mooney | when it comes to the pipelining | 13:47 |
| sean-k-mooney | but for normal audits yes | 13:47 |
| dviroel | in audit-pipeline I will reuse the cluster data model | 13:47 |
| sean-k-mooney | the point of caching in the pipeline is that the metric for a give hsot for the rist stragy shoudl be ruesed for the second | 13:47 |
| dviroel | but the strategy neves starts with a model == None in my case | 13:48 |
| sean-k-mooney | i.e. the subsequet step in the pipelien soudl not get new raw values | 13:48 |
| dviroel | yeah, I need to carefully check that | 13:48 |
| sean-k-mooney | if there are cased valeus | 13:48 |
| sean-k-mooney | ack | 13:48 |
| amoralej | even in that case, this cache should be correct | 13:48 |
| sean-k-mooney | ok well we can revist when lookign at the pipeline part | 13:49 |
| dviroel | yes | 13:49 |
| amoralej | as i understand the operations on the model are done with the model methods migrate_instance, etc... | 13:49 |
| dviroel | amoralej: correct | 13:49 |
| amoralej | anyway, note that this cache is only for the used resouces by nodes | 13:49 |
| dviroel | the simulation uses cdm migrate_instance | 13:49 |
| amoralej | it can be cleaned and automatically recalculated at any time | 13:50 |
| dviroel | amoralej: yes, i have a different cache for metrics | 13:50 |
| amoralej | it's only performance | 13:50 |
| amoralej | yep | 13:50 |
| dviroel | +1 | 13:50 |
| jgilaber | ok to move to next review? | 13:50 |
| dviroel | jgilaber: yes | 13:50 |
| jgilaber | I have a quick one | 13:51 |
| dviroel | do we can cover the following ones | 13:51 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher/+/999241 | 13:51 |
| jgilaber | it changes the default storage engine for mysql | 13:51 |
| sean-k-mooney | i was wonderign if we shoudl be more explcity about hte default CHARACTER SET as well but tis proably fine | 13:52 |
| dviroel | just checking that https://codesearch.opendev.org/?q=default_storage_engine%3DMYISAM&i=nope&literal=nope&files=&excludeFiles=&repos= | 13:52 |
| dviroel | which seems the default in many other projects | 13:53 |
| sean-k-mooney | MYSIAM technially does not supprot atopic tranasctions | 13:53 |
| sean-k-mooney | so it not somethign that we would ever use in proudction for watcher | 13:53 |
| jgilaber | it looks like the script was copied from project to project | 13:53 |
| jgilaber | we actually noticed when I copied the script to cyborg | 13:54 |
| sean-k-mooney | MYISAM is faster as a result but InnoDB is what we shoudl be using | 13:54 |
| dviroel | yeah, just wondering why other projects didn't migrate | 13:54 |
| sean-k-mooney | it works so we just never got aroudn to changing it | 13:55 |
| amoralej | i though there may be something is oslo-db or something forcing to use innodb tbh | 13:55 |
| sean-k-mooney | there is no reason beyond that | 13:55 |
| dviroel | ok | 13:55 |
| sean-k-mooney | amoralej: i dont think so but i think we can spcyfiy it vai alembic | 13:55 |
| amoralej | https://github.com/openstack/watcher/blob/a1b113ce54a5c4c1dae794488227af11e8e4499f/watcher/conf/db.py#L32-L34 ? | 13:55 |
| sean-k-mooney | anyway this is a harmle test only patch, i asked jgilaber ot use InnoDB for cybrog when teim suggeted it | 13:56 |
| amoralej | yep | 13:56 |
| jgilaber | thank folks, let's continue discussing in the patch so we can finish with the agenda | 13:56 |
| sean-k-mooney | yes | 13:56 |
| dviroel | ack, jgilaber sure | 13:56 |
| sean-k-mooney | that will default it for use too | 13:56 |
| jgilaber | last review patch is from morenod | 13:56 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher-dashboard/+/996067 | 13:57 |
| morenod | this is just to verify that we see the same information on horizon than api for goals and strategies | 13:57 |
| dviroel | i was reviewing this one atm | 13:57 |
| dviroel | checking the snapshots and so | 13:57 |
| dviroel | i will vote soon in this one | 13:58 |
| dviroel | thanks morenod | 13:58 |
| morenod | thanks dviroel | 13:58 |
| jgilaber | we have one bug to triage, do we want to try quickly or leave it for next week? | 13:58 |
| jgilaber | I think we can leave it for next week, since we're out of time | 13:59 |
| jgilaber | last thing | 13:59 |
| jgilaber | #topic Volunteers to chair next week | 13:59 |
| jgilaber | any volunteer? I'll be out next week | 13:59 |
| dviroel | I can | 13:59 |
| jgilaber | thanks dviroel ++ | 14:00 |
| jgilaber | and thanks all for joining! | 14:00 |
| jgilaber | #endmeeting | 14:00 |
| opendevmeet | Meeting ended Thu Jul 30 14:00:26 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:00 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/watcher/2026/watcher.2026-07-30-13.01.html | 14:00 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/watcher/2026/watcher.2026-07-30-13.01.txt | 14:00 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/watcher/2026/watcher.2026-07-30-13.01.log.html | 14:00 |
| dviroel | thank jgilaber++ | 14:01 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Fix race condition in DefaultLoader._reload_config() https://review.opendev.org/c/openstack/watcher/+/994845 | 15:50 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add functional test framework for Watcher https://review.opendev.org/c/openstack/watcher/+/988389 | 15:50 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add Nova/Placement emulators and host_maintenance functional tests https://review.opendev.org/c/openstack/watcher/+/993352 | 15:50 |
| amoralej | dviroel, sean-k-mooney ^ i've sent new PSs with your feedback ^ I will not update the prometheus and cinder ones until you approve the approach | 15:53 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add functional test framework for Watcher https://review.opendev.org/c/openstack/watcher/+/988389 | 16:11 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add Nova/Placement emulators and host_maintenance functional tests https://review.opendev.org/c/openstack/watcher/+/993352 | 16:11 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add Nova/Placement emulators and host_maintenance functional tests https://review.opendev.org/c/openstack/watcher/+/993352 | 16:35 |
| dviroel | ack amoralej | 16:47 |
| opendevreview | Merged openstack/watcher-dashboard master: Add Playwright tests to validate goals and strategies https://review.opendev.org/c/openstack/watcher-dashboard/+/996067 | 17:11 |
| opendevreview | Merged openstack/python-watcherclient master: Add default_parameters support to Audit Template https://review.opendev.org/c/openstack/python-watcherclient/+/993661 | 17:50 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!