| corvus | clarkb, fungi: i'd like to restart zuul now | 00:01 |
|---|---|---|
| clarkb | corvus: ok I'm around | 00:02 |
| corvus | looks like image promotion is good | 00:02 |
| corvus | i don't see any release activity | 00:02 |
| clarkb | I don't see any openstack release activity | 00:02 |
| clarkb | yup | 00:02 |
| corvus | running the pull playbook now | 00:03 |
| corvus | seems to actually be doing something :) | 00:04 |
| clarkb | ya the pull runs much longer when it has to download images :) | 00:04 |
| corvus | okay, ready. i'm restarting now | 00:04 |
| clarkb | rgr | 00:05 |
| corvus | #status log restarted all of zuul on commit d87a9a8b8f00887c37bb382271181cb5d703ba3f | 00:07 |
| opendevstatus | corvus: finished logging | 00:07 |
| fungi | that was quick! | 00:09 |
| fungi | i'm around to help keep an eye on things post-restart at least | 00:09 |
| clarkb | I don't think it has loaded configs yet | 00:09 |
| corvus | yeah, was just filling the dead time with status update :) | 00:09 |
| corvus | it's doing cat jobs for openstack now. | 00:11 |
| corvus | is openstack keeping branches open for longer? | 00:12 |
| clarkb | corvus: sort of. The extended mainteance of branches has allowed branches to stick around | 00:13 |
| corvus | was just thinking this is taking longer than it used to, and i saw the word "ocata" a lot. | 00:13 |
| clarkb | but more recently we gave openstack release managers permission to delete those branches as they get really bitrotted and they have done that for some projects | 00:13 |
| fungi | more than intended. elodilles has been working on eol'ing and deleting branches which have fallen out of extended maintenance | 00:13 |
| corvus | gotcha | 00:14 |
| fungi | and yeah, now that it's accessible via a specific access-controlled api call, we've granted that to the openstack release managers over just the openstack/ namespace projects | 00:14 |
| fungi | and removed their previous global extra access to some stuff too | 00:15 |
| fungi | so now they're no longer special in our global config | 00:15 |
| TheJulia | is zuul down? | 00:16 |
| clarkb | TheJulia: yes see above | 00:16 |
| corvus | well, technically no | 00:16 |
| clarkb | it is being restarted to fix a bug where jobs could lose their node assignments causing them to hang out forever | 00:16 |
| corvus | but it's not running jobs right now :) | 00:16 |
| clarkb | ah fair enough it is working through loading its config so not fully operational | 00:16 |
| TheJulia | fun | 00:17 |
| clarkb | I see the openstack tenant in the list now | 00:17 |
| corvus | it's queing; total perceived event outage time should be like 10 seconds | 00:18 |
| corvus | and re-enqueue is running now | 00:18 |
| clarkb | at least one job has started | 00:19 |
| corvus | re-enqueue done | 00:22 |
| corvus | TheJulia: should be gtg now | 00:22 |
| TheJulia | corvus: thanks | 00:40 |
| Clark[m] | ianw: did you have an opinion on planning/scheduling for the Gerrit restart and mariadb jdbc connector flip? | 01:32 |
| Clark[m] | I think we want to land the first one and make sure the images promote. Then we can land the second. Then we do a pull and restart? I can work on getting changes landed tomorrow then we do the restart towards the end of my day maybe? | 01:33 |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 02:17 |
| ianw | Clark[m]: yeah nothing in particular. I guess we know what to look for and can switch back quickly if issues | 03:21 |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 03:24 |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 03:57 |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 04:25 |
| *** ykarel|away is now known as ykarel | 04:47 | |
| *** marios is now known as marios|ruck | 06:31 | |
| *** jpena|off is now known as jpena | 07:08 | |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 07:26 |
| *** rpittau|afk is now known as rpittau | 07:47 | |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 08:20 |
| *** diablo_rojo is now known as Guest3371 | 08:27 | |
| opendevreview | Merged opendev/yaml2ical master: Update hacking version https://review.opendev.org/c/opendev/yaml2ical/+/800944 | 09:16 |
| opendevreview | Merged opendev/yaml2ical master: Report which week a meeting occurs. https://review.opendev.org/c/opendev/yaml2ical/+/799691 | 09:20 |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 09:35 |
| opendevreview | xinliang proposed zuul/zuul-jobs master: Fix install podman error on Ubuntu aarch64 Bionic https://review.opendev.org/c/zuul/zuul-jobs/+/803413 | 10:40 |
| *** dviroel|out is now known as dviroel | 11:25 | |
| *** jpena is now known as jpena|lunch | 11:37 | |
| iskull | hi all, | 11:46 |
| iskull | I am new to openstack and need to setup 6 server infrastructure that can be scaled out | 11:47 |
| iskull | any one who can share some good documentation or guide | 11:47 |
| iskull | also want to use SAN storage for the Block storage and VM root volume | 11:47 |
| iskull | have checked with Charmed OpenStack team they require 12 servers and I have to go with 6 due to limitation of hardware | 11:49 |
| *** Guest3371 is now known as diablo_rojo | 11:56 | |
| *** jpena|lunch is now known as jpena | 12:39 | |
| clarkb | fungi: did you want to review https://review.opendev.org/c/opendev/system-config/+/803372 before I approve it? That will rebuild our gerrit images which ensures we have the openid fix and the mariadb fix allowing us to switch over to mariadb jdbc connector (the child change does that) | 14:38 |
| clarkb | fungi: I'll plan to approve that change after some breakfast probably with intention of restarting gerrit later today or tomorrow depending on how timing goes | 14:38 |
| clarkb | looks like the zanata user with trouble logging in has requested that we delete their account. I'll give that a go after breakfast too (I've never done it before so who knows what will happne :) ) | 14:42 |
| fungi | i'll take a look thanks | 14:44 |
| fungi | clarkb: you refreshed some plugin versions for 3.3, i guess there were no plugin updates for 3.2? | 14:47 |
| fungi | the commit message doesn't really mention them either way | 14:47 |
| clarkb | fungi: we do that periodically beacuse we track the stable-3.3 branch for gerrit to build our 3.3 images. If plugins get new versions we want to update to match | 14:49 |
| clarkb | I don't know if they got new features or bug fixes though it is technically possibly | 14:49 |
| clarkb | basically we don't want to build a new 3.3 image on new gerrit with old plugins | 14:50 |
| mordred | I have recently discovered the Storm Weather Shanty Choir. In case anyone is in need of a good Shanty | 14:51 |
| fungi | i'll look them up. right now i only have the port isaac's fisherman's friends collection | 14:52 |
| fungi | which are great, but feel over-produced at times | 14:52 |
| clarkb | fungi: if every plugin maintained a stable branch or did reliable release tags this issue would go away. As is its a mix of behaviors and we try to keep up with periodic updates | 14:52 |
| fungi | clarkb: yeah, i get why we update the plugins when rebuilding, just wondering at the lack of 3.2 plugin updates... but maybe that's to be expected | 14:52 |
| clarkb | I think mordred and corvus worked out some sort of system for doing upstream CI testing that we could try and incorporate ourselves | 14:53 |
| *** ykarel is now known as ykarel|away | 14:53 | |
| clarkb | fungi: 3.2 hasn't had a release since the last time we updated its list | 14:53 |
| clarkb | fungi: we are already running the most up to date versions of plugins there | 14:53 |
| fungi | also is there a script for finding those latest versions, or are you reviewing them all and adding them manually? | 14:53 |
| clarkb | fungi: I scan the https://gerrit.googlesource.com/ repos manually as sometimes new stable branches show up or plugin stable releases don't get new tags | 14:54 |
| fungi | ahh, okay. thanks | 14:54 |
| mordred | Yeah, it would be easier if they always branched plugins | 14:56 |
| mordred | But they don't. Yeah though, i do think there might be opportunity to port in the upstream stuff. I can't remember what's where tbh | 14:56 |
| corvus | related, they want to start building the actual releases with zuul, so we need to set up a post pipeline for them. as part of that we can make a json file with build/branch/version info. that might help us downstream? | 14:57 |
| clarkb | corvus: oh ya I bet we could reconsume that to produce our images using the same refs | 14:58 |
| clarkb | then branches and tags don't really matter as much | 14:58 |
| corvus | that's a volunteer activity i haven't had time to start on yet, in case any one wants to pitch in on it with me | 14:58 |
| * mordred can try to help on that some next week | 14:59 | |
| corvus | kk thx | 15:00 |
| clarkb | I think the stable-3.4 merge up change is going to land soon if it hasn't landed already which is good | 15:01 |
| corvus | it is raining here. oO | 15:01 |
| clarkb | we aren't using 3.3 or 3.4 but it feels weird to deploy on 3.2 with fixes that may not be on the later branches then you regress when you upgrade | 15:01 |
| clarkb | #status Log A user account has been deleted from Zanata in an effort to fix the issue described in http://lists.openstack.org/pipermail/openstack-discuss/2021-August/023972.html | 15:02 |
| opendevstatus | clarkb: finished logging | 15:02 |
| clarkb | Looks like we will get a gitea 1.14.6 before 1.15.0 | 15:08 |
| clarkb | the changelog update for 1.14.6 was approved half an hour ago. | 15:08 |
| clarkb | I don't expect any issues doing that update though | 15:09 |
| clarkb | yay for getting the non 1.15.0 fixes landed separately | 15:09 |
| *** jpena is now known as jpena|off | 15:15 | |
| clarkb | https://gerrit-review.googlesource.com/c/gerrit/+/313842/ is the 3.4 upstream merge | 15:19 |
| opendevreview | Merged opendev/system-config master: Rebuild gerrit images to include fixes https://review.opendev.org/c/opendev/system-config/+/803372 | 15:53 |
| clarkb | now if the deploy jobs run successfully for that I can land the next chagne | 15:53 |
| clarkb | https://review.opendev.org/c/opendev/system-config/+/803373 is the next change if you want to review that one too fungi | 15:54 |
| clarkb | its WIP because I don't want it landing until we're sure images have the bug fix | 15:54 |
| fungi | sure | 15:55 |
| clarkb | https://zuul.opendev.org/t/openstack/build/b467c31396884344a71d2e4aac27be4f did succeed so I think we're good there | 15:56 |
| clarkb | I;m about to pop out for some exercise but can work on landing the mariadb config chagne when I get back. I was thinking that it would be good to restart with ianw around in case we have to debug that further, but I guess we can always ninja revert to the mysql jdbc connector and try again later if we have to | 15:57 |
| clarkb | mordred: ^ not sure if you followed that mariadb thing at all but gerrit orm code has conditions to catch duplicate entry errors from the database and ignore them rather than doing an insert .. on duplicate key update type deal | 16:00 |
| clarkb | debugging that was fun | 16:00 |
| *** marios|ruck is now known as marios|out | 16:08 | |
| *** sshnaidm is now known as sshnaidm|afk | 16:29 | |
| *** rpittau is now known as rpittau|afk | 16:41 | |
| clarkb | I have approved https://review.opendev.org/c/opendev/system-config/+/803373/1 to switch us back to the mariadb connector | 17:47 |
| clarkb | the mysql connector is still on the images if we need to roll back again | 17:48 |
| clarkb | and now I promised to do more zuul reviews /me finds a comfy spot to do that | 17:48 |
| fungi | Subject: Element Matrix Services - Your trial is about to expire | 18:39 |
| fungi | just received that today at our infra-root address | 18:39 |
| mordred | clarkb: gross | 18:44 |
| opendevreview | Merged opendev/system-config master: Use the mariadb jdbc connector in gerrit https://review.opendev.org/c/opendev/system-config/+/803373 | 18:44 |
| clarkb | fungi: I think that is expected they gave us a month free but there should be billing info in there. Was there any indication we'll be cancelled or shut down? | 18:45 |
| fungi | i'll re-read it | 18:46 |
| clarkb | fungi: probably what we want to do is confirm with jbryce after billing happens that all looked well on his side | 18:46 |
| clarkb | fungi: looks like its just a heads up and no indication we'll be turned off | 18:46 |
| fungi | yeah, it's just a request for feedback, looks like | 18:46 |
| fungi | i'm going to try to get a bit of yardwork done over the next hour or so, but will try to keep an eye on the computer when i stop to catch my breath | 18:49 |
| fungi | 803373 has deployed, what was the plan for restarting gerrit? | 19:13 |
| Clark[m] | We need to pull the new image and then up -d. I'm eating lunch but can start that after. In say an hour | 19:15 |
| Clark[m] | Then we test toggling the reviewed flag functions | 19:16 |
| fungi | wfm, i'm caught up on yardwork now and just enjoying the breeze on the patio, so free to help whenever | 19:50 |
| fungi | though in an hour-ish i'll likely start to be distracted by dinner prep | 19:50 |
| fungi | i guess i can docker-compose pull on review.o.o while the service is up, right? so we only need to down/up when the time comes | 19:51 |
| clarkb | yes | 19:56 |
| clarkb | fungi: did you want to do the docker-compose pull? I just need a few more minutes | 19:57 |
| fungi | yep, on it now | 19:59 |
| fungi | all three images have been successfully pulled | 20:00 |
| fungi | opendevorg/gerrit 3.2 f991f86f3054 5 hours ago 811MB | 20:00 |
| clarkb | ok I'm at the keybaord properly now after lunch. I'm ready if you are | 20:00 |
| clarkb | the openstack gate queues are empty and I see no release jobs that might be sad if talking to gerrit while it is down | 20:01 |
| clarkb | 9d0dcdc652a2 is the current image | 20:01 |
| fungi | status notice The Gerrit service on review.opendev.org is going down for a quick restart to adjust its database connection configuration, and should return to service momentarily | 20:01 |
| clarkb | fungi: lgtm | 20:01 |
| fungi | does that wording look okay? | 20:01 |
| fungi | thanks | 20:01 |
| fungi | #status notice The Gerrit service on review.opendev.org is going down for a quick restart to adjust its database connection configuration, and should return to service momentarily | 20:01 |
| opendevstatus | fungi: sending notice | 20:01 |
| -opendevstatus- NOTICE: The Gerrit service on review.opendev.org is going down for a quick restart to adjust its database connection configuration, and should return to service momentarily | 20:02 | |
| fungi | once that's hit most channels i'll down and up -d | 20:02 |
| clarkb | and ya if you do a docker-compose up -d it should udpate the gerrit container because it has a new image | 20:02 |
| fungi | okay, i think the status notice is done, it doesn't seem to let you know when it finishes other than logging, can't recall if it ever did | 20:05 |
| fungi | ready? | 20:05 |
| clarkb | I'm ready | 20:05 |
| fungi | down it goes | 20:05 |
| fungi | and starting again | 20:05 |
| fungi | webui is up again | 20:07 |
| clarkb | I am able to pull up a file I have previously marked reviewed and it doesn't throw an error at me | 20:07 |
| clarkb | If I mark it unreviewed and then open it again it successfully transitions to reviewed | 20:07 |
| clarkb | That was what threw errors previously with the mariadb connector so that looks good | 20:08 |
| fungi | i can't remember how to toggle or view the reviewed state for a file | 20:08 |
| clarkb | fungi: its on the top level page of the change it shows "Reviewed" on the lines for each file. If you move your mouse to the right of that you get a mark unrevewied button | 20:08 |
| clarkb | I'm going to log out and log back in again just to be sure openid is working as expected as well since that got some updates too (I tested that on the test machien but can't be too careful) | 20:09 |
| clarkb | I will not be testing my ability to delete my openid's email address because if that doesn't stop me like it should I'll have a bad afternoon | 20:09 |
| fungi | interesting, it doesn't show that state on the file list for me | 20:09 |
| fungi | oh! | 20:09 |
| fungi | i'm not logged in ;) | 20:09 |
| clarkb | that will do it :) | 20:10 |
| clarkb | fwiw logigng out and back in again worked great for me | 20:10 |
| fungi | yeah, seems to be nonbroken | 20:11 |
| clarkb | this all looks good to me | 20:12 |
| clarkb | I guess early next week we can land the change to remove the mysql connector from the image if nothing pops up | 20:12 |
| clarkb | https://review.opendev.org/c/opendev/system-config/+/803374 that chagne | 20:13 |
| fungi | yup, i'll go ahead and review that now while it's fresh in my mind | 20:13 |
| clarkb | There is a gitea 1.14.6 I'll get a change up for that shortly (need to double check the diffs) | 20:14 |
| fungi | clarkb: on 803374 the code comment above it still says mysql-connector | 20:15 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update to gitea v1.14.6 https://review.opendev.org/c/opendev/system-config/+/803650 | 20:18 |
| clarkb | fungi: ya our role supported both for review01 and review02 | 20:19 |
| clarkb | fungi: there is further cleanup we can probably do | 20:19 |
| clarkb | infra-root I think we should probably try and upgrade to 803650 as the changelog indicates a very minimal delta | 20:19 |
| clarkb | fungi: note I don't think it is a comment? the comment was removed as part of teh switch back to mariadb. But there is the ability to select mysql in the current role because it had to run against the old and new server | 20:20 |
| fungi | clarkb: the comment i marked says "Download mysql-connector so that gerrit doens't download it during init." and we're no longer downloading that after your change merges | 20:21 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update gitea project creation to only use the REST API https://review.opendev.org/c/opendev/system-config/+/803367 | 20:22 |
| opendevreview | Clark Boylan proposed opendev/system-config master: WIP Prepare gitea 1.15.0 upgrade https://review.opendev.org/c/opendev/system-config/+/803231 | 20:22 |
| clarkb | fungi: oh sorry I was looking at the chagne that merged not the one that is still open for review | 20:22 |
| clarkb | fungi: ya I can update that. We might also want to start removing the mysql support from the role in the same change. Why don't I check with ianw to see what he thinks about that and I can push a new patchset one way or another | 20:23 |
| fungi | wfm, this is entirely non-urgent cleanup after all | 20:23 |
| clarkb | I'd like to approve 803650 now to upgrade gitea. Any objections? | 21:43 |
| clarkb | I've approved it. You have however long it takes to gate to stop it :) | 21:57 |
| ianw | clarkb: thanks for running upgrade, looks like things are working | 22:10 |
| clarkb | ianw: yup everything seems happy. Though I did not intentionally try to break my account and see if gerrit stops me | 22:10 |
| clarkb | fungi: ^ you might be willing to test that with your testing account though? | 22:10 |
| clarkb | ianw: on https://review.opendev.org/c/opendev/system-config/+/803374 do you want me to maybe try and pulling out the mariadb vs mysql support in the ansible role there too? | 22:11 |
| clarkb | ianw: and we can land a cleanup for all that stuff in one go? | 22:11 |
| clarkb | note I haven't done the ssh known hosts change yet because I wanted to get the restart done, but at this point I think I'll have to do that one tomorrow (but it is on my todo list0 | 22:12 |
| ianw | clarkb: i feel like i might have removed the mysql bits when review01 was removed? | 22:13 |
| clarkb | ianw: they are still in the secure.conf file at least. See the diff in that chagne (at the top of the file) | 22:13 |
| clarkb | ya gerrit_reviewdb_database_type is still in the role and everything. | 22:14 |
| ianw | but yeah, i'd support any cleanup there; as elon says, the best part is no part :) | 22:14 |
| clarkb | alright I'll do that update in that chagne and we can land it next week after we're happy mysql connector bits won't be necessary | 22:14 |
| clarkb | er I guess it is the diff of the parent change | 22:15 |
| ianw | i've got a few quick ones hanging around if people have some time | 22:17 |
| ianw | https://review.opendev.org/c/opendev/system-config/+/802979 was from the other day when hound wasn't indexing zuul | 22:17 |
| ianw | https://review.opendev.org/c/opendev/system-config/+/802980 tests openafs setup on centos-8-stream | 22:17 |
| ianw | (for wheels) | 22:17 |
| ianw | https://review.opendev.org/c/opendev/system-config/+/803418 blocks a paste endpoint that is giving errors | 22:18 |
| ianw | https://review.opendev.org/c/opendev/system-config/+/803417 was changes i used to run the prune on backup02, which just makes it automatically log the output | 22:18 |
| clarkb | ianw: do we use h2 in testing or mariadb? | 22:19 |
| clarkb | we must use mariadb because it is set as a host var? | 22:19 |
| ianw | we should be using mariadb | 22:19 |
| ianw | fungi: other ones of interest are https://review.opendev.org/c/openstack/openstack-ansible/+/803404 and 803405 which removes debian-stable from openstack-ansible. so far i haven't been able to get reviews on them, and additionally stable/train is broken for other reasons | 22:21 |
| ianw | not sure what to do here. force merge might be an option as we already merged https://review.opendev.org/c/openstack/openstack-ansible-tests/+/803127 | 22:23 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Remove the mysql support from our gerrit role and image https://review.opendev.org/c/opendev/system-config/+/803374 | 22:24 |
| clarkb | bah I forgot to make the update fuingi suggested | 22:24 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Remove the mysql support from our gerrit role and image https://review.opendev.org/c/opendev/system-config/+/803374 | 22:25 |
| clarkb | ianw: ^ can you check my note in there about tesing (the dockerfile comment says we predownload so that gerrit.war init doesn't do it for us but now I wonder if gerrit assumes mysql or if it will properly see mariadb is there) | 22:26 |
| ianw | i actually think they removed the auto-downloading .jar bits ... | 22:29 |
| ianw | i have a vague memory of finding that change when setting this up, let me dig | 22:30 |
| clarkb | ianw: I have approved the paste change | 22:31 |
| ianw | clarkb: https://gerrit-review.googlesource.com/c/gerrit/+/250498 is what i was thinking of | 22:37 |
| clarkb | from the commit message I'm not entirely sure what the effect of that is. but it does remove library management from init things | 22:38 |
| ianw | oh another fairly easy one is https://review.opendev.org/c/openstack/project-config/+/803411 which adds the publish jobs for centos-8-strem wheels. i've already created /mounted the volumes | 22:41 |
| fungi | clarkb: sorry, post-dinner fog... test what with my testing account? | 22:48 |
| clarkb | fungi: attempting to delete the email address associated with that accounts openid | 22:55 |
| clarkb | fungi: previously gerrit allowed you to od this and the account would be orphaned. But now it should return an error and not do it. However, there are a cuple of other gotchas including: you cannot delete your preferred email address so you need to add a second email address and make it preferred then try the delete | 22:56 |
| clarkb | fungi: I'm happy to walk you through it tomorrow if interested via a screenshare | 22:56 |
| fungi | yeah, i'm happy to test that tomorrow. for tonight i'm no longer in immediate vicinity of a machine which can easily authenticate to that account | 22:57 |
| clarkb | no worries | 23:00 |
| clarkb | the gitea upgrade change has about 13 minutes left according to zuul and then it has to wait behind the hourly deploy jobs | 23:03 |
| opendevreview | Merged opendev/system-config master: Update to gitea v1.14.6 https://review.opendev.org/c/opendev/system-config/+/803650 | 23:13 |
| opendevreview | Merged opendev/system-config master: lodgeit: disable getRecent API endpoint https://review.opendev.org/c/opendev/system-config/+/803418 | 23:13 |
| clarkb | I'm going to need to move some furniture around (new couch tomorrow) but I'll keep an eye on the gitea upgrade | 23:14 |
| fungi | congrats on achieving new seating | 23:26 |
| clarkb | there is quite the production and shipping backlog on things like that right now | 23:31 |
| clarkb | we bought it months ago | 23:31 |
| fungi | indeed | 23:36 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!