| @udayendu.kar:matrix.org | Hi, I am having some issue logging into https://review.opendev.org/ . I can get into ubuntu one account and reaching to the page where its sharing the personal data like name and email to https://review.opendev.org/ but after clicking 'yes,log me' in button, its giving 'Not Found' in https://review.opendev.org/SignInFailure,SIGN_IN,Contact+site+administrator . Any suggestion ? | 01:16 |
|---|---|---|
| -@gerrit:opendev.org- OpenStack Proposal Bot proposed: [openstack/project-config] 994582: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/994582 | 02:11 | |
| @harbott.osism.tech:regio.chat | Udayendu: there may be an issue if you used the same email address with another gerrit account earlier. I am seeing an account matching what looks to be your email address based on your matrix ID, which was created in 2017. if you no longer have access to the same ubuntu one account that this old gerrit account was created with, I think we will have to deactivate the old account, but I'll defer that action to some more experienced gerrit admins who will hopefully be around later today | 07:52 |
| @udayendu.kar:matrix.org | [Jens Harbott](https://matrix.to/#/@harbott.osism.tech:regio.chat) I dont have access to the old account. So please deactivate that account. | 09:50 |
| @udayendu.kar:matrix.org | uskar88 - this is my latest id and I am using it for all my work now. | 09:52 |
| @udayendu.kar:matrix.org | Kindly ping me once the changes are done, so that I can start using the new account. | 09:52 |
| @harbott.osism.tech:regio.chat | infra-root I'm approving https://review.opendev.org/c/opendev/base-jobs/+/994048 now and will watch for results in case this goes wrong | 10:53 |
| -@gerrit:opendev.org- Zuul merged on behalf of Dr. Jens Harbott: [opendev/base-jobs] 994048: Revert "Disable job log uploads to ovh swift" https://review.opendev.org/c/opendev/base-jobs/+/994048 | 10:58 | |
| -@gerrit:opendev.org- Dr. Jens Harbott proposed: [openstack/project-config] 994689: Drop config for retired Sunbeam single charm repositories https://review.opendev.org/c/openstack/project-config/+/994689 | 12:20 | |
| @harbott.osism.tech:regio.chat | not sure why zuul didn't throw an error on checking the parent for ^^ | 12:21 |
| -@gerrit:opendev.org- Brian Rosmaita proposed: [openstack/project-config] 993184: Retire cinderlib https://review.opendev.org/c/openstack/project-config/+/993184 | 12:44 | |
| -@gerrit:opendev.org- Brian Rosmaita proposed: [openstack/project-config] 993184: Retire cinderlib https://review.opendev.org/c/openstack/project-config/+/993184 | 12:50 | |
| @harbott.osism.tech:regio.chat | looks like gerrit is having a hard time once again ... | 13:30 |
| @fungicide:matrix.org | load average is low, apache has plenty of open worker slots | 13:32 |
| @fungicide:matrix.org | webui is loading quickly for me | 13:32 |
| @fungicide:matrix.org | Jens Harbott: did it come back for you, or is it maybe something network-path-related? | 13:33 |
| @harbott.osism.tech:regio.chat | yeah, just did take very long | 13:34 |
| @fungicide:matrix.org | i'll try to keep a closer eye on it | 13:35 |
| @fungicide:matrix.org | thanks for the heads up | 13:35 |
| @harbott.osism.tech:regio.chat | maybe it was just all the ppl in the kolla meeting looking at the same big patch that was mentioned all at the same time | 13:35 |
| @fungicide:matrix.org | meeting-induced thundering herd | 13:36 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [zuul/zuul-jobs] 990850: Add resolute and trixie to zuul-jobs test matrix https://review.opendev.org/c/zuul/zuul-jobs/+/990850 | 14:19 | |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [zuul/zuul-jobs] 994751: Remove use_buildset_registry argument from ensure-docker https://review.opendev.org/c/zuul/zuul-jobs/+/994751 | 14:23 | |
| @fungicide:matrix.org | okay, i just caught apache on review03 with almost all of its worker slots busy in the "reading request" state again, so it's getting overwhelmed (or at least whelmed) | 14:29 |
| @clarkb:matrix.org | Jens Harbott: Udayendu I see the error messages in the gerrit log which I think gives me sufficient info to process the old account deactivation. I'll work on that after breakfast | 14:37 |
| @clarkb:matrix.org | fungi: Jens Harbott I think we probably can increase the number of apache slots slightly but Gerrit has its own thread limits so we may find that doesn't help dramatically | 14:38 |
| @clarkb:matrix.org | if anyone is interested in following along on the gerrit account cleanup I'm happy to try and make that work. Alternatively I've captured logs for this work in the past in my gerrit server home dir as an informal way of capturing the process. I'll put a new log file in there for this one too when I'm done | 14:43 |
| @clarkb:matrix.org | ok similar to the last cleanup I did the conflict isn't with the openid external id only with an additional email address. But that additional email address is set to the preferred email on the conflicting account. Given this I'm going to proceed with a full account disablement and removal of that preferred email address | 15:01 |
| @fungicide:matrix.org | i'll note that it's not just gerrit getting hammered harder today, i've been unable to get the wiki to load at all. looking into it now | 15:10 |
| @clarkb:matrix.org | Udayendu: ok I have retired the old account and removed associated email address records from it. If you login now I would expect it to succeed and create a new account for you | 15:16 |
| @clarkb:matrix.org | I've stuck the logs/notes for this in my homedir next to the others | 15:19 |
| @udayendu.kar:matrix.org | sure, let me try | 15:31 |
| @udayendu.kar:matrix.org | I am able to login. Thanks [Clark](https://matrix.to/#/@clarkb:matrix.org) for the fix. | 15:34 |
| -@gerrit:opendev.org- melanie witt proposed wip: [zuul/zuul-jobs] 994521: Add optional IPv6 address support to multi-node-bridge https://review.opendev.org/c/zuul/zuul-jobs/+/994521 | 15:53 | |
| @fungicide:matrix.org | i've manually synced a copy of our `ua-filter.conf` contents into the vhost config on the wiki server | 15:55 |
| @mordred:waterwanders.com | super random question I should know the answer to - dashboards that are defined in All-Projects - do we manage those in git somewhere? | 15:55 |
| @mordred:waterwanders.com | not very important, I just noticed that my "Important" dashboard which I haven't loaded in a while throws: ``` | 15:57 |
| Error 400 (Bad Request): Unrecognized value: draft | ||
| Endpoint: /changes/ | ||
| Trace Id: 1782316590613-9644cd4e | ||
| ``` | ||
| and the dashboard is defined in my list as ```/projects/All-Projects,dashboards/important-changes:important-changes-dashboard``` | ||
| @mordred:waterwanders.com | * not very important, I just noticed that my "Important" dashboard which I haven't loaded in a while throws: | 15:57 |
| ``` | ||
| Error 400 (Bad Request): Unrecognized value: draft | ||
| Endpoint: /changes/ | ||
| Trace Id: 1782316590613-9644cd4e | ||
| ``` | ||
| and the dashboard is defined in my list as ```/projects/All-Projects,dashboards/important-changes:important-changes-dashboard``` | ||
| ```` | ||
| @mordred:waterwanders.com | * not very important, I just noticed that my "Important" dashboard which I haven't loaded in a while throws: | 15:57 |
| ``` | ||
| Error 400 (Bad Request): Unrecognized value: draft | ||
| Endpoint: /changes/ | ||
| Trace Id: 1782316590613-9644cd4e | ||
| ``` | ||
| and the dashboard is defined in my list as | ||
| ``` | ||
| /projects/All-Projects,dashboards/important-changes:important-changes-dashboard | ||
| ``` | ||
| @mordred:waterwanders.com | so I thought I'd nerd-snipe myself into trying to fix that before I just delete it :) | 15:58 |
| @clarkb:matrix.org | I think you configure those through your settings it's the same place where you select if you want to open in a new tab by default | 15:58 |
| @clarkb:matrix.org | But I'm not positive as I don't use custom dashboards | 15:59 |
| @clarkb:matrix.org | And yes drafts went away forever ago | 15:59 |
| @mordred:waterwanders.com | yeah. it's the dashboard in my custom list referencing /projects/All-Projects,dashboards made me think "there's apparently some dashboards defined centrally that are out of date" - the internet says they're in refs/meta/dashboards which I don't seem to have access to clone | 16:00 |
| @mordred:waterwanders.com | oh well - not super important. I'll just walk away slowly. doesn't seem like we have an "important-changes" dashboard lurking somewhere in git being applied with jeepyb | 16:02 |
| @clarkb:matrix.org | does it show up at https://review.opendev.org/settings/#Menu ? | 16:02 |
| @mordred:waterwanders.com | yeah - that's where it's showing up for me | 16:03 |
| @mordred:waterwanders.com | And it's the one that's not defined in terms of a query | 16:03 |
| @mordred:waterwanders.com | I'm not sure I even realized there were centrally defined dashboards like that | 16:04 |
| @clarkb:matrix.org | oh wow ok | 16:04 |
| @mordred:waterwanders.com | ikr? | 16:04 |
| @clarkb:matrix.org | so I think what you've got is a holdover from when we had the forked change for important changes filtering | 16:04 |
| @clarkb:matrix.org | I don't remember if that autoamtically created that dashboard for you or maybe you created it using those terms? | 16:04 |
| @clarkb:matrix.org | but I think deleting it is appropriate | 16:04 |
| @mordred:waterwanders.com | yeah. I think you're right | 16:05 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [opendev/base-jobs] 994146: Rename opendev-buildset-registry-consumer to opendev-use-container-image https://review.opendev.org/c/opendev/base-jobs/+/994146 | 16:13 | |
| @mordred:waterwanders.com | corvus: sake of argument ^^ based on your comments from yesterday, that might be a clean approach? | 16:13 |
| @clarkb:matrix.org | fungi: looking at gerrit apache logs I think what we're seeing is things making tcp/tls connections and then not continuning to make http requests | 16:30 |
| @clarkb:matrix.org | so they are parking the slot and dos'ing us | 16:30 |
| @clarkb:matrix.org | system load drops because we aren't really doing any work | 16:30 |
| @clarkb:matrix.org | and then once things timeout and get freed up we go back to doing work again? | 16:34 |
| @clarkb:matrix.org | but I'm still digging in logs and comparing to ss output and trying to make sense of it all | 16:36 |
| @clarkb:matrix.org | oh heh I was looking at the wrong vhost log file. That explains the lack of logging so my above interpretation is probably wrong | 16:38 |
| @fungicide:matrix.org | i sort of feel the same with what i've been observing, but i suspect it may actually be related to how mod_proxy works and we're just not seeing what it's blocking on | 16:45 |
| @fungicide:matrix.org | like is reading request state in limbo if apache is still trying to open a socket to gerrit? | 16:45 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 16:46 | |
| - [opendev/zuul-providers] 994782: Revert "Increase rax-classic 16GB flavor to next size up" https://review.opendev.org/c/opendev/zuul-providers/+/994782 | ||
| - [opendev/zuul-providers] 994783: Test v3-starter-8 on vexxhost https://review.opendev.org/c/opendev/zuul-providers/+/994783 | ||
| @clarkb:matrix.org | fungi: oh you think maybe the gerrit side is runing out of threads? that could be and the gerrit task/ps stuff should tell us I think? | 16:47 |
| @jim:acmegating.com | mordred: lgtm if you fix it up | 16:49 |
| @fungicide:matrix.org | Clark: "think" is a strong word, but i am not ruling it out yet | 16:49 |
| @mordred:waterwanders.com | corvus: cool | 16:49 |
| @fungicide:matrix.org | these things are often not what they seem on the surface | 16:50 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [opendev/base-jobs] 994146: Rename opendev-buildset-registry-consumer to opendev-use-container-image https://review.opendev.org/c/opendev/base-jobs/+/994146 | 16:52 | |
| -@gerrit:opendev.org- Clark Boylan proposed: | 16:54 | |
| - [opendev/system-config] 994785: Fix Gerrit robots.txt and clean up static file rules https://review.opendev.org/c/opendev/system-config/+/994785 | ||
| - [opendev/system-config] 994786: Stop including robots.txt in the Gerrit container image https://review.opendev.org/c/opendev/system-config/+/994786 | ||
| @clarkb:matrix.org | fungi: ack. fwiw ^ we discussed doing this previously and maybe having a crawl delay set for the better behaved bots will help us | 16:54 |
| @clarkb:matrix.org | I need to transition over to writing some emails for zuul tenant splitting. I'm around if we want to proceed with those changes or the etherpad upgrade or the gitea upgrade | 16:57 |
| @jim:acmegating.com | Clark: trade you a review on the providers changes above? :) | 16:57 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [opendev/zuul-providers] 994782: Revert "Increase rax-classic 16GB flavor to next size up" https://review.opendev.org/c/opendev/zuul-providers/+/994782 | 16:59 | |
| @clarkb:matrix.org | corvus: done I went ahead and approved both since neither should affect any production workloads so single reviewer seems fine | 17:00 |
| @jim:acmegating.com | thanks, i'll get some jobs running soon to collect data | 17:00 |
| @fungicide:matrix.org | Clark: looking | 17:01 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [opendev/zuul-providers] 994783: Test v3-starter-8 on vexxhost https://review.opendev.org/c/opendev/zuul-providers/+/994783 | 17:01 | |
| @clarkb:matrix.org | `gerrit show-caches --show-threads` gives some thread count breakdowns including for ssh and http | 17:30 |
| @clarkb:matrix.org | that should give us some idea if we're using the max threads count in gerrit for http or if the problem is on the frontend only | 17:34 |
| @mnasiadka:matrix.org | Ideally we should gather these metrics in Prometheus in future | 17:50 |
| @mnasiadka:matrix.org | That reminds me to update ansible_host :) | 17:50 |
| @clarkb:matrix.org | mnasiadka: as an alternative you can rebase on my change that tests the /etc/hosts public ips | 17:57 |
| @clarkb:matrix.org | mnasiadka: in theory that would make your existing change work as is I think | 17:57 |
| @mnasiadka:matrix.org | Can we merge the Prometheus patch, so I’ll rebase only the node exporter one? | 18:00 |
| @mnasiadka:matrix.org | https://review.opendev.org/c/opendev/system-config/+/980840/12 | 18:01 |
| @mnasiadka:matrix.org | It already has 2 +2s | 18:01 |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 994798: Increase the Gerrit apache worker limits https://review.opendev.org/c/opendev/system-config/+/994798 | 18:01 | |
| @clarkb:matrix.org | mnasiadka: yes I think you can approve it since it isn't adding any new servers it should noop outside of the ci environment | 18:02 |
| @clarkb:matrix.org | and if it has the +2s then it should be good to go as is. | 18:02 |
| @mnasiadka:matrix.org | Clark: rebase node exporter on https://review.opendev.org/c/opendev/system-config/+/994565 ? because the /etc/hosts one is not going to make a big difference I think? (https://review.opendev.org/c/zuul/zuul-jobs/+/994564) | 18:13 |
| @clarkb:matrix.org | mnasiadka: yes it should be on 994565 which depends on 994564 and applies it to the system-config test jobs | 18:14 |
| @mnasiadka:matrix.org | 565 failed on gitea, but I assume it's some transient error | 18:15 |
| @fungicide:matrix.org | looks like the lists server is getting overloaded now, even with using anubis | 18:34 |
| @fungicide:matrix.org | i'm just going to assume that the web is dying | 18:40 |
| @clarkb:matrix.org | mnasiadka: I think we may need to update the gitea haproxy template to also use ansible_host instead of private ips | 18:40 |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 994565: Make public ip address public not private https://review.opendev.org/c/opendev/system-config/+/994565 | 18:49 | |
| @clarkb:matrix.org | something like that | 18:49 |
| -@gerrit:opendev.org- Zuul merged on behalf of Mohammed Naser: [opendev/system-config] 980840: Add Prometheus monitoring service https://review.opendev.org/c/opendev/system-config/+/980840 | 18:50 | |
| @jim:acmegating.com | Clark: v3-starter-8 looks great: https://review.opendev.org/988175 | 18:59 |
| @jim:acmegating.com | i think that means our next step is to decide whether we want to look into the bfv legacy rax flavors, or ... something else | 19:00 |
| @clarkb:matrix.org | Awesome, so maybe we consider if we do the bigger flavor in rax classic at all or try boot from volume there | 19:00 |
| @clarkb:matrix.org | Like maybe just not using the bigger flavor is an option there? | 19:01 |
| @jim:acmegating.com | yeah, an option made less concerning by the fact that it's "legacy" -- presumably the problem will solve itself eventually? | 19:01 |
| @mordred:waterwanders.com | corvus: that does look great | 19:02 |
| @fungicide:matrix.org | granted we keep hearing flex will have ipv6 "eventually" too | 19:02 |
| @fungicide:matrix.org | i've got personal servers i'm ready to rebuild in flex the moment i can have global v6 addressing there, so seems like the "legacy" cloud is probably sticking around for a while still | 19:03 |
| @jim:acmegating.com | okay, so plan A) dont use rax for 16gb nodes; plan B) (if we find we need more clouds supporting 16gb) bump to 30GB nodes if we need it; plan C) try to use bfv | 19:04 |
| and: don't do plan C. :) | ||
| @jim:acmegating.com | * okay, so plan A) dont use rax for 16gb nodes; plan B) (if we find we need more clouds supporting 16gb) use rax legacy 30GB nodes if we need it; plan C) try to use bfv | 19:04 |
| and: don't do plan C. :) | ||
| @clarkb:matrix.org | ya that seem reasonable | 19:04 |
| @fungicide:matrix.org | wfm, sure | 19:04 |
| @jim:acmegating.com | cool, i'll make same changes to tidy things up and remove the test labels/flavors | 19:05 |
| @jim:acmegating.com | after lunch | 19:05 |
| -@gerrit:opendev.org- Brian Rosmaita proposed: [openstack/project-config] 993184: Retire cinderlib https://review.opendev.org/c/openstack/project-config/+/993184 | 19:23 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: | 19:38 | |
| - [opendev/system-config] 994785: Fix Gerrit robots.txt and clean up static file rules https://review.opendev.org/c/opendev/system-config/+/994785 | ||
| - [opendev/system-config] 994786: Stop including robots.txt in the Gerrit container image https://review.opendev.org/c/opendev/system-config/+/994786 | ||
| @clarkb:matrix.org | 980840 is running manage projects and that seems to be stuck on slowness on gitea13 | 19:41 |
| @fungicide:matrix.org | i uess it does gitea servers before gerrit | 19:43 |
| @clarkb:matrix.org | yes. It is working through the openstack/fuel* repos according to strace | 19:46 |
| @clarkb:matrix.org | I think it is making progress just not super quickly | 19:46 |
| @clarkb:matrix.org | its roughly halfway through the list if my estimate from vim in projects.yaml is accurate | 19:47 |
| @fungicide:matrix.org | mailman eventually recovered and is responding promptly again now that the crawlers have gotten bored and moved on | 19:55 |
| @clarkb:matrix.org | gitea13 is working through the puppet repos now | 19:59 |
| @clarkb:matrix.org | so it is progressing but as mentioned slowly | 19:59 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [opendev/zuul-providers] 994818: Use v3-starter-8 for vexxhost 16GB and remove rax 16GB https://review.opendev.org/c/opendev/zuul-providers/+/994818 | 20:13 | |
| @clarkb:matrix.org | ok the manage-projects deploy timed out and now the hourly jobs have the lock. So it will be a few more minutes before we actually update apache on review | 20:21 |
| -@gerrit:opendev.org- Scott Little proposed: | 20:25 | |
| - [openstack/project-config] 981169: Revert "Temporarily add the power to delete a starlingx branch." https://review.opendev.org/c/openstack/project-config/+/981169 | ||
| - [openstack/project-config] 994821: add repo build-config for StarlingX https://review.opendev.org/c/openstack/project-config/+/994821 | ||
| @fungicide:matrix.org | https://review.opendev.org/robots.txt exists now, thanks for fixing that! | 20:32 |
| -@gerrit:opendev.org- Zuul merged on behalf of Scott Little: [openstack/project-config] 981169: Revert "Temporarily add the power to delete a starlingx branch." https://review.opendev.org/c/openstack/project-config/+/981169 | 20:46 | |
| -@gerrit:opendev.org- Scott Little proposed: | 20:48 | |
| - [openstack/project-config] 994821: add repo build-config for StarlingX https://review.opendev.org/c/openstack/project-config/+/994821 | ||
| - [openstack/project-config] 994823: Remove branch deletion powers from three new repo https://review.opendev.org/c/openstack/project-config/+/994823 | ||
| @clarkb:matrix.org | fungi: you approved https://review.opendev.org/c/openstack/project-config/+/981169 and it is deploying now. keep in mind that it is likely to fail for the same reason that manage-projects for 980840's manage-projects build failed. But 980840 was a noop for manage projects 981169 is not so we may need to figure that out | 20:55 |
| @clarkb:matrix.org | hrm actually while the job times out we may actually keep running ansible on the remote end | 20:58 |
| @clarkb:matrix.org | which may be a different problem but I think even if that job fails it will update gerrit acls | 20:58 |
| @clarkb:matrix.org | Jens Harbott: I have sent email to three airship maintainer emails that were shared with me and to the starlingx discuss mailing list to kick off separate discussions for each of them to move into their own dedicated zuul tenants | 21:12 |
| @mordred:waterwanders.com | gerrit may have had a sad again? | 21:22 |
| @clarkb:matrix.org | yes but this is different now load is high implying it is doing real work | 21:24 |
| @mordred:waterwanders.com | yay work | 21:24 |
| @fungicide:matrix.org | more threads means more gerrit | 21:28 |
| @fungicide:matrix.org | fwiw traffic to mailman also picked back up and knocked it offline again, even with the up-to-date version of our ua filters in place | 21:29 |
| @fungicide:matrix.org | but maybe increasing threads there is ~okay if we assume they'll mainly go through to anubis and get turned away rather than hitting django/postorius/hyperkitty | 21:30 |
| @clarkb:matrix.org | right the ua filters still require apache slots to process | 21:36 |
| @clarkb:matrix.org | mordred: fwiw I blocked twitter because they were ddosing us | 21:36 |
| @clarkb:matrix.org | mordred: load is dropping and I think things may be slowly recovering | 21:36 |
| @mordred:waterwanders.com | wow. what the heck twitter | 21:46 |
| @fungicide:matrix.org | the twits must flow | 21:47 |
| @mordred:waterwanders.com | must they? | 21:49 |
| @fungicide:matrix.org | according to someone anyway | 21:49 |
| @mordred:waterwanders.com | does anyone here still review pbr patches? I just pushed up 2 small ones that fix warnings that get spit out when using pbr with new setuptools and pyproject.toml https://review.opendev.org/c/openstack/pbr/+/994826 and https://review.opendev.org/c/openstack/pbr/+/994827 - I've verified that they work locally - I have no idea how to write a test that might show either the breakage or that it's fixed. | 22:03 |
| @fungicide:matrix.org | stephenfin did a lot of the recent pyproject.toml support implementation and may have suggestions | 22:03 |
| @fungicide:matrix.org | though probably not at this time of night | 22:04 |
| @mordred:waterwanders.com | yeah - he's got a few larger patches up to do some more stuff in this area. I tried to keep these *super* small and targetted | 22:04 |
| @mordred:waterwanders.com | I'm glad he did work on that - I never would have been able to stomach doing it | 22:05 |
| @mordred:waterwanders.com | but I'm happy that I can continue to use pbr in the new world of our enforced obsession with toml overlords, and that it's possible to keep my toml exposure really minimal. so - you know - hats off there | 22:06 |
| @mordred:waterwanders.com | wow. that became a rabbithole. there seems to have been a new release of something (setuptools?) that has caused things to get much pickier. here's hoping stephenfin doesn't wake up and throw things at me | 22:46 |
| @fungicide:matrix.org | unless it's cans of pbr | 22:56 |
| @mordred:waterwanders.com | oh golly | 22:57 |
| @mordred:waterwanders.com | "silence some warnings" turned in to "make tox unittests work again" | 22:57 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!