| NeilHanlon | not that we have windows systems (I hope?) but... https://msrc.microsoft.com/blog/2024/01/microsoft-actions-following-attack-by-nation-state-actor-midnight-blizzard/ | 01:20 |
|---|---|---|
| frickler | so I've generated a new API token for github now with the opendev admin user and updated the ansible group_var on bridge with it, I'm assuming it should be rolled out by the next hourly cycle, will watch for a bit | 16:34 |
| frickler | I've followed the github recommendation to not create a non-expiring token, so it has a lifetime of 1y now, which was the maximum I could set. I've added a calendar reminder for me to refresh it in time | 16:35 |
| fungi | non-expiring = 1y ? gotta love their definitions | 16:46 |
| frickler | no, there was an option to not expire at all, but with a warning that one should better not use it | 16:47 |
| fungi | oh, to *not* create a non-expiring token, sorry i misread | 16:47 |
| frickler | 1y is the longest time for expiry, which is already hidden as "custom". default expiry is 90d maximum | 16:48 |
| fungi | still i wonder what suddenly caused the one we'd been using to become invalid, seems like it lasted at least 2.5 years | 16:49 |
| mordred | "reasons" | 16:49 |
| fungi | github definitely has those | 16:50 |
| corvus | after it deploys, could try a tenant reconfig and if that doesn't work, since it's the weekend, a full reconfig | 16:51 |
| corvus | actually, strike that, should probably restart the schedulers and web. that might be necessary and sufficient, but if not, then a full reconfig after doing that. | 16:54 |
| frickler | corvus: config seems to be updated. it was tenant reconfig where I was seeing the failures yesterday, do we want to retry that or do a restart at once? | 18:11 |
| frickler | trying "zuul-scheduler tenant-reconfigure openstack" now | 18:32 |
| corvus | frickler: i have a strong suspicion that it won't reload the api key without a restart (possibly a full reconfig might do it, but that's very slow), so i think the rolling restart of scheduler/web is the most efficient way to start. | 18:32 |
| frickler | corvus: ack, still seeing the 401s. do you have time to run the restarts and watch things? else I'd rather do it tomorrow as it is getting late here | 18:38 |
| corvus | yes i can restart | 18:39 |
| corvus | 2024-01-20 18:41:50,629 DEBUG zuul.GithubRequest: GET https://api.github.com/repos/ansible/ansible/branches?per_page=100 result: 200, size: 10358, duration: 361 | 18:43 |
| corvus | (restart is onging; i don't expect error resolution until its complete) | 18:43 |
| frickler | seems to have worked and the errors are gone, yay | 19:21 |
| corvus | #status log restarted zuul schedulers/web to pick up new github api key | 19:22 |
| opendevstatus | corvus: finished logging | 19:22 |
| corvus | i left the mergers and executors as they were; i believe the only change merged since their last restart was a doc update | 19:22 |
| frickler | also there's a new warning, at least I don't think I've seen it before: "Multiple Project Configurations" | 19:26 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!