| opendevreview | Merged opendev/system-config master: Remove /var/lib/zuul/times backup exclusion https://review.opendev.org/c/opendev/system-config/+/972931 | 00:14 |
|---|---|---|
| opendevreview | Merged opendev/system-config master: Remove zuul status backups https://review.opendev.org/c/opendev/system-config/+/972932 | 00:14 |
| corvus | the crontabs appear to be gone after that change. i deleted the backup files and approved the next changes. | 01:03 |
| opendevreview | James E. Blair proposed opendev/system-config master: Finish removing zuul-status-backup https://review.opendev.org/c/opendev/system-config/+/972989 | 01:05 |
| corvus | one more cleanup for that series | 01:06 |
| clarkb | corvus: thanks again! I'll review that last one then I'm calling it a day | 01:06 |
| opendevreview | Merged opendev/system-config master: Remove /var/lib/zuul/backup exclusion https://review.opendev.org/c/opendev/system-config/+/972933 | 02:06 |
| opendevreview | Merged opendev/system-config master: Refactor zuul-scheduler borg backup excludes https://review.opendev.org/c/opendev/system-config/+/972946 | 02:07 |
| opendevreview | Merged opendev/system-config master: Finish removing zuul-status-backup https://review.opendev.org/c/opendev/system-config/+/972989 | 02:45 |
| opendevreview | Dr. Jens Harbott proposed openstack/project-config master: Update remaining AFS keytabs https://review.opendev.org/c/openstack/project-config/+/973000 | 15:33 |
| frickler | fyi there's still quite a number of "dummy" entries in ^^, like e.g. for rtd, zanate, weblate, puppetforge | 15:38 |
| frickler | infra-root: debian mirroring seems broken. last released 2 months ago :( | 15:51 |
| frickler | "The lock file '/afs/.openstack.org/mirror/debian/db/lockfile' already exists." | 15:51 |
| Clark[m] | frickler I suspect some of those dummy values can be cleaned up and we can start doing that in zuul as long as we also do it on the backend too | 15:53 |
| frickler | Clark[m]: not sure I understand what you're saying. the dummy values are there because that's how I replaced the old secrets that could no longer be decrypted. what backend are you referring to? | 15:57 |
| frickler | is it o.k. to just remove the mirror lockfile and see what happens? or does someone want to do more digging first? | 16:06 |
| Clark[m] | frickler I mean we should either delete unused principals in Kerberos or rotate their key tabs if we remove them from zuul config | 16:09 |
| Clark[m] | For the lock file I'm not sure. I think that is a reprepro lock file and there may be a recovery process for it | 16:10 |
| Clark[m] | I guess those dummy secrets may be for things other than afs but similar processes should be used regardless. Disabled or invalidate the thing the secret accesses and then drop the secret from zuul | 16:18 |
| fungi | i can work on fixing the debian mirroring first thing after lunch | 16:28 |
| fungi | probably (hopefully) just needs its reprepro lock cleared | 16:28 |
| fungi | and then manual re-run of reprepro keeping an eye on it so it doesn't timeout | 16:29 |
| frickler | Clark[m]: ah, for the keytabs I created secrets now with all the new keytabs that you had prepared on kdc03. if some of them are no longer actually used, we can of course clean them up, but in terms of AFS/kerberos we should now have reached the same state as before the incident. unless we have some secrets hidden in other repos or files. the remaining stuff in that file is for other services | 16:51 |
| Clark[m] | Got it | 16:53 |
| opendevreview | Merged openstack/project-config master: Update remaining AFS keytabs https://review.opendev.org/c/openstack/project-config/+/973000 | 20:30 |
| fungi | i'll note that clearing the reprepro lockfile for the debian mirror did seem to work, but with two months' backlog (and whatever pending amount prior that likely ran afoul of the timeout previously), my manual update is still running in a screen session on the mirror-update server. i'll check it again in the morning | 23:17 |
| opendevreview | Jeremy Stanley proposed opendev/system-config master: Update documentation for Matrix move https://review.opendev.org/c/opendev/system-config/+/972510 | 23:21 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!