| opendevreview | Albin Vass proposed zuul/zuul-jobs master: Revert "Revert "tox: update lint regex to not require column"" https://review.opendev.org/c/zuul/zuul-jobs/+/728912 | 07:04 |
|---|---|---|
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Ratchet down and simplify linting rules https://review.opendev.org/c/opendev/engagement/+/945212 | 13:32 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Update project boilerplate https://review.opendev.org/c/opendev/engagement/+/945151 | 14:21 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Import old who-approves.py script https://review.opendev.org/c/opendev/engagement/+/945152 | 14:21 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Ratchet down and simplify linting rules https://review.opendev.org/c/opendev/engagement/+/945212 | 14:21 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: Add upload-image-s3 role https://review.opendev.org/c/zuul/zuul-jobs/+/944813 | 14:51 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Rename who-approves.py to maintainers.py https://review.opendev.org/c/opendev/engagement/+/945224 | 14:55 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Add a convenience entrypoint for maintainers.py https://review.opendev.org/c/opendev/engagement/+/945225 | 14:55 |
| clarkb | infra-root I've just looked at nodepool dib-image-list records and everything still looks good for cleaning up the old servers. I am going to proceed with that now | 14:55 |
| fungi | perfect, thanks! | 14:56 |
| clarkb | #status log Deleted nb04.opendev.org (8cdc7729-1a51-4c7b-9841-fb03f7873f9e) and its associated volume (abd54a83-b31d-4400-b6bd-6a64409726cd). nb07 has replaced it. | 15:01 |
| opendevstatus | clarkb: finished logging | 15:01 |
| clarkb | the volume is actually still in a deleting state. I'll have to check on that later and followup with osuosl if it doesn't go away | 15:01 |
| clarkb | now to do the x86 builders | 15:01 |
| clarkb | #status log Deleted nb01.opendev.org (cd068bef-5da1-4d43-b9c6-6e55b5000ed7) and its associated volumes (0ba0c1b7-f2df-42dd-a3a3-ac42585c5cb8, 07683dcc-8139-4d53-93cf-cb2e90cb1ac8). nb05 has replaced it. | 15:06 |
| opendevstatus | clarkb: finished logging | 15:06 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: Add upload-image-s3 role https://review.opendev.org/c/zuul/zuul-jobs/+/944813 | 15:06 |
| clarkb | #status log Deleted nb02.opendev.org (f5b26d10-260e-4479-b639-42d4fff81ffb) and its associated volumes (219a3cf2-6c07-4625-b926-87bb4411d397, 0ed5b6f5-9db0-4d1f-81b0-e771177e9992). nb06 has replaced it. | 15:10 |
| opendevstatus | clarkb: finished logging | 15:10 |
| clarkb | the arm64 builder volume eventually deleted on the cloud side too so I think thats all good | 15:10 |
| clarkb | next I'm going to boot a new arm64 mirror and get that ball rolling | 15:11 |
| clarkb | that is in progress. When the boot is complete I'll attach the volume and configure it and push some changes up to have ansible deploy it | 15:20 |
| opendevreview | Clark Boylan proposed opendev/zone-opendev.org master: Add new mirror in osuosl https://review.opendev.org/c/opendev/zone-opendev.org/+/945230 | 15:51 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Add new mirror in osuosl https://review.opendev.org/c/opendev/system-config/+/945231 | 15:56 |
| clarkb | ok those two changes should do it. The new server is up and running with the volume attached, formatted, and mounted to the two different cache paths | 15:56 |
| clarkb | feel free to ssh in and take a look and dbouel check | 15:57 |
| clarkb | now that nb01, nb02, and nb04 are not in our inventory and have been deleted I will clean up the emergency file on bridge | 15:59 |
| clarkb | fungi: do we want to try deploying the new arm64 mirror in osuosl? https://review.opendev.org/c/opendev/zone-opendev.org/+/945230 then https://review.opendev.org/c/opendev/system-config/+/945231 could use reviews if so | 17:44 |
| fungi | yeah, lemme wrap up what i'm in the middle of and i can take a look in a few minutes | 17:46 |
| clarkb | thanks | 17:46 |
| clarkb | I'm going to start working on that newsletter draft now | 17:47 |
| clarkb | fungi: I assume that verify your keys email is in response to you generating new keys/ | 17:50 |
| fungi | yeah, just finished validating it and moved those messages to the openpgp inbox folder | 17:50 |
| fungi | working on exporting the signing subkey now | 17:51 |
| clarkb | ack | 17:51 |
| clarkb | ok draft is in place at https://etherpad.opendev.org/p/opendev_newsletter | 18:06 |
| opendevreview | Merged opendev/zone-opendev.org master: Add new mirror in osuosl https://review.opendev.org/c/opendev/zone-opendev.org/+/945230 | 18:20 |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Replace 2025.1/Epoxy key with 2025.2/Flamingo https://review.opendev.org/c/openstack/project-config/+/945247 | 18:50 |
| opendevreview | Merged opendev/system-config master: Add new mirror in osuosl https://review.opendev.org/c/opendev/system-config/+/945231 | 19:16 |
| clarkb | I'll check that as soon as the mirror job is done. Hopefully we can swap over DNS today too | 19:18 |
| clarkb | almost there. I just remembered we'll have to compile openafs though and on arm that may take a while so don't be in a hurry | 19:41 |
| clarkb | https://mirror02.regionone.osuosl.opendev.org/ is serving content now | 19:54 |
| opendevreview | Clark Boylan proposed opendev/zone-opendev.org master: Flip osuosl mirror over to the new noble node https://review.opendev.org/c/opendev/zone-opendev.org/+/945254 | 19:55 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Cleanup old osuosl mirror https://review.opendev.org/c/opendev/system-config/+/945256 | 19:59 |
| opendevreview | Clark Boylan proposed opendev/zone-opendev.org master: Cleanup old arm64 mirrors https://review.opendev.org/c/opendev/zone-opendev.org/+/945257 | 19:59 |
| clarkb | that series of changes switches us to the new mirror, removes the old mirror from the inventory, then removes the old mirror and a couple others I saw taht can go away from dns | 20:00 |
| fungi | all 3 lgtm | 20:42 |
| clarkb | I'll start approving them in sequence | 20:44 |
| opendevreview | Merged opendev/zone-opendev.org master: Flip osuosl mirror over to the new noble node https://review.opendev.org/c/opendev/zone-opendev.org/+/945254 | 20:45 |
| fungi | thanks, i mainly just didn't want to approve them if you weren't ready to check behind them as they deploy | 20:48 |
| clarkb | ya no problem. I should be around. Maybe even with enough time to delete the old server. We'll see | 20:50 |
| clarkb | `dig mirror.regionone.osuosl.opendev.org` shows me the new record already so I'll proceed with the inventory cleanup | 20:50 |
| clarkb | its still ni check but I'ev approved it anyway | 20:51 |
| opendevreview | Merged opendev/system-config master: Cleanup old osuosl mirror https://review.opendev.org/c/opendev/system-config/+/945256 | 21:27 |
| clarkb | I'm approving the dns cleanup now | 21:38 |
| opendevreview | Merged opendev/zone-opendev.org master: Cleanup old arm64 mirrors https://review.opendev.org/c/opendev/zone-opendev.org/+/945257 | 21:41 |
| clarkb | that is queued up behind the deployment for 945256. Once dns is cleaned up I'll delete the server | 21:42 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Ratchet down and simplify linting rules https://review.opendev.org/c/opendev/engagement/+/945212 | 22:16 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Rename who-approves.py to maintainers.py https://review.opendev.org/c/opendev/engagement/+/945224 | 22:16 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Add a convenience entrypoint for maintainers.py https://review.opendev.org/c/opendev/engagement/+/945225 | 22:16 |
| opendevreview | Jeremy Stanley proposed opendev/engagement master: Rewrite maintainers.py functionality https://review.opendev.org/c/opendev/engagement/+/945262 | 22:16 |
| clarkb | digging against the nameservers doesn't resolve mirror01 anymore. I'm going to delete it and its volume now | 22:16 |
| clarkb | #status log Deleted mirror01.regionone.osuosl (f0f1ed9d-98a8-4713-8d74-69b168c4c996) and its volume (5abcb319-1b3a-4d3c-9dba-532899554da1). mirror02.regionone.osuosl has replaced it. | 22:19 |
| opendevstatus | clarkb: finished logging | 22:20 |
| clarkb | glad I was able to get that done and minimize the amount of time we were using twice the resources | 22:21 |
| clarkb | this is related to debugging zuul unittests but bringing it up here bceause it is a bigger question around our sudo removal. I'm running a bunch of zuul unittests on a throwaway change to gather data on some mariadb failures. normally zuul uses mysql and its fine. I have realized I can't journalctl -u mariadb > logfile in the job post-run playbook beacuse we've revoked sudo access at | 22:23 |
| clarkb | that point | 22:23 |
| clarkb | any one have good idaes for working around that? maybe the best thing is to drop the revoke sudo in the throwaway change? Or maybe journalctl has some group membership I can add before dropping sudo to see logs later? | 22:23 |
| fungi | we could add a sudoers rule to allo that | 22:23 |
| clarkb | ah yup that is an idea | 22:23 |
| fungi | pretty simple pattern match | 22:24 |
| clarkb | that might be the simplest thing. The test-setup.sh script can add a rule to /etc/sudoers.d/foo and then I can sudo journalctl -u mariadb later | 22:24 |
| clarkb | any idea what that rule looks like? | 22:25 |
| fungi | normally i'd create a file like /etc/sudoers.d/mariadb-logs and then put a line in it like `zuul ALL ALL ALL = (root:root) NOPASSWD: /usr/bin/journalctl -u mariadb` | 22:28 |
| clarkb | `zuul ALL=/usr/bin/journalctl -u mariadb NOPASSWD:ALL` maybe | 22:28 |
| fungi | er `zuul ALL = (root:root) NOPASSWD: /usr/bin/journalctl -u mariadb` | 22:28 |
| fungi | adapted from a similar rule on my systems for something else | 22:29 |
| clarkb | now I have to figure out what it means | 22:29 |
| fungi | anyway, easy enough to test | 22:29 |
| fungi | man 5 sudoers | 22:29 |
| fungi | it has examples | 22:29 |
| clarkb | ya the manpage is like a 20 page long language specification | 22:30 |
| * fungi is knocking off for the evening, need to prep for an overnight trip tomorrow | 22:30 | |
| clarkb | enjoy | 22:30 |
| clarkb | looks like (root:root) means that you can run the command as root:root | 22:31 |
| corvus | i wonder if we should put a general sudo rule to allow journalctl in the image, so any opendev job can do that | 22:50 |
| Clark[m] | I like that idea | 22:54 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!