Saturday, 2024-12-14

clarkbthe cirros ssl cert appears to have renewed00:03
fungiwith 3 days to spare00:05
fungiokay, reprepro update completed, and then i ran it a second time just to make sure it was roughly a no-op. now i'm copying the local db files that was using back into afs and will try running the update script again as soon as that's finished00:15
clarkbevery change in topic:podman-prep passes now00:21
corvusfungi: I just read Berkeley db and afs. I have no context for what you're doing but if there is a Berkeley db involved with multiple actors and locking is important then afs may be a problem due to incomplete posix locking.00:21
clarkbI think that proves it out pretty well. Gerrit and mailman are the other services sufficicently different/odd/complicated that should probably be checked against before we commit to this but I think this is all pointing in the right directly00:21
clarkbs/directly/direction/00:22
fungicorvus: indeed, it's possible we somehow had two reprepro process run concurrently (we run under a flock and reprepro itself has a lockfile of its own too so that shouldn't ever happen, but...)00:22
corvusI haven't paged any of that into my brain for a while. It's just that those keywords triggered a red flag memory.00:22
fungibut anyway, it seems like i was able to successfully get reprepro to rebuild its databases from scratch00:23
corvusYeah I'm guessing it's NBD because we should only have one actor.00:23
corvusOk good00:23
fungiand the only impact so far has been that arm64 ubuntu nodes have been pulling almost-month-old packages from our mirrors00:24
corvusclarkb: woo I'll take a look before Monday00:24
clarkbcorvus: thanks!00:24
clarkbthe actual hack part of this is actually pretty small. Its basically disable dockerd sufficiently so that podman can listen on the docker socket path. Then make a docker-compose command shim00:25
clarkbI guess that ignore sthat we're running docker compose with podman as the backend but that seems like a supported piece of functionality00:25
fungireprepro mirror script is still running, i have a feeling from the warnings it's emitted so far that the check is going to exit nonzero and we're going to need to kick out some files that got truncated so it will pull fresh copies, but i'll deal with that in the morning01:01
clarkbone thing that occurred to me about configuring the podman.socket unit to listen on docker's socket path is that `podman` commands may not work. However running sudo podman ps -a shows the same containers with the same id's as sudo docker ps -a. sudo podman info shows it is using a default podman socket. I think this means the socket doesn't determine the context but is just another16:53
clarkbway to talk to a common backend16:53
clarkbI think that alleviates any concerns about unexpected behaviors when interacting with systems as an operator. Its a bit unconventional and hacky but seems to do the right thing and work16:54
fungiif you're using socket activation in systemd, then the process isn't really listening on the socket, systemd is and then it ferries bytes from its listening socket to the stdin of the "activated" process it spins up19:45
Clark[m]fungi: ya the docker compose and docker invocations will use the docker.sock that podman is configured to listen on but the podman command doesn't appear to do so20:24
Clark[m]However it isn't an issue for the podman command it still sees the same resources so it all works out20:24

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!