Friday, 2026-01-30

-@gerrit:opendev.org- Maksim Malchuk proposed: [openstack/diskimage-builder] 975122: Fix issue with 'DIB_DISTRIBUTION_MIRROR_UBUNTU_INSECURE' https://review.opendev.org/c/openstack/diskimage-builder/+/97512214:09
@fungicide:matrix.orglists.o.o has all its apache worker slots in use again14:19
@fungicide:matrix.orgi'm restarting apache as hopefully a quick fix in case any of those are stale/hung14:20
@fungicide:matrix.orgseems like it helped14:50
@fungicide:matrix.orgunrelated, the ubuntu-ports reprepro checksums.db rebuild seems to still be in progress14:51
@fungicide:matrix.orgpropose-translation-update and upstream-translation-update jobs are working again after the zanata secret rotation15:06
@fungicide:matrix.orgtaking this last opportunity to pop out for lunch before we get snowed in for days here, back in a bit16:38
@fungicide:matrix.orgoh hey, the ubuntu-ports reprepro checksums.db rebuild finished while i was at lunch17:52
@fungicide:matrix.orgnow to see if a manual mirror update will succeed17:52
@fungicide:matrix.orgin progress17:53
@fungicide:matrix.org`File "pool/main/l/linux-gcp-6.8/linux-modules-6.8.0-1044-gcp-64k_6.8.0-1044.47~22.04.1_arm64.deb" is already registered with different checksums!`17:56
@fungicide:matrix.orgi guess we still need a little more repair work, i'll see what i can do17:57
@clarkb:matrix.orgfungi: I seem to recall that can happen if we downloaded a corrupted file then reprepro won't redownload it thinking it is lready there?17:57
@fungicide:matrix.orgyeah, in the past i've manually pulled the file down to replace the corrupt one18:00
@fungicide:matrix.orgworking on it18:00
@fungicide:matrix.orgclearly https://packages.ubuntu.com/jammy-updates/arm64/linux-modules-6.8.0-1044-gcp-64k/download has broken when it comes to listing mirror locations. this is fun18:03
@clarkb:matrix.orgare you saying `deb http:// jammy-updates main` isn't a valid mirror :)18:04
@fungicide:matrix.orgheh18:04
@fungicide:matrix.orgi think the problem is finding a mirror of ubuntu-ports18:09
@clarkb:matrix.orgfungi: there is the one we sync from right?18:09
@clarkb:matrix.orgpresumably the data isn't actually corrupt there now but instead there was a problem with the transfer or the data whenever things broke?18:09
@fungicide:matrix.orgyeah18:10
@fungicide:matrix.orghttp://ports.ubuntu.com/ubuntu-ports/pool/main/l/linux-gcp-6.8/linux-modules-6.8.0-1044-gcp-64k_6.8.0-1044.47~22.04.1_arm64.deb18:15
@fungicide:matrix.orgredownloaded that and then forced a checksum update of just that file into the database18:17
@clarkb:matrix.orgfungi: is forcing a checksum update a reprepro command?18:18
@clarkb:matrix.orgalso I don't thnik there is a rush to update the Gerrit images via https://review.opendev.org/c/opendev/system-config/+/975176 but if infra-root have time to look at that and weigh in on whether or not we want to build off of tags as proposed or keep building from the stable branch that would be great. mordred you may have thoughts too since you set up the original system18:19
@fungicide:matrix.orgyeah, it's the `rereference` command documented in https://docs.opendev.org/opendev/system-config/latest/reprepro.html18:23
@fungicide:matrix.orgi just piped the path for that one file through a call to that reprepro subcommand after redownloading it18:24
@clarkb:matrix.orgthanks18:28
@fungicide:matrix.orgessentially the same command i used to rebuild the checksums database, i merely re-updated a single entry in it that time18:26
@fungicide:matrix.organyway, it's been a few minutes refreshing the mirror and hasn't complained again, so this seems like progress18:30
@fungicide:matrix.orgmy client seems to not think my last two messages made it to the channel. i guess it's missing an ack or something18:46
@clarkb:matrix.orgthe message about same command then refreshing the mirror? I got them18:47
@clarkb:matrix.org* the messages about `same command` then `refreshing the mirror`? I got them18:47
@fungicide:matrix.orgyeah, element webclient says they arrived to the channel, so my weechat plugin is just confused18:48
@fungicide:matrix.orgthough it shows they arrived after your "thanks" so maybe there was some lag/interruption18:49
@clarkb:matrix.orgI got them after the "thanks" as well so that could be18:49
@fungicide:matrix.orginterestingly, i have timestamps turned on in the webclient and it shows your thanks at 18:28 utc followed by my "essentially..." at 18:26 utc18:50
@fungicide:matrix.orgso matrix must do display ordering separate from timestamping18:51
@clarkb:matrix.orgoh yup it goes 10:24 then 10:28 then 10:26 and 10:30 for the messages around that time18:52
@fungicide:matrix.organyway, the ununtu-ports mirror update is still progressing, so i *think* it's back on track finally18:54
@fungicide:matrix.orgi'm distracted by winter storm prep, so monitoring this is probably all i have bandwidth for before the weekend18:56
@fungicide:matrix.orgthat and the debate raging on python community discourse about the wheel variant pep18:59
@clarkb:matrix.orgI've been fiddling with my local wifi setup periodically trying to determine if I can find a specific AP location that produces the best results for my distribution of devices18:59
@clarkb:matrix.orgfar less exciting than preparing for a noreasting18:59
@clarkb:matrix.org* far less exciting than preparing for a noreaster18:59
@fungicide:matrix.orgi have a mesh of 3 access points at the moment. my house is not large, but it's apparently a radio signal sponge19:00
@clarkb:matrix.orgMy neighborhood has a ton of intentional wifi radio noise. But I also suspect that my heat pump may create noise as well?19:02
@clarkb:matrix.orgI may just need to start hardwiring more stuff, but I'm hopeful that if I try enough places I'll find a good AP location that works more reliably19:02
@fungicide:matrix.orgor wrap your heat pump in a faraday cage19:03
@clarkb:matrix.orgI have to say PoE is the greatest thing as it makes moving around really easy. 25' cat 6 cable and painters tape and I can go anywhere with a flat surface without too much fuss19:32
@clarkb:matrix.orgthen if I find a great spot I can figure out how to run a permanent cable19:32
@fungicide:matrix.orgi have a led-lit usb flex scope on a several-meter-long cable, and a one-meter flexible auger bit for drilling through floors/ceilings/studs inside wall plenum19:34
@fungicide:matrix.orgbut my walls are all stuffed with fiberglass insulation batting, which makes any of that a challenge19:34
@clarkb:matrix.orgThe spot I want to put it is a weird ledge at the approximate center of the house and I think there is a small void behind a closet under that ledge so I think I can go through the crawlspace up into that void and pop out on the ledge and not have to deal with insulation at all. But I want to test the AP here for a few days before I commit to that19:37
@clarkb:matrix.orgbecause I'm not entirely sure what causes the interference so need to give it enough time to observe if I'ev mitigated that problem19:37
@fungicide:matrix.orgyeah, and i agree wrt poee, i have a poee ubiquiti long-range access point which i run off a battery-backed poee switch, so one cable and added stability when there are power fluctuations19:38
@fungicide:matrix.orgthat's the one wired to my gateway, and then i have a couple of the in-wall (though i didn't put mine inside the wall) ubiquity poee access points meshed to it to extend the range into problematic parts of the house19:42
@mordred:waterwanders.comI think building off tags seems like a fine idea. I think the original thinking about stable branches was to allow for "easy" rebuilding for security/point-releases within a stable series, inspired by how we'd think about that in OpenStack. But I don't think that has wound up actually being the lived experience, as you point out. :) 20:04
@clarkb:matrix.orgYa it mostly works except that restarting Gerrit is expensive so we don't want to have to do that for every change upstream we might build20:05
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 975319: Update python base images https://review.opendev.org/c/opendev/system-config/+/97531920:44
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 975320: Build the gitea images on debian trixie https://review.opendev.org/c/opendev/system-config/+/97532020:46
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 975321: Upgrade limnoria and bump its container to trixie https://review.opendev.org/c/opendev/system-config/+/97532120:50
@fungicide:matrix.orgaccording to discussion in the launchpad matrix room just now, ports.ubuntu.com is experiencing issues and has been since late last year, so this may be related to our mirroring woes and could take a few tries to get back on track20:51
@clarkb:matrix.orgI decided that we should go ahead and start replacing bookworm with trixie everywhere20:56
@clarkb:matrix.orgit is working well in the places we have updated and there isn't much reason to keep bookworm around if trixie is happy20:56
@fungicide:matrix.orgi see, makes sense to me20:56
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/statusbot] 975324: Update container images to trixie and python3.12 https://review.opendev.org/c/opendev/statusbot/+/97532421:00
@clarkb:matrix.orgremote:   https://review.opendev.org/c/openstack/ptgbot/+/975325 Update ptgbot containers to python3.12 and trixie21:04
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/gerritbot] 975331: Update container image to python3.12 and trixie https://review.opendev.org/c/opendev/gerritbot/+/97533121:07
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/lodgeit] 975332: Update the container image to trixie https://review.opendev.org/c/opendev/lodgeit/+/97533221:09
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/grafyaml] 975334: Update to python3.12 trixie container https://review.opendev.org/c/opendev/grafyaml/+/97533421:11
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/gear] 975335: Update the container image to python3.12 and trixie https://review.opendev.org/c/opendev/gear/+/97533521:15
@clarkb:matrix.orgfor ^ it might be time to start thinking about winding the project down.21:15
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 975320: Build the gitea images on debian trixie https://review.opendev.org/c/opendev/system-config/+/97532021:19
@clarkb:matrix.orgheh all of these irc bots use the same ssl setup which is broken by python3.12. Time to figure that out21:22
@fungicide:matrix.orgi feel like i already fixed them once not too long ago21:23
@fungicide:matrix.orgi really wish that api didn't keep changing21:33
@clarkb:matrix.orgfungi: we fixed some of them. accessbot for example. But statusbot, gerritbot, and ptgbot seem to use the old setup still21:35
@clarkb:matrix.orgI just pushed an update for ptgbot as it failed first now to see if the others complain similarly21:35
-@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 975320: Build the gitea images on debian trixie https://review.opendev.org/c/opendev/system-config/+/97532021:37
@fungicide:matrix.orgah cool21:38
@clarkb:matrix.orgfungi: it doesn't look like gerritbot or statusbot have test coverage of irc connectivity so their tests pass21:52
@clarkb:matrix.orgbut they seem to use the old ib3.SSL construct that ptgbot was using that exploded in ptgbot tests21:52
@clarkb:matrix.orgI may have to punt on those for now. Ill make them work in progress os they don't accidentally merge21:53
@fungicide:matrix.orgreprepro succeeded and vos release started about 20 minutes ago22:04
@fungicide:matrix.orgi'll restart the script once this pass finishes, just to make sure it's fully caught up22:05
@fungicide:matrix.orgfinished exiting zero, started again now22:25
-@gerrit:opendev.org- Clark Boylan proposed:22:52
- [opendev/statusbot] 975324: Update container images to trixie and python3.12 https://review.opendev.org/c/opendev/statusbot/+/975324
- [opendev/statusbot] 975343: Use irc bot connection factory for SSL https://review.opendev.org/c/opendev/statusbot/+/975343
-@gerrit:opendev.org- Clark Boylan proposed:22:53
- [opendev/gerritbot] 975331: Update container image to python3.12 and trixie https://review.opendev.org/c/opendev/gerritbot/+/975331
- [opendev/gerritbot] 975344: Use irc bot connection factory for SSL https://review.opendev.org/c/opendev/gerritbot/+/975344
@clarkb:matrix.orgfungi: that is totally untested, but I think the general shape is correct if something doesn't quite work it shouldn't take much to get it working22:54
@fungicide:matrix.orgcool, i may or may not get time to look at it this weekend. we're expecting gusts close to hurricane speeds along with the blizzard now, so i give it 50/50 odds that at least one of either my electricity or internet service may be offline for a few days22:56
@clarkb:matrix.orgya I don't think this is urgent22:56
@clarkb:matrix.orgmostly I wanted to get changes up while it was still fresh after doing ptgbot updates22:56
@fungicide:matrix.orgmakes sense, thanks for working on it!22:56
@clarkb:matrix.orgits interesting that they fixed this in ib3 in 2023 a few days after their last release. Then they never made a new release22:57
@clarkb:matrix.orgthe other big bookworm -> trixie move is zuul. corvus https://review.opendev.org/c/zuul/zuul/+/960682 not sure if the usage from the opendev side is sufficient for you to consider updating at this point23:01
@clarkb:matrix.orgcurrently opendev has etherpad, gerrit, the statsd sidecars and a couple of the matrix/irc bots using those trixie images23:02
-@gerrit:opendev.org- Clark Boylan proposed:23:04
- [opendev/gerritbot] 975344: Use irc bot connection factory for SSL https://review.opendev.org/c/opendev/gerritbot/+/975344
- [opendev/gerritbot] 975331: Update container image to python3.12 and trixie https://review.opendev.org/c/opendev/gerritbot/+/975331
@fungicide:matrix.orgyeah, i think i peppered some files in places with pointers to the missing commit in code comments saying we're waiting for a new release23:05
-@gerrit:opendev.org- Clark Boylan proposed:23:08
- [opendev/statusbot] 975343: Use irc bot connection factory for SSL https://review.opendev.org/c/opendev/statusbot/+/975343
- [opendev/statusbot] 975324: Update container images to trixie and python3.12 https://review.opendev.org/c/opendev/statusbot/+/975324
-@gerrit:opendev.org- Clark Boylan proposed:23:09
- [opendev/gerritbot] 975344: Use irc bot connection factory for SSL https://review.opendev.org/c/opendev/gerritbot/+/975344
- [opendev/gerritbot] 975331: Update container image to python3.12 and trixie https://review.opendev.org/c/opendev/gerritbot/+/975331

Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!