Wednesday, 2023-07-26

opendevreviewMerged opendev/system-config master: Update to latest tag for Limnoria
fungikeeping an eye on that ^ deploy to make sure the bot reconnects01:00
fungiwelcome back, opendevmeet!01:02
fricklerfungi: ianw: I renamed the two .conf files on meetpad01, iiuc it will take until tomorrow before the next run happens09:38
*** amoralej is now known as amoralej|lunch11:01
*** amoralej|lunch is now known as amoralej12:07
fungithanks frickler! i guess we can check in on it again tomorrow and see if the problem has recurred or it successfully updated12:21
amorinhey all, is the location for openinfra summit 2024 already known? Did I miss the notification for it?14:33
*** d34dh0r5- is now known as d34dh0r5315:01
fungiamorin: not to my knowledge. i think the plan is to discuss it at the board of directors meeting on tuesday:
amorinack, will wait then :)15:19
fungiany time!15:20
ildikovHi, I have a quick question. I know that there were discussions around moving OpenDev container images over from DockerHub to Quay. Have you all looked into other options as well before more seriously considering Quay? Did you have a list of criteria that you used for evaluation?15:45
ildikovBackground: the StarlingX community has been having issues with DockerHub recently, so the idea to switch came up. Nexus Repository and Quay came up as options, but people don't have a lot of experience, especially with the latter.15:47
fungiildikov: i don't think we looked very hard at other options besides quay or hosting our own registry. the main reason we rolled back off quay was because the docker client has baked-in fallback policies which favor dockerhub in ways which make our current test model (relying on jobs that can depend on other proposed changes to container images) is impossible with the docker client, and16:15
fungiswitching to other clients is something we're still working on16:16
fungianother community worth looking at is scs, they started hosting their own image registry for their distribution16:16
fungiin unrelated news, just saw this on "We are currently in read-only mode. That means that at the moment you can't create new accounts, update your personal information, or request password reminders."16:16
fungiheads up, in case anyone runs into problems16:17
ildikovthanks fungi, it's great info!16:35
fungiildikov: maybe also worth noting, zuul moved their container image hosting to quay, but they don't need the same degree of dependent testing (for opendev's case it's more related to replicating production deployments of preview images to mock servers and interacting with them in subsequent jobs/changes)16:37
ildikovthat makes sense16:37
ildikovfungi: Zuul is using Quay as a service and it's not a self-hosted platform, right?16:38
fungicorrect, specificially16:39
fungii think what scs is using to host their own registry is software called "harbor" according to ttps://
fungiunfortunately, goes straight to a login so it's hard to tell for sure16:42
ildikovOh, cool, I’ll try to dig into that17:25
fungiwe're about 90 minutes out from around when we planned to do the gerrit restart. the opendevorg/gerrit:3.7 tag seems to indicate a successful upload from the change merged yesterday:
fungii've done a docker-compose pull and image list shows the updated images, so i think we're all set. down the service, recursively delete /home/gerrit2/review_site/data/replication/ref-updates, then up everything again19:41
Clark[m]I think we may just need to delete the waiting/ dir under that path? If you ls | wc -l you can find the big dir19:44
fungiah, okay19:44
Clark[m]And moving it aside and deleting later may be much quicker?19:44
fungialso i'll do a status notice about an hour ahead (15 minutes form now)19:44
fungican do19:44
fungipresumably gerrit will recreate that directory19:45
Clark[m]Yes it should since before the bind mount it was starting without the dir every time19:45
Clark[m]clarification: the delete won't be any faster but we can perform it after Gerrit is restarted if we mv rather than rm while Gerrit is down19:46
clarkbildikov: the alternative registry options we considered were github's registry or quay for hosted versions and for self hosted gitea and zuul registry were considered. We prefered hosted for simplicity reasons (we don't have to deal with storage and bw and pruning)19:57
clarkbquay is open source and the hosted version is free for open source projects so that was a straightforward choice (with the acknowledgement that we could end up where docker had initially put us in the future)19:58
fungi#status notice The Gerrit service on will be offline briefly for a minor upgrade at 21:00 utc, approximately an hour from now20:01
opendevstatusfungi: sending notice20:01
-opendevstatus- NOTICE: The Gerrit service on will be offline briefly for a minor upgrade at 21:00 utc, approximately an hour from now20:01
opendevstatusfungi: finished sending notice20:04
ildikovclarkb: +1, thank you for sharing some more details!20:05
clarkbfungi: there are "only" 29k entries in the waiting/ dir and ya that is the dir20:06
clarkbI expected more20:06
clarkbfungi: and if we mv waiting to /home/gerrit2/tmp/$dir/waiting we'll stay on the same fs and a mv should be almost free20:06
fungiright, will definitely save some time20:08
clarkb90cc4cec43ba is the current gerrit image20:37
fungiopendevorg/gerrit   <none>    90cc4cec43ba   2 months ago    790MB20:44
fungiand we're restarting on20:45
fungiopendevorg/gerrit   3.7       3a2e576abc94   23 hours ago    793MB20:45
fungi#status notice The Gerrit service on will be offline briefly for a minor upgrade, but should return shortly21:00
opendevstatusfungi: sending notice21:00
-opendevstatus- NOTICE: The Gerrit service on will be offline briefly for a minor upgrade, but should return shortly21:00
fungii'll start momentarily21:00
clarkbcool I can follow along21:00
opendevstatusfungi: finished sending notice21:03
fungii'll `mv /home/gerrit2/review_site/data/replication/ref-updates/waiting /home/gerrit2/tmp/upgrade-3.7.4/` once the containers are down21:03
fungitaking the containers down now21:03
fungimoving the replication queue21:03
fungiupping containers now21:04
fungiwebui is loading for me now and says "Powered by Gerrit Code Review (3.7.4-10-g56c0b3d9ce-dirty)"21:05
clarkbI see that too21:05
clarkband the waiting dir was recreated as expected21:05
fungicurrent /home/gerrit2/review_site/data/replication/ref-updates/waiting is there and empty, yep21:05
clarkbpushing a change/patchset if anyone has one ready would be a good adidtional test but this lgtm21:07
fungii've got some changes i need to fix, i'll try to do one real quick21:07
fungii've self-approved some trivial changes instead, that'll still get us replication events once they merge21:12
fungialso looks like this is a good time to approve 88811121:13
fungi886641 in the gate is just waiting for a node to run a docs build21:15
fungithough it's been waiting 6 minutes for that node already21:15
fungii probably need to update the nodeset for the ossa repo's jobs21:16
fungilots of stuff that got approved after it is already running jobs21:16
clarkbit is running now21:19
fungiyeah, and that's from a standard openstack job template, so using ubuntu-jammy21:19
fungijust got unlucky with boot failures i guess21:19
fungiat least it's wrapping up and uploading logs already21:20
clarkb this one just merged21:20
fungi reflects it21:21
fungiso replication is working21:21
clarkbthe starlingx/monitoring commit replicated as well21:22
fungiperfect, i'm going to go do some very exciting yardwork while i wait for the latest meetbot image update to go21:22
clarkbsound sgood. Thank you for pushing this along21:23
clarkbI too will resume morning activities21:23
opendevreviewMerged openstack/project-config master: Switch from deprecated require-approval to require
opendevreviewMerged opendev/system-config master: Update ircbot and matrix-eavesdrop to python-3.11
opendevreviewMerged opendev/system-config master: Add $self to the list of files for python images.
opendevreviewMerged opendev/system-config master: Update docker-images/python* to remove '.*'

Generated by 2.17.3 by Marius Gedminas - find it at!