| opendevreview | Yoshi Kadokawa proposed openstack/project-config master: Add Cinder Huawei charm https://review.opendev.org/c/openstack/project-config/+/867589 | 05:06 |
|---|---|---|
| opendevreview | Yoshi Kadokawa proposed openstack/project-config master: Add Cinder Huawei charm https://review.opendev.org/c/openstack/project-config/+/867589 | 05:45 |
| opendevreview | Merged opendev/system-config master: Add rabbitmq caching mirrors https://review.opendev.org/c/opendev/system-config/+/931897 | 11:13 |
| Clark[m] | fungi: I'm getting sorted after the school run. If you get a chance can you look at the gitea upgrade change? I think we can proceed with that if we are happy with it then depending on timing decide if we want to do Gerrit after as well | 14:49 |
| fungi | sure, looking... | 14:54 |
| opendevreview | Merged opendev/system-config master: Cleanup python 3.10 bullseye images https://review.opendev.org/c/opendev/system-config/+/932102 | 15:00 |
| clarkb | thanks! | 15:03 |
| fungi | gitea change lgtm, i'm disappearing to grab lunch but should be around later to help test | 15:07 |
| clarkb | fungi: thanks, should I approve it in like half an hour or so to try and get it to land when you're back? | 15:09 |
| clarkb | I'll operate under that plan unless I haer otherwise. It takes a bit to gate too so we can always -2 it during that time | 15:10 |
| fungi | i'm back and available to help once it's in | 16:50 |
| fungi | looks like you approved it around an hour ago | 16:51 |
| clarkb | yup I've been waiting patiently | 16:51 |
| clarkb | Once that is done we can decide if we want to proceed with the gerrit update too: https://review.opendev.org/c/opendev/system-config/+/932314 I suspect we can do this Monday afternoon ish after PTG things as an alternative | 16:54 |
| fungi | i'm around for the rest of today, so happy to help keep an eye on the gerrit upgrade too. today seems like a better choice than monday if we have time collectively | 17:01 |
| clarkb | ya it should work for me as well | 17:02 |
| clarkb | oh I'm noticing there may be a node hold for the gerrit update too | 17:04 |
| clarkb | fungi: https://200.225.47.13/ is the held node if that is useful in your reviewing | 17:06 |
| fungi | nice. sorry i've been kinda out of touch for the past few days but seems like tonyb was on the ball with this | 17:10 |
| fungi | seems working | 17:11 |
| fungi | the other thing that didn't help is that for some reason my workstation, which i rely on as a remote bastion when i'm travelling, apparently senses if the monitors get turned off and powers itself down completely, i guess it's something deep inside systemd power management but haven't remembered to investigate | 17:12 |
| fungi | so the last few times i've gone anywhere, when i turn my monitors off manually, i forget that's going to result in my workstation shutting down | 17:13 |
| clarkb | mine does not do this. However I do have the occasionaly (like once a quarter) kernel panic due to some bug in the amd driver when the monitor goes to sleep | 17:13 |
| clarkb | I don't think you use a desktop environment but some of them have fancy power management settings that could do stuff like that too | 17:14 |
| fungi | that too sounds bad, but i don't think that's what i'm seeing because it's cleanly shut down and fully powered off | 17:14 |
| fungi | and yeah, the closest thing i have to a desktop environment is ratpoison, but that's more of a window manager | 17:14 |
| fungi | startx is basically my desktop environment | 17:14 |
| fungi | and whatever commands i stick in my ~/.xinitrd | 17:14 |
| clarkb | the gitea change should merge around when the hourly jobs end. Good timing | 17:16 |
| opendevreview | Merged opendev/system-config master: Update Gitea to 1.22.3 https://review.opendev.org/c/opendev/system-config/+/931966 | 17:27 |
| clarkb | hourly jobs are done too so this should move fairly quickly | 17:27 |
| clarkb | promote job was successful the infra-prod job is starting now. | 17:28 |
| clarkb | gitea09 has updated. I can browse system-config there | 17:29 |
| clarkb | I am able to clone system-config from gitea09 as well | 17:32 |
| clarkb | this seems to be generally working. Just need to make sure all 6 are happy when done and also that replication is working | 17:32 |
| fungi | yeah, 09 is working for me | 17:37 |
| clarkb | all 6 are upgraded now | 17:37 |
| clarkb | https://zuul.opendev.org/t/openstack/build/869e83a4103949adb9de4d514f73654e and the job was successful. So I think we just need to look for replication success | 17:38 |
| clarkb | nothing has merged sicne the upgrade but we could try to fetch specific change refs /me looks for new patchests | 17:38 |
| clarkb | https://review.opendev.org/c/openstack/keystone/+/925020 patchset 15 would've been uploaded right around when gitea09 updated but I'm not finding anything newer | 17:39 |
| clarkb | I guess we wait for someone to push something then check. In the meantime fungi I guess you can review https://review.opendev.org/c/opendev/system-config/+/932314 and if that looks good to you we can approve it then plan to do a manual gerrit update later today after the images publish? | 17:40 |
| fungi | yep, also lgtm, approved | 17:42 |
| clarkb | the process for that is wait for image promotion then docker-compose pull, docker-compose down, mv gerrit replication plugin waiting queue dir aside, docker-compose up -d, then wait for gerrit to start and then wait for diffs to render again | 17:44 |
| clarkb | https://review.opendev.org/c/starlingx/utilities/+/932555 I'm checking if this new patchset replicated now | 17:47 |
| clarkb | `git fetch origin refs/changes/55/932555/6` then `git show FETCH_HEAD` works and the sha matches the new patchset sha | 17:48 |
| clarkb | replication seems ot be working | 17:48 |
| fungi | looks right | 17:56 |
| clarkb | current timing says the change should merge in about 17 minutes. If that is accurate we can probably get this done before I find lunch | 18:17 |
| opendevreview | Merged opendev/system-config master: Update Gerrit images to 3.9.7 and 3.10.2 https://review.opendev.org/c/opendev/system-config/+/932314 | 18:30 |
| clarkb | fungi: ^ those need to promote but then I think we can proceed | 18:30 |
| clarkb | https://hub.docker.com/layers/opendevorg/gerrit/3.9/images/sha256-712af444c67136ec5b21dc3c46059698da795871a4d447a5526cc8e39df69921?context=explore image updated. Now we wait for the infra-prod-service-review job to end I guess just so that we don't interfere with one another | 18:33 |
| clarkb | I started a screen | 18:33 |
| fungi | yeah, was waiting to get the notifications from the promote pipeline | 18:33 |
| fungi | connected | 18:33 |
| clarkb | opendevorg/gerrit 3.9 d78266d324d0 is the existing image | 18:33 |
| fungi | lgtm | 18:33 |
| clarkb | now to figure out the waiting queue path | 18:34 |
| fungi | yeah, "2 months ago" so should refresh when the promote finishes | 18:34 |
| fungi | (and after a pull) | 18:35 |
| clarkb | looks like the project-config chnge that merged shouldn't involve gerrit either | 18:35 |
| clarkb | er that is about to merge | 18:35 |
| clarkb | fungi: I think we are ready to proceed? I've prepped the waiting queue move command in another window that I can copy over when that step is reached | 18:39 |
| clarkb | I also notified the openstack release team | 18:39 |
| fungi | agreed, promote/deploy finished | 18:39 |
| clarkb | ok I'll pull the new image, down gerrit, move the waiting queue, then start gerrit again | 18:40 |
| clarkb | opendevorg/gerrit 3.9 9ba502024800 new image | 18:41 |
| clarkb | proceeding now | 18:41 |
| clarkb | [2024-10-18T18:42:35.312Z] [main] INFO com.google.gerrit.pgm.Daemon : Gerrit Code Review 3.9.7-5-g71c50e4e33-dirty ready | 18:42 |
| clarkb | I can load changes but not all diffs yet | 18:43 |
| fungi | that looks right | 18:43 |
| clarkb | ya the version looks good and early startup functionality is as expected (half working but not fully) | 18:43 |
| clarkb | in another couple of minutes everything else should be happy too I hope | 18:44 |
| clarkb | [DiskCache-Prune-0] INFO com.google.gerrit.server.cache.h2.H2CacheImpl : Cache jdbc:h2:file:///var/gerrit/cache/gerrit_file_diff size (2.44g) is greater than maxSize (128.00m), pruning | 18:45 |
| clarkb | I noticed this in the logs. I suspect that we should consider increasing that cache size? Maybe to 512MB? | 18:45 |
| clarkb | I get diffs now on a random change I hadn't oepned in a long time | 18:46 |
| clarkb | and in random change I have never loaded before | 18:46 |
| fungi | yeah, that does seem fairly small | 18:46 |
| clarkb | [DiskCache-Prune-0] INFO com.google.gerrit.server.cache.h2.H2CacheImpl : Cache jdbc:h2:file:///var/gerrit/cache/git_modified_files size (290.89m) is greater than maxSize (128.00m), pruning | 18:46 |
| clarkb | this is a different cache that maybe should be increased in size too | 18:46 |
| clarkb | I'll grep for DiskCache-Prune and see if there are any others then push up a change after lunch | 18:47 |
| clarkb | https://review.opendev.org/c/airship/promenade/+/932740 this change goet a new patchset after the restart | 18:47 |
| clarkb | so I think that covers being able to push changes. I'll check that new patchset replicated to gitea now too | 18:48 |
| clarkb | yup I'm able to `git fetch origin refs/changes/40/932740/2` and origin is: origin https://opendev.org/airship/promenade (fetch) | 18:49 |
| clarkb | fungi: anything else you think we should check? | 18:50 |
| clarkb | git show FETCH_HEAD matches the ps2 sha | 18:50 |
| fungi | not that i can think of, everything seems to be working for me | 18:51 |
| clarkb | https://paste.opendev.org/show/bk4pTIuQLCsWaF3dVVF7/ this is my grep for the cache pruning. A number of those are floating at the limit which I think we can leave alone | 18:52 |
| clarkb | I'll just bump up those that are obviously not keeping up | 18:52 |
| clarkb | fungi: should I close the screen session? | 18:53 |
| fungi | yeah, i think we're good now | 18:55 |
| clarkb | done | 18:55 |
| clarkb | tonyb: thank you for preparing the change and holding a node to make this quick and easy | 18:55 |
| opendevreview | Merged openstack/project-config master: Add #openstack-eventlet-removal to accessbot https://review.opendev.org/c/openstack/project-config/+/931983 | 18:56 |
| clarkb | ok lunch now then I'll work on that cache change later | 18:57 |
| JayF | Would it be possible to get https://review.opendev.org/c/openstack/project-config/+/931991 merged today and the initial groups populated (I am a reasonable choice for that if you want)? I'd like to have this new core structure in place before the end of the PTG. | 19:08 |
| fungi | make it so | 19:15 |
| fungi | er, engage? | 19:15 |
| fungi | i can add rpittau as the initial user in those groups once the deploy job completes | 19:15 |
| opendevreview | Merged openstack/project-config master: Proposed new Ironic core structure https://review.opendev.org/c/openstack/project-config/+/931991 | 19:20 |
| JayF | thanks fungi | 20:03 |
| JayF | rpittau: ^ those groups should be created now, if you want to get them populated I can do the follow-up changes for project-config. | 20:03 |
| fungi | i should be able to add him as the initial member after i get out of the meeting i'm in | 20:06 |
| JayF | he's hopefully gone until monday morning at this point, given his tz | 20:07 |
| JayF | so no rush | 20:07 |
| JayF | that message was left expecting he reads it Monday :D | 20:07 |
| fungi | perfect! | 20:07 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Increase size limits for some Gerrit caches https://review.opendev.org/c/opendev/system-config/+/932763 | 20:53 |
| clarkb | I tried to base that on what the docs say we should aim for while still being conservative about memory. In theory we should set those even larger but since memory is a scarce resource and the old defaults are 10m I suspect this will still be a big improvement | 20:54 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!