| -@gerrit:opendev.org- hushuai proposed: [zuul/zuul-jobs] 982189: ensure-golangci-lint: Check installed version, as we do for ensure-go https://review.opendev.org/c/zuul/zuul-jobs/+/982189 | 04:03 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 982231: Add support for building Ubuntu Resolute (26.04) https://review.opendev.org/c/openstack/diskimage-builder/+/982231 | 12:07 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 982231: Add support for building Ubuntu Resolute (26.04) https://review.opendev.org/c/openstack/diskimage-builder/+/982231 | 13:25 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 982231: Add support for building Ubuntu Resolute (26.04) https://review.opendev.org/c/openstack/diskimage-builder/+/982231 | 13:51 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [opendev/zuul-providers] 982182: Add Ubuntu resolute image build job https://review.opendev.org/c/opendev/zuul-providers/+/982182 | 14:22 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 982231: Add support for building Ubuntu Resolute (26.04) https://review.opendev.org/c/openstack/diskimage-builder/+/982231 | 15:07 | |
| -@gerrit:opendev.org- Jason Paroly proposed: [openstack/diskimage-builder] 982098: Fix unused function arguments https://review.opendev.org/c/openstack/diskimage-builder/+/982098 | 15:25 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [opendev/system-config] 982263: Add mnasiadka to statusbot users https://review.opendev.org/c/opendev/system-config/+/982263 | 15:38 | |
| @clarkb:matrix.org | I went ahead and approved ^ so that mnasiadka can record that the openmetal mirror has been replaced with statusbot | 15:40 |
|---|---|---|
| @fungicide:matrix.org | oh, thanks i had missed that one | 15:43 |
| @clarkb:matrix.org | but that also reminds me we discussed making the variable private. I think that may be as simple as adding all the current public values to private vars on bridge and then adding new values privately and removing the public values | 15:44 |
| @fungicide:matrix.org | yes, that should just work | 15:56 |
| @clarkb:matrix.org | https://etherpad.opendev.org/p/B8O2w60ffoOA3JdCeE5O how does this look for announcing the bionic cleanup/removal | 16:00 |
| @fungicide:matrix.org | lgtm. thanks! | 16:02 |
| @clarkb:matrix.org | awesome I'll get that sent out shortly after there has been a change for additional feedback | 16:04 |
| @clarkb:matrix.org | that announcement should land in inboxes momentarily | 16:39 |
| @fungicide:matrix.org | confirmed, it's in mine already | 16:41 |
| @clarkb:matrix.org | did anyone submit a ticket to ovh per arnaudm's request? | 16:45 |
| @clarkb:matrix.org | I wasn't really around during the problem so I'm not sure I'm the best person to try and capture the issue | 16:45 |
| @clarkb:matrix.org | Luca indicates they are not using the incubated vector api in the jvm for their Gerrit deployments. I think we don't need to be the experimenters for that. If no vector stuff si good enough for them then it is probably good enoug hfor us | 16:51 |
| -@gerrit:opendev.org- Zuul merged on behalf of Michal Nasiadka: [opendev/system-config] 982263: Add mnasiadka to statusbot users https://review.opendev.org/c/opendev/system-config/+/982263 | 16:53 | |
| @mnasiadka:matrix.org | Deploy pipeline failed on ^^ | 17:06 |
| @clarkb:matrix.org | hrm the base job failed for some reason so the other jobs didn't run | 17:07 |
| @clarkb:matrix.org | mnasiadka: the log for that job is in /var/log/ansible/base.yaml.log on bridge | 17:07 |
| @fungicide:matrix.org | i usually search forward for `failed=[^0]` to find the task and host, then search backword for that host to find the error message | 17:08 |
| @clarkb:matrix.org | mirror01.iad3.openmetal.opendev.org is still in the inventory for some reason | 17:09 |
| @clarkb:matrix.org | and is now unreachable (this is expected because mnasiadka and I just deleted ti) | 17:09 |
| @fungicide:matrix.org | s/backword/backward/ | 17:09 |
| @clarkb:matrix.org | oh heh https://review.opendev.org/c/opendev/system-config/+/981958 didn't actually merge | 17:09 |
| @clarkb:matrix.org | I've rechecked it. In the meantime I think we can add mirror01.iad3.openmetal.opendev.org to the emergency file and then reenqueue the deployment for the eavesdrop statusbot update? | 17:10 |
| @clarkb:matrix.org | or we can just let daily runs update things and mnasiadka can do the status log tomorrow after the 0200 ish daily run? | 17:11 |
| @clarkb:matrix.org | I know its late for mnasiadka so not sure we need to try and solve this immediately | 17:11 |
| @clarkb:matrix.org | mnasiadka: do you have a preference? | 17:11 |
| @clarkb:matrix.org | (I'm happy to wait and let you enjoy your evening. Getting that unmerged change should land the broader issue and then daily runs will sync up statusbot | 17:13 |
| @clarkb:matrix.org | I have tested the gerrit 3.12 -> 3.11 downgrade process and captured notes about the process in https://etherpad.opendev.org/p/gerrit-upgrade-3.12 During the upgrade I failed to test that users are logged out (expected due to caches being recreated). I logged in prior to the downgrade and confirmed that you do need to log back in afterwards so I expect the same behavior on upgrade. The main consideration with the downgrade is that index versions change and we need an offline reindex. But otherwise the process seems straightforward | 17:21 |
| @clarkb:matrix.org | I also don't want to get overly excited but even on the test node the h2 v2 backing db files appear smaller than the v1 files | 17:23 |
| @clarkb:matrix.org | The server barely has any content in it though so its hard to extrapolate that to our production system. But it does give me hope that h2 v2 improves upon its disk consumption | 17:23 |
| @fungicide:matrix.org | neat | 17:24 |
| @clarkb:matrix.org | the new MVStore system (the default for h2 v2) says this "It is intended to be fast, simple to use, and small." https://www.h2database.com/html/mvstore.html | 17:26 |
| @clarkb:matrix.org | here's hoping this produces good results for us | 17:26 |
| @clarkb:matrix.org | I think that upgrade planning document is in a state where it can be reviewed by others now. I'm not fully ready to do the upgrade. In particular I've got that stack of ~4 changes now that I want to aldn and restart prod gerrit 3.11 on before we go to 3.12. Then I want to retest the upgrade and downgrade process using the new 3.11.10 and 3.12.6 images (I don't expect changes but good for completeness). And at that point if nothing new comes up I think we'll be ready | 17:29 |
| @clarkb:matrix.org | so I think April 12/13 is looking good for the upgrade. The weekend after (17-20) is not good for me as my daughter is going to have surgery Friday with recovery planned over the weekend. | 17:30 |
| @clarkb:matrix.org | * so I think April 12/13 is looking good for the upgrade. The weekend after (17-20) is not good for me as my daughter is going to have surgery Friday the 17th with recovery planned over the weekend. | 17:31 |
| @fungicide:matrix.org | i plan to be around on the 12th and 13th | 17:32 |
| @clarkb:matrix.org | cool I think I will continue to target those days as I should be able to get images and config updated in prod and rerun upgrade tests quickly all before the 12th and we'll be ready by then | 17:33 |
| @clarkb:matrix.org | I'll make it official on Monday if nothing else comes up between now and then when is about 2 weeks of notice | 17:33 |
| @fungicide:matrix.org | thanks! | 17:33 |
| @fungicide:matrix.org | i'm disappearing for a while, christine's favorite restaurant out here is reopening for the season and it's about a 45-minute drive up the island, so will be gone for a few hours but plan to check back in later | 18:14 |
| @mnasiadka:matrix.org | Clark: sorry, missed your message - happy to wait until tomorrow for the statusbot message ;) | 18:15 |
| @clarkb:matrix.org | mnasiadka: ok enjoy your evening. I'll keep an eye on that chaneg and try to get it in. It looks like it may timeout again. I think the current job is running in ovh gra1 | 18:15 |
| @clarkb:matrix.org | not sure if that is due to the openafs compilations being slow or what. But I'll try to take a lcoser look after the logs post and recheck ti at the same time as theoretically another cloud may perform differently | 18:16 |
| @clarkb:matrix.org | fungi: enjoy. I guess winter is officially over if the good spots are reopening for the season | 18:16 |
| @clarkb:matrix.org | fwiw I think the check job just snuck in under the time limit. I'll look at the logs for it anyway | 18:30 |
| @clarkb:matrix.org | it almost looks like apt package installation is just slow. The test infra tox run which installs from pip and then runs our testinfra test cases runs relatively quickly. This cloud region is the one most distant from our openafs fileservers. Maybe the caches are cold and the openafs cost for high rtt is hitting us in ovh gra right now? | 18:34 |
| @clarkb:matrix.org | it took 8 minutes and 21 seconds to install openafs kernel modules. This is the step that runs the dkms build so not super surprising | 18:39 |
| @clarkb:matrix.org | but even installing our base set of packages from the base role took almost 4 minutes. So I suspect a good amount of time is not spent compiling but rather downloading packges? | 18:40 |
| -@gerrit:opendev.org- Zuul merged on behalf of Michal Nasiadka: [opendev/system-config] 981958: Remove mirror01.iad3.openmetal https://review.opendev.org/c/opendev/system-config/+/981958 | 19:03 | |
| @clarkb:matrix.org | I half expected https://review.opendev.org/c/opendev/system-config/+/982180 to fail due to all the changes in python3.13 and python3.14, but I guess it is good news that it didn't. If we think skipping 3.13 entirely is a reasonable thing to do (its one less set of things to work through) then I think it should be safe to proceed with this change to add 3.14 images. Granted you should still review the change and make sure I didn't do anything silly in it | 20:07 |
| @clarkb:matrix.org | I'm always worried I will accidentally a tag name and overwrite 3.12 images with 3.14 content or something along those lines | 20:07 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 982306: Add python314 unittest jobs https://review.opendev.org/c/zuul/zuul-jobs/+/982306 | 20:22 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | 20:25 | |
| @clarkb:matrix.org | That is a couple of changes to exercise this all a bit more and get more information | 20:27 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 982306: Add python314 unittest jobs https://review.opendev.org/c/zuul/zuul-jobs/+/982306 | 20:28 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | 20:29 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 982306: Add python314 unittest jobs https://review.opendev.org/c/zuul/zuul-jobs/+/982306 | 20:43 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | 20:44 | |
| @jim:acmegating.com | https://review.opendev.org/c/openstack/diskimage-builder/+/982231 from mnasiadka to add resolute to dib looks like it's mainly adding test jobs, so the success result on my change https://review.opendev.org/982182 to build it is probably a good/expected result, right? | 20:47 |
| @jim:acmegating.com | if so, should we go ahead and merge that? | 20:48 |
| @jim:acmegating.com | or do we want to hold it and just use pyenv for eager beavers like zuul for now? | 20:48 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 20:49 | |
| - [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | ||
| - [opendev/lodgeit] 982311: Uncap Werkzeug for python 3.14 compatibility https://review.opendev.org/c/opendev/lodgeit/+/982311 | ||
| @clarkb:matrix.org | corvus: yes the dib change is only adding testing it didn't modify the ubuntu-minimal element that we rely on. I agree we probably can proceed with 982182. | 20:50 |
| @clarkb:matrix.org | I mostly wanted to get both streams moving forward as they don't strictly depend on one another | 20:50 |
| @clarkb:matrix.org | I've +2'd the zuul-providers change now too | 20:51 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 20:58 | |
| - [opendev/lodgeit] 982311: Uncap Werkzeug for python 3.14 compatibility https://review.opendev.org/c/opendev/lodgeit/+/982311 | ||
| - [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | ||
| @fungicide:matrix.org | okay, back | 21:04 |
| @fungicide:matrix.org | looking at the z-p change | 21:06 |
| @fungicide:matrix.org | lgtm, in it goes | 21:07 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 21:28 | |
| - [opendev/lodgeit] 982311: Uncap Werkzeug for python 3.14 compatibility https://review.opendev.org/c/opendev/lodgeit/+/982311 | ||
| - [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | ||
| @clarkb:matrix.org | it is a bit early to declare victory, but I have to say I am really impressed with Werkzeug's code compatibility over time. It really seems like they've maintained the apis/interfaces and the only problems are moving things around. We'll see if those changes find problems with arguments and such that need updating though. I really hope not because I want to be impressed | 21:29 |
| @clarkb:matrix.org | bah foiled by the community secure-cookie package for werkzeug. The issue is fixed by https://github.com/pallets-eco/secure-cookie/pull/90 but they haven't made a release in years https://github.com/pallets-eco/secure-cookie/issues/101. I'm thinking maybe we just vendor the code? | 21:39 |
| @fungicide:matrix.org | looks like there's about 900 lines of python in that library if you strip away packaging asnd tests... how much would we vendor? | 21:46 |
| @fungicide:matrix.org | presumably we only need a fraction of the routines from it? | 21:47 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 21:54 | |
| - [opendev/lodgeit] 982311: Uncap Werkzeug for python 3.14 compatibility https://review.opendev.org/c/opendev/lodgeit/+/982311 | ||
| - [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | ||
| @clarkb:matrix.org | fungi: ^ I think that much at a maximum. If we want to trim it down further that may be possible. But I went for the copy the actual secure cookie implementation and the things it imports at a file level approach | 21:54 |
| @clarkb:matrix.org | its possible we could trim things down further to only the stuff we actually need but doing it at a file level makes easily traceable | 21:55 |
| @clarkb:matrix.org | Looks like there is further incompatibility that hasn't been fixed so now we get to modify the forked copy ugh | 21:59 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 22:03 | |
| - [opendev/lodgeit] 982311: Uncap Werkzeug for python 3.14 compatibility https://review.opendev.org/c/opendev/lodgeit/+/982311 | ||
| - [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | ||
| @clarkb:matrix.org | maybe this is the secret to making your code super future portable: just kick out all the bits of the code base that will need updates :) I actually don't think that is what happened here. But it is funny to think about as a possible reason for that | 22:03 |
| @clarkb:matrix.org | I need to pop out soon for a bike ride while the sun is out so I'm not sure how many more iterations I'll do. This does seem like ti is probably close and possible | 22:04 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 22:14 | |
| - [opendev/lodgeit] 982311: Uncap Werkzeug for python 3.14 compatibility https://review.opendev.org/c/opendev/lodgeit/+/982311 | ||
| - [opendev/lodgeit] 982307: Start testing python3.14 https://review.opendev.org/c/opendev/lodgeit/+/982307 | ||
| @clarkb:matrix.org | ya its still failing now on the type of the serialized cookie. Werkzeug wants a string and secure-cookie as vendored provides bytes | 22:23 |
| @clarkb:matrix.org | But I'm going to pop out now | 22:23 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!