| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/project-config] 988041: Switch the requirements-constraints job to py311 https://review.opendev.org/c/openstack/project-config/+/988041 | 06:55 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/project-config] 988041: Switch the requirements-constraints job to py311 https://review.opendev.org/c/openstack/project-config/+/988041 | 06:58 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/project-config] 988041: Switch the requirements-constraints job to py311 https://review.opendev.org/c/openstack/project-config/+/988041 | 06:58 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Michal Nasiadka: [openstack/project-config] 988041: Switch the requirements-constraints job to py311 https://review.opendev.org/c/openstack/project-config/+/988041 | 07:53 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Bartosz Bezak: [openstack/project-config] 986493: kayobe: Add reviewer tier with +2 rights https://review.opendev.org/c/openstack/project-config/+/986493 | 09:31 | |
| -@gerrit:opendev.org- Stephen Finucane proposed: [openstack/project-config] 988094: Add nova-approvers to openstack/nova ACL https://review.opendev.org/c/openstack/project-config/+/988094 | 11:10 | |
| -@gerrit:opendev.org- Stephen Finucane proposed: [openstack/project-config] 988094: Add nova-reviewers to openstack/nova ACL https://review.opendev.org/c/openstack/project-config/+/988094 | 11:13 | |
| @jim:acmegating.com | Clark: it looks like there may be a problem with the temporary image storage: i see "401 Client Error: Unauthorized for url" in the zuul-launcher logs | 14:44 |
|---|---|---|
| @jim:acmegating.com | i'm looking into it | 14:44 |
| @fungicide:matrix.org | temporary image storage is flex swift? | 14:47 |
| @jim:acmegating.com | yep | 14:47 |
| @clarkb:matrix.org | corvus: the error with the previous container was 404 not found. It would show up when listing containers with openstack client but running container show also returned 404 | 14:49 |
| @clarkb:matrix.org | so 401 is a new one to me at least | 14:49 |
| @jim:acmegating.com | i *think* we may just need to flip the public access bit | 14:49 |
| @jim:acmegating.com | of course i can't compare to the prev container :| | 14:49 |
| @clarkb:matrix.org | oh are we anonymously reading the data back when fetching it rahter than authenticating? | 14:50 |
| @clarkb:matrix.org | if so then flipping that bit does make sense to me | 14:50 |
| @jim:acmegating.com | yep | 14:50 |
| @jim:acmegating.com | ok i will do so | 14:50 |
| @fungicide:matrix.org | makes sense, and would explain 401 unauthorized responses | 14:51 |
| @clarkb:matrix.org | fungi: for https://review.opendev.org/c/opendev/system-config/+/985834 did you check the held node? I think I'd like to proceed with that change if someone else has checked the held node. | 14:52 |
| @clarkb:matrix.org | fungi: then separately there is the therpad 2.7.3 upgrade whcih I haven't checked the held node (I checked 2.7.2 but haven't made it to the 2.7.3 update yet) | 14:52 |
| @jim:acmegating.com | the default in zuul-launcher is to assume public urls. the aws driver does have a code path where it can download images via s3 using authentication. we could add that to the openstack driver, but -- i think for opendev that public url is a benefit (anyone can just fetch our images), so i think we want to keep using public urls here. | 14:52 |
| @jim:acmegating.com | i flipped the switch and started anonymously downloading urls | 14:53 |
| @clarkb:matrix.org | ya I think in this case it should be fine. We don't treat these images as secret data or anything like that | 14:53 |
| @fungicide:matrix.org | i agree, public is better. it also aligns with our usual stance on openness and transparency in operations | 14:55 |
| @fungicide:matrix.org | Clark: i did not check the held node at the time, but will in a moment and can then approve if you're ready for it | 14:55 |
| @clarkb:matrix.org | also as a heads up I have to do a bunch of afternoon errands today (conflicting appointments and only one car mean I get to play taxi driver). Depending on how that goes I may not get the meeting agenda out until late, But I should still be able to get it out today | 14:55 |
| @clarkb:matrix.org | fungi: that would be great thank you | 14:56 |
| @clarkb:matrix.org | fungi: the child change is the one that held a node | 14:56 |
| @jim:acmegating.com | i think the launchers may need some prodding to retry these uploads; i will restart them | 14:56 |
| @fungicide:matrix.org | looks like 200.225.47.56 is the held haproxy | 14:57 |
| @clarkb:matrix.org | yup that is the IP I see in the log | 14:58 |
| @jim:acmegating.com | #status log set public access bit on zuul image storage swift container in rax-flex and restarted launchers to fix image uploads | 14:58 |
| @status:opendev.org | @jim:acmegating.com: finished logging | 14:58 |
| @jim:acmegating.com | okay looks like image uploads are doing their thing | 15:01 |
| @jim:acmegating.com | it may be a while until we actually see resolute images | 15:02 |
| @clarkb:matrix.org | thank you for fixing that | 15:06 |
| @fungicide:matrix.org | `release-volumes.py` has been running on mirror-update since some time yesterday, releasing the project.tarballs volume. this is blocking all other static volume updates. i'm lokking into it now | 16:21 |
| @fungicide:matrix.org | and looks like the ubuntu-ports mirror has exceeded its quota | 16:27 |
| @fungicide:matrix.org | i don't see any significant change/increase in usage for project.tarballs over the past week | 16:30 |
| @fungicide:matrix.org | `fs listquota` doesn't show much difference between the rw and ro volume use either (not even 1k), but maybe it's almost done syncing now | 16:32 |
| @clarkb:matrix.org | 104.239.175.4 is the held etherpad 2.7.3. I'm going to create a clarkb-test pad there now and start checking it | 16:43 |
| @clarkb:matrix.org | This is interesting 2.7.3 gives you a pad deletion token the first time you open a new pad if you are the creator of a pad | 16:45 |
| @clarkb:matrix.org | my second browser did not get the token when opening the same pad. I suspect this is fine. It might catch some people off guard, but functioanlly I think this is probably safe enough | 16:46 |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [opendev/system-config] 985834: Update Gitea to 1.26.1 https://review.opendev.org/c/opendev/system-config/+/985834 | 16:51 | |
| @clarkb:matrix.org | gitea09 has just updated | 16:55 |
| @clarkb:matrix.org | my client hits gitea10 so I'll check things momentarily once gitea10 updates | 16:55 |
| @clarkb:matrix.org | browsing gitea10 seems to work for me | 16:57 |
| @clarkb:matrix.org | all 6 backends should be upgraded now | 17:05 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/openstack/buildset/491239ae82ae4f06ad48a95ec6e7ac7a reports success. I am also able to clone system-config | 17:06 |
| @clarkb:matrix.org | Now to find something that has replicated after the upgrade | 17:06 |
| @fungicide:matrix.org | yeah, looks like it's working to me | 17:07 |
| @clarkb:matrix.org | https://review.opendev.org/c/starlingx/app-prometheus/+/986843 has replicated to https://opendev.org/starlingx/app-prometheus/commit/839ca6b0a62691bb4d2f96d5ff7ef1aa60841419 | 17:13 |
| @clarkb:matrix.org | so replication also looks good | 17:14 |
| @fungicide:matrix.org | agreed | 17:14 |
| @clarkb:matrix.org | I've done some very early meeting agenda updates. Please let me know if there are other updates you'd like to see made. And as noted earlier I probably won't send that out until very late today, but should still get it out for some value of today (probably in my timezone at least) | 17:53 |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [opendev/system-config] 987871: Patch postorius https://review.opendev.org/c/opendev/system-config/+/987871 | 19:03 | |
| @fungicide:matrix.org | #status log Increased the mirror.ubuntu-ports AFS volume quota by 100GB (from 1TB to 1.1TB) bringing utilization down to 91% | 19:46 |
| @status:opendev.org | @fungicide:matrix.org: finished logging | 19:46 |
| @fungicide:matrix.org | i've got the lock for that volume held and am manually running the mirror update script now | 19:46 |
| @fungicide:matrix.org | it's only behind by ~1.5 weeks so hopefully shouldn't take long to complete | 19:47 |
| @fungicide:matrix.org | the project.tarballs release completed roughly half an hour ago, unblocking the other volumes | 19:48 |
| @fungicide:matrix.org | looks like reprepro may have made a ton of zero-length package writes when the quota was exceeded, so i'll probably have to do a ton of manual surgery to get it updating successfully again | 20:05 |
| @fungicide:matrix.org | yeah, in the end it reports an incorrect size on two debs. i'll have to pick that up tomorrow, but at least the quota should be sufficient again for now | 20:57 |
| @clarkb:matrix.org | thank you for digging into that | 20:58 |
| @clarkb:matrix.org | I've made it through the Gerrit 3.13 release notes and edited https://etherpad.opendev.org/p/gerrit-upgrade-3.13 with info. Mostly generated a large TODO list more than anything else | 21:09 |
| @clarkb:matrix.org | the next step is going to be working through that stuff with a held node to test and then finalizing the actual upgrade process | 21:09 |
| @clarkb:matrix.org | If possible I'd like to do that testing against https://review.opendev.org/c/opendev/system-config/+/987375 built images if we can maybe plan to land that sometime this week and update 3.12 to 3.12.7 | 21:10 |
| @clarkb:matrix.org | I don't see anything in there that super concerns me. Lots of things to double check just to amke sure they do what I expect them to. But on the whole this upgrade looks pretty straightforward. Which is good because I think that means late May/early June is still a possibility | 21:11 |
| @clarkb:matrix.org | we can discuss timing of the Gerrit image change in our meeting tomorrow (its on the agenda now) | 21:12 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!