| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 00:27 |
|---|---|---|
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 01:10 |
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 02:08 |
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 03:54 |
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 04:32 |
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 05:18 |
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] testing backup purge idea https://review.opendev.org/c/opendev/system-config/+/933700 | 06:36 |
| opendevreview | Ian Wienand proposed opendev/system-config master: backups: add retirement and purge lists https://review.opendev.org/c/opendev/system-config/+/933700 | 08:22 |
| opendevreview | Ian Wienand proposed opendev/system-config master: backups: add retirement and purge lists https://review.opendev.org/c/opendev/system-config/+/933700 | 09:04 |
| ianw | infra-root: can we keep an eye on https://github.com/readthedocs/readthedocs.org/issues/11733 for https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/933740 | 09:05 |
| ianw | i probably missed discussion of the debugging, sorry :/ have we manually tried a POST with the openstackci credentials and know what the RTD API gives us back? the jobs are no_log: so not much help | 09:07 |
| *** dhill is now known as Guest8048 | 12:24 | |
| fungi | ianw: i don't think anyone has tested manually, but tracing the code it looks like after https://github.com/readthedocs/readthedocs.org/pull/11083 it should give a http 400 bad request error with detail starting "This webhook doesn't have a secret configured..." and referencing https://blog.readthedocs.com/security-update-on-incoming-webhooks/ | 12:48 |
| fungi | the code doesn't seem to check whether the request has http basic auth credentials supplied | 12:49 |
| fungi | definitely worth confirming though, yes | 12:49 |
| amorin | hey, can someone give me a hint on what is wrong here: https://review.opendev.org/c/openstack/mistral-tempest-plugin/+/932960 | 14:51 |
| fungi | amorin: because your depends-on is to a change in another project which doesn't share a change queue, you have to wait until it merges before approving that change | 14:58 |
| amorin | ah!! | 14:59 |
| amorin | ok thanks | 14:59 |
| clarkb | re video how tos I realized probably the easiest one for me to bootstrap with is just a straightforward git for opendev/gerrit video. The reason behind that is I should be able to do everything as a demo in a single terminal window which simplifies the production side. I figured I'd talk about cloning from gitea but pushing to gerrit a quick git review setup thing and then show how I | 15:13 |
| clarkb | use git rebase to edit changes in stacks or insert new changes for test fixes within existing stacks. Are there other common git things people do when dealingwith gerrit? | 15:13 |
| fungi | that sounds ideal, yep | 15:37 |
| fungi | stepping out to lunch, bbiab | 15:37 |
| clarkb | this is not concrete diagnosis of ci registry problems of any sort but I see it handling tls negotiation problems and continuing on. In these cases it appears to be related to tls version mismatches between client and server | 16:17 |
| clarkb | theory time: Maybe it wasn't handling these issues with clients properly before which wedged its ability to reuse those threads for subsequent clients or something | 16:17 |
| clarkb | I see version too low and unsupported cipher messages | 16:18 |
| clarkb | just a hunch at this point but time should tell if this is more reliabkle | 16:18 |
| clarkb | and just to rule out cherrypy doing a weird thing and using really old tls I've got openssl s_client negotiating tls 1.3 so I don't think that is the cse | 16:34 |
| clarkb | I'm looking at Gerrit upgrade planning and given November is looking full of distractions what do we think about aiming for the first week of december? | 17:59 |
| fungi | that works fine for me | 17:59 |
| clarkb | December 6th looks like would be the Friday | 17:59 |
| clarkb | I think that gives us time to work through the change log and test what we need/want to test through November too | 17:59 |
| clarkb | there are a number of new features and chagnes that I think are potentially interesting to us (like configuring the cache pruning time instead of 0100, more configurable log file rotation and so on) | 18:01 |
| clarkb | so definitely some things we should look into between now and then | 18:01 |
| fungi | yeah, the 6th is clear on my schedule | 18:01 |
| clarkb | there are also new limitations on submit requirements like a max of 50 label predicates per rule. I don't think we have any with that many | 18:02 |
| clarkb | I'll start putting an upgrade document together I guess | 18:04 |
| fungi | seems unlikely in our case, yes | 18:05 |
| clarkb | upgrading the server is also on the todo list but I'm thinking do the service upgrade first since that doesn't come with IP addr changes and is something that is downgradable if necessary | 18:06 |
| fungi | makes sense | 18:06 |
| clarkb | 3.10 does also require java 17 but we've already upgraded | 18:08 |
| clarkb | oh interesting they have a minimum 17 minor version too of 17.0.5 /me checks the debian bookworm package | 18:08 |
| clarkb | https://packages.debian.org/bookworm/openjdk-17-jdk that is 17.0.13 + debian patches so we should be good and our CI testing hasn't complained | 18:10 |
| clarkb | despite the large number of changes compared to the previous upgrade (I didn't actually count this is just my perception) the upgrade seems straightforward. Every index gets a new version but things can reindex online | 18:11 |
| clarkb | slowly starting to build out content in https://etherpad.opendev.org/p/gerrit-upgrade-3.10 | 18:19 |
| clarkb | ok that document should be fairly complete without additional testing/information. Next step is holding a node then using that to test. I'll start on that after lunch today | 18:53 |
| clarkb | I'm collecting another show-caches against review.opendev.org now ~24 hours ish after the prior restart | 20:07 |
| clarkb | cache disk usage has gone up a bit. I believe that is expected bceause we're only pruning once a day and we're allowing a large floor to exist so we prune to that larger floor then grow from there | 20:11 |
| clarkb | memory usage overall looks sane and is not growing unsustainably from where we started post restart | 20:11 |
| clarkb | and we exclude caches from backups so the bigger cache files shouldn't impact us there | 20:12 |
| fungi | awesome, sounds like you guessed close enough on the first attempt | 20:12 |
| clarkb | overall I think this is working as expected and in theory we'll get better performance out of it | 20:12 |
| opendevreview | Clark Boylan proposed opendev/system-config master: DNM Forced fail on Gerrit to test the 3.10 upgrade https://review.opendev.org/c/opendev/system-config/+/893571 | 20:14 |
| clarkb | I've got an autohold in place for each gerrit run (3.9 and 3.10) against ^ and I cleaned up the etherpad hold from that upgrade testing | 20:14 |
| corvus | clarkb: re registry, yes my previous analysis led me to think that we gradually lost threads until the pool was exhausted, so your theory comports with that. is the only change so far the rebuild with newer libs? | 22:36 |
| ianw | fungi: per https://github.com/readthedocs/readthedocs.org/issues/11733 perhaps we need to add a blank dict to the | 22:49 |
| ianw | ... POST? but seems from that it _should_ work | 22:50 |
| clarkb | corvus: that was the only change I made yes | 23:01 |
| clarkb | corvus: I think the container itself is newer too so newer openssl c lib as well as newer cheroot server | 23:01 |
| corvus | ++ | 23:04 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!