| @priteau:matrix.org | Hello. Is there an ongoing issue with Zuul? Some builds have been in queued state for many hours | 09:45 |
|---|---|---|
| @mnasiadka:matrix.org | That's probably a fallout of the DDoS that is/was happening towards Gerrit, we've implemented some user agent filtering measures - it should be a bit better now. | 09:56 |
| @fungicide:matrix.org | Pierre Riteau: which builds specifically? are they all arm64 builds? it looks like the provider we get them from may be in trouble at the moment | 13:39 |
| @priteau:matrix.org | Indeed, it appears to affect aarch64 jobs specifically | 13:40 |
| @fungicide:matrix.org | yeah, we have only one arm provider these days (osuosl), so if we can't boot server instances there then jobs will wait indefinitely for them | 13:45 |
| @fungicide:matrix.org | i'll see if i can figure out what's happening with it, but folks in oregon are probably not at their desks this early | 13:46 |
| @mordred:waterwanders.com | what's a desk? | 13:52 |
| @clarkb:matrix.org | I think it may be finals week | 13:53 |
| @fungicide:matrix.org | oof | 13:55 |
| @clarkb:matrix.org | Confirmed today is the last day of finals. Commencement ceremonies begin tomorrow | 13:57 |
| @clarkb:matrix.org | Gerrit says they made stable branches for its-base. So we can try to use those instead of my hacky workaround | 14:06 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 15:32 | |
| - [opendev/system-config] 991946: Add Gerrit 3.14 image builds and testing https://review.opendev.org/c/opendev/system-config/+/991946 | ||
| - [opendev/system-config] 992342: Test Gerrit 3.13 to 3.14 upgrades https://review.opendev.org/c/opendev/system-config/+/992342 | ||
| @clarkb:matrix.org | I checked the stable branches look right on its-base and they do so updated ^. I will abandon my workaround change now | 15:33 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [opendev/system-config] 993149: Block git clients from Gerrit's REST API https://review.opendev.org/c/opendev/system-config/+/993149 | 16:09 | |
| @fungicide:matrix.org | private e-mail tends to be a bit of a black hole for me, since i already have random project contributors e-mailing me directly about things that they should be using mailing lists or irc/matrix for | 16:12 |
| @fungicide:matrix.org | er, that was meant for a different room | 16:12 |
| @mordred:waterwanders.com | it's still true though | 16:24 |
| @scott.little:matrix.org | Starlingx is seeing a lot of build failures today . We are wondering if anything changes on the opendev side .... Seems like it denies access to git client version 2.43.0 specifically, which is exactly what we have. Looks like a filtering rule on the server side for the User-Agent header. Probably some kind of anti-abuse measure or something. | 16:39 |
| This fails: | ||
| curl -H "User-Agent: git/2.43.0" -H "Accept-Encoding: deflate, gzip, br, zstd" -H "Accept-Language: en-US, *;q=0.9" -H "Pragma: no-cache" -H "Git-Protocol: version=2" -vvv https://review.opendev.org/starlingx/jenkins-pipelines.git/info/refs?service=git-upload-pack | head -c 200 | ||
| But any other git version string succeeds. | ||
| @clarkb:matrix.org | scott.little: yes we had to do that to mitigate a ddos | 16:40 |
| @clarkb:matrix.org | scott.little: 993149 is the more graceful solution that should get thinsg working for you again | 16:41 |
| @clarkb:matrix.org | however | 16:41 |
| @clarkb:matrix.org | you should not be talking to gerrit directly from your CI jobs | 16:41 |
| @clarkb:matrix.org | Zuul jobs should rely on the local git repos provided by zuul | 16:41 |
| @clarkb:matrix.org | everyone else should be relying on the gitea farm | 16:41 |
| @scott.little:matrix.org | 993149 ? | 16:42 |
| @clarkb:matrix.org | scott.little: change 993149 which is linked just a bit above this discussion by gerrit bot | 16:42 |
| @scott.little:matrix.org | We were using the gerrit as a way to access jenkins pipeline changes that were up for review ... and run them through a test build. Is there an alternative url where we can pull the same content? | 16:48 |
| @clarkb:matrix.org | yes, as mentioned you should use gitea for that. So https://opendev.org/starlingx/jenkins-pipelines | 16:48 |
| @clarkb:matrix.org | and if you switch to that now it should work while we sort out the rules for Gerrit proper | 16:49 |
| @scott.little:matrix.org | so the paths to review content will work there as well? Is there an propogation delay significant enough to worry about ? | 16:52 |
| @fungicide:matrix.org | it's generally a fraction of a second | 16:53 |
| @clarkb:matrix.org | yes and yes. Gerrit replication does have a slight delay but it is quite minimal. I would expect that if you waited 10 seconds you'd never have a problem | 16:53 |
| @fungicide:matrix.org | between when someone pushes a change/revision to gerrit and when that same ref appears on the gitea backends | 16:53 |
| @clarkb:matrix.org | (that does scale with the size of the repo so if you were pulling from nova it would be longer) | 16:53 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: | 16:59 | |
| - [opendev/system-config] 993149: Block git clients from Gerrit's REST API https://review.opendev.org/c/opendev/system-config/+/993149 | ||
| - [opendev/system-config] 993155: Don't build gerrit images when changing tests https://review.opendev.org/c/opendev/system-config/+/993155 | ||
| @clarkb:matrix.org | ok I'm +2 on both of those changes. Still don't know yet if the testinfra test cases are happy in 993149, but if they are I'm good to proceed | 17:28 |
| @fungicide:matrix.org | in theory they prove it works as intended and doesn't break anything | 17:36 |
| -@gerrit:opendev.org- Brian Rosmaita proposed: [openstack/project-config] 993177: [cinder] Make Review-Priority sticky https://review.opendev.org/c/openstack/project-config/+/993177 | 19:54 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 993155: Don't build gerrit images when changing tests https://review.opendev.org/c/opendev/system-config/+/993155 | 20:16 | |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [opendev/system-config] 993183: Don't trigger system-config-run-gitea for Gerrit https://review.opendev.org/c/opendev/system-config/+/993183 | 20:46 | |
| -@gerrit:opendev.org- Brian Rosmaita proposed: [openstack/project-config] 993184: Retire cinderlib https://review.opendev.org/c/openstack/project-config/+/993184 | 20:57 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 993149: Block git clients from Gerrit's REST API https://review.opendev.org/c/opendev/system-config/+/993149 | 21:32 | |
| -@gerrit:opendev.org- Clark Boylan proposed: | 21:35 | |
| - [opendev/system-config] 991946: Add Gerrit 3.14 image builds and testing https://review.opendev.org/c/opendev/system-config/+/991946 | ||
| - [opendev/system-config] 992342: Test Gerrit 3.13 to 3.14 upgrades https://review.opendev.org/c/opendev/system-config/+/992342 | ||
| @clarkb:matrix.org | those two new patchsets are a rebase onto 993149 so that the new 3.14 image builds also don't trigger on testinfra updates | 21:35 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!