| @dan_with:matrix.org | I'm PTO tomorrow, but the federated api key problem is probably related to what we are seeing with other accounts: unknown reason for account lockouts | 01:08 |
|---|---|---|
| @mordred:waterwanders.com | "unknown reason" is the best reason | 01:59 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [opendev/system-config] 1004970: Start mirroring node:26-trixie https://review.opendev.org/c/opendev/system-config/+/1004970 | 07:38 | |
| @mordred:waterwanders.com | fwiw - nobody anywhere in codesearch is using any of the -bookworm versions we're mirroring. I'm not sure if there is a policy or thinking about removing things. It's probably not much to keep those around - but just though I'd mention. | 07:39 |
| @clarkb:matrix.org | mordred: yes zuul was the last holdout iirc. We've been pretty aggressive about dropping things once we don't have any immediate network users | 08:52 |
| @mordred:waterwanders.com | Do we clear from quay? Or just stop mirroring? | 09:01 |
| @mnasiadka:matrix.org | Is it me, or the networking issues in VexxHost are still visible? | 09:36 |
| @fungicide:matrix.org | mnaser: yeah, seems so... | 12:47 |
| @fungicide:matrix.org | er mnasiadka (but mnaser too i suppose) | 12:47 |
| @fungicide:matrix.org | `PING review.opendev.org (2604:e100:1:0:f816:3eff:fe31:926a) 56 data bytes | 12:47 |
| From 2604:e100:1::1 icmp_seq=1 Destination unreachable: Address unreachable` | ||
| @mnasiadka:matrix.org | Maybe we need to raise a proper case with VexxHost support team? | 12:48 |
| @fungicide:matrix.org | that's what i get from anywhere on the internet i've tried | 12:48 |
| @fungicide:matrix.org | oddly i can still `ssh -6` to it just fine | 12:48 |
| @mnasiadka:matrix.org | In Magnum we have also been observing jobs failing in VexxHost tenant when fetching kubectl (https://zuul.opendev.org/t/openstack/build/7f211e717dbf4290b7a9375bf6063f09/log/job-output.txt#28713) | 13:59 |
| @fungicide:matrix.org | so getting a nonzero (4?) exit code from wget when attempting to fetch https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl | 14:01 |
| @fungicide:matrix.org | hard to know what exactly happened since wget's -q option is set | 14:02 |
| @fungicide:matrix.org | looks like it's set to try twice and continue partial downloads | 14:04 |
| @fungicide:matrix.org | also i guess the earlier wget to the same site worked a fraction of a second before | 14:05 |
| @fungicide:matrix.org | dl.k8s.io seems to be a dns round-robin between 4 different ipv6 and 4 different ipv4 addresses, from what i can see, hosted at the fastly cdn | 14:07 |
| @fungicide:matrix.org | exit status 4, per wget's manpage, means "Network failure." | 14:08 |
| @mnasiadka:matrix.org | sounds like similar v6 issues as Gerrit? | 14:12 |
| @mnasiadka:matrix.org | We probably can force v4 to work around it | 14:13 |
| @fungicide:matrix.org | it could be, hard to know without more output from wget | 14:13 |
| @fungicide:matrix.org | you could consider dropping the `-q` option from that command in a dnm change and then rechecking until you catch the same failure mode. we could also set an autohold if it's reproducible without too many unrelated failures triggering the hold first | 14:14 |
| -@gerrit:opendev.org- Stephen Finucane marked as active: [openstack/project-config] 1003848: Add nova-approvers to openstack/nova ACL https://review.opendev.org/c/openstack/project-config/+/1003848 | 14:21 | |
| @jim:acmegating.com | fungi: mnasiadka we set an autohold a few days ago, but it only caught non-network-related failures. https://zuul.opendev.org/t/openstack/autohold/0000000303 but if you need any "control" nodes to test anything out, there are two vexxhost nodes you can play with. | 14:52 |
| @fungicide:matrix.org | i have a feeling it's luck of the draw in getting scheduled to an affected hypervisor host | 15:07 |
| -@gerrit:opendev.org- Dmitriy Rabotyagov proposed: [opendev/system-config] 1005081: Update mirrored Ceph versions https://review.opendev.org/c/opendev/system-config/+/1005081 | 16:06 | |
| -@gerrit:opendev.org- Stephen Finucane proposed: [openstack/project-config] 1003848: Add nova-approvers to openstack/nova ACL https://review.opendev.org/c/openstack/project-config/+/1003848 | 16:49 | |
| -@gerrit:opendev.org- Stephen Finucane proposed: | 17:17 | |
| - [openstack/project-config] 1003840: Remove references to ironic-core https://review.opendev.org/c/openstack/project-config/+/1003840 | ||
| - [openstack/project-config] 1005094: Add approvers to the "special" groups https://review.opendev.org/c/openstack/project-config/+/1005094 | ||
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!