| -@gerrit:opendev.org- OpenStack Proposal Bot proposed: [openstack/project-config] 976707: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/976707 | 02:22 | |
| -@gerrit:opendev.org- Zuul merged: [openstack/project-config] 976707: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/976707 | 14:21 | |
| @fungicide:matrix.org | i'm looking into the broken ipv6 in vexxhost sjc1. i can't reach any of our systems there over ipv6 whether it's ping, ssh, https... tested from various parts of the internet | 14:27 |
|---|---|---|
| @fungicide:matrix.org | ipv4 is working fine though | 14:27 |
| @fungicide:matrix.org | from mirror03.sjc1.vexxhost i can ping both routers that it says are its default routes, but traceroutes outbound die at the very first hot | 14:34 |
| @fungicide:matrix.org | hop | 14:34 |
| @fungicide:matrix.org | trying to traceroute in, the last hop that responds is 2001:438:ffff::407d:1e11 registered to abovenet/zayo according to whois. it has no reverse dns so hard to tell where in their network that is, but the hop before it for me is my isp peering with zayo in atlanta | 14:37 |
| @fungicide:matrix.org | mnaser: are you aware of an incident in sjc1? i haven't checked any looking glasses yet but wonder if you're not announcing the prefix for there | 14:39 |
| @fungicide:matrix.org | maybe that's not it, hurricane electric's looking glass sees 2604:e100:3::/48 announced from AS33028 | 14:41 |
| @fungicide:matrix.org | inside sjc1 i can communicate between global v6 addresses, e.g. the mirror server and gitea lb | 14:43 |
| @fungicide:matrix.org | just don't seem to be able to get out of sjc1 over v6 at all | 14:44 |
| @fungicide:matrix.org | i also reached out in vexxhost's slack channel, in case that gets seen sooner | 14:48 |
| @fungicide:matrix.org | tobiash: you mentioned it in irc at 08:56 utc, was that when you first saw the problem, or were you digging into it for a while before that? | 14:50 |
| @tobias.henkel:matrix.org | fungi: we stumbled accross thus by accident while validating zuul jobs in an ipv6-only network | 14:52 |
| @tobias.henkel:matrix.org | * fungi: we stumbled accross this by accident while validating zuul jobs in an ipv6-only network | 14:52 |
| @fungicide:matrix.org | some users yesterday were reporting a similar-sounding problem, but i wasn't able to reproduce it from anywhere i had access to at the time. now i'm wondering if something died on, like, one core router and their flows were dying on it while mine were getting hashed to working routers in the core there | 14:52 |
| @tobias.henkel:matrix.org | The first occurrence I see in our jobs was actually yesterday 2026-02-12 13:04:04 UTC | 14:55 |
| @fungicide:matrix.org | thanks tobiash! | 14:56 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: | 15:16 | |
| - [opendev/zone-opendev.org] 976744: Temporarily drop IPv6 for Gitea https://review.opendev.org/c/opendev/zone-opendev.org/+/976744 | ||
| - [opendev/zone-opendev.org] 976745: Revert "Temporarily drop IPv6 for Gitea" https://review.opendev.org/c/opendev/zone-opendev.org/+/976745 | ||
| @fungicide:matrix.org | if this persists much longer, we can merge the first one there ^ (and i've marked the second wip for now) | 15:17 |
| @mnaser:matrix.org | [@fungicide:matrix.org](https://matrix.to/#/@fungicide:matrix.org) hi! Could you or anyone if they haven't already open a ticket and we'll check it out? One of our providers have been wonky for a bit and I'm not sure why thr advertisement isnt out but I can check this. | 15:18 |
| @mnaser:matrix.org | hmm, the prefix is being advertised to all peers | 15:31 |
| @mnaser:matrix.org | i wonder if its the gateway | 15:31 |
| @mnaser:matrix.org | fungi: can you or someone try and see if they can reach the ipv6 gateway from the vms ? | 15:32 |
| @mnaser:matrix.org | fungi: should be back in action now | 15:35 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [openstack/project-config] 976752: Put wandertracks repos into a shared queue https://review.opendev.org/c/openstack/project-config/+/976752 | 15:47 | |
| @clarkb:matrix.org | mnaser: I just checked a simple ping and that works. I'll let others who hit the problems confirm that it is working for them again. Thank you for looking at it | 15:47 |
| @fungicide:matrix.org | mnaser: thanks! sorry i had stepped away for a quick shower | 15:56 |
| @fungicide:matrix.org | tobiash: ^ it should be working again now hopefully | 15:57 |
| -@gerrit:opendev.org- Zuul merged on behalf of Monty Taylor https://matrix.to/#/@mordred:inaugust.com: [openstack/project-config] 976752: Put wandertracks repos into a shared queue https://review.opendev.org/c/openstack/project-config/+/976752 | 16:01 | |
| @mordred:waterwanders.com | thanks fungi | 16:09 |
| @tobias.henkel:matrix.org | yes, works again | 16:32 |
| @fungicide:matrix.org | i see the same security scanning company hit our tripwire on static02 again, but from a different ip address this time | 19:13 |
| @clarkb:matrix.org | If at first you don't succeed. Try try again. Is that the only hit so far? | 19:26 |
| @fungicide:matrix.org | yeah :/ | 19:27 |
| @clarkb:matrix.org | It does prove the hypothesis that some crawlers are ignoring the robots.txt entirely. But maybe not as many as we thought | 19:57 |
| @fungicide:matrix.org | or at least not as often, or maybe they're not looking at docs.opendev.org much compared to the other vhosts on that server, or other services of ours | 20:05 |
| @clarkb:matrix.org | Ya that was the other thought I had. We could try expanding to Openstack docs or lists maybe as those both seemed to get crawled pretty aggressively | 20:08 |
| @fungicide:matrix.org | the waf/robots change for lists.o.o is still awaiting approval, but yeah i could also look at how to do something similar to the mod_sed rule we used on docs.o.o but with mod_proxy there | 20:11 |
| @clarkb:matrix.org | fungi: if you get a chance https://review.opendev.org/c/opendev/gear/+/975335 and https://review.opendev.org/c/opendev/system-config/+/976693 should be safe updates as part of the trixie transition (in this case stopping image builds rather than updating them). The second is slightly less safe as it may trigger a gitea deployment (using the current gitea images not a rebuild) | 20:52 |
| @fungicide:matrix.org | looking | 20:53 |
| @fungicide:matrix.org | 975335 seemed like it had some consensus in here yesterday, so i went ahead and single-core approved | 20:55 |
| @fungicide:matrix.org | same for 976693 which is a no-brainer | 21:05 |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [opendev/gear] 975335: Drop container image builds https://review.opendev.org/c/opendev/gear/+/975335 | 21:13 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [opendev/system-config] 976693: Drop jinja-init and gitea-init container images and jobs https://review.opendev.org/c/opendev/system-config/+/976693 | 21:14 | |
| @clarkb:matrix.org | It did not run the gitea deployment job so that looks good to me | 21:17 |
| @fungicide:matrix.org | yep, i concur | 21:18 |
| @fungicide:matrix.org | friday cleanup ought to become a tradition | 21:19 |
| @fungicide:matrix.org | takin' out the trash | 21:19 |
| @fungicide:matrix.org | (not that our old work is trash, by any definition) | 21:19 |
| @fungicide:matrix.org | ((well, maybe some of *my* old work is trash, but i'm allowed to think that)) | 21:20 |
| @clarkb:matrix.org | I'd say spring cleaning but it's finally cold enough the geese are flying south in mid February. Feels like winter | 21:34 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!