| opendevreview | Ian Wienand proposed opendev/system-config master: gerrit: update OpenDev theme CSS installation https://review.opendev.org/c/opendev/system-config/+/881682 | 01:58 |
|---|---|---|
| opendevreview | Merged opendev/system-config master: Remove Gerrit 3.6 image builds and test jobs https://review.opendev.org/c/opendev/system-config/+/881596 | 02:14 |
| opendevreview | Merged opendev/system-config master: Add Gerrit 3.8 image builds and test jobs https://review.opendev.org/c/opendev/system-config/+/881599 | 02:42 |
| opendevreview | Merged opendev/system-config master: Add Gerrit 3.7 -> 3.8 upgrade job https://review.opendev.org/c/opendev/system-config/+/881647 | 02:55 |
| opendevreview | Merged opendev/system-config master: Switch to nodepool images on quay.io https://review.opendev.org/c/opendev/system-config/+/881591 | 02:55 |
| opendevreview | Merged opendev/system-config master: Update Hound to Use Python 3.11 Base Images https://review.opendev.org/c/opendev/system-config/+/880908 | 02:55 |
| opendevreview | Merged opendev/system-config master: Cleanup unused nodepool-base-legacy role https://review.opendev.org/c/opendev/system-config/+/881594 | 02:55 |
| opendevreview | Ching Kuo proposed opendev/system-config master: Build haproxy-statsd with Python 3.11 Base Images https://review.opendev.org/c/opendev/system-config/+/881687 | 03:15 |
| opendevreview | Ian Wienand proposed opendev/system-config master: gerrit: update OpenDev theme CSS installation https://review.opendev.org/c/opendev/system-config/+/881682 | 03:31 |
| opendevreview | Ian Wienand proposed opendev/system-config master: gerrit: update OpenDev theme CSS installation for Gerrit 3.8 https://review.opendev.org/c/opendev/system-config/+/881689 | 03:38 |
| opendevreview | Ian Wienand proposed opendev/system-config master: [dnm] trying to get some logs for openafs on centos builds https://review.opendev.org/c/opendev/system-config/+/881528 | 04:02 |
| opendevreview | Ian Wienand proposed opendev/system-config master: openafs-client: get logs better https://review.opendev.org/c/opendev/system-config/+/881528 | 06:12 |
| ianw | all the nodepool bits restarted successfully with quay.io images afaics | 06:17 |
| ianw | "This is a notice that your app - openstack_statusbot - has been suspended from accessing the Twitter API. However, you can self-serve reactivate your app for free." ... yeah, nah | 06:26 |
| opendevreview | Ian Wienand proposed openstack/project-config master: tools/normalize_acl.py: Add some human readable output https://review.opendev.org/c/openstack/project-config/+/880898 | 06:53 |
| apevec | wut, Lon Muck banned us ;) | 09:29 |
| dpawlik | fighting with bots requires sacrifices | 09:41 |
| *** amoralej_ is now known as amoralej|lunch | 11:01 | |
| fungi | but whatever, we can always make more killbots | 12:41 |
| *** amoralej|lunch is now known as amoralej_ | 13:13 | |
| clarkb | ianw: thank you for looking into the gerrit theming issue. https://review.opendev.org/c/opendev/system-config/+/881682 looks great | 14:50 |
| clarkb | ianw: comparing the 3.7 change 1 screenshot to the 3.8 change 1 screenshot I think that maybe the green checkmark and red x aren't working under 3.8 with the zuul summary tab. A minor detail but similar in nature to the theme issues I'm guessing | 14:53 |
| clarkb | infra-root if we get https://review.opendev.org/c/opendev/system-config/+/881592 in before end of day tomorrow UTC time the zuul restart process should operate on quay images for zuul instead of docker hub | 15:07 |
| clarkb | the other big item I've got is the ensure-quay-repo + opendev/base-jobs + opendev/system-config stack to start publishing opendev images to quay with zuul. Reviews on that very much appreciated (and the reviews I'vegotten so far have really helped make this better I think) | 15:17 |
| opendevreview | Elod Illes proposed openstack/project-config master: Prevent recreate EOL'd branch https://review.opendev.org/c/openstack/project-config/+/881731 | 15:18 |
| opendevreview | Elod Illes proposed openstack/project-config master: Prevent recreate EOL'd branch https://review.opendev.org/c/openstack/project-config/+/881731 | 15:22 |
| clarkb | https://gerrit-review.googlesource.com/c/plugins/zuul-results-summary/+/371674 I think this is the zuul results summary fix. WOrking on a depends on change to check it now | 16:47 |
| opendevreview | Clark Boylan proposed opendev/system-config master: DNM testing the depends on change upstream https://review.opendev.org/c/opendev/system-config/+/881749 | 16:50 |
| clarkb | I don't think I'm brave enough but we could theoretically upgrade to gerrit 3.8 before it even releases | 16:53 |
| *** amoralej_ is now known as amoralej|off | 17:04 | |
| corvus | fungi: can you +3 https://review.opendev.org/881593 ? | 17:12 |
| clarkb | CSC says authoritative dns for opendev.org and zuulci.org should be updated now | 17:20 |
| fungi | whois seems to reflect that, checking tld nameservers | 17:23 |
| fungi | a.root-servers.net is still returning the old ns records though | 17:24 |
| fungi | could take time to propagate | 17:24 |
| fungi | er, sorry was from a0.org.afilias-nst.info | 17:26 |
| fungi | the tld not the root | 17:26 |
| fungi | b0.org.afilias-nst.org has the updated ns though | 17:28 |
| fungi | okay now a0.org.afilias-nst.info is returning updated results too | 17:28 |
| fungi | and b2.org.afilias-nst.org | 17:29 |
| fungi | and d0.org.afilias-nst.org | 17:29 |
| fungi | seems like at least most of them probably have updated already | 17:29 |
| fungi | c0.org.afilias-nst.info lgtm now too | 17:30 |
| fungi | clarkb: here's what i saw from one of the tld nameservers a few minutes ago though: https://paste.opendev.org/show/bLCcuj3CaZ9NqBIse9PW/ | 17:34 |
| fungi | since the ttl on that response was an hour, we should probably give it some time in case any resolvers cached that | 17:35 |
| fungi | registrars used to caution you to expect something like 48 hours for nameserver updates to propagate completely, though in my experience these days it's on the order of minutes or a few hours tops | 17:37 |
| corvus | we expect glue records to be in place? | 17:37 |
| fungi | good point, i wasn't even checking that | 17:38 |
| corvus | well, porkbun seems happy now. i have switched the auth dns servers for zuul-ci.org to ns03 and 04. so i'm guessing "yes" is the answer to that :) | 17:39 |
| clarkb | I don't see glue records from here but I may not be looking correctly | 17:39 |
| clarkb | I would expect the registrar to know they need to add those, but I also wouldn't be surprise if they don't ... | 17:40 |
| fungi | i'm not seeing glue records returned from the tld nameservers at the moment, only ns records | 17:40 |
| clarkb | fungi: should we clarify with csc? | 17:40 |
| fungi | here's an example from my personal domain which is hosted from dns servers within itself so needs glue records: https://paste.opendev.org/show/bWtlvTQjR0mpoCDDqNnM/ | 17:42 |
| clarkb | oh querying that server I see A records for ns03 and ns04 in the additional section | 17:43 |
| fungi | okay, now i'm seeing glue records for opendev.org (ipv4 only): https://paste.opendev.org/show/bFgcJTe5MFahLxBLZHcQ/ | 17:43 |
| clarkb | so maybe we wait a bit to see if AAAA show up and if not we followup to get those added, but this should be functional in the interim | 17:44 |
| corvus | me too | 17:44 |
| fungi | okay, yeah the v4 glue was there earlier, i was just thrown by not seeing any v6 glue | 17:44 |
| fungi | as long as v6-only machines are going through recursive resolvers that have access to the v4 internet, that ought to suffice | 17:45 |
| clarkb | fungi: well we know we have ipv6 only test nodes so fixing that is a good idea | 17:47 |
| clarkb | but it isn't an emergency | 17:47 |
| fungi | those test nodes are forwarding their queries to other resolvers though | 17:47 |
| fungi | it's only recursive resolvers hitting the tld nameservers directly which need to be able to follow the glue | 17:48 |
| clarkb | ah right | 17:48 |
| fungi | so, like, i run a (non-forwarding) caching recursive resolver on my openbsd firewalls. they do have ipv4 access. any v6-only machines on my internal network can still get the v6 addresses of our nameservers if they want them because the resolver they're going through has v4 access | 17:52 |
| clarkb | My zuul results summary update worked except for I lost color indication. New ps tries to fix that but I'm quickly running up against my lack of understanding of ts and js and css and html | 17:56 |
| fungi | that's all word soup to me | 18:01 |
| clarkb | fungi: tl;dr web development and making things pretty is hard | 18:02 |
| opendevreview | Merged opendev/system-config master: Switch zuul container images to quay.io https://review.opendev.org/c/opendev/system-config/+/881592 | 18:04 |
| opendevreview | Merged opendev/system-config master: Switch the zuul-registry image location to quay.io https://review.opendev.org/c/opendev/system-config/+/881593 | 18:04 |
| clarkb | woot my half a stab in the dark doing css stuff seems to have worked | 18:47 |
| clarkb | I'm sure there are other things to address before planning a 3.8 upgrade, but I think the only other thing we know about is double checking the gitweb gitea links | 18:50 |
| fungi | infra-root: so we had a retired openstack repo mistakenly set to read-only because the wrong acl was picked, and needed undoing. a change to switch to the correct acl merged but jeepyb wasn't allowed to push that. i used the webui as a member of administrators to set it back to active, but rerunning manage-projects for it was a no-op because the cache indicated no change to the acl (presumably | 19:20 |
| fungi | because of the earlier failure). i edited /opt/lib/jeepyb/project.cache to make the checksum for that project incorrect and ran m-p for it again, which did work as intended | 19:20 |
| fungi | in other news, a tentative release date for debian bookworm has been announced: 2023-06-10 | 19:29 |
| fungi | so ~6 weeks out | 19:29 |
| ianw | thanks for sorting out the DNS stuff. I think that's pretty much it, everything is pointing at the new servers now | 21:14 |
| ianw | clarkb: cool, update looks good. I briefly looked at moving that plugin to Lit. It looks like it's pretty similar to polymer conceptually but of course the way to initialize and reference everything is totally different | 21:16 |
| fungi | wait, is polygerrit no longer using polymer? | 21:32 |
| fungi | that's like the new change screen being old | 21:32 |
| ianw | haha yeah it's moved to https://lit.dev/docs/ | 21:49 |
| fungi | i really wish webui frameworks would stop reinventing themselves every 1-2 years. this is getting tiresome | 21:56 |
| fungi | sorry, not getting, it's been tiresome for at least a decade | 21:56 |
| clarkb | ianw: next up is updating NS records in our zones and the SOA records? I could check the etherpad I Guess | 22:23 |
| clarkb | corvus: ianw: if you get a chance can you review the trio of https://review.opendev.org/c/zuul/zuul-jobs/+/881521 https://review.opendev.org/c/opendev/base-jobs/+/881522 and https://review.opendev.org/c/opendev/system-config/+/881285 these are the next steps for quay publishing in opendev | 22:25 |
| clarkb | ianw: for the zuul application in opendevorg on quay.io was a token generate for it? and for the real account you created to act as a bot is the docker cli password saved somewhere? Those are the two bits I need to encrypt as secrets | 22:26 |
| fungi | clarkb: yeah, the remaining changes remove the old ns records for ns1/ns2 and update the soa records to adns02 | 22:30 |
| fungi | it's probably safe to do that now, i just didn't want to merge them immediately after registrar updates because tld record changes tend to take time to propagate | 22:30 |
| clarkb | makes sense | 22:31 |
| clarkb | ianw: I just rebased my zuul-results-summary change because gerrit said it was in merge conflict | 22:31 |
| clarkb | c git had no trouble with it but it should be mergeable now I think | 22:31 |
| clarkb | ianw: do you think we need to recheck my testing change? | 22:31 |
| ianw | so was just looking at that; i merged the js bundle thing as our concerns were about 3.4 branches which is long gone | 22:33 |
| ianw | i also tried to upload but got | 22:33 |
| ianw | HEAD -> refs/for/main (cannot add patch set to 371674.) | 22:33 |
| clarkb | ianw: it let me push. Maybe they don't allow other people to update someones change on that gerrit | 22:34 |
| clarkb | it is a permission thing iirc and we allow it? | 22:34 |
| ianw | yeah "add patchset" is inherited through, refs/for/* ALLOWS it from "gerrit-trusted-users" | 22:34 |
| ianw | i don't know what makes people a trusted user | 22:34 |
| ianw | anyway, looks fine, i merged it | 22:35 |
| clarkb | thanks! | 22:35 |
| ianw | gosh it feels weird to click submit :) | 22:36 |
| fungi | dangerous even | 22:36 |
| ianw | there's nothing else in the queue; the only thing would be updating to Lit which I imagine has to happen at some point, but seems ok for now | 22:36 |
| clarkb | I'm happy to chip away at the least amount of effort that keeps this working | 22:37 |
| clarkb | I seem to recall that 3.8 is also removing robot comment support which will require zuul updates | 22:37 |
| ianw | clarkb: i don't think i created a docker cli password for that real-bot account | 22:38 |
| fungi | also it'll throw our engagement reporting out of whack since i was using that to differentiate between human-added and ci comments | 22:38 |
| ianw | however, there was a token created for it; that should be recorded in the usual place | 22:39 |
| clarkb | ianw: cool I should be able to generate a docker cli password and reuse that token | 22:39 |
| ianw | ++ | 22:39 |
| clarkb | I've been punting on that aspect of my system-config change as it seems like a minor detail compared to getting all the roles and jobs sorted out | 22:39 |
| ianw | i don't see AAAA glue records for opendev.org | 22:43 |
| ianw | and zuulci.org still has the records -- but as discussed this might be something the registrar does and we have no control over | 22:43 |
| clarkb | ianw: I think we can request AAAA glue records but I suspect they didn't add them. And yes they seem to have done glue records on their own | 22:44 |
| clarkb | fungi: is there any reason to not request AAAA glue records? I guess if they don't automatically add them then it might not be something they are familiar with and could go wrong? | 22:44 |
| fungi | yeah, that's the only counterargument i can think of | 22:45 |
| ianw | corvus: zuul-ci.org seems similarly to have only A glue records (dig +noall +authority +additional +norecurse @a0.org.afilias-nst.info. NS zuul-ci.org) | 22:45 |
| fungi | i mean, it's a registrar who apparently still expects you to request nameserver changes by e-mail | 22:45 |
| corvus | erm, why do we think that zuul-ci.org has any glue records? | 22:46 |
| fungi | zuul-ci.org shouldn't need glue records | 22:46 |
| fungi | the tld may be returning the glue it has from opendev.org | 22:46 |
| corvus | it's probably the opendev.org glue records that the authoritative server is returning with a zuul-ci.org query? | 22:46 |
| fungi | yes, that | 22:47 |
| corvus | fungi: yeah that :) | 22:47 |
| * fungi gets back to his very important beer | 22:47 | |
| ianw | interesting | 22:48 |
| ianw | dig +noall +authority +additional +norecurse @a0.org.afilias-nst.info. NS zuul-ci.org | 22:48 |
| corvus | so i think if we ask csc to add aaaa records then the tld will start returning them for any soa query. or.. they will screw it up and break everything. :) | 22:48 |
| ianw | dig +noall +authority +additional +norecurse @ns-tld1.charlestonroadregistry.com. NS gating.dev | 22:48 |
| ianw | i see A records for zuul-ci like that, but not gating.dev | 22:49 |
| fungi | dev is a different tld than org | 22:49 |
| fungi | with different operators | 22:49 |
| fungi | the dev tld is owned and operated by google, fwiw | 22:49 |
| corvus | so a recursive resolver would have to ask .org who ns03.opendev.org is after getting the SOA for gating.dev from google | 22:49 |
| ianw | ahh, interesting. my .org domain has .com nameservers, and I don't get A records | 22:50 |
| corvus | i'm guessing the .org tld servers don't want resolvers coming back for a second query asking who ns03.opendev.org is after asking for zuul-ci.org, so it includes the glue records. | 22:50 |
| ianw | so I guess you get back the A/AAAA records if your nameservers are also in .org | 22:51 |
| fungi | that's up to the nameserver implementation usually | 22:51 |
| ianw | i was under the assumption that dig command was returning the configured records, if that makes sense | 22:52 |
| fungi | some nameservers (e.g. bind) aggressively return additional records that they think you might ask for next with your initial query, though unsolicited responses like that have also be leveraged in cache poisoning attacks so recursive resolvers have had to grow filters to make sure they only cache additional responses that make sense in the context of the original query | 22:52 |
| corvus | clarkb: those changes lgtm! | 22:55 |
| ianw | i've updated https://etherpad.opendev.org/p/2023-opendev-dns to reflect on all this | 23:03 |
| ianw | i'll merge https://review.opendev.org/c/opendev/zone-gating.dev/+/880907 first to make sure everything is ok | 23:04 |
| ianw | Released volume project.tarballs successfully | 23:49 |
| opendevreview | Merged zuul/zuul-jobs master: Update ensure-quay-repo to run opportunistically https://review.opendev.org/c/zuul/zuul-jobs/+/881521 | 23:57 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!