| -@gerrit:opendev.org- Goutham Pacha Ravi proposed: [openstack/project-config] 989053: Add devstack-plugin-lustre notifications https://review.opendev.org/c/openstack/project-config/+/989053 | 04:44 | |
| -@gerrit:opendev.org- Goutham Pacha Ravi proposed: [openstack/project-config] 989054: Add devstack-plugin-lustre to Zuul tenant https://review.opendev.org/c/openstack/project-config/+/989054 | 05:00 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Goutham Pacha Ravi: [opendev/irc-meetings] 987415: Add schedule for the TC weekly meeting (2026.2) https://review.opendev.org/c/opendev/irc-meetings/+/987415 | 07:22 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Elod Illes: [openstack/project-config] 988968: Fix publish-openstack-releasenotes-python3 job https://review.opendev.org/c/openstack/project-config/+/988968 | 11:47 | |
| @fungicide:matrix.org | as mentioned in #openstack-infra on oftc, mirroring has gone stale for the mirror.debian volume as of the past several days, coinciding with the 13.5 stable point release which seems to have caused reprepro to hit our timeout leaving its lockfile behind. i'm working to get it back on track now | 13:50 |
|---|---|---|
| @fungicide:matrix.org | i cleared `/afs/.openstack.org/mirror/debian/db/lockfile` and am rerunning reprepro in a root screen session now | 13:51 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [opendev/zuul-providers] 989136: Drop haveged and urandom fiddling from images https://review.opendev.org/c/opendev/zuul-providers/+/989136 | 14:11 | |
| @fungicide:matrix.org | oh, also the afs quota for mirror.debian has been exceeded, guess i'll increase that too | 14:18 |
| -@gerrit:opendev.org- Dmitriy Rabotyagov proposed: [opendev/infra-manual] 989143: Add ACL change mention to retirement guide https://review.opendev.org/c/opendev/infra-manual/+/989143 | 14:49 | |
| -@gerrit:opendev.org- Dmitriy Rabotyagov proposed: [openstack/project-config] 989144: Change ACLs for Venus to retired https://review.opendev.org/c/openstack/project-config/+/989144 | 14:55 | |
| -@gerrit:opendev.org- Clark Boylan proposed wip on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 15:27 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | 15:27 | |
| @clarkb:matrix.org | fungi: looking in the zuul console the data structure looks right but the file we copy at the end of the job has the old content in it. So I've added more debugging to figure out if we ever write the correct file or not and we can take it from there. But I can see that the nested inventory value is updated to set ansible_host to the ipv4 addr but the gate-hosts.yaml file we log still shows the ipv6 value so maybe the final write out is broken or something resets it | 15:28 |
| -@gerrit:opendev.org- Zuul merged on behalf of Goutham Pacha Ravi: | 15:37 | |
| - [openstack/project-config] 989053: Add devstack-plugin-lustre notifications https://review.opendev.org/c/openstack/project-config/+/989053 | ||
| - [openstack/project-config] 989054: Add devstack-plugin-lustre to Zuul tenant https://review.opendev.org/c/openstack/project-config/+/989054 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of cid: [openstack/project-config] 988655: [EOL]: Remove VirtualPDU repository references https://review.opendev.org/c/openstack/project-config/+/988655 | 15:47 | |
| -@gerrit:opendev.org- Will Szumski proposed: [openstack/diskimage-builder] 989169: Adds DIB_IMAGE_VERSION to centos element https://review.opendev.org/c/openstack/diskimage-builder/+/989169 | 15:51 | |
| -@gerrit:opendev.org- Clark Boylan proposed wip on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 15:57 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | 15:57 | |
| -@gerrit:opendev.org- Clark Boylan proposed wip on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 16:21 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | 16:21 | |
| @fungicide:matrix.org | the debian mirror ended up with 23 truncated package files sp o' | 16:25 |
| @fungicide:matrix.org | er, so i've replaced them with official copies and am rerunning reprepro | 16:25 |
| -@gerrit:opendev.org- Elod Illes proposed: [openstack/project-config] 989179: [release-tool] check-approval use only governance data https://review.opendev.org/c/openstack/project-config/+/989179 | 16:43 | |
| @clarkb:matrix.org | fungi: https://zuul.opendev.org/t/openstack/build/5dc49f77010a4055b66d648b17f983ab/console#2/3/7/bridge99.opendev.org here you can see the update_fact task seems to update the fact. But then when we debug the fact not much later it shows the old values: https://zuul.opendev.org/t/openstack/build/5dc49f77010a4055b66d648b17f983ab/console#2/3/8/bridge99.opendev.org | 16:44 |
| @clarkb:matrix.org | that also exposes another problem: public_v4 for the rax nodes is the private 10/8 address ... but one problem at a time | 16:45 |
| @mordred:waterwanders.com | Clark: public_v4 is the private address in the return value from rax? Or sdk is doing the mapping wrong? | 16:45 |
| @fungicide:matrix.org | Clark: is it a fact cache problem? different playbooks? | 16:46 |
| @clarkb:matrix.org | mordred: it may also be something that we did intentionally due to rackspace networking being weird | 16:47 |
| @clarkb:matrix.org | mordred: the `public_v6: ''` values are likely coming from sdk if you're interested in debugging something related | 16:47 |
| @clarkb:matrix.org | mordred: we've seen that for rax classic and ovh | 16:47 |
| @clarkb:matrix.org | fungi: maybe the tasks are in the same playbook and are basically adjacent to one another | 16:47 |
| @clarkb:matrix.org | fungi: I see the issue after rereading docs | 16:48 |
| @clarkb:matrix.org | update_fact returns a new value and doesn't update in place. Poorly named imo | 16:49 |
| -@gerrit:opendev.org- Elod Illes proposed: [openstack/project-config] 989179: [release-tool] check-approval use only governance data https://review.opendev.org/c/openstack/project-config/+/989179 | 16:50 | |
| -@gerrit:opendev.org- Clark Boylan proposed wip on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 16:54 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | 16:54 | |
| @clarkb:matrix.org | mordred: but osuosl nodes don't have the `public_v6: ''` issue so this isn't a problem across the board. It is weird | 16:55 |
| @clarkb:matrix.org | `self.public_ipv4 = server.get('public_v4')` that makes me think this is coming from openstacksdk too | 17:03 |
| @clarkb:matrix.org | there may be a number of bugs here. But I have only observed this through zuul so far and haven't use the sdk directly. I think that is the next step | 17:03 |
| @clarkb:matrix.org | ok I wrote a little script on bridge in my homedir that others can run like ` sudo /usr/launcher-venv/bin/python ./get_server_with_sdk.py np979f624eec954` it gets the server in `argv[1]` in the dfw zuul cloud account. This is one of the regions observed to have problems: https://zuul.opendev.org/t/openstack/build/daf42ad2dca14a1688eaf095b26e9bd5/console#2/3/3/bridge99.opendev.org but running that script I am not able to reproduce | 17:13 |
| @fungicide:matrix.org | different sdk version? | 17:14 |
| @clarkb:matrix.org | that venv has the current openstacksdk release in it which is from yesterday (4.13.0). I Think zuul may be using an older openstacksdk? | 17:14 |
| @clarkb:matrix.org | so ya it could be that this broke in 4.12.0 but got fixed after? | 17:14 |
| @clarkb:matrix.org | or the bug is in zuul somewhere? | 17:15 |
| @jim:acmegating.com | can you try your script with the older sdk | 17:15 |
| @clarkb:matrix.org | yes I should be able to set up a venv in my homedir that installs the old version. I'll do that next | 17:15 |
| @jim:acmegating.com | launchers have openstacksdk==4.12.0 | 17:16 |
| @clarkb:matrix.org | corvus: `sudo /home/clarkb/sdkvenv/bin/python ./get_server_with_sdk.py np979f624eec954` also does not reproduce. That has openstacksdk 4.12.0 installed into the venv | 17:20 |
| @clarkb:matrix.org | `public_v4=104.239.142.18, public_v6=2001:4800:7818:104:be76:4eff:fe02:d32e` is printed | 17:21 |
| @clarkb:matrix.org | so the v6 address isn't `''` and the public_v4 address is the public not private address | 17:21 |
| @clarkb:matrix.org | corvus: oh looking at zuul we use list servers to get the info I think. I'm using get_server in my little script | 17:23 |
| @jim:acmegating.com | i'm working on a zuulish reproducer | 17:24 |
| @clarkb:matrix.org | and we call `openstack.cloud.meta.add_server_interfaces` so maybe the issue here is in the call path specifically? | 17:24 |
| @clarkb:matrix.org | corvus: ack I think that is the next step since my naive script is not quite the same as what zuul does | 17:24 |
| @clarkb:matrix.org | my ansible update to modify the inventory to prefer ipv4 if not all hosts have ipv6 is "working" now. But it is failing due to the private address now | 17:27 |
| @clarkb:matrix.org | it is interesting that the playbook was able to function until a relatively late task https://eeadb1b3a17ef6b83258-4d11a9e45a4dfc6cfba428233616e283.ssl.cf2.rackcdn.com/openstack/d8fbc386f35e48b38f35baac63a6051b/bridge99.opendev.org/ara-report/playbooks/3.html?status=failed&status=unreachable#results then it complains about connecting to the private address | 17:31 |
| @clarkb:matrix.org | corvus: I wonder actually if write-inventory is the problem | 17:33 |
| @clarkb:matrix.org | corvus: that role is in zuul-jobs but its how we "fetch" the inventory from the zuul side to the nested side. It just occurred to me that zuul may be doing the correct thing here and I think https://zuul.opendev.org/t/openstack/build/daf42ad2dca14a1688eaf095b26e9bd5/log/zuul-info/inventory.yaml may show this is the case for public_v4 but not public_v6? | 17:34 |
| @jim:acmegating.com | i'm confused -- that says public_v6='' -- that's the debug inventory from zuul -- aren't you saying that's wrong? | 17:36 |
| @clarkb:matrix.org | corvus: yes I'm saying public_v6 is wrong, but public_v4 there is the public address not the private one | 17:36 |
| @clarkb:matrix.org | corvus: sorry I meant that maybe write-inventory is to blame for the public_v4 mixup but I don't think it explains the public_v6 empty string problem | 17:37 |
| @jim:acmegating.com | ah ok | 17:37 |
| @clarkb:matrix.org | but also write-inventory seems to be a pretty direct port of the inventory it isn't modifying key value pairs as far as I can tell so write-inventory may not explain public_v4's weird values | 17:37 |
| @jim:acmegating.com | Clark: the test server is gone now; i tried to find a new one, but my test script is showing no problem; maybe it doesn't always happen; how did you find a test server? | 17:38 |
| @fungicide:matrix.org | i guess i should have kept my autohold, since that was where we first spotted the problem | 17:39 |
| @clarkb:matrix.org | corvus: I just picked out of openstack server list. But on the zuul side these are all happening in fungi's mixed nodeset jobs. However, I looekd beyond them and found a random gra job that also exhibited it. I think its doing it to all the rax classic and ovh nodes but haven't done an exhaustive check (specifically the public_v6 empty string thing) | 17:39 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/openstack/build/f333659af7b84f27a9e998c05226d674/log/zuul-info/inventory.yaml this is a random rax iad nodeset for a pep8 job that has the empty string | 17:40 |
| @jim:acmegating.com | but that node wasn't held right? | 17:41 |
| @clarkb:matrix.org | correct | 17:41 |
| @jim:acmegating.com | i picked npb593977296804 | 17:41 |
| @clarkb:matrix.org | I have some held nodes for gerrit testing let me check them and mabe they are a good case | 17:42 |
| @jim:acmegating.com | that has 'public_v6': '2001:4800:7818:104:be76:4eff:fe03:5b88' at the end of zuul's wacky server list thingy | 17:42 |
| @jim:acmegating.com | (i'm allowed to call it that i wrote it) | 17:42 |
| @jim:acmegating.com | ~corvus/get_server_with_sdk.py has my port of the relevant code | 17:43 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/openstack/build/0becef01bc434d009fa9f169bcbe9a73/log/job-output.txt#32-54 these two held nodes in ovh gra1 have ipv6 addrs in openstack server show output but not at https://zuul.opendev.org/t/openstack/build/0becef01bc434d009fa9f169bcbe9a73/log/zuul-info/inventory.yaml#66 | 17:44 |
| @jim:acmegating.com | so if all that is legit, then either the problem may be elsewhere, or there may be a time component to this (eventual consistency?) | 17:44 |
| @clarkb:matrix.org | ya I'm wondering if our use of listings to update the server makes us more prone to timing issues? | 17:45 |
| @clarkb:matrix.org | but that also doesn't explain the public_v4 private address mixup | 17:45 |
| @clarkb:matrix.org | but since zuul's inventory seems to capture them separately they may be independent issues | 17:45 |
| @clarkb:matrix.org | I see the public_v4 vs private_v6 issue | 17:46 |
| @clarkb:matrix.org | it is in system-config so only public_v6 to worry about on the sdk/zuul side of things | 17:46 |
| @jim:acmegating.com | i'm not sure we know enough to blame server listing vs just being fast | 17:46 |
| @jim:acmegating.com | (like, it may be that if we did a server get very quickly we would see the same problem -- assuming it is timing related) | 17:47 |
| -@gerrit:opendev.org- Clark Boylan proposed wip on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 17:49 | |
| -@gerrit:opendev.org- Clark Boylan proposed: | 17:49 | |
| - [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | ||
| - [opendev/system-config] 989185: Make public_v4 in our nested ansible inventory the actual public address https://review.opendev.org/c/opendev/system-config/+/989185 | ||
| @clarkb:matrix.org | if anyone is curious ^ that is the public_v4 mixup fixed. Whether or not we did that intentionally for some reason previously isn't clear to me | 17:49 |
| @fungicide:matrix.org | oh neat | 17:51 |
| @fungicide:matrix.org | unrelated, reprepro for mirror.debian ran clean, i'm running it once more just to make sure it's essentially a no-op | 17:52 |
| @clarkb:matrix.org | https://opendev.org/opendev/system-config/src/branch/master/playbooks/zuul/templates/group_vars/gitea-lb.yaml.j2#L7-L10 I guess I ran into this previously and just worked around it | 17:53 |
| @fungicide:matrix.org | hah | 17:54 |
| @clarkb:matrix.org | meetpad also has some handling around this | 17:54 |
| @clarkb:matrix.org | the proposed change is running a fairly full set of jobs. I expect gitea and meetpad will fail and everything else should be ok based on some quick git greps | 17:55 |
| @clarkb:matrix.org | I need to switch over to meeting prep mode shortly. But feel free to push updates on anything in that stack if I'm busy with other things | 17:55 |
| @fungicide:matrix.org | looks like it may have been a typo in https://review.opendev.org/733409 that none of us has spotted for the subsequent 6 years? | 17:56 |
| @clarkb:matrix.org | fungi: no it was intentional according to the commit message | 17:57 |
| @fungicide:matrix.org | oh, NOT a typo! | 17:57 |
| @fungicide:matrix.org | yes | 17:57 |
| @clarkb:matrix.org | so that is actually helpful. Apparently we're writing these names to the /etc/hosts file | 17:57 |
| @clarkb:matrix.org | this may actually take some effort to untangle. I'm thinking the easiest thing in the short term may be to write actual_public_v4 to the v4 address and then we can use that? | 17:58 |
| @clarkb:matrix.org | then figure out lceaning up where we rely on public being private elsewhere later? | 17:59 |
| @clarkb:matrix.org | if someone else wants to update 989185 and 989022 to do that that would be great otherwise I'll try to get to it after our meeting and lunch later today | 17:59 |
| @mnasiadka:matrix.org | Oh boy, good rabbit hole | 18:01 |
| @clarkb:matrix.org | but also that may explain some of the connectivity issues if /etc/hosts points to the private addr and we're in a mixed nodeset situation | 18:01 |
| @clarkb:matrix.org | its possible that fixing the inventory is only part of the problem | 18:01 |
| @mnasiadka:matrix.org | Clark: I won't be there for the meeting, but I'll go back to polishing Prometheus patches from tomorrow, got sucked into a downstream vortex for a couple of days... | 18:03 |
| @clarkb:matrix.org | thank you for the heads up and I understand. I've been sucked into random vortexes myself recently | 18:03 |
| @mnasiadka:matrix.org | If there's any server upgrade to be done - count me in, just tell me which one and we can do that together (I think you mentioned some backup server) | 18:04 |
| @clarkb:matrix.org | mnasiadka: yup lets work on a new backup server. Maybe Thursday? | 18:04 |
| @mnasiadka:matrix.org | And maybe I'll be able to go back to the mediawiki stuff, because that's probably one of the oldest servers | 18:04 |
| @clarkb:matrix.org | I'm flexible timing wise but have some extra meetings tomorrow that I normally wouldn't so Thursday is probably easiest for me | 18:04 |
| @mnasiadka:matrix.org | Thursday looks good | 18:04 |
| @fungicide:matrix.org | rerun of reprepro for mirror.debian completed quickly and successfully as well, so i've closed out the root screen session for it on mirror-update now | 18:20 |
| @fungicide:matrix.org | #status log Increased AFS quota for the mirror.debian volume by 100GB (from 0.9TB to 1TB) to make room for the 13.5 stable point release | 18:24 |
| @status:opendev.org | @fungicide:matrix.org: finished logging | 18:25 |
| @fungicide:matrix.org | we're currently running very close to quota on mirror volumes for ubuntu-ports, centos-stream, debian, and debian-security; also the project.docs volume is almost at quota | 18:27 |
| @fungicide:matrix.org | Clark: system-config-run-base-arm64 passed! | 18:29 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org marked as active: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 18:30 | |
| @clarkb:matrix.org | fungi: great so I think my suggestion to add an actual public v4 car to the inventory should sort things out without first untangling everything | 18:33 |
| @clarkb:matrix.org | * @fungicide:matrix.org: great so I think my suggestion to add an actual public v4 var to the inventory should sort things out without first untangling everything | 18:33 |
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/system-config] 988698: Use mixed arches for system-config-run-base-arm64 https://review.opendev.org/c/opendev/system-config/+/988698 | 19:01 | |
| -@gerrit:opendev.org- Clark Boylan proposed: | 19:01 | |
| - [opendev/system-config] 989022: Handle mixed provider nodests and ipv6 availability https://review.opendev.org/c/opendev/system-config/+/989022 | ||
| - [opendev/system-config] 989185: Make public_v4 in our nested ansible inventory the actual public address https://review.opendev.org/c/opendev/system-config/+/989185 | ||
| @clarkb:matrix.org | there's the new var update with the change from private to public for public_v4 at the end so we can iterate on that as a followup | 19:01 |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/zone-opendev.org] 989024: Remove records for static02 and static04 https://review.opendev.org/c/opendev/zone-opendev.org/+/989024 | 19:45 | |
| @fungicide:matrix.org | mentioned during the meeting in passing, but https://github.com/ether/pad is a console-based tui client for etherpad | 20:16 |
| @fungicide:matrix.org | though needs etherpad 3.x or later | 20:17 |
| @fungicide:matrix.org | also https://github.com/ether/etherpad-desktop for a graphical etherpad client application | 20:17 |
| @fungicide:matrix.org | ooh, this is an interesting feature in the etherpad 3.0 changelog... "Scrub history in-place on the pad URL. A long-edited pad can now have its history rewritten in place (e.g. for compliance or to drop accidentally-pasted secrets), without changing the pad URL or breaking deep-links" | 20:19 |
| @fungicide:matrix.org | also it can do .docx file exports without libreoffice now | 20:20 |
| @fungicide:matrix.org | #status log Deleted the unused static02.opendev.org and static04.opendev.org servers, now that all AFS-backed sites have been on static03.opendev.org for over a month | 20:27 |
| @status:opendev.org | @fungicide:matrix.org: finished logging | 20:27 |
| @clarkb:matrix.org | fungi: the gitea job is failing on my ansible inventory rewrite change. I dont' fully understand why yet as we seem to rely on fetching things against localhost on the load balancer but it is saying there aren't any backends? In theory my change to avoid changing the public address value should sidestep those issues, but maybe not fully? | 20:52 |
| @clarkb:matrix.org | or there is just a bug in there | 20:52 |
| @clarkb:matrix.org | In any case I'm going to pop out for a bike ride now. But I'll try to look more closely when I get back | 20:52 |
| -@gerrit:opendev.org- Stephen Finucane proposed: | 21:49 | |
| - [opendev/git-review] 988966: Use configparser utils to parse types https://review.opendev.org/c/opendev/git-review/+/988966 | ||
| - [opendev/git-review] 987712: Revert "Clean up all references to branchauthor after removal of usage" https://review.opendev.org/c/opendev/git-review/+/987712 | ||
| - [opendev/git-review] 987713: Add gitreview.autotopic git config flag https://review.opendev.org/c/opendev/git-review/+/987713 | ||
| -@gerrit:opendev.org- Stephen Finucane proposed: | 21:58 | |
| - [opendev/git-review] 988966: Use configparser utils to parse types https://review.opendev.org/c/opendev/git-review/+/988966 | ||
| - [opendev/git-review] 987712: Revert "Clean up all references to branchauthor after removal of usage" https://review.opendev.org/c/opendev/git-review/+/987712 | ||
| - [opendev/git-review] 987713: Add gitreview.autotopic git config flag https://review.opendev.org/c/opendev/git-review/+/987713 | ||
| - [opendev/git-review] 989239: Remove support for global/system git-review config files https://review.opendev.org/c/opendev/git-review/+/989239 | ||
| @clarkb:matrix.org | fungi: oh nevermind I mixed up the changes. https://review.opendev.org/c/opendev/system-config/+/989022/16 and child both pass and seem to do the correct thing. Its the followup that redefines public_v4 to not be private_v4 equivalent that fails. And I expected that | 23:06 |
| @clarkb:matrix.org | so I think we can probably land the first two chagnes tomorrow as well as the ansible-role-puppet fixup and then hopefully the ansible 9 fallout will be completely dealt with | 23:06 |
| @fungicide:matrix.org | sounds good. i'm about checked out for the evening, but plan to be around all day tomorrow and can monitor for breakage once those merge | 23:07 |
| @clarkb:matrix.org | sounds good. I should be around as well. I have that AI wg meeting and then also will try to work through the gerrit upgrade todos | 23:08 |
| @fungicide:matrix.org | yeah, that's the only meeting on my schedule for tomorrow (so far) | 23:08 |
| @jim:acmegating.com | it looks like the server that zuul-ci.org points to is gone | 23:41 |
| @jim:acmegating.com | zones/zuul-ci.org/zone.db:@ IN A 104.130.253.131 | 23:41 |
| @jim:acmegating.com | i think that was static02 | 23:42 |
| @jim:acmegating.com | i know stuff moved around; is everything on one static host now? do we just need to update to whatever that is? | 23:42 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [opendev/zone-zuul-ci.org] 989306: Update address records for domain root https://review.opendev.org/c/opendev/zone-zuul-ci.org/+/989306 | 23:43 | |
| @fungicide:matrix.org | corvus: ^ oops! | 23:43 |
| @fungicide:matrix.org | sorry i didn't remember it was that way | 23:44 |
| @jim:acmegating.com | https://review.opendev.org/540029 has the reason why it's that way | 23:45 |
| @jim:acmegating.com | (i didn't remember either, just had to go look it up :) | 23:45 |
| @fungicide:matrix.org | i mean, i know why, it just didn't dawn on me to check those domains | 23:46 |
| @fungicide:matrix.org | checking the vhosts on static03, i think i need to do the same for devstack.org and gating.dev | 23:48 |
| @fungicide:matrix.org | working on those now | 23:49 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [opendev/zone-gating.dev] 989307: Update address records for domain root https://review.opendev.org/c/opendev/zone-gating.dev/+/989307 | 23:51 | |
| @clarkb:matrix.org | corvus: is fast on that approval button :) | 23:51 |
| @jim:acmegating.com | i wrote a whole program for it | 23:52 |
| @jim:acmegating.com | i typo "alt+3" | 23:52 |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [opendev/zone-zuul-ci.org] 989306: Update address records for domain root https://review.opendev.org/c/opendev/zone-zuul-ci.org/+/989306 | 23:52 | |
| @fungicide:matrix.org | okay, dns management for devstack.org is still living in rackspace so i've manually updated its a and aaaa records there | 23:55 |
| @fungicide:matrix.org | not that anybody probably actually uses that domain these days | 23:56 |
| @clarkb:matrix.org | zuul-ci.org and zuulci.org should be updated in just another minute or so | 23:56 |
| @fungicide:matrix.org | everything else we're hosting from static is a subdomain so presumably uses a cname to static.opendev.org already (or if it doesn't, should be fixed to do so) | 23:56 |
| @clarkb:matrix.org | I didn't have the old records cached and I can hit zuul-ci.org now | 23:56 |
| @fungicide:matrix.org | cool, and apologies once again for the disruption! i guess it's been down since 20:27 when static02 was deleted in rackspace, so approximately 3.5 hours | 23:58 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!