| clarkb | ok getting the agenda sent now | 00:06 |
|---|---|---|
| clarkb | corvus: also thats cool I'm not sure I realized that (the cd stuff) | 00:06 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add option to force docker.io addresses to IPv4 https://review.opendev.org/c/opendev/system-config/+/943216 | 00:13 |
| tonyb | clarkb: Something like ^^ ? | 00:14 |
| clarkb | there is also a cloudflare hosted name that has to go in there. | 00:29 |
| clarkb | but ya something like that. Though now I'm wondering where we should host it. I think we mostly only need ti in ci for now? | 00:29 |
| clarkb | like maybe that makes sense in zuul-jobs somewhere and then system-config and everyone else that flips a flag to enable it can take advantage? I dunno what makes sense for that | 00:30 |
| corvus | clarkb: re inaugust; i think at least the website is netlify and published from post. | 00:30 |
| clarkb | neat | 00:30 |
| corvus | clarkb: do we use any zuul-jobs roles in prod playbooks? that might be a new thing. | 00:32 |
| clarkb | no, but we primarily have trouble with the system-config-run jobs not the infra-prod jobs. They both run the same playbooks but I think the system-config-run stuff has setup taht may dip into zuul-jobs? | 00:34 |
| clarkb | (we hit the rate limits once in prod that I know of. Once when I tried to update the gerrit image) | 00:34 |
| clarkb | system-config-run-containers has parent opendev-buildset-registry-consumer so that might be a location where we can force docker onto ipv4 addrs | 00:35 |
| corvus | ack | 00:35 |
| clarkb | its kinda "funny" that for years we wished docker would be more ipv6 aware and now that it is it craetes more problems for us :) | 00:38 |
| corvus | "not like that" | 00:39 |
| tonyb | Ahh okay I need to be more complete with the information borrowed from: https://docs.docker.com/desktop/setup/allow-list/ | 00:40 |
| tonyb | Okay so did you settle on where we want it? opendev.org/zuul/zuul-jobs? opendev.org/opendev/system-config? both? | 00:43 |
| clarkb | I think we should put it in zuul-jobs if we can find a spot for it. System-config isn't the only thing pulling from docker in CI | 00:47 |
| clarkb | and other users of zuul may have similar issues | 00:47 |
| clarkb | but I guess the problem there is a natural location for it would be zuul-jobs/roles/ensure-docker but system-config doesn't use those roles | 00:48 |
| clarkb | so we're back to doing it both places or just system-config? So maybe to start we just do it with system-config and take it from there? | 00:48 |
| clarkb | also any reason to use a state conditional rather than a when conditional? I think I have a slight preference for when because that would completely avoid running code on the remote side in prod if we don't enable it there wheras setting something to absent would create side effects on the remote side even if they noop. But not sure if there is a reason for that pattern | 00:50 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add option to force docker.io addresses to IPv4 https://review.opendev.org/c/opendev/system-config/+/943216 | 00:52 |
| tonyb | I used the state so we can unwind the change in prod by flipping the boolean | 00:53 |
| tonyb | but I can switch to when: if that explanation isn't convincing | 00:54 |
| tonyb | I don't have a strong preference | 00:54 |
| Clark[m] | Ah ok allows us to opt in then opt out. Sorry switched clients as dinner is almost ready. One last thought maybe set that var in system-config/playbooks/zuul/templates/group_vars/all.yaml or whatever the path is to set it on test nodes | 00:54 |
| tonyb | Yeah, I can set the var in the next revision. Yeah the opt out was the reason. I'm flexible. It's fairly easy to use an ad-hoc mode to clean up later if we choose to | 00:58 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add option to force docker.io addresses to IPv4 https://review.opendev.org/c/opendev/system-config/+/943216 | 00:59 |
| ianw | it's 2025 and instead of increasing the bits used to address a remote system we're forcefully reducing it. i don't think this was the future we were promised :) | 01:51 |
| fungi | clarkb: sorry, stepped away for evening tasks, but i can work on merging old flex mirror cleanup changes tomorrow | 01:56 |
| JayF | ianw: somewhere a dancing turtle is very sad | 04:38 |
| ianw | JayF: i remember being legitimately excited when i got things working to make that turtle dance. I think we're showing our age! :) | 07:13 |
| amorin | corvus ack, I answered. My bad, that was not clear at all, but this was a new proposition to not create oif-x flavors, stay with the old naming convention and create only a new flavor with less memory | 08:20 |
| *** liuxie is now known as liushy | 09:51 | |
| corvus | amorin: got it -- would it be possible to have one with 8vcpu/16g ? | 14:53 |
| amorin | corvus: yes, we can also add another with bigger memory | 14:53 |
| amorin | so I hope you understand that we need first to get rid of our max_on_host stuff, I will ask the team to work on that first then. | 14:54 |
| corvus | yeah, that's making sense now, and an outage to do that is understandable. how long do you think that would be? we probably want to make sure the openstack release process has settled down before we schedule that. | 14:58 |
| clarkb | ++ I think that is the primary timing consideration from our side | 15:25 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Pull the selenium standalone-firefox image from quay https://review.opendev.org/c/opendev/system-config/+/943326 | 15:33 |
| clarkb | tonyb: I left a comment on https://review.opendev.org/c/opendev/system-config/+/943216 I think there is a subtle bug in play there for long lived production servers that we should aviod for safety. Anyawy thoughts on the change | 15:44 |
| clarkb | all of the CI jobs for that change apssed though whcih I think is likely a good indicator it helps in CI | 15:44 |
| opendevreview | Pierre Riteau proposed openstack/diskimage-builder master: rocky-container: use Quay.io instead of Docker Hub https://review.opendev.org/c/openstack/diskimage-builder/+/943337 | 17:42 |
| *** isaacvicente is now known as isaacvicente_ | 19:11 | |
| *** isaacvicente is now known as isaacvicente_ | 22:36 | |
| clarkb | I've rechecked the selenium image location update change in hopes that this time of day is better for docker hub rate limits | 23:15 |
| clarkb | fungi: should we approve the sjc3 mirror removal now? | 23:26 |
| clarkb | I'll be around if you'd like to | 23:26 |
| fungi | i've been trying to work it through the gate for a while | 23:40 |
| fungi | it bounced out once due to a timeout trying to reach our openafs ppa in launchpad | 23:40 |
| clarkb | oh heh I should'ved looked. I just checked the scrollback merge messages | 23:40 |
| corvus | i was just sending out the zuul release email and was reminded that i wanted to bring a new feature to our attention; see the second bullet point on https://zuul-ci.org/docs/zuul/11.3.0/releasenotes.html | 23:42 |
| corvus | basically, i think we may be able to get rid of some file matchers in system-config | 23:42 |
| fungi | oh neat, i had missed that landing | 23:43 |
| corvus | (i'm not sure, i haven't looked at the details; this is more of a "let's keep this in mind for the next time we're looking at the job graph" kind of thing) | 23:43 |
| clarkb | ++ | 23:44 |
| corvus | but, hand-wavey example: we can force the letsencrypt job to run just by having a job that depends on it run. so we don't need to put file matchers for everything on letsencrypt. no idea if that's a real example, just more of a theoretical example. | 23:44 |
| clarkb | fwiw I've been looking at gitea13 to get a sense for how memecached is doing and the memecached stats report we are using almost the full gig of allocated memory, top reports 1gb resident too. Free -m reports what appears to be a healthy service memory wise so I don't think we've negatively impacted server health | 23:45 |
| clarkb | then from a usage standpoint access logs seem to show continued persistent requests from the ai crawlers as expected but any explicit requests I've made against the server in my browser remain consistently quick (around 100-120ms for /zuul/zuul) | 23:46 |
| fungi | very cool, seems solid | 23:46 |
| clarkb | ya I'm pretty happy with that | 23:46 |
| clarkb | I suspect there is tuning we could do, but for now I think the best effort guesses I made are working | 23:47 |
| opendevreview | Merged opendev/system-config master: Remove Ansible for old Rackspace Flex SJC3 mirror https://review.opendev.org/c/opendev/system-config/+/943195 | 23:55 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!