| *** __ministry is now known as Guest62 | 07:51 | |
| *** elodilles is now known as elodilles_pto | 08:37 | |
| *** ralonsoh_ is now known as ralonsoh | 10:04 | |
| opendevreview | Artem Goncharov proposed openstack/pbr master: add python 3.12 support https://review.opendev.org/c/openstack/pbr/+/924216 | 17:00 |
|---|---|---|
| gmann | frickler: ack | 17:41 |
| gmann | just saw Aodh enabled it 2 days back and there are not many projects/jobs who disable it https://codesearch.opendev.org/?q=GLOBAL_VENV%3A%20false&i=nope&literal=nope&files=&excludeFiles=&repos= | 17:49 |
| kozhukalov | Hi infra team. We've started experiencing docker hub request rate limit issues while running test jobs. Were there any changes recently that could affect this? What are the recommendations to deal with this? | 18:46 |
| clarkb | kozhukalov: we have noticed too. The best theory I have at this point is that we may have usage pattern changes (eg more requests) and since the limits are per IP but we still try to be good and cache through the proxy cache we're tripping over those limits | 18:47 |
| clarkb | the workaround (which I hate) would be to stop using the proxy cache so that we get to use all the other IPs we've got | 18:47 |
| clarkb | one other thing I noticed is that even requests to "standard" images like library/alpine hit limits. I thought those images were supposed to not have limited but I guess its more about who makes the requests than the upstream target. However, maybe that changed upstream and we were getting those for "free" and now we don't | 18:48 |
| clarkb | oh other things to consider requests are limited by manifest so having "shallow" image builds would hit limits less. Also quay.io does not have limits which is another alternative | 18:49 |
| kozhukalov | I was also thinking about pulling all necessary images and putting them to a buildset registry before starting the test job. This could also help for multi-jobs pipelines and multi-node envs. | 18:51 |
| clarkb | yes that is what the buildset registry is for | 18:53 |
| clarkb | for our (opendev) jobs how it typicalyl works is we build an image or three and push them into the buildset registry then that is what we fetch for the other jobs | 18:54 |
| clarkb | but if you have oither images you may need to push them manually so that they are available | 18:54 |
| clarkb | (I guess we sometimes use mariadb and could do the same for those images) | 18:54 |
| fungi | sounds like a good idea, especially if the jobs in that buildset might collectively pull down the mariadb container more than once | 18:55 |
| cardoe | sean-k-mooney: So I used go install as you suggested in, https://review.opendev.org/c/openstack/openstack-helm/+/934698 but not sure how to get "go" to be installed for the docs job. | 19:39 |
| fungi | cardoe: you can add a docs profile in your bindep.txt, any distro packages specified there will get installed at the start of the job | 19:42 |
| cardoe | it's not a distro package. | 19:43 |
| cardoe | oh go | 19:43 |
| sean-k-mooney | i belive we have a ensure_go role too | 19:43 |
| fungi | sorry, the profile name is "doc" looks like: https://zuul.opendev.org/t/openstack/build/c8f07acdf1b74580b875bfd830fad8a3/console#1/0/14/ubuntu-jammy | 19:43 |
| sean-k-mooney | but ya bindeps would be the way to go | 19:44 |
| sean-k-mooney | * the way to declar golang as a depency | 19:44 |
| fungi | though we also install any packages from the "compile" profile if that would be more appropriate | 19:44 |
| sean-k-mooney | you can list them under multiple tags | 19:44 |
| sean-k-mooney | docs test compile ectra | 19:44 |
| fungi | sure, just noting that as long as it appears under at least compile or doc profiles then it will get preinstalled for the docs jobs | 19:45 |
| opendevreview | Artem Goncharov proposed openstack/pbr master: add python 3.12 support https://review.opendev.org/c/openstack/pbr/+/924216 | 20:40 |
| cardoe | Wondering if this mirror is setup wrong? https://www.irccloud.com/pastebin/rRTG6n8y/ | 23:37 |
| cardoe | https://zuul.opendev.org/t/openstack/build/19d11f769c3a4f5c8ba3b42dbba968b1 | 23:37 |
| clarkb | if you navigate to the root of that mirror it looks like something is sad | 23:42 |
| clarkb | the mirro remounted its cache device read only and lvs outputs no info | 23:43 |
| clarkb | The last time this happened we believe it was related to changes to cinder in the cloud (encryption or something similar iirc) and we ended up provisioning an entirely new device and replacing it. I don't know that I can get to that right now so I'll push up a change to disable the cloud instead | 23:44 |
| opendevreview | Clark Boylan proposed openstack/project-config master: Disable raxflex cloud https://review.opendev.org/c/openstack/project-config/+/935575 | 23:46 |
| clarkb | cardoe: ^ that is the short term mitigation then we need to followup with rackspace to see if this is another unexpected user impact and/or replace the cinder volume | 23:46 |
| clarkb | cardoe: side note all of our mirrors should have https enabled | 23:47 |
| cardoe | So I'll tell them data's gone missing. | 23:48 |
| clarkb | its all ephemeral in our case so not a huge deal from that perspective. The main issue is that lvm seems to have lost track of the device and consequently things remounted ro and we can't cache and that makse things sad | 23:50 |
| clarkb | last time we couldnt' read sector 0x0 but this time I think that may not be the same symptom | 23:50 |
| clarkb | actuall nope same error | 23:51 |
| cardoe | Not good though. They should be more stable. | 23:51 |
| clarkb | kernel: I/O error, dev vdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 32 prio class 0 | 23:52 |
| cardoe | sean-k-mooney: You got an example somewhere that go install works? I'm getting. https://zuul.opendev.org/t/openstack/build/59075f796f43477d80eefe528e411ac1 | 23:52 |
| clarkb | cardoe: let me see if I can find an example from the zuul-jobs testsuite | 23:54 |
| cardoe | I'm not a Go guy so I'm just gonna google. | 23:54 |
| clarkb | https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/go this is what the standard golang jobs in zuul-job do. ensure-go in pre-run and run playbook runs the go role | 23:56 |
| clarkb | what those all do under the hood I'm not sure | 23:56 |
| sean-k-mooney | https://github.com/openstack-k8s-operators/nova-operator/blob/main/Makefile#L233-L236 | 23:56 |
| sean-k-mooney | that how we install envets | 23:56 |
| sean-k-mooney | or ginkgo https://github.com/openstack-k8s-operators/nova-operator/blob/main/Makefile#L242 | 23:56 |
| sean-k-mooney | GOBIN=$(LOCALBIN) go install github.com/onsi/ginkgo/v2/ginkgo | 23:57 |
| sean-k-mooney | GOBIN is where ot install it | 23:57 |
| sean-k-mooney | https://github.com/norwoodj/helm-docs/blob/master/Makefile#L4C1-L5 that what they have in there makefile | 23:59 |
| sean-k-mooney | im trying `GOBIN=/home/smooney/.local/bin/ go install github.com/norwoodj/helm-docs/cmd/helm-docs@latest` | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!