| opendevreview | Goutham Pacha Ravi proposed openstack/project-config master: Add manila-core to osc/sdk repo config https://review.opendev.org/c/openstack/project-config/+/894605 | 00:25 |
|---|---|---|
| opendevreview | Goutham Pacha Ravi proposed openstack/project-config master: Add manila-core to osc/sdk repo config https://review.opendev.org/c/openstack/project-config/+/894605 | 00:26 |
| opendevreview | Bernhard Berg proposed zuul/zuul-jobs master: prepare-workspace-git: Add ability to define synced pojects https://review.opendev.org/c/zuul/zuul-jobs/+/887917 | 08:47 |
| *** amoralej is now known as amoralej|lunch | 12:16 | |
| opendevreview | Merged openstack/project-config master: Add manila-core to osc/sdk repo config https://review.opendev.org/c/openstack/project-config/+/894605 | 12:55 |
| *** amoralej|lunch is now known as amoralej | 12:58 | |
| opendevreview | daniel.pawlik proposed zuul/zuul-jobs master: Add feature to set --vm-driver name for minikube https://review.opendev.org/c/zuul/zuul-jobs/+/894755 | 13:35 |
| opendevreview | daniel.pawlik proposed zuul/zuul-jobs master: Add feature to set --vm-driver name for minikube https://review.opendev.org/c/zuul/zuul-jobs/+/894755 | 13:38 |
| opendevreview | daniel.pawlik proposed zuul/zuul-jobs master: Add feature to set --vm-driver name for minikube https://review.opendev.org/c/zuul/zuul-jobs/+/894755 | 14:18 |
| opendevreview | daniel.pawlik proposed zuul/zuul-jobs master: Add feature to set --vm-driver name for minikube https://review.opendev.org/c/zuul/zuul-jobs/+/894755 | 14:23 |
| clarkb | frickler: any luck with the inmotion image(s) and possibly leaked instances keeping the images around? | 14:46 |
| frickler | clarkb: not yet, I just noticed that there are even more images in glance with status "queued" than pending delete in nodepool, but didn't check in more detail so far | 14:49 |
| clarkb | I think those may be leaks that we can delete as normal users. I can give that a try after breakfast | 14:50 |
| clarkb | the queued ones specifically | 14:50 |
| frickler | one general note, victoria is pretty old, is there any plan/provisions to update that cluster? or would that imply a redeployment (which might be even easier when we want to go to something recent)? | 14:52 |
| frickler | maybe we could skip the cleanup that way, too | 14:53 |
| clarkb | frickler: no hard plans but yes it should be updated. The path forward there is to redeploy | 14:54 |
| clarkb | frickler: yuriys isn't in this channel right now but in vancouver said that our deployment was one of the first and we helped them work through a lot of the initial problems and learning. He thought it would be easier to redeploy at this point if we go that route | 14:55 |
| clarkb | I think for planning a redeployment we mostly need to determine if we can deploy a new cloud alongside the old one or if we need to stop the current one and replace it (for quota/resource reasons) | 14:56 |
| clarkb | then make note of the general architecture. I think it is 3 converged control nodes + 4 compute nodes of varying size and we run instances on provider networking to avoid losing IPs to router interfaces | 14:56 |
| clarkb | one of the main things we worked with them on was ssl+certs (I think we are doing self signed so we have an explicitly trusted cert in our clouds.yaml config). I expect that may have improved sicne we did the deployment though | 14:58 |
| frickler | ceph seems to be running 1 nvme osd on each of the hosts | 14:58 |
| frickler | I think after the openstack release we should be fine with dropping that capacity for a couple of days if needed | 15:00 |
| clarkb | agreed, there tends to be a lull in activity after openstack makes its release | 15:00 |
| * clarkb scribbles a note to contact yuriys for any other inputs | 15:01 | |
| clarkb | fungi: ^ you may be interested too. yuriys shares a timezone with you | 15:02 |
| frickler | I'll try to look at the kolla configuration to see if I notice anything else noteworthy. but I think I'd prefer to focus on redeployment then and ignore the image cleanup until then | 15:02 |
| clarkb | wfm | 15:02 |
| frickler | ceph seems to be at ~35% usage, so no lack of capacity there | 15:03 |
| clarkb | frickler: our main limitation was cpu and memory iirc | 15:04 |
| clarkb | actually no just cpu | 15:04 |
| clarkb | we tried overcommitting and it just didn't work for the CI workloads and so ended up on a 1:1 or close to 1:1 cpu ratio for vms | 15:04 |
| jrosser | is this for aarch64 capacity? | 15:09 |
| clarkb | jrosser: no that cloud is x86_64 | 15:10 |
| frickler | maximum 51 nodes, so roughly 10% of our total capacity | 15:15 |
| fungi | will a redeploy preserve our mirror instance, or will we need to recreate it too? | 15:19 |
| clarkb | corvus: looking at the list of github repos without apps I wonder if some would be willing to provide an install with fewer perms. Do you know what perms are necessary to do the advisory type testing we do? I think we have the app set up to request perms that could potentially be used for gating but we aren't doing that with these repos | 15:19 |
| clarkb | fungi: it will need to be redeployed too | 15:19 |
| clarkb | specifically thinking projects like gnocchi and sqlalchemy/dogpile.cache might be open to that as we know their maintainers and can explain the use case | 15:21 |
| clarkb | I suspect it would be a hard sell to get any app installed on ansible or cryptography | 15:21 |
| clarkb | fungi: do you know who to get in touch with in the monasca project about removing the elastic/kibana zuul integratoin? | 15:23 |
| clarkb | that project is no longer OSI compatible and I'd like to remove it from our zuul config (note I don't think CI use cases are in violation of the lciense, I just don't want to be in a situation where non open source software in our system is making our lives more difficult so it should be removed) | 15:24 |
| fungi | clarkb: i think it's probably a question for the openstack tc. looks like there's not even a ptl candidate for that team | 15:26 |
| clarkb | fungi: ack I'll bring it up there | 15:26 |
| frickler | there's a potential candidate https://review.opendev.org/c/openstack/election/+/893033 | 15:26 |
| frickler | just noone with actual contributions | 15:26 |
| fungi | they have an irc channel but nobody using it other than bot notifications | 15:26 |
| frickler | the electorate for that project would have been elodilles and me, but just for cleanup patches | 15:27 |
| fungi | if there are no substantive changes being tested for their repos, then it's pretty easy to make the argument that they're not doing anything with that job integration anyway | 15:28 |
| frickler | yes, so just proposing a patch for cleanup and force-merging with TC approval if needed would sound fine to me | 15:31 |
| clarkb | corvus: searching hound for pravega/zookeeper-operator shows the only instance of that string is in the zuul tenant config itself. I think that can be removed? Pinging you as I suspect that was in use by zuul operator testing at some point or was intended to be | 15:33 |
| clarkb | frickler: the github.com/eventlet/eventlet inclusion was added I think for testing with neutron. Do you recall if that was the case? I can't find it in hound (so maybe it is on older branches?) | 15:35 |
| clarkb | but I think eventlet/eventlet can just be removed too if it isn't actually used anywhere | 15:35 |
| corvus | clarkb: i don't think i have an exhaustive list, but my guess would be just "read" for "repo admin", "repo contents", "issues", and "prs". maybe "statuses". from https://zuul-ci.org/docs/zuul/latest/drivers/github.html#configure-github | 15:35 |
| frickler | clarkb: I'm not sure, multiple projects come to my mind that might do testing on eventlet latest | 15:36 |
| clarkb | corvus: cool maybe we can work with someone like zzzeek to test that | 15:36 |
| corvus | clarkb: yeah, i think we can remove the zk operator | 15:36 |
| clarkb | frickler: ok git logs can probably point me more directly at the people who started with it at least | 15:37 |
| clarkb | I need to pause for breakfast but I should be able to push changes up for some of this soon | 15:38 |
| fungi | skimming codesearch, i don't see any jobs setting eventlet/eventlet as a required project | 15:38 |
| clarkb | fungi: ya I couldnt' find any examples either | 15:39 |
| clarkb | git logs say it was swift doing python3 in ~2019 | 15:39 |
| frickler | git blame finds https://review.opendev.org/c/openstack/project-config/+/656889 | 15:39 |
| frickler | timburke: ^^ is that still being used somewhere? | 15:39 |
| clarkb | looks like in the related chagne the required-project was used in older patchsets then eventually removed before the change merged | 15:40 |
| clarkb | I suspect we can remove eventlet too | 15:41 |
| clarkb | I can get a change up for zookeeper-operator and eventlet together then another separate one for kibana | 15:41 |
| fungi | looks like there's going to be a python 3.12.0rc3 on monday with some more bug fixes... getting close! | 16:07 |
| corvus | clarkb: i think https://review.opendev.org/794688 and generating a personal access token for the account would help | 16:22 |
| corvus | i just tested my own personal access token and verified that i get the full 5k rate limit using that, and was able to load the opendev configuration as currently written. | 16:23 |
| corvus | so i think we just need a change like that to handle automatic app -> token fallback while still having app configured | 16:24 |
| corvus | (alternatively, we could drop the app config, set a token, and run without that patch today) | 16:24 |
| frickler | did something change about github api limits recently? I think I'm seeing more issues downstream, too | 16:38 |
| frickler | seems we've had a spam message pass on service-discuss: https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/message/JLGRB7TNXJK2W3ELRXMOTAK3NH5TNYI3/ | 16:53 |
| clarkb | corvus: ok looks like there are some things that need addressing in the change. I suspect ianw may not have time for that now. I can try and push on it | 16:53 |
| clarkb | frickler: I think fungi developed a reaction process to that when it happened to zuul. | 16:54 |
| corvus | clarkb: sounds good; i think they are fairly straightforward | 16:55 |
| clarkb | I still think removing unneeded projects is a good idea, but good to know we can improve this through zuul tool | 16:55 |
| corvus | yeah. best outcome is we have the 5k ops available to projects that actually use them. and we don't have to have a high bar for adding projects. | 16:57 |
| opendevreview | Clark Boylan proposed openstack/project-config master: Remove two unnused github projects from tenant config https://review.opendev.org/c/openstack/project-config/+/894813 | 17:02 |
| opendevreview | Clark Boylan proposed openstack/project-config master: Remove github.com/elastic/kibana from Zuul config https://review.opendev.org/c/openstack/project-config/+/894814 | 17:02 |
| clarkb | I suspect that first one is good to go. For the second I'm hoping I can make use of the TC meeting to get an ack that this is also going to happen | 17:02 |
| corvus | did i read that kibana is only used by a dormant project? | 17:03 |
| clarkb | corvus: yes | 17:04 |
| clarkb | but apparently part of the raeson it is dormant is that no one can get the CI working. Of course my response to that would be start over on the CI jobs but people seem scared of doing that | 17:04 |
| corvus | ah | 17:04 |
| clarkb | the number of branches they have makes the github branch viewer sad when I scroll through it too fast | 17:05 |
| corvus | maybe they thought it was a kpi metric to optimize. let's get the number of stars and branches super high! | 17:10 |
| clarkb | corvus: if you remove a project from a pipeline config it shouldn't report zuul config errors via that pipeline's reporter correct? | 17:20 |
| opendevreview | Bernhard Berg proposed zuul/zuul-jobs master: prepare-workspace-git: Add ability to define synced pojects https://review.opendev.org/c/zuul/zuul-jobs/+/887917 | 17:27 |
| JayF | infra-root: can someone please hold a failing node for ironic-tempest-bios-redfish-pxe on https://review.opendev.org/c/openstack/ironic/+/894460 | 17:33 |
| JayF | You can put the keys from https://github.com/jayofdoom.keys on it when it gets caught | 17:34 |
| corvus | clarkb: yes | 17:35 |
| JayF | let me know when the hold is in place so I can recheck | 17:37 |
| clarkb | JayF: the hold is in place | 17:37 |
| clarkb | let us know when it fails and we can update ssh keys | 17:37 |
| JayF | ack | 17:37 |
| JayF | ty | 17:37 |
| JayF | clarkb: I think we got a winner, https://github.com/jayofdoom.keys https://zuul.opendev.org/t/openstack/build/2f3f200d942c48ac94bbef27d70cbb25 | 19:08 |
| clarkb | JayF: I can add them after the infra team meeting | 19:08 |
| JayF | I am about to lunch so there is no real rush, just wanted to send the message so I wouldn't forget :) | 19:09 |
| JayF | thank you | 19:09 |
| fungi | #status log Deleted a stray spam post from the service-discuss list archives | 20:26 |
| opendevstatus | fungi: finished logging | 20:26 |
| fungi | i guess if there's something to document, it's https://gitlab.com/mailman/hyperkitty/-/issues/261 | 20:26 |
| fungi | log into the webui with superuser credentials and message/thread deletion is glaringly obvious | 20:26 |
| clarkb | JayF: root@173.231.255.247 | 20:28 |
| fungi | oh, i missed that there's recent progress on the message deletion issue in hyperkitty: https://gitlab.com/mailman/hyperkitty/-/merge_requests/534 | 20:30 |
| clarkb | fungi: when you did the deletion you did so as the django superuser then? | 20:51 |
| clarkb | I'm going to pop out for a bike ride but when I return I'm planning to land the zuul tenant config cleanups and draft that email to yuriys | 20:53 |
| fungi | clarkb: yeah, i opened a browser tab in a new "container" so that it wouldn't be logged in as my list owner account, logged into mai | 21:08 |
| fungi | lman with the admin account in our private hostvars on bridge, and then there were "delete post" and "delete thread" buttons next to the spam message in hyperkitty | 21:09 |
| fungi | hopefully the author of that hyperkitty mr finishes up the work and gets it into an upcoming release | 21:10 |
| fungi | because ideally the list owners or moderators should have permission to take those actions | 21:10 |
| fungi | but even moreso, i hope this just doesn't come up all that often | 21:12 |
| clarkb | I guess that doesn't sound too bad but ya less than ideal | 22:53 |
| clarkb | anyone want to review https://review.opendev.org/c/openstack/project-config/+/894813/1 and child before I proceed with approving them? | 22:53 |
| fungi | yeah, i can | 22:56 |
| fungi | reviewed and approved | 22:56 |
| clarkb | thanks | 22:57 |
| clarkb | https://etherpad.opendev.org/p/VvYeKbEBSpVo0TU2vGqe something like this for the inmotion cloud deployment? | 22:59 |
| opendevreview | Merged openstack/project-config master: Remove two unnused github projects from tenant config https://review.opendev.org/c/openstack/project-config/+/894813 | 23:08 |
| opendevreview | Merged openstack/project-config master: Remove github.com/elastic/kibana from Zuul config https://review.opendev.org/c/openstack/project-config/+/894814 | 23:08 |
| fungi | clarkb: draft e-mail lgtm. the only other thing that comes to mind is whether we should wait for kolla-ansible to release its 2023.2 version too and then deploy that | 23:19 |
| fungi | which ties into your "what versions are available" question | 23:20 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!