Thursday, 2024-01-18

opendevreviewIan Wienand proposed opendev/system-config master: [wip] small overhaul of rpm builds
opendevreviewIan Wienand proposed opendev/system-config master: [wip] small overhaul of rpm builds
ianw^ at least that tells us clearly why it was failing01:53
ianw*** Running kernel does not have headers01:53
ianwkernel-devel-uname-r = 4.18.0-535.el8.x86_6401:53
ianw| centos-8-stream-d1380f065f264bc0af1193466458adcc           | centos-8-stream           | | qcow2,raw,vhd | ready    | 13:06:52:50  |01:57
ianw| centos-8-stream-447e693eca05446ea9e6c4f33accbaaf           | centos-8-stream           | | qcow2,raw,vhd | ready    | 06:04:04:59  |01:57
Clark[m]Oh I bet those images refresh weekly now02:00
Clark[m]That would explain at least some of the delay02:00
ianw-rw-r--r-- 1 nodepool nodepool 1.3M Dec 21 15:22 centos-8-stream-2a2b6da68bef4c4dad1eebea9c0de0b2.log02:05
ianw-rw-r--r-- 1 nodepool nodepool 1.3M Dec 28 17:13 centos-8-stream-7696746ed6004390b0dd7746188c73b1.log02:05
ianw-rw-r--r-- 1 nodepool nodepool 1.3M Jan  4 19:03 centos-8-stream-d1380f065f264bc0af1193466458adcc.log02:05
ianwthat would explain the cadence ...02:05
ianwwell i've kicked off two fresh builds of those02:10
ianw906013 is likely to pass with new images02:11
fricklerupdating the cert seems to have worked tonight, yay08:18
fricklersame for review.opendev.org08:19
*** benj_4 is now known as benj_08:24
ianw ... seems we're still behind on the centos-8-arm64 image, but when that appears i think we should be back to working openafs08:32
opendevreviewIan Wienand proposed opendev/system-config master: roles/openafs-client: small overhaul of rpm builds
*** dmellado747 is now known as dmellado7410:58
*** dmellado745 is now known as dmellado7411:07
*** priteau_ is now known as priteau12:14
clarkbOIL episode is done. I'm going to find braekfast then be back in an hour or so. I'm going to look at maybe approving those gitea changes today16:09
fungiyeah, heading out to lunch here, and then need to finish up some paperwork but i'll be around again after ~17:30 utc if i'm needed16:16
*** tosky_ is now known as tosky16:41
clarkbI'ev approved which I wanted to go in before the robots.txt fix if possible17:21
clarkbfungi: maybe when you get back from lunch you want ot approve the robots.txt update?17:21
fungisure, can do17:54
jrosseris there any possibility to have `job.collections` in a similar manner to `job.roles`?17:54
clarkbhaproxy has made new releases I'm going to push up a change to test if they fix our issue. Not beacuse I think we should get off of the lts tag but because I want ot make sure that 2.9 eventually fixes things as I assume one day it will be an lts17:55
clarkbjrosser: that is currently unimplemented but something like that has been talked about in the past. There may even be a zuul spec for it17:55
clarkbmay be worth discussing in the zuul matrix room if you are in that room.17:56
jrosseri am - i can do that17:57
opendevreviewClark Boylan proposed opendev/system-config master: DNM Use latest haproxy image to test newer versions
TheJuliahey folks, eventlet has unfortnately put one of ironic's repos in a not great state for the docs job. Is there any clean/happy/preferred way to make our docs job non-voting specifically when we're using a template from the system18:00
clarkbTheJulia: I'm not positive of this but I think you may be able to list the job in your jobs list and set it to non voting and have that override things18:01
clarkbif I'm wrong that should throw an error about duplicate jobs or something along those lines and return quickly I hope18:01
TheJuliaclarkb: That is kind of what I was thinking as well18:01
* TheJulia gives it a spin18:01
clarkband in that case you'd have to drop the template and list the jobs explicitly18:01
fungiTheJulia: another possibility is that something is not quite right with the tox config on that repo and is causing newer eventlet to be installed than what's in the requirements upper-constraints.txt18:05
fungidocs builds often have an extra bit of configuration which was problematic in tox v4, and may have gone unnoticed until now18:06
TheJuliait might be fine in u-c, the underlying issue regardless is a bug which is not common and we don't have a forward path on yet, either.18:06
fungiyeah, i mean you may have two bugs: constraints not being applied to your docs builds and also docs builds not working with latest eventlet (which will become relevant anyway once constraints updates for that)18:07
JayFI suspect we have more like 17 bugs and we're up to like, 3 of them identified ;) 18:08
JayFsome combo of werkzeug + new eventlet + sphinx autodoc18:08
TheJuliaWe also don't care *as* much, since the specific repo is on the exit path in the grand scheme of things18:08
fungiwell, yes, i didn't mean to imply that was the upper limit of your bug count, merely the lower limit of bugs affecting your docs builds ;)18:09
JayFfungi: I'm only talking about the docs build 18:09
fungihah, okay18:09
JayFI have stared into the sphinx, it gave me a riddle, I said "I'm only A MAN!" and it was like "correct"18:09
JayFand now I have all these bugs to fix18:10
TheJuliawell, zuul didn't glare at me immediately18:10
TheJuliaso there *is* hope18:10
fungiriddle me this18:11
TheJuliawait, who is batman in this context?18:11
fungidunno, but i figure the sphinx and the riddler have something in common18:11
JayFbatman is the one walking on zero legs and one grappling hook18:11
* TheJulia wonders if JayF is describing an incorrectly assembled or "new age" office chair18:13
JayFI'm trying to make a joke referencing the riddle of the Sphinx (What walks on four legs in the morning, two legs in the afternoon, and three legs at night?)18:14
JayF(the answer is "a man/human/person")18:14
clarkblooking at zuul status it appears my suggested method did work18:20
* clarkb hopes to remember that for next time18:20
TheJuliait.. also... passwed.18:31
fungithis too shall pass18:36
TheJuliaWell, my mind is more in the mood for T'pol saying "facinating" or "intruiging"18:37
fungior t'pau saying: "if both survive the lirpa, combat will continue with the ahn woon"18:39
fungi(most iconic tos episode evar)18:39
TheJuliaI actually remarked about the main scene to someone this past weekend18:40
fungii only recently got clearance from my better half that we could start watching strange new worlds together, and i was rather impressed with the relevance and accuracy of original series references18:41
* fungi did not have high hopes going into it18:41
opendevreviewMerged opendev/system-config master: Disable gitea's update checker cron job
clarkbreview.o.o did trip the certcheck alert. But looking at it in my browser the cert is updated. That one may need an apache restart if it doesn't clear out the older worker on its own18:46
fungii thought the alert had merely raced fixing the base deploy job18:47
fungiwasn't going to pay it much heed unless it happened yet again18:47
clarkbseems 905020 did not restart services (this was expected just confirming this)19:11
clarkband new haproxy doesn't break afte 4k connections:
opendevreviewMerged opendev/system-config master: Update our Gitea robots.txt from's
clarkbthinking out loud for preptg planning. Maybe something like wednesday + thursday in a to be determined week. I would do Monday + Tuesday but Tuesdays tend to be filled with other meetings for me. Then the next questions are how to deal with timezones and which week do we ant to do. Also this is just brainstorming feel free to suggest alternatives19:30
opendevreviewMerged openstack/project-config master: Retire heat-cfnclient: End Project Gating
clarkbif I stay up late we can get not compeltely terrible overlap with my timezone (and corvus') as well as tonyb and frickler. Something like 06:00-08:00 UTC. But that excludes fungi. If we do time around 14:00UTC-16:00UTC that excludes tonyb but should work for everyone else. And if we did say 21:00-23:00 UTC we'd exclude frickler but be somewhat reasonabel for everyone else19:34
clarkbtalking out loud here maybe we pick two of those opptions each day for a total of 4 hours a day over two days? I'll keep trying to come up with something better19:34
fungias long as i don't have anything else pressing immediately before/after, i can adjust my sleep schedule to accommodate either19:35
clarkbgitea09's gitea service restarted a few minutes ago19:37
clarkbso ya the robots.txt update appears to be doing what we want19:37
clarkbalso I think there is a new gitea bugfix release19:38
fungiyep, is showing up for me at this point, looks great19:54
fungiwell, maybe not great, but looks like what we configured anyway19:54
opendevreviewClark Boylan proposed opendev/system-config master: Update gitea to 1.21.4
fungiclarkb: was removal of the search templates intentional? it's neither mentioned in the commit message nor the changelog afaict20:18
fungiwere our copies identical to upstream's or something?20:18
Clark[m]Oh sorry yes those were the files we patched to fix the code search 500 errors. I should've called that out20:43
clarkbfungi: should I update the commit message?20:53
opendevreviewClark Boylan proposed opendev/system-config master: Update gitea to 1.21.4
clarkbI decided it was important enough for future us if digging through history to get that right and udpated the commit message21:00
fungiclarkb: yeah, clearly i had already forgotten why those were there in the previous upgrade. thanks!21:46
ianwok has succeeded ; which means all the centos nodes should be able to build openafs now.  review of the changes proposed there to the checking/build/log collection welcome, but aren't required to build the modules now it's working (just to help us next time they're not)22:53
JayFI am having trouble exercising the "eventlet in required_projects" support that was mentioned in the last few days; both '' and 'eventlet/eventlet' both come up as unknown projects in zuul:
JayFI've checked out just about every repo that I can tell is involved in config, and a grep for 'eventlet' across them does not find anything interesting (including: openstack-zuul-jobs, project-config, system-config) ... so I'd not only like the correct answer, but I'd love to figure out how I coulda self-served this23:12
tonybJayF: looking gimme a few23:21
JayFack not urgent :) thank you23:21
Clark[m]It's possible we removed eventlet. Or maybe we aren't loading it into zuul properly23:21
Clark[m] is the file23:22
JayFYeah; I think it may be worth the effort to plumb it back up if we can given current circumstances if it's not far out of reach. I certainly was going to add a unit test / docs / tempest job to Ironic's projects experimental queues23:22
Clark[m]I remember now we removed eventlet because it had a significant number of branches which incurred zuul startup penalties23:22
Clark[m]It appears to have 21 branches now which seems far less than it must've been before. We can probably add it back. Also zuul got improved since then for dealing with GitHub rate limits23:25
tonybJayF: So adding it back would be a patch here:, then using it would be most like 906070,PS123:26
opendevreviewJay Faulkner proposed openstack/project-config master: Add eventlet to projects available from github
tonybIn terms self service, you can look at: to see which projects and drivers the openstack tenant has configured23:28
JayFClark[m]: Yeah, it's all been ... what's the word ... maintained :D since them23:29
JayFtonyb: yeah, I mainly wanted to make sure there wasn't a bonus-config repo or something hiding somewhere ;) 23:30
tonybJayF: fair enough.  That page will show you "all the things" finding where they're defined .... not so much23:30
clarkbJayF: ya I feel like the count was in the thousands range when we removed it23:30
clarkbwhich caused zuul to dig deep into github rate limits to process them all23:31
JayFI'd believe it, but have no direct knowledge :)23:31
JayFif that becomes true now, we have friends with brooms who can help clean up lol23:31
clarkbwe probably should try and curate that list a bit though. Like is anyone doing dpdk stuff these days?23:32
tonybJayF: There is a "zuul app" you can add to eventlet but I *think* that's only if you want zuul to do more than comment?23:32
JayFIf there is value to you all to do that, we can pursue that with Herve and that crew. I'm happy to be ambassador for those discussions23:33
clarkbtonyb: I think the main issue with the app (and this is something we could investigate adjusting too) is that I think it may request push/merge perms since when it was set up we thought it might do things like that for some projects23:33
clarkbsince we don't do gating and merging for anything we can remove that permission from the required list and then hopefully projects will be more inclined to add the app as it does make a huge difference for rate limits23:34
tonybJayF: I don't have a strong opinion there.  I feel like it's a time will tell situation23:34
JayFI'm just letting you know I'm happy to be the complaint box23:35
JayF*for this specific kind of complaint*23:35
JayFeverything else OpenStack goes to the TC Chair ;)23:35
tonybclarkb: Oh yeah that might help.  I wonder if we could have 2 "apps".  one for comments/tests and another for comments/tests/gating+merging23:35
tonybJayF: LOL23:36
clarkbtonyb: I think we can probably minimize perms for the existing app now since we're just trying to do informative responses. But then ya if that changes a second app may make sense for doing more23:39
clarkbpart of the problem there is I think the apps perms aren't always intuitive and there was a bit of trial and error we may have to go through again23:40
clarkbnot that gerrit is better with some of its acls. Push tag is distinct from push signed tag and whether or not you can delete branches is affected by your push permissions. etc23:41
tonybclarkb: Yeah, it's all a bit of a mess23:42

Generated by 2.17.3 by Marius Gedminas - find it at!