Tuesday, 2022-07-19

fungino, not yet00:00
ianwok, i'll run a zuul_pull and start them in that screen then00:02
*** rlandy|bbl is now known as rlandy00:35
*** rlandy is now known as rlandy|out01:49
opendevreviewJie Niu proposed opendev/system-config master: add computing force network mailling list for computing force network working group  https://review.opendev.org/c/opendev/system-config/+/85026803:31
*** ysandeep|out is now known as ysandeep05:21
*** tkajinam is now known as Guest543507:14
*** Guest5435 is now known as tkajinam07:36
*** ysandeep is now known as ysandeep|lunch07:57
*** ysandeep|lunch is now known as ysandeep09:30
opendevreviewMerged opendev/system-config master: centos 7 mirror: switch upstream provider  https://review.opendev.org/c/opendev/system-config/+/85026009:55
ianw^ i've manually run that, and the volume has released now10:22
*** rlandy|out is now known as rlandy10:23
*** dviroel|out is now known as dviroel11:27
*** ysandeep is now known as ysandeep|break12:02
*** ysandeep|break is now known as ysandeep|mtg12:33
*** dhill is now known as Guest545812:49
*** ysandeep|mtg is now known as ysandeep12:58
*** dasm|off is now known as dasm|ruck13:31
*** ysandeep is now known as ysandeep|dinner14:28
opendevreviewClark Boylan proposed opendev/system-config master: WIP Update to Gitea 1.17.0-rc1  https://review.opendev.org/c/opendev/system-config/+/84720414:50
opendevreviewClark Boylan proposed opendev/system-config master: WIP Update to Gitea 1.17.0-rc2  https://review.opendev.org/c/opendev/system-config/+/84720414:51
*** ysandeep|dinner is now known as ysandeep15:02
*** marios is now known as marios|out15:58
clarkbI'm not seeing any recent POST_FAILURES in our hourly deploy pipeline. Looking much better now that all the executors are updated16:08
fungiso we think the glibc update has taken care of it? did the ansible version override get rolled back yet?16:10
*** ysandeep is now known as ysandeep|afk16:18
clarkbfungi: yes the ansible version override was rolled back a day or two ago. I think this must fix it since we were able to find an example of the issue with only 50 % of the executors on old glibc yesterday16:21
*** ysandeep|afk is now known as ysandeep|out16:38
corvusAlbin Vass: ^ 🍺17:30
AlbinVass[m]Cool :)17:37
*** rlandy is now known as rlandy|ruck17:40
clarkbwe'll definitely continue to check it just in case but ya good find17:44
*** dasm|ruck is now known as dasm|off17:48
AlbinVass[m]Btw since ansible closed the issue with "the way to fix this is to upgrade glibc" it seems likely to pop up in the future again as long as they use os.fork to spawn new processes.17:55
clarkbinfra-root did anyone else want to weigh in on whether or not I should update https://review.opendev.org/c/opendev/system-config/+/850059 to be a role?17:55
clarkbAlbinVass[m]: ya I guess on older rhel they expect you to use ansible 2.9 so isn't a problem there17:56
clarkbI half expected them to want to figure out a way to fix it until I realized this17:56
ianwclarkb: https://review.opendev.org/c/openstack/project-config/+/850252/2/tools/check_valid_gerrit_projects.py -- i guess that's functionally the same as hitting an api end point like /repos/ or something?20:05
clarkbianw: yup, except that the gitea api requires auth20:05
clarkbianw: using the api was my first good idea until I realized that. I think checking the web ui for a 404 is simple and robust here20:06
ianwit does just check for a generic error20:06
ianwi guess if we broke things (so it was 500 instead of 404, say) , broken ssh would be about the same20:08
clarkbyup, but all that code does in that case is check the upstream remote can be cloned20:08
clarkbwhich means running in an unexpected failure case should be safe enough (we'll just do a potentially unnecessary git clone from the upstream remote)20:08
ianwclarkb/fungi: if you have time to go through the stack @ https://review.opendev.org/q/topic:selfsigned-shared-ca that would be good.  the base change has had reviews, but just like some eyes on the bits on top20:09
clarkbianw: I'll try to get to that today. SHould be able to20:10
ianwclarkb: if you could also double check https://review.opendev.org/c/opendev/system-config/+/845316 for the haproxy logs that you already looked at; i can deploy that and watch it closely 20:10
opendevreviewMerged openstack/project-config master: HTTP check existing repositories  https://review.opendev.org/c/openstack/project-config/+/85025220:21
clarkbianw: https://review.opendev.org/c/opendev/system-config/+/845316 lgtm but didn't approve as unsure when you wanted to keep an eye on it. Feel free to +A20:23
clarkbStarted there as I had previously reviewed it. Will look at the CA stuff shortly20:24
ianwthanks, yeah i'll do it a little later just to watch it closely20:24
ianwthanks; certainly no rush on that stuff20:25
*** Guest5201 is now known as diablo_rojo_phone20:57
clarkbianw: question on https://review.opendev.org/c/opendev/system-config/+/848562 I'm sure I'm just not understanding some bash behavior there but figured I'd ask anyway21:39
ianwclarkb: yeah the args are really weird there; left a comment but basically each arg is a spaced-deliminated string21:44
fungii too had to stare at that for a bit to realize that's how it was being parsed out21:47
fungiseems sort of like relying on an implementation detail, but it's not a terrible way to avoid double-accounting that list21:47
clarkboh so when we've got multiple certs we'll get multiple compound arg lists21:47
clarkbis that worth a comment in the actual code (followup is fine)21:48
ianwi feel like there may be a comment to that effect, but maybe not at that point21:48
ianwon line 57, but yeah it is obscure21:49
clarkbianw: ok I think I've reviewed the whole stack. A few nits here and there, but nothing crazy22:14
ianwthanks, will look soon.  just writing up the changelog for another one for you to chew on wrt the zuul console logs :)22:15
*** dviroel is now known as dviroel|dinner22:22
*** dviroel|dinner is now known as dviroel|out23:20

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!