15:00:05 <gmann> #startmeeting tc
15:00:06 <mnaser> o/
15:00:07 <openstack> Meeting started Thu Apr  1 15:00:05 2021 UTC and is due to finish in 60 minutes.  The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:11 <openstack> The meeting name has been set to 'tc'
15:00:13 <gmann> #topic rollcall
15:00:16 <yoctozepto> \o/
15:00:17 <dansmith> o/
15:00:26 <gmann> o/
15:00:28 <dansmith> sorry, I'm not as excited as yoctozepto
15:00:37 <gmann> :)
15:00:52 <belmoreira> o/
15:01:55 <diablo_rojo> o/
15:01:57 <gmann> let's start
15:02:07 <gmann> #topic Follow up on past action items
15:02:17 <gmann> no action items from previous meeting
15:02:32 <gmann> #topic  PTG
15:02:43 <gmann> #link https://etherpad.opendev.org/p/tc-xena-ptg
15:03:07 <gmann> we have room booked now, thanks diablo_rojo for arranging new room for Thursday
15:03:29 <gmann> I have added the information in etherpad
15:04:02 <gmann> also I sent invitation to k8s steering committee also for cross community sessions.
15:04:17 <gmann> I have not got any reply yet so let's wait
15:04:46 <gmann> and keep adding the topics  in etherpad you would like to discuss in PTG.
15:04:59 <mnaser> +1
15:05:04 <gmann> that's all on PTG planning form my side.
15:05:09 <diablo_rojo> gmann, no problem!
15:05:10 <gmann> anything else from any one?
15:05:40 <mnaser> nothing to add from me
15:06:04 <gmann> ok, let's move to next topic
15:06:08 <gmann> #topic Gate performance and heavy job configs (dansmith)
15:06:13 <dansmith> I have updated numbers!
15:06:18 <dansmith> https://termbin.com/n7yl
15:06:31 <dansmith> most every project has dropped their footprint, tripleo being the biggest
15:06:44 <gmann> nice
15:06:49 <dansmith> I think the nova +1 bump is likely just noise, as is the kolla -2, since node counts are unchanged
15:07:06 <dansmith> not much movement on the heaviest one, neutron, but some
15:07:13 <mnaser> wow neutron @ 34h sounds wild
15:07:23 <dansmith> I'm looking for clark to update me with the percent usage again and then will send something to the list
15:07:30 <mnaser> that's how many hours of testing each patchset gets right/
15:07:35 <dansmith> mnaser: yeah they're still really big :/
15:07:38 <dansmith> mnaser: yes
15:08:33 <dansmith> other than that,
15:08:42 <dansmith> I think the biggest thing hurting us is still spurious fails
15:09:00 <dansmith> I've got a patch up to fix the only known one in glance, but I think cinder is still the biggest offender,
15:09:16 <gmann> may be we can check with neutron team on optimizing the number of jobs runs for related file/code change? if not yet done
15:09:21 <dansmith> although my "seat of the pants" data has been less accurate since the release was cut and things have slowed
15:09:50 <dansmith> gmann: yeah, or just ask them to do less per patch. I think they already have a lot of per-file rules
15:10:16 <dansmith> we'll see what the usage percent values look like
15:10:18 <fungi> we've also had a spate of mass build retries this week because of what we suspect is a slow scheduler memory leak in zuul's master branch since 4.1.0, and when we get under memory pressure we wind up with zookeeper connection timeouts which result in running builds being retried
15:10:19 <gmann> dansmith not sure if that we should do or not, we can skip/or skip-on-failure volume test ?
15:10:42 <dansmith> gmann: well, I'd really rather see those get fixed of course
15:10:57 <gmann> yeah, fixing if good but if that is taking time
15:11:23 <dansmith> maybe jungleboyj knows how progress is going on the cinder side?
15:11:34 <fungi> we've been doing some live debugging of the memory leak by collecting object graphs from the running process, but it's tens of gigabytes of python structures to process and that tends to slow down the scheduler to the point where the debugging is introducing similar zk connection timeouts too
15:11:58 <fungi> hopefully we'll have it nailed down sometime today
15:12:49 <slaweq> gmann: hi, we already changed lists of irrelevant files for most of the jobs
15:12:57 <fungi> but if you see lots of build retries, or some builds ending in retry_limit results, that's probably why
15:13:03 <slaweq> and limited number of jobs run on check/gate queues
15:13:12 <gmann> slaweq: i see,
15:13:22 <slaweq> I think it may be due to some instability in our jobs so we had to recheck patches a lot
15:13:28 <slaweq> but we are working to improve that
15:13:34 <slaweq> I hope it will be better soon
15:13:42 <dansmith> slaweq: the numbers we're looking at are per check not related to total usage
15:13:49 <dansmith> slaweq: so these don't go up or down based on instability
15:13:50 <slaweq> ahh, ok
15:14:11 <dansmith> but, each recheck means re-using 34h of time.. so instability surely impacts the overall usage
15:15:06 <dansmith> gmann: anyway, that's all I had on the topic, will continue to monitor
15:15:11 <ricolin> sorry I'm late
15:15:46 <gmann> yeah, let's keep monitor those , thanks dansmith
15:15:46 <dansmith> things definitely suck less these days, but we're in a bit of a lull, so hard to really say I think
15:16:04 <gmann> true
15:16:06 <gmann> anything else on this topic ?
15:16:50 <gmann> let's move next then
15:16:52 <gmann> #topic PTL assignment for Xena cycle leaderless projects (gmann)
15:17:09 <gmann> we left with keystone only and patch is up for DPL model
15:17:15 <gmann> #link https://review.opendev.org/c/openstack/governance/+/784102
15:17:30 <gmann> please vote on this who has not voted yet
15:17:54 <mnaser> candidates look good
15:17:58 <mnaser> +1 from me
15:17:59 <gmann> thanks knikolla and gagehugo for stepping forward for the the help and liaisons.
15:18:01 <gmann> yeah
15:19:19 <gmann> let's move next
15:19:21 <gmann> #topic Election for one Vacant TC seat (gmann)
15:19:36 <ricolin> 5 votes on Keystone patch now:)
15:20:00 <gmann> I think we have one candidacy until now for TC vacant seat
15:20:06 <gmann> ricolin: yeah,
15:20:32 <ricolin> gmann, so no need to run election I assume?
15:20:44 <fungi> we still have a week remaining until the nomination deadline
15:20:46 <gmann> nomination open until 8th April so no action item until then
15:20:46 <mnaser> ^
15:20:50 <mnaser> yep
15:20:56 <ricolin> right
15:21:00 <gmann> yeah, let's wait until 8th april
15:21:06 <fungi> i plan to send another reminder later today to let folks know they've got a week left to propose candidacies
15:21:24 <gmann> fungi: +1, thanks
15:21:50 <spotz> I also let folks at RH know during a meeting yesterday
15:22:07 <fungi> thanks!
15:22:09 <fungi> that helps
15:23:02 <spotz> NP
15:23:09 <gmann> spotz: +1 thanks
15:23:24 <gmann> #topic Elect Vice Chair
15:23:45 <gmann> we have two members volunteer for vice chair
15:23:53 <gmann> #link https://review.opendev.org/c/openstack/governance/+/783409
15:23:59 <gmann> #link https://review.opendev.org/c/openstack/governance/+/783411
15:24:10 <gmann> ricolin: and yoctozepto .
15:24:19 <gmann> this is first time i think.
15:24:20 <yoctozepto> indeed
15:24:51 <gmann> and as per charter, this is internal role in TC so we do not need to have a official election or so
15:25:12 <gmann> as discussed in this week early, we decided to vote on gerrit itself and then we can choose one
15:25:22 <gmann> is that work for all?
15:25:40 <mnaser> i think i'll personally abstain from the vote
15:26:54 <gmann> yeah, that is also option so please vote by today end of the day and then I will merge one with highest positive vote
15:27:44 <gmann> but does this need to be formal-vote ? or merge directly
15:27:54 <gmann> it has been mixed way in past. mnaser ?
15:28:22 <mnaser> gmann: it's ultimately your decision as chair to decide who becomes chair, so you can just merge it anytime
15:28:34 <mnaser> the tc doesn't get to decide (you're giving us the luxury of doing that =))
15:29:00 <gmann> ok, sure.
15:29:17 <gmann> tc-members: please vote on both patch if you would like by end of today.
15:29:34 <ricolin> I think yoctozepto and myself will be fine to whatever gmann call for this, so let's fix this today and continue with rest works:)
15:29:37 <gmann> but good to see two members step up for help
15:29:45 <gmann> ricolin: yup
15:29:54 <yoctozepto> ricolin ++
15:30:04 <gmann> let's move to next topic
15:30:07 <gmann> #topic Community newsletter: "OpenStack project news" snippets
15:30:10 <ricolin> yoctozepto, :)
15:30:18 <gmann> #link https://etherpad.opendev.org/p/newsletter-openstack-news
15:30:50 <gmann> this is monthly (if i am not wrong) newsletter and as OpenStack community we need to add few highlights as requested by diablo_rojo
15:30:55 <diablo_rojo> The newsletter will go out next week
15:31:14 <gmann> I have added few, feel free to add more or in Future section if you have any
15:31:14 <diablo_rojo> I will draft from what is in the etherpad to our internal working doc early next week.
15:31:53 <gmann> diablo_rojo cool, do we have a consistent monthly date  for publication ?
15:32:10 <gmann> so that I can keep it in that week meeting agenda
15:32:52 * ricolin is putting DPL information now
15:33:18 <gmann> something like if it is published every 1st week of month then we can add in our meeting agenda in last week of every month or so
15:33:39 <gmann> ricolin: +1
15:35:52 <gmann> anyways I will check with diablo_rojo later on dates. so let's move next
15:35:57 <gmann> #topic Open Reviews
15:36:09 <gmann> #link https://review.opendev.org/q/project:openstack/governance+is:open
15:36:35 <gmann> there is no other open review which we have not discussed today.
15:37:03 <gmann> that's all from my today, any other topic to discuss or we can close early?
15:37:17 <mnaser> nothing to report from my side
15:37:25 <gmann> I will be on off from work tomorrow due to good friday which i think most of us.
15:38:32 <gmann> seems nothing else for today.
15:38:38 <gmann> Thanks everyone for joining. Have a good Friday and Easter everyone.
15:38:47 <gmann> #endmeeting