15:00:05 <gmann> #startmeeting tc 15:00:06 <mnaser> o/ 15:00:07 <openstack> Meeting started Thu Apr 1 15:00:05 2021 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:11 <openstack> The meeting name has been set to 'tc' 15:00:13 <gmann> #topic rollcall 15:00:16 <yoctozepto> \o/ 15:00:17 <dansmith> o/ 15:00:26 <gmann> o/ 15:00:28 <dansmith> sorry, I'm not as excited as yoctozepto 15:00:37 <gmann> :) 15:00:52 <belmoreira> o/ 15:01:55 <diablo_rojo> o/ 15:01:57 <gmann> let's start 15:02:07 <gmann> #topic Follow up on past action items 15:02:17 <gmann> no action items from previous meeting 15:02:32 <gmann> #topic PTG 15:02:43 <gmann> #link https://etherpad.opendev.org/p/tc-xena-ptg 15:03:07 <gmann> we have room booked now, thanks diablo_rojo for arranging new room for Thursday 15:03:29 <gmann> I have added the information in etherpad 15:04:02 <gmann> also I sent invitation to k8s steering committee also for cross community sessions. 15:04:17 <gmann> I have not got any reply yet so let's wait 15:04:46 <gmann> and keep adding the topics in etherpad you would like to discuss in PTG. 15:04:59 <mnaser> +1 15:05:04 <gmann> that's all on PTG planning form my side. 15:05:09 <diablo_rojo> gmann, no problem! 15:05:10 <gmann> anything else from any one? 15:05:40 <mnaser> nothing to add from me 15:06:04 <gmann> ok, let's move to next topic 15:06:08 <gmann> #topic Gate performance and heavy job configs (dansmith) 15:06:13 <dansmith> I have updated numbers! 15:06:18 <dansmith> https://termbin.com/n7yl 15:06:31 <dansmith> most every project has dropped their footprint, tripleo being the biggest 15:06:44 <gmann> nice 15:06:49 <dansmith> I think the nova +1 bump is likely just noise, as is the kolla -2, since node counts are unchanged 15:07:06 <dansmith> not much movement on the heaviest one, neutron, but some 15:07:13 <mnaser> wow neutron @ 34h sounds wild 15:07:23 <dansmith> I'm looking for clark to update me with the percent usage again and then will send something to the list 15:07:30 <mnaser> that's how many hours of testing each patchset gets right/ 15:07:35 <dansmith> mnaser: yeah they're still really big :/ 15:07:38 <dansmith> mnaser: yes 15:08:33 <dansmith> other than that, 15:08:42 <dansmith> I think the biggest thing hurting us is still spurious fails 15:09:00 <dansmith> I've got a patch up to fix the only known one in glance, but I think cinder is still the biggest offender, 15:09:16 <gmann> may be we can check with neutron team on optimizing the number of jobs runs for related file/code change? if not yet done 15:09:21 <dansmith> although my "seat of the pants" data has been less accurate since the release was cut and things have slowed 15:09:50 <dansmith> gmann: yeah, or just ask them to do less per patch. I think they already have a lot of per-file rules 15:10:16 <dansmith> we'll see what the usage percent values look like 15:10:18 <fungi> we've also had a spate of mass build retries this week because of what we suspect is a slow scheduler memory leak in zuul's master branch since 4.1.0, and when we get under memory pressure we wind up with zookeeper connection timeouts which result in running builds being retried 15:10:19 <gmann> dansmith not sure if that we should do or not, we can skip/or skip-on-failure volume test ? 15:10:42 <dansmith> gmann: well, I'd really rather see those get fixed of course 15:10:57 <gmann> yeah, fixing if good but if that is taking time 15:11:23 <dansmith> maybe jungleboyj knows how progress is going on the cinder side? 15:11:34 <fungi> we've been doing some live debugging of the memory leak by collecting object graphs from the running process, but it's tens of gigabytes of python structures to process and that tends to slow down the scheduler to the point where the debugging is introducing similar zk connection timeouts too 15:11:58 <fungi> hopefully we'll have it nailed down sometime today 15:12:49 <slaweq> gmann: hi, we already changed lists of irrelevant files for most of the jobs 15:12:57 <fungi> but if you see lots of build retries, or some builds ending in retry_limit results, that's probably why 15:13:03 <slaweq> and limited number of jobs run on check/gate queues 15:13:12 <gmann> slaweq: i see, 15:13:22 <slaweq> I think it may be due to some instability in our jobs so we had to recheck patches a lot 15:13:28 <slaweq> but we are working to improve that 15:13:34 <slaweq> I hope it will be better soon 15:13:42 <dansmith> slaweq: the numbers we're looking at are per check not related to total usage 15:13:49 <dansmith> slaweq: so these don't go up or down based on instability 15:13:50 <slaweq> ahh, ok 15:14:11 <dansmith> but, each recheck means re-using 34h of time.. so instability surely impacts the overall usage 15:15:06 <dansmith> gmann: anyway, that's all I had on the topic, will continue to monitor 15:15:11 <ricolin> sorry I'm late 15:15:46 <gmann> yeah, let's keep monitor those , thanks dansmith 15:15:46 <dansmith> things definitely suck less these days, but we're in a bit of a lull, so hard to really say I think 15:16:04 <gmann> true 15:16:06 <gmann> anything else on this topic ? 15:16:50 <gmann> let's move next then 15:16:52 <gmann> #topic PTL assignment for Xena cycle leaderless projects (gmann) 15:17:09 <gmann> we left with keystone only and patch is up for DPL model 15:17:15 <gmann> #link https://review.opendev.org/c/openstack/governance/+/784102 15:17:30 <gmann> please vote on this who has not voted yet 15:17:54 <mnaser> candidates look good 15:17:58 <mnaser> +1 from me 15:17:59 <gmann> thanks knikolla and gagehugo for stepping forward for the the help and liaisons. 15:18:01 <gmann> yeah 15:19:19 <gmann> let's move next 15:19:21 <gmann> #topic Election for one Vacant TC seat (gmann) 15:19:36 <ricolin> 5 votes on Keystone patch now:) 15:20:00 <gmann> I think we have one candidacy until now for TC vacant seat 15:20:06 <gmann> ricolin: yeah, 15:20:32 <ricolin> gmann, so no need to run election I assume? 15:20:44 <fungi> we still have a week remaining until the nomination deadline 15:20:46 <gmann> nomination open until 8th April so no action item until then 15:20:46 <mnaser> ^ 15:20:50 <mnaser> yep 15:20:56 <ricolin> right 15:21:00 <gmann> yeah, let's wait until 8th april 15:21:06 <fungi> i plan to send another reminder later today to let folks know they've got a week left to propose candidacies 15:21:24 <gmann> fungi: +1, thanks 15:21:50 <spotz> I also let folks at RH know during a meeting yesterday 15:22:07 <fungi> thanks! 15:22:09 <fungi> that helps 15:23:02 <spotz> NP 15:23:09 <gmann> spotz: +1 thanks 15:23:24 <gmann> #topic Elect Vice Chair 15:23:45 <gmann> we have two members volunteer for vice chair 15:23:53 <gmann> #link https://review.opendev.org/c/openstack/governance/+/783409 15:23:59 <gmann> #link https://review.opendev.org/c/openstack/governance/+/783411 15:24:10 <gmann> ricolin: and yoctozepto . 15:24:19 <gmann> this is first time i think. 15:24:20 <yoctozepto> indeed 15:24:51 <gmann> and as per charter, this is internal role in TC so we do not need to have a official election or so 15:25:12 <gmann> as discussed in this week early, we decided to vote on gerrit itself and then we can choose one 15:25:22 <gmann> is that work for all? 15:25:40 <mnaser> i think i'll personally abstain from the vote 15:26:54 <gmann> yeah, that is also option so please vote by today end of the day and then I will merge one with highest positive vote 15:27:44 <gmann> but does this need to be formal-vote ? or merge directly 15:27:54 <gmann> it has been mixed way in past. mnaser ? 15:28:22 <mnaser> gmann: it's ultimately your decision as chair to decide who becomes chair, so you can just merge it anytime 15:28:34 <mnaser> the tc doesn't get to decide (you're giving us the luxury of doing that =)) 15:29:00 <gmann> ok, sure. 15:29:17 <gmann> tc-members: please vote on both patch if you would like by end of today. 15:29:34 <ricolin> I think yoctozepto and myself will be fine to whatever gmann call for this, so let's fix this today and continue with rest works:) 15:29:37 <gmann> but good to see two members step up for help 15:29:45 <gmann> ricolin: yup 15:29:54 <yoctozepto> ricolin ++ 15:30:04 <gmann> let's move to next topic 15:30:07 <gmann> #topic Community newsletter: "OpenStack project news" snippets 15:30:10 <ricolin> yoctozepto, :) 15:30:18 <gmann> #link https://etherpad.opendev.org/p/newsletter-openstack-news 15:30:50 <gmann> this is monthly (if i am not wrong) newsletter and as OpenStack community we need to add few highlights as requested by diablo_rojo 15:30:55 <diablo_rojo> The newsletter will go out next week 15:31:14 <gmann> I have added few, feel free to add more or in Future section if you have any 15:31:14 <diablo_rojo> I will draft from what is in the etherpad to our internal working doc early next week. 15:31:53 <gmann> diablo_rojo cool, do we have a consistent monthly date for publication ? 15:32:10 <gmann> so that I can keep it in that week meeting agenda 15:32:52 * ricolin is putting DPL information now 15:33:18 <gmann> something like if it is published every 1st week of month then we can add in our meeting agenda in last week of every month or so 15:33:39 <gmann> ricolin: +1 15:35:52 <gmann> anyways I will check with diablo_rojo later on dates. so let's move next 15:35:57 <gmann> #topic Open Reviews 15:36:09 <gmann> #link https://review.opendev.org/q/project:openstack/governance+is:open 15:36:35 <gmann> there is no other open review which we have not discussed today. 15:37:03 <gmann> that's all from my today, any other topic to discuss or we can close early? 15:37:17 <mnaser> nothing to report from my side 15:37:25 <gmann> I will be on off from work tomorrow due to good friday which i think most of us. 15:38:32 <gmann> seems nothing else for today. 15:38:38 <gmann> Thanks everyone for joining. Have a good Friday and Easter everyone. 15:38:47 <gmann> #endmeeting