15:00:10 #startmeeting tc 15:00:10 Meeting started Thu Jul 15 15:00:10 2021 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:10 The meeting name has been set to 'tc' 15:00:24 #topic Roll call 15:00:24 o/ 15:00:33 o/ 15:00:49 hola 15:00:51 o/ 15:01:17 we have 3 members absent today. 15:01:18 yoctozepto on PTO 15:01:24 spotz on PTO 15:01:30 jungleboyj on PTO 15:02:21 let's start 15:02:26 #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda_Suggestions 15:02:35 ^^ today agenda 15:02:50 #topic Follow up on past action items 15:02:59 two AI from last meeting 15:03:00 clarkb to convey the ELK service shutdown deadline on ML 15:03:15 clarkb send it to ML #link http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023578.html 15:03:28 gmann to send ML to fix warning and oslo side changes to convert them to error 15:03:31 #link http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023646.html 15:04:24 gibi mentioned about sqlAlchemy warning also which need keystone fix to merge to get oslo.db 10.0.0 in g-r 15:04:29 O/ 15:04:41 #link https://review.opendev.org/c/openstack/keystone/+/799672 15:04:53 seems less active member in kesytone. 15:05:03 knikolla: ^^ if you see this msg 15:05:53 stephen already pinged keystone team on keystone channel so let's see if we can merge it soon 15:06:02 #topic Gate health check (dansmith/yoctozepto) 15:06:17 dansmith: any update you would like to share? 15:06:17 gate has seemed fairly good to me lately, hard to complain much 15:07:15 also check-arm64 was blocked last week, but back to normal now 15:07:20 one issue i am aware of and is fixed now. tempest-full-py3 was broken on ussuri due to python3 disable via base job 15:07:29 oh, 15:07:29 +1 15:07:30 (tempest-slow-py3) 15:07:39 yeah tempest-slow-py3 15:07:46 not really gate, but is the depends-on is still broken? 15:08:05 dansmith: that is fixed now 15:08:10 i don't believe so, i saw the message about it mention an immediate revert 15:08:11 okay cool 15:08:15 worked for tempest-slow-py3 fix testing 15:08:41 yes as soon as we identified the issue we pushed and landed a revert of the change that broke depends-on. Then restarted as soon as that had applied to the servers 15:08:44 don't believe it to still be broken, i meant 15:08:56 clarkb: +1 15:09:02 okay I thought it was broken for a while 15:09:27 dansmith: from Sunday evening to about Tuesday Noonish relative to our timezone 15:09:37 ah okay 15:10:06 anyway, nothing else gate-ish from me 15:10:17 ok, let's move next then 15:10:30 #topic Migration from 'Freenode' to 'OFTC' (gmann) 15:10:59 while doing this for deprecation repo, i found few repo not deprecated or retired properly. also some setup in project-config side need update 15:11:06 project-config side things are merged 15:11:50 for retired repo, I am leaving the OFTC ref update because 1. they are many repo 2. need to add setup in project-config to get it updated to github repo 15:12:06 if anyone has time I will not object. 15:12:49 #topic PTG Planning 15:13:05 Doodle poll for slot selection 15:13:25 please vote your availability/preference 15:14:07 Will do today. 15:14:13 thanks 15:14:22 ricolin: jungleboyj you too 15:14:36 we need to book slot by 21st July 15:15:08 I thought I already vote, but will check again 15:15:20 also I sent doodle poll for TC+PTL interaction session which is 2 hrs either on Monday or Tuesday 15:15:22 #link https://doodle.com/poll/ua72h8aip4srsy8s 15:15:33 ricolin: i think you voted on TC+PTL sessions not on TC PTG 15:15:36 please check 15:15:46 you're right 15:15:51 will vote right now 15:15:59 too many doodles 15:16:01 ricolin: thanks 15:16:44 for TC sessions, I am thinking to book slot for two days and 4 hrs each day ? 15:16:58 that should be enough? what you all say ? 15:17:24 I'll have a hard time making all of that, as usual, but sure 15:17:32 done 15:18:21 gmann, I think that's good enough 15:18:30 k 15:18:33 and this is etherpad to collect the topic #link https://etherpad.opendev.org/p/tc-yoga-ptg 15:18:56 please start adding the topic you would like to discuss 15:19:28 anything else on PTG thing? 15:19:31 I assume we also want to coordinate with the k8s folks for some time? 15:20:15 diablo_rojo_phone, +1 15:20:56 sure, last time k8s folks did not join but we are always fine if they would like to. we can have 1 hr slot for that if ok for them 15:21:04 Something to keep in mind. 15:21:34 Yeah I think with more heads up and if we dictate a time to them and put an ical on their ml we should get more engagement. 15:22:12 sure, i did last time on ML also. I can do this time too. 15:22:27 IMO we like to include that, maybe we need more than 8 hours(4 a day) 15:22:45 ricolin: time slot is not issue I think. 15:22:50 s/we like/if we like/ 15:22:55 gmann, Okay 15:22:59 but yes we can extend if needed 15:24:07 added this in etherpad 15:24:09 anything else ? 15:24:33 #topic ELK services plan and help status 15:24:36 Help status 15:24:59 I think there is no help yet. clarkb fungi anything you heard from anyone ? 15:25:09 I have not 15:25:21 k 15:25:22 Reducing the size of the existing system 15:25:29 clarkb: ^^ go ahead 15:25:31 Since increasing the log workers to 60% of our total it is keeping up much better than before 15:26:24 On the Elasticsearch cluster side of things we are using ~2.4TB of disk right now. We have 6TB total but only 5TB is usuable. The reason for this is we have 6 nodes with 1TB each and we are resilient to a single node failure which means we need to fit within the disk available on 5 instances 15:26:55 Given that the current disk usage is 2.4TB or so we can probably reduce the cluster size to 5 nodes. Then we would have 5TB total and 4TB useable. 15:27:15 If we reduce to 4 nodes then we get 3TB usable and I think that is too close for comfort 15:27:58 One thing to keep in mind is that growing the system again if we shrink and it falls over is likely to be difficult. For this reason I think we can take our time. Keep monitoring usage patterns a bit before we commit to anything 15:28:23 But based on the numbers available today I would say we should shrink the log workers to 60% of their total size now and reduce the elasticsearch cluster size by one instance 15:28:38 2.4 TB is usual usage like during peak time of release etc or just current one ? 15:29:12 gmann: just during the current usage. Its hard to look at numbers historically like that because cacti doesn't give us great resolution. But maybe fungi or corvus have tricks to find that data more ccurately 15:29:31 'shrink the log workers to 60% of their total size' - shrink or increase ? 15:30:00 initially you mentioned increasing 15:30:10 gmann: shrink. We have 20 instances now. I have disabled the processes on 8 of them and we seem to be keeping up. That means we can shrink to 60% I think 15:30:29 gmann: last week I had it set to 50% and we were not keeping up so I increased that to 60% but that is still a shrink compared to 100% 15:30:39 ohk got it 15:31:00 i though 60% more from what we had:) 15:31:15 8thought 15:31:31 I think this is reasonable proposal. 15:31:41 Anyway that is what the current data says we can do. Lets watch it a bit more and see if more data changes anything 15:31:54 But if this stays consistent we can probably go ahead and make those changes more permanent 15:32:45 clarkb: is it fine to monitor until Xena release ? 15:32:56 or you think we should decide early than that ? 15:33:36 its probably ok to monitor until then. Particularly during feature freeze as that is when demand tends to be highest 15:33:45 yeah 15:35:02 that was all I had. We can watch it and if those numbers hold up make the changes after the xena release (or maybe after feature freeze) 15:35:20 +1 sounds perfect 15:35:56 clarkb: anything else you would like to keep discussing on this in TC meeting or is it fine to remove it from agenda for now and re-add during Xena release ? 15:36:25 Should be fine to remove for now 15:36:31 ok 15:36:48 thanks a lot clarkb for reporting on data and help on this. 15:37:14 #topic Open Reviews 15:37:17 #link https://review.opendev.org/q/projects:openstack/governance+is:open 15:37:34 many open reviews, let's check them quickly and vote accordingly 15:38:11 will do 15:38:15 tc-members please vote on the Yoga testing runtime #link https://review.opendev.org/c/openstack/governance/+/799927 15:38:39 which is same as what we had in Xena 15:38:58 centos-stream9 can be added later once that is released 15:39:21 gmann: are you planning to support both 8 and 9? 15:39:37 my selfish preference is that you pick only the one (as it allows us to delete images more quickly) 15:39:38 clarkb: no, one, means updating 8->9 15:39:41 got it 15:40:56 need one more vote on this project-update #link https://review.opendev.org/c/openstack/governance/+/799826 15:42:12 others are either having enough required vote or waiting for depends-on /zuul fix. 15:42:14 voted 15:42:17 thanks 15:42:31 ricolin: this quick one for governance-sig https://review.opendev.org/c/openstack/governance-sigs/+/800135 15:42:47 anything else we need to discuss for today meeting? 15:42:54 done 15:43:00 thanks 15:43:03 yes 15:43:33 one thing, sorry for the delay but I sended the ML for collect pain point out http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023659.html 15:44:08 +1 15:44:09 for the eliminate pain point idea 15:44:28 Let's see if we can get valid pain point feedback from teams 15:45:07 thanks ricolin for doing that. 15:45:17 NP, will keep tracking 15:45:25 sure 15:45:52 anything else? 15:46:20 thanks all for joining, let's close meeting 15:46:23 #endmeeting