18:00:07 <JayF> #startmeeting tc
18:00:07 <opendevmeet> Meeting started Tue Nov 28 18:00:07 2023 UTC and is due to finish in 60 minutes.  The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:07 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:07 <opendevmeet> The meeting name has been set to 'tc'
18:00:11 <JayF> #topic Roll Call
18:00:13 <JayF> o/
18:00:15 <gmann> o/
18:00:19 <frickler> \o
18:00:24 <JayF> Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.
18:00:27 <JayF> Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee.
18:00:32 <JayF> There are no noted absenses in the agenda for today's meeting.
18:00:33 <slaweq> o/
18:00:51 <jamespage> o/
18:00:51 <rosmaita> o/
18:00:55 <knikolla> o/
18:00:58 <JayF> Going to wait until :05 or until all TC members have checked in
18:01:00 <dansmith> o/
18:04:45 <JayF> Aight, going to get started. We have 8 that is quorum.
18:04:52 <JayF> #topic Follow up on tracked action items
18:05:02 <spotz[m]> o/
18:05:08 <rosmaita> 9!
18:06:31 <JayF> ping?
18:06:41 <JayF> okay, sorry, wifi network went down locally, but I appear to still have wired connectivity
18:07:19 <JayF> #info Rosmaita to propose amendment to unmaintained branch resolution allowing a single review group
18:07:24 <JayF> #info Rosmaita to email mailing list about progress implementing unmaintained branch resolution
18:07:32 <JayF> rosmaita: I grouped these together since they are related; do you have an update?
18:07:41 * rosmaita actually did all his action items for once
18:07:56 <rosmaita> #link https://review.opendev.org/c/openstack/governance/+/900940
18:08:04 <rosmaita> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/ZYAZG43BLJJVXCYZVPYQX5733BYDVVNL/
18:08:16 <rosmaita> so, no response to the email, so i guess everyone is on board
18:08:17 <knikolla> #success rosmaita did all his action items
18:08:20 <opendevstatus> knikolla: Added success to Success page (https://wiki.openstack.org/wiki/Successes)
18:08:27 <rosmaita> :)
18:08:54 <JayF> That governance change appears to have wide consensus; please make time to review it if you haven't and I will take a pass at landing eligible governance patches this afternoon.
18:08:54 <rosmaita> the resolution has achieved ripeness, and i think has enough votes to pass
18:09:12 <gmann> amendment to resolution looks good to me, thanks rosmaita for preparing that
18:09:13 <dansmith> jsut added one more
18:09:21 <rosmaita> \o/
18:09:45 <JayF> Is there anything further we should mention about unmaintained branches before moving on? I don't believe we have another agenda item for it.
18:09:48 <jamespage> ditto (and I managed to actually hit the right option this time)
18:09:58 <JayF> What is our next step, assuming this governance change lands in the next hour or two
18:10:17 <rosmaita> i think just to help the release team get this implemented
18:10:42 <JayF> Is someone on TC doing or helping coordinate that work?
18:10:46 <rosmaita> and i guess we need opendev infra team to set up the meta-acl for gerrit
18:11:15 <frickler> that should be a patch to project-config
18:11:16 <fungi> yes, if you can get me details on what you need i can propose a patch
18:11:35 <rosmaita> great
18:11:37 <JayF> Instead of hashing those details out here, can someone take the action to work with fungi on this?
18:11:39 <fungi> it'll just be an edit to the existing gerrit/acls/openstack/meta.config acl file
18:11:40 <JayF> And we can move on?
18:12:03 <rosmaita> #action rosmaita work with fungi on the gerrit meta-acls
18:12:13 <JayF> #undo
18:12:13 <opendevmeet> Removing item from minutes: #action rosmaita work with fungi on the gerrit meta-acls
18:12:16 <JayF> #redo
18:12:24 <JayF> #action rosmaita work with fungi on the gerrit meta-acls
18:12:34 <JayF> I didn't think anyone but the person who started the meeting could do that :)
18:12:43 <rosmaita> :D
18:12:44 <JayF> Thanks for that
18:12:50 <JayF> #topic Gate Health Check
18:12:53 <JayF> How is the gate?
18:13:21 <JayF> Things have been mostly calm on the Ironic side. Some failures that we tracked down to real issues in one of our libraries, contained to Ironic-related projects.
18:13:49 <gmann> one failure there in novaclient fucntional job with neutron new rbac enabled
18:14:19 <dansmith> really not great on the nova side
18:14:24 <gmann> we have enabled neutron rbac by default in devstack and that broke novaclient job, not sure why but it is disabled in that job and worked around
18:14:30 <dansmith> I'm at 14 rechecks for a patch that has been approved for weeks
18:15:16 <dansmith> many of the failures are related to volume tests, per usual, but some other instabilities have also crept up, but no giant smoking guns that I've seen
18:16:28 <JayF> Is there anything at a TC level we can or should do? I would say can we raise awareness but if it's bad enough to need >a dozen rechecks; I imagine there's decent awareness.
18:16:50 <dansmith> well, tbh, I think most people are happy to recheck 20 times to get their patches in
18:16:51 <gmann> I am sure those are not blind recheck
18:16:55 <dansmith> so awareness is maybe not enough
18:17:32 <dansmith> gmann: mine aren't for sure, except for yesterday when zuul wasn't loading log results, so it was hard to examine for root cause
18:17:34 <gmann> i think we know what failing like volume tests and need someone to deep dive into those
18:17:52 <gmann> dansmith: yeah
18:17:54 <dansmith> agree.. before last week, the cinder grenade phase was failing for me a *lot*
18:18:11 <dansmith> haven't seen that yet this week, but the resize vol-backed test has bitten me a few times
18:18:48 <frickler> there was an issue with cinder, grenade and tooz, I added that as dedicated agenda item
18:18:51 <rosmaita> the grenade seems to fail after it creates the third volume and then tries to ssh into the server
18:19:08 <rosmaita> at least that's what i've seen a few times
18:19:23 <rosmaita> but i don't know what's happening to the server
18:19:33 <dansmith> rosmaita: okay I haven't seen that, it's usually waiting for the volume to become in-use, but that was 2+ weeks ago
18:19:43 <dansmith> it'
18:19:54 <dansmith> it's meaning "what I was seeing"
18:20:16 <dansmith> anyway, agree with gmann that some deep dive is needed for sure
18:20:22 <rosmaita> well, there is something bad happening during that phase
18:20:37 <dansmith> I'm pretty burned out on that stuff myself, as I spent months on it after I came back from holiday break this jan, when things were really bad
18:20:42 <JayF> Unless folks in the TC want to dig this personally, I'd suggest putting some of these research results out onto the list and trying to recruit additional help.
18:20:47 <dansmith> so I hope someone else can really jump in while I'm out
18:21:05 <JayF> We've gotta try to increase the number of people engaged with these kind of problems, and the only way I know to try and do that is to increase communication around them.
18:21:47 <dansmith> I'm worried that the pain level has to be higher before people will really care, but .. yeah
18:22:06 <gmann> true
18:22:21 <JayF> I agree, based on the results of the similar approach I took with eventlet, but ensuring folks know where status is IMO is part of what we're supposed to do
18:22:33 <JayF> we can only do the work of 9 people, and there's a heck of a lot more to do to keep things going than that
18:22:41 <dansmith> that's not true
18:22:49 <dansmith> I know for a fact that gmann can do the work of three people :P
18:23:09 <gmann> you should know that I am getting old :)
18:23:09 <JayF> :) moving on
18:23:13 <JayF> #topic Leaderless projects
18:23:20 <JayF> Well, one of those three gmann's gets to lead this topic :D
18:23:34 <JayF> #link https://etherpad.opendev.org/p/2024.1-leaderless
18:23:39 <gmann> we have two project PTL appointment proposed for ling time and we should take decision soon i will say in this meeting
18:23:49 <gmann> sahara: #link https://review.opendev.org/c/openstack/governance/+/899782
18:24:09 <gmann> sahara is marked as inactive but PTL appointment and give them a chance to make it active does not hurt
18:24:18 <gmann> 2nd is rally: #link https://review.opendev.org/c/openstack/governance/+/898228
18:24:51 <gmann> it has voting in favor and not in favor too but we should decide
18:25:39 <gmann> I think they are open in gerrit for enough time, I would like to request tc-members who have not voted to do that
18:25:51 <JayF> Yes, please do, especially since one of those does have a negative vote
18:26:04 <JayF> I'd really rather not land something with a plurality if it has a negative vote
18:26:05 <gmann> other 3 project remaining for PTL appointment is no progress, I will send email today to their previous PTL or any active maintained i get to know
18:26:47 <gmann> JayF: I will say let's get the remaining vote today and we should either merge them or abandon them this week max
18:27:19 <JayF> Yep, that's extremely reasonable, and abandoning isn't really a choice unless a project without a PTL is an OK state
18:27:37 <JayF> I don't think "we take no action" is a good state here; we need to land these *or* take some other action to resolve the project's lack of leadership
18:27:49 <gmann> but better than not deciding about it and PTL volunteer just wait for us to say yes or not to appointment
18:28:12 <spotz[m]> Having password issues but will vote if I haven’t
18:28:15 <gmann> especially there is no other candidate or even maitainers
18:28:16 <JayF> Yeah, you're right, and I appreciate you pointing it out and will take that advice -- I'll land these if eligible to land by EOW, even if only a plurality vote to land if still eligible
18:28:28 <gmann> thanks.
18:28:38 <JayF> Is there anything else on leaderless projects before wee move on?
18:28:41 <gmann> that is all form my side on this.
18:29:03 <JayF> Next topic was Implementation of Unmaintained branch statuses -- but I think we covered that during action items
18:29:06 <JayF> so I'm going to skip it
18:29:11 <JayF> #topic 2024.1 TC Tracker
18:29:17 <JayF> #link https://etherpad.opendev.org/p/tc-2024.1-tracker
18:29:27 <JayF> If there are any updated on TC Tracker items; please give them now.
18:30:50 <JayF> OK, moving on.
18:31:00 <JayF> #topic Upgrade issue between tooz and cinder
18:31:06 <JayF> #link https://review.opendev.org/c/openstack/requirements/+/901131
18:31:11 <JayF> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/L72QU3SR2VFVYOFXVYH74V7HGMQ3YJRU/
18:31:29 <JayF> Unsure who added this to the agenda; but there's clearly an issue around Tooz upgrades that broke things, it got reverted but there are still issues here to resolve I believe
18:31:58 <JayF> frickler: ^ looks like this was your topic
18:32:25 <frickler> yes, that was me, I wanted to check whether we need to discuss some general guidance regarding interactions between these projects
18:33:03 <clarkb> naively it seems like tooz could sort out what version of etcd it is talking to then use the appropriate api paths
18:33:47 <fungi> i.e. support multiple versions of etcd in tooz (noting that coinstalling services that needed those different versions would be challenging still)
18:34:50 <frickler> yes, that's what https://review.opendev.org/q/topic:%22bug/2043810%22 does
18:35:19 <JayF> I'm looking some stuff up, Ironic had similar failures and fixed it with devstack configuration
18:35:34 <JayF> mainly because I want to ensure however it gets fixed that we all align in the same direction
18:36:01 <frickler> having grenade upgrade also etcd would be another option
18:37:12 <frickler> but also I added the topic before those tooz patches were proposed, so maybe all is well for now. and the next topic is even more interesting ;)
18:37:59 <JayF> Hmm https://review.opendev.org/c/openstack/networking-generic-switch/+/895973 is the change Ironic landed
18:38:07 <JayF> I think it's in the same realm but not the same mechanism
18:38:35 <JayF> frickler: it sounds like there's not much else to talk abuot here though, things got moving over the holiday?
18:38:42 <frickler> ack
18:38:56 <JayF> #topic Declare projects inactive for lack of ...?
18:39:15 <JayF> #info (list from agenda) Working py311 unit tests, Support for sqla2, Broken CI since 2023.2 release, Do we want/need to look at specific deliverables or only whole projects?
18:39:38 <frickler> also added by me
18:39:54 <dansmith> definitely not sqla2 yet, IMHO
18:40:03 <frickler> my main concern is what we can do to get sqla2 into u-c this cycle
18:40:11 <dansmith> the other two seem reasonable
18:41:09 <frickler> if we have projects that do not support sqla2, but depend on global upper-constraints, what other options do we have?
18:41:12 <gmann> gate broken even due to py 3.11 or any other reason is reasonable way to mark them inactive
18:41:46 <dansmith> frickler: because it's a lot of work for some projects and 2024.1 won't even be supported on a distro that has 2.0 AFAIK
18:42:10 <gmann> I think we need to go with community wide goal for sqla2 work and maybe multicycle effort.
18:42:10 <dansmith> it's fine to have it be a requirement or goal or whatever, but marking a project as inactive because it hasn't crossed that line yet is way too large of a hammer, IMHO
18:42:15 <dansmith> yeah
18:42:26 <JayF> While I mostly agree with frickler that we have to draw a line at some point, and it's getting to be that time, it seems late in the cycle to draw that line TBH.
18:42:34 <gmann> agree, I think we discussed the same in gerrit in some change
18:42:42 <JayF> and I suspect if we apply the other criteria, we'll catch clearly inactive projects without catching any that might just be late with SQLA 2.0 migrations
18:42:54 <gmann> marking project inactive on this is little too strong and inconsistent in our process
18:42:55 <dansmith> yeah
18:43:06 <JayF> I'm thinking of this like a graduated filter; sqla2.0 is a more fine filter
18:43:10 <JayF> lets use the larger, coarser filters first
18:43:18 <JayF> which more clearly say "these should not be in 2024.1"
18:43:36 <JayF> and perhaps decide (not today; but this cycle) that next cycle is where we draw the SQLA 2.0 line
18:43:43 <clarkb> (it is R-18 according to the schedule)
18:43:48 <dansmith> sqla2.0 is a point in time hurdle.. it's a filter right now, but won't be later, where CI brokenness and py$current unit tests are more of a reasonable timeless filter, IMHO
18:44:55 <frickler> o.k., so I will propose some inactivity marks based on the latter
18:45:02 * dansmith notes he's going to have to do his 15th recheck on that patch
18:45:15 <frickler> and then I'll add sqla2 planning as a new topic
18:45:39 <gmann> ++
18:45:48 <JayF> frickler++ Thank you for that, I put up (and abandoned) a governance change in that direction last cycle, it may be useful as a starting point or at least to review preexisting discussion
18:46:03 <frickler> (I'm also not sure that it'll be sqla2.0 still ;)
18:46:14 <JayF> Anything else on project inactivity before we move on?
18:47:09 <JayF> #topic Open Discussion and Review
18:47:11 <JayF> #link  https://review.opendev.org/q/status:open+repo:openstack/governance
18:47:26 <JayF> as mentioned earlier; please prioritize governance reviews; I will be making a pass in a couple of hours to land things eligible to land
18:47:33 <JayF> and it's nice if a majority of us have voted on those changes
18:47:46 <JayF> dansmith: I think you had something for open discussion
18:48:19 <dansmith> Just FYI to all, I'm out for the rest of the year starting tomorrow
18:48:58 <slaweq> dansmith: have a great holiday time :)
18:49:00 <JayF> Enjoy your holiday!
18:49:15 <JayF> Are there any other items for open discussion?
18:49:15 <dansmith> I shall endeavor to do so, thanks
18:49:23 <gmann> Enjoy! and forget about gate :)
18:49:30 <spotz[m]> Have a good break
18:49:38 <dansmith> gmann: trust me, I shan't be thinking about the gate until Jan :)
18:49:41 <fungi> just don't break the gate!
18:49:47 <gmann> dansmith: ++
18:50:23 <JayF> Last call for items for open discussion
18:51:21 <JayF> Thanks for coming to the TC meeting; see you all in 1 week for our monthly video meeting o/
18:51:23 <JayF> #endmeeting