17:00:46 <gouthamr> #startmeeting tc
17:00:46 <opendevmeet> Meeting started Tue Sep  9 17:00:46 2025 UTC and is due to finish in 60 minutes.  The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:46 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:46 <opendevmeet> The meeting name has been set to 'tc'
17:01:03 <gouthamr> Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.
17:01:06 <gouthamr> Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee
17:01:11 <gouthamr> #topic Roll Call
17:01:16 <mnasiadka> o/
17:01:25 <seunghunlee> o/
17:01:25 <frickler> \o
17:01:39 <spotz[m]> o/
17:01:46 <cardoe> o/
17:01:47 <noonedeadpunk> o/
17:02:09 <spotz[m]> jonathanspw meeting!
17:02:18 <JonathanWright[m]> o/
17:02:30 <spotz[m]> :)
17:03:12 <gouthamr> noted absence: g t e m a, b a u z a s
17:03:59 <gouthamr> courtesy ping: gmaan,  jbernard
17:04:49 <frickler> gmaan is on pto afaict
17:05:05 <spotz[m]> Yeah he is
17:05:13 <gouthamr> ah true, i was wondering if he meant to join
17:05:22 <gouthamr> lets get started
17:05:28 <gouthamr> #topic Last week's AIs
17:06:07 <gouthamr> update on leaderless projects (Vitrage, OpenStack Charms, Venus, Monasca)
17:06:42 <gouthamr> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/WR277VO5VNJP2Z5WPZ7RPWRSW2Y7F7CT/ (charms)
17:06:42 <gouthamr> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/22GL7TB54WY4T2JDEVKFQCEKQY4AAVAN/ (vitrage)
17:06:42 <gouthamr> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/G4FSLQDGJBRY6G4JQ3SSIPN2UVX7BC4K/ (venus)
17:07:09 <gouthamr> ^ i sent these out under 24 hours ago, so we'll give folk a week to respond
17:07:26 <gouthamr> #link https://etherpad.opendev.org/p/2026.1-leaderless
17:08:09 <gouthamr> the next AI we took was around - reminding people to opt-in to CIVS emails
17:08:47 <gouthamr> ianychoi did this a few days ago, you can see the chat in #openstack-election for details/brainstorming that went on..
17:09:07 <gouthamr> hopefully this translates to more people exercising their vote
17:11:01 <gouthamr> a related AI was around the reduced electorate - which we suspected to be due to OIF membership renewal requirements
17:11:33 <ianychoi> Note: the voters actually cast so far are: 66 out of 71 for TC and 9 out of 21 for Horizon election
17:11:38 <gouthamr> i think this is worth surfacing to the foundation-board email list post the elections
17:11:40 <ianychoi> s/71/171
17:11:54 <gouthamr> woah, i celebrated for a second! thanks for the update ianychoi
17:11:55 <spotz[m]> I was going to say great turn out
17:12:46 <gouthamr> the number of people that voted has exceeded 2025.1, which is good! https://civs1.civs.us/cgi-bin/results.pl?id=E_079cc0f0dda5010c
17:12:54 <noonedeadpunk> 71 for TC vote is not a thing to celebrate...
17:13:29 <fungi> still a week to go, i haven't cast my ballot yet ;)
17:13:34 <ianychoi> Note: the voters actually cast so far are: 66 out of 171 for TC and 9 out of 21 for Horizon election
17:13:44 <spotz[m]> Weel 66 out of 71 is, just not 66 out of 171
17:13:47 <ianychoi> noonedeadpunk: it was my typo - 171, not 71
17:13:57 <noonedeadpunk> yeah, I got that :)
17:13:58 <gouthamr> sorry, i'd like to be positive around this.. i've seen this slowly/steadily improve over the past few cycles
17:15:32 <fungi> one thing i don't think anyone has analyzed yet is what percentage of contributors were openinfr foundation individual members for the last election vs what percentage of contributors are oif individual members for this election
17:16:11 <gouthamr> yes, i'd like to pursue that and related analyses after the polls have closed
17:16:16 <fungi> a drop in electorate size can be attributed to fewer contributors people reestablishing their membership, or can be attributed to a drop in contributor count
17:16:44 <ianychoi> I agree with gouthamr. And I believe OIF renew membership impacted on this election - from data: There are 171 items using "yq 'to_entries | map(select(.value.member)) | length' _all_owners.yaml", and 486 items using "yq 'to_entries | map(select(.value.member == null)) | length' _all_owners.yaml"
17:17:03 <spotz[m]> Very valid point, I would also go back to the last election around this time to be more accurate vs the other cycle of the year
17:17:08 <fungi> my guess is it's a mix of the two, hard to know without closer review which is predominately responsible
17:17:24 <gouthamr> and loop folks from the foundation as we discussed earlier to boost renewals, and actually educating folks what it means to be a foundation individual member vs a community member, etc
17:18:41 <ianychoi> Also, not sure at moment to dig into now, but might it worth if we go for election although there is just 1 candidate? One Horizon electorate reached me via e-mail and I was personally surprised that she/he did not recognize the existence of election, since I believe there were no elections for Horizon for recent several cycles
17:18:48 <fungi> ianychoi: keep in mind that in recent years less than 50% of contributors signed up to be individual members anyway
17:19:45 <ianychoi> (Actually, her/his electorate was invalid due to late renew to OIF)
17:20:09 <gouthamr> ianychoi: ah! these two concerns were from the same contributor
17:20:30 <gouthamr> apologies for not having responded there so far, but i wanted to consult with folks that have been here longer than i have
17:21:19 <gouthamr> the problem they pointed out was that the electoral rolls were compiled at a time when they were a "community" member, but, they converted to a "foundation individual" member soon after
17:21:47 <gouthamr> they were suggesting that our deadlines weren't clear as to when someone has to be a foundation individual member to qualify for a ballot
17:21:50 <ianychoi> Thank you for detailed explanation on the situation gouthamr
17:21:59 <fungi> i guess the suggestion is to run 40+ polls for a single ptl in each project regardless of whether they have multiple candidates?
17:22:14 <fungi> in order to make it clear there are the possibility for ptl elections?
17:22:28 <spotz[m]> Back when I joined everyone had to be a Foundation member, when Community only became a thing we were very specific that voting was the one thing they couldn't do in OUI presentations
17:22:30 <gouthamr> there's a contribution period that's well defined, but, we haven't stated if we will check for foundation individual member at the same time
17:23:25 <fungi> the check for foundation membership happens at the time when the electorate is built, essentially, so after the nomination period but before the end of campaigning
17:24:01 <gouthamr> ack, its implied in the same deadline - but they contended that folks could change their membership with a click of a button
17:24:03 <ianychoi> I think anyone can change anytime between Foundation and Community membership
17:24:05 <fungi> or in the case of nominations, membership for each nominee is checked at the time their candidacy is proposed
17:25:22 <spotz[m]> But you can then fix and re-propose yourself, I think I've seen that in the past
17:25:32 <gouthamr> (ianychoi completed the rest of the concern)
17:25:53 <gouthamr> spotz[m]: yes, this happened quite a bit during the nomination period..
17:26:04 <clarkb> in my local government elections when casting the ballot I have to swear under penalty of perjury that I am still a valid elector
17:26:14 <gouthamr> but, electors usually react late and can't retroactively fix things
17:26:14 <clarkb> you're always going to have a race condition here.
17:27:14 <gouthamr> yes, some of it has to be based on trust/honor.. i'd want to commend the person for pursuing this.. i think they want clarity in the deadlines/docs so they/others don't get caught up on this
17:27:53 <gouthamr> its different for candidates - they get feedback on their nomination proposals and can make membership changes to get their nominations accepted
17:29:00 <gouthamr> electors cannot vote in elections if they  fat-fingered something during the OIF membership renewal/making routine profile updates
17:29:11 <frickler> is there a legal reason to require OIF membership for voting?
17:29:23 <spotz[m]> I believe so
17:29:23 <fungi> when we first dropped oif individual membership checks from the cla signing in gerrit, i recall including reminders in election communications for how potential voters can check their member status
17:29:38 <gouthamr> yes, our charter explicitly states it - and that language came from the erstwhile foundation bylaws i think
17:31:19 <fungi> while i don't know for sure, i expect the reason is that it provides a legal assertion that the voter is represented by a specific, non-duplicated individual
17:31:51 <frickler> well nobody really can verify that for OIF memberships, either?
17:31:56 <fungi> so that there is ostensible legal recourse if someone merges changes under a bunch of different gerrit accounts and uses them to multi-vote
17:33:43 <fungi> the foundation has (or at least had when not part of the lf) a legal responsibility for the integrity of its own board elections and ballot measures, which rested on at least some minimal legal strength for its individual member agreement
17:34:55 <fungi> tc elections were originally described by the foundation's own bylaws, but even once separated were able to piggy-back on some of that assurance
17:35:51 <gouthamr> +1
17:35:52 <fungi> if the tc wants to pursue removing the requirement for electors and/or candidates to be oif individual members, i can ask about having the foundation consult their counsel on our behalf
17:36:27 <gouthamr> this is an area to explore if we _need to_ - not the time for that discussion atm imho; lets take this to the PTG perhaps
17:36:27 <spotz[m]> I think it has value but then I've always been a member:)
17:37:04 <spotz[m]> Good call on PTG
17:37:13 <gouthamr> lets do the election retro, and start a ML thread with foundation folks, and discuss this further
17:37:33 <ianychoi> +1 :)
17:37:56 <gouthamr> does any one else have any other AI to highlight from the past week/s
17:38:15 <spotz[m]> I brought someone from Alma as requested:)
17:38:44 <gouthamr> ++ thanks, its the topic right after the mysql changes one
17:39:04 * gouthamr notes that he pinged jbernard regarding the TC resolution for phone-home from OpenStack
17:39:10 <JonathanWright[m]> spotz[m]: Hiiii
17:39:11 <gouthamr> he plans to follow up post RC
17:39:27 <gouthamr> #topic Default MySQL/MariaDB charset/collations changes
17:39:32 <gouthamr> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/VR7XUI7V7QGFNHKFOXLFJJP6P4GGKOM3/
17:40:05 <gouthamr> seunghunlee: thanks for raising this topic on the ML
17:40:58 <seunghunlee> Thanks for bring this to today's topic
17:41:15 <noonedeadpunk> ++
17:41:27 <noonedeadpunk> yeah, sorry, I promised to write ML but -ENOTIME
17:41:37 <noonedeadpunk> this was a really good summary
17:42:20 <seunghunlee> thanks
17:43:10 <seunghunlee> So, what does TC think on this matter? how should we approach?
17:43:17 <gouthamr> i think some more discussion happened here wrt Ironic specifically
17:43:40 <noonedeadpunk> frankly, I'd love to have a community goal regaridg change of default charset to mb4
17:43:58 <noonedeadpunk> that feels like most of folks are in alignment with atm
17:44:12 <clarkb> yes basically the assertion is that ironic cannot do mb3 with some hand wavyness about that. I'm fairly confident that the problem has to do with index sizes and ironic's indexes
17:44:34 <clarkb> innodb limits the size of an index to either 767 or 3072 byes depending on the row format
17:44:35 <noonedeadpunk> I'm not sure about collations though. As it seems that there're really different defaults and available set of collactions, except the generic one, depending on the engine
17:44:45 <noonedeadpunk> like mariadb default != mysql
17:44:50 <noonedeadpunk> and I have not checked percona
17:45:02 <clarkb> and mb4 can easily push an index over 767 byes (255 * 3 = 765 but 255 * 4 > 767)
17:45:33 <JayF> I know that TheJulia did a lot of work in this area around the bug, and our fixes were to fix an actual operator issue. We should be very, very careful about what actions we do -- especially what actions TC dictates -- here.
17:45:40 <noonedeadpunk> I can recall somebody said it was already solved?
17:46:00 <noonedeadpunk> but not really sure
17:46:01 <clarkb> noonedeadpunk: the solution was to not allow mb4
17:46:06 <noonedeadpunk> ah
17:46:36 <JayF> This is the bug Ironic fixed, that from the current perspective, makes Ironic seemingly more broken (although I'm not sure if either perspective is universal)
17:46:55 <JayF> (explicitly using _mb3 so utf8 wouldn't be _mb4)
17:47:17 <frickler> iiuc openstack cannot enforce this, since we do not deploy the DB and things are different for maria/psql/others anyway
17:47:24 <noonedeadpunk> yeah
17:47:26 <noonedeadpunk> #link https://opendev.org/openstack/ironic/src/commit/5a33e8dbe7faaeb751e4e4cddb8f9c06c67a072c/ironic/db/sqlalchemy/alembic/versions/2581ebaf0cb2_initial_migration.py#L41
17:47:35 <frickler> so not sure what that community should would look like
17:47:37 <JayF> frickler: at table create time you can specify charset
17:47:46 <clarkb> and yes care should be taken to not make things worse. But in theory switching innodb rowformats (I don't know what the implications of this are it probably impacts performance) so that indexes can be 3072 byes, then switching to utf8mb4, then switching collation is probably the path to take if wanting to address both type and collation
17:47:49 <noonedeadpunk> so eventually, MariaDB has announced a plan to remove `UTF8` alias to MB3 soonish
17:48:02 <frickler> *community goal
17:48:06 <JayF> frickler: the initial ironic bug was we specified utf8, a default changed the pointer of that to utf8_mb4 and $something broke (I think the identity of the $something is still being archeologied)
17:48:10 <noonedeadpunk> so we need to anyway ensure projects are explicit with using charset in their migrations
17:48:30 <noonedeadpunk> which might be a good opprotunity for some project to use different charsets
17:49:50 <noonedeadpunk> for instance, placement not defining charset at all
17:49:54 <noonedeadpunk> #link https://opendev.org/openstack/placement/src/branch/master/placement/db/sqlalchemy/alembic/versions/b4ed3a175331_initial.py
17:51:16 <noonedeadpunk> nova uses `utf8` which is mb3, but also is about to be removed
17:51:18 <noonedeadpunk> #link https://opendev.org/openstack/nova/src/commit/2952c109489f9cbe082f9a9e79d4ac5b15dba290/nova/db/api/migrations/versions/d67eeaabee36_initial_version.py#L59
17:51:32 <noonedeadpunk> while some are using latin1
17:52:10 <noonedeadpunk> so I guess even if we're talking about some community goal - it should be about being explicit about mb3 vs mb4 and stop using aliases
17:52:16 <clarkb> utf8mb3 will be removed in future releases of mysql too
17:52:31 <clarkb> which is why I think it is important to consider changing the type as well as the collation
17:53:19 <noonedeadpunk> right....
17:53:49 <noonedeadpunk> I wonder if latin1 should be used for places where mb4 is impossible?
17:53:55 <noonedeadpunk> like Ironic usecase?
17:54:00 <fungi> in other software, we've seen some obvious and severe bugs related to users inputting 4-byte codepoints and the database backend not being configured to cope with storing them
17:54:06 <clarkb> I don't think ironic's case is impossible
17:54:50 <clarkb> but I think that is probably the first thing someone needs to sort out
17:55:07 <noonedeadpunk> but if we can't get common denominator for charsets, we probably should just leave collations alone
17:55:09 <clarkb> know that with confidence one way or another then use that to decide whether or not both type and collation should change or just collation
17:55:19 <noonedeadpunk> and do recommendation of some generic_ci or smth
17:56:20 <gouthamr> noob question: if no code changes were made to Flamingo (and earlier supported releases), would deployment tooling be able to handle this like the way you've proposed in openstack-ansible?
17:56:43 <noonedeadpunk> yes
17:56:56 <noonedeadpunk> but
17:57:08 <clarkb> for the charset/collation is there an accounting of what each project is already doing by default?
17:57:20 <clarkb> that way we can understand if case sensitive or insensitive is more common etc?
17:57:28 <noonedeadpunk> this was a specific thing about upgrade of MariaDB version default version
17:57:40 <gouthamr> (the code changes i am understanding will help clean things up for the future, and we could use the TC goals framework to define/recommend stuff to project teams and have this in Gazpacho if possible)
17:57:41 <noonedeadpunk> so we were able to handle this case in context of this
17:57:49 <clarkb> its possible that you'll have a large cohort with basically the same roles for charset/collation and they can all standardize. Then deal with the odd ones separately?
17:57:53 <fungi> clarkb: i think seunghunlee's post to openstack-discuss had some analysis along those lines
17:57:54 <noonedeadpunk> but it also opened can of worms
17:58:53 <noonedeadpunk> and potentially with the next release - most of existing migrations will just fail at the very begining
17:58:54 <clarkb> fungi: the email distinguishes those taht set the value but not what they set it to
17:59:00 <noonedeadpunk> if alias for `utf8` is removed
17:59:15 <clarkb> I guess I'm suggesting that someone draw up a big table with those concrete details in order to determine if there is a rough standard we're already adhering to
17:59:25 <gouthamr> JonathanWright[m]: spotz[m]: apologies, but it looks like our agenda was loaded; do you mind the Alma discussion occurring right after we wrap up here? I can still capture the discussion and follow up in further weeks
17:59:42 <spotz[m]> It's up to him:)
17:59:48 <noonedeadpunk> when I looked through project, most are using utf8mb3 atm
17:59:59 <noonedeadpunk> nobody defines collations explicitly
18:00:15 <clarkb> noonedeadpunk: the email says ` services such as Nova, Keystone, Cinder, designate provides charset when creating tables`
18:00:33 <JonathanWright[m]> Sure
18:00:34 <JonathanWright[m]> I don't have any other committments right now
18:00:45 <noonedeadpunk> yes, charset is provided by most. I think that placement is close to only exception
18:00:48 <gouthamr> ty JonathanWright[m] spotz[m]
18:01:08 <noonedeadpunk> I can build some rought table for the next time
18:01:40 <clarkb> noonedeadpunk: cool I think that would be helpful to understand the scope of the problem and whether or not everythign is already super disjoint or has lots of informal consistency
18:01:49 <mnasiadka> Before we get to that Gazpacho cycle goal (or later) - should we set a recommendation to users to avoid MariaDB 10.5 and higher in the install guide? (https://docs.openstack.org/install-guide/environment-sql-database.html)
18:02:28 <gouthamr> we'd begin with that ^ at the moment, i don't know if noonedeadpunk's solution works with kolla-*, or any other installer atm
18:02:39 <clarkb> then separately for mb3 vs mb4 it would probably be helpful to identify which projects have conflicts wit hthe larger type (we already know ironic does; anyone else?) and then start determining if there is a path forward to address that (different rowformat, different indexes, whatever)
18:02:55 <noonedeadpunk> mnasiadka: 11.5 :)
18:03:13 <mnasiadka> noonedeadpunk: right, I have a bad day today ;-)
18:03:16 <noonedeadpunk> fwiw, current LTS release for MariaDB is 11.8
18:03:23 * gouthamr we're a few mins over already..
18:03:30 <mnasiadka> noonedeadpunk: although devstack is testing 10.11
18:03:37 <mnasiadka> at least in centos jobs
18:03:47 <noonedeadpunk> but 11.4 is supported for quite a while anyway
18:04:24 <noonedeadpunk> but frankly - I'd won't say to avoid >11.5
18:04:49 <gouthamr> noonedeadpunk seunghunlee: if you would, can you please add your findings to an etherpad or a change on gerrit - you could just put it up as a proposed goal and let the discussion occur - i think it'd be helpful to brainstorm all of this
18:04:50 <noonedeadpunk> as in OSA we're planning to make 11.l8 as default, and with the patch in mind - it works nicely
18:05:29 <mnasiadka> We decided to stick to 11.4 in Kolla for now
18:05:50 <seunghunlee> Sure which etherpad should I use?
18:06:32 <gouthamr> we can make one up
18:06:35 <gouthamr> http://etherpad.opendev.org/p/gazpacho-collations-charsets
18:06:48 <seunghunlee> thanks
18:07:03 <noonedeadpunk> I think they closed some compatability gap with MySQL 8 in 11.8?
18:07:19 <gouthamr> there's nothing there, but, we can use this as reference to drive further brainstorm
18:07:34 <gouthamr> i'll wrap up the meeting now.. any thing else to add to the minutes today?
18:07:46 <noonedeadpunk> not sure though
18:08:19 <gouthamr> we'll discuss Alma Linux here in a few
18:08:25 <gouthamr> thank you all for attending
18:08:49 <gouthamr> any other discussion topics will be (re)-visited next week!
18:08:51 <gouthamr> #endmeeting