13:00:26 <mnasiadka> #startmeeting kolla 13:00:26 <opendevmeet> Meeting started Wed Aug 20 13:00:26 2025 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:00:26 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 13:00:26 <opendevmeet> The meeting name has been set to 'kolla' 13:00:29 <mnasiadka> #topic rollcall 13:00:29 <mnasiadka> o/ 13:00:33 <mmalchuk> \o 13:00:41 <bbezak> o/ 13:00:48 <amir58118> \o 13:01:34 <AlexWelsh[m]> o/ 13:01:55 <yusufgungor_> o/ 13:02:09 <frickler> \o 13:02:21 <opendevreview> Merged openstack/kolla stable/2025.1: Enable Barbican build on Ubuntu ARM64 https://review.opendev.org/c/openstack/kolla/+/958020 13:02:25 <opendevreview> Merged openstack/kolla stable/2024.2: Enable Barbican build on Ubuntu ARM64 https://review.opendev.org/c/openstack/kolla/+/958021 13:02:32 <mnasiadka> #topic agenda 13:02:32 <mnasiadka> * CI status 13:02:32 <mnasiadka> * Release tasks 13:02:32 <mnasiadka> * Current cycle planning 13:02:32 <mnasiadka> * Additional agenda (from whiteboard) 13:02:33 <mnasiadka> * Open discussion 13:02:36 <mnasiadka> #topic CI status 13:02:40 <mnasiadka> overall sort of green 13:02:50 <mnasiadka> Octavia removed wsgi script, so we're moving to a file 13:02:58 <mnasiadka> #link https://review.opendev.org/c/openstack/kolla-ansible/+/958070 13:03:51 <mnasiadka> debian-upgrade jobs fail sometimes, we have lined up two patches - one that bumps debian jobs to 16GB, the other one stops proxysql monitoring on single node jobs (which might overwhelm mariadb a bit on constrained env and then we have problems) 13:03:58 <mnasiadka> We'll see how these go 13:04:04 <mnasiadka> Did I miss anything? 13:04:45 <fungi> the default ansible version in zuul increased this week 13:05:03 <fungi> (from 9 to 11) 13:05:28 <mnasiadka> Yeah, I don't think that broke anything on our side 13:05:29 <fungi> fallout has been fairly minimal for openstack projects, and in your case probably most of the ansible you care about is nested anyway so not affected 13:05:58 <mnasiadka> thanks for reminding fungi 13:06:07 <mnasiadka> #topic Release tasks 13:06:11 <mnasiadka> Let me check the calendar 13:06:33 <mnasiadka> R-6 13:06:38 <bbezak> I think I saw some ironic jobs failing in stable branches, and we've got ubuntu ipv6 in disk full state for some time 13:06:50 <mnasiadka> R-8: Switch images to current release 13:07:23 <mnasiadka> we should switch RDO and UCA and Debian OpenStack to some current in-development release 13:07:29 <mnasiadka> But I think RDO is not going to release Flamingo 13:07:35 <mnasiadka> Since they switched to do only SLURP 13:07:45 <mnasiadka> Any volunteer to switch UCA? 13:08:25 <bbezak> I can take a look 13:08:36 <bbezak> or will find somebody :) 13:09:34 <bbezak> can't see flamingo yet here - https://openstack-ci-reports.ubuntu.com/reports/cloud-archive/epoxy_versions.html 13:10:19 <mnasiadka> Ok well then, let's check again next week 13:10:32 <bbezak> http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/noble-updates/flamingo/ 13:10:32 <mnasiadka> #topic Current cycle planning 13:10:36 <bbezak> ok it seems there is 13:10:58 <mnasiadka> ok, then, bbezak will do it or find somebody to do it 13:11:10 <bbezak> are we ok with releases for stable branches? 13:11:35 <bbezak> (monthly ones0 13:11:38 <bbezak> ) 13:11:48 <mnasiadka> I think we still have the sec bug to fix, and we haven't reverted to latest mariadb on 2025.1 13:11:50 <mnasiadka> so not really 13:12:01 <bbezak> ok 13:12:10 <mnasiadka> sec bug fix had to be reverted, because it masked too many things, not only that (because we copy things from docker_common_opts) 13:13:04 <mnasiadka> I think I went with frickler's recommendation and checked my hosts where we run Kayobe (and Kolla-Ansible through that) and a dev VM with Kolla-Ansible - and can't really find that in my syslog 13:13:16 <mnasiadka> Which might mean we only get that with debug mode enabled 13:13:29 <mnasiadka> I'll dig through our CI Ansible configs to find the difference 13:13:59 <mnasiadka> But I still think we should get rid of docker_common_opts since it's not parsed or verified in any sense 13:14:13 <mnasiadka> And we might find a more nasty bug with that in the future 13:15:14 <mnasiadka> bbezak, frickler: any comments? 13:16:26 <bbezak> ideally it would be good not to touch all roles. but if that is not possible then I'm ok with that 13:16:46 <frickler> I'll take a look at https://review.opendev.org/c/openstack/kolla-ansible/+/957228 , but if we only log sensitive data in debug mode, then it should not be considered a security issue 13:17:16 <mnasiadka> True, at least we would not have to backport that 13:18:01 <mnasiadka> During looking how complicated is it to do docker_common_options drop I found out kolla_container module has wrong documentation (we claim some things are defaulting to X, but then we default them to Y or empty value) 13:18:09 <bbezak> backporting such change would be highly suboptimal 13:18:26 <mnasiadka> And only docker_common_options with correct values in group_vars are safeguarding deployments from a disaster 13:19:05 <mnasiadka> I tried using module_defaults, but it does not append to roles, we would need to do a block in each role and set the defaults, which is a nightmare 13:19:26 <opendevreview> Bartosz Bezak proposed openstack/kolla master: [release] Use Flamingo Ubuntu Cloud Archive https://review.opendev.org/c/openstack/kolla/+/958081 13:19:35 <mnasiadka> Ok then, let's move on 13:19:47 <fungi> the commit message starts out with "Due to a security bug..." but the bug is not linked 13:20:26 <mnasiadka> fungi: yes, that's WIP - I'll update it thoroughly after I get to a state that we do minimal changes needed 13:20:41 <fungi> okay, just making sure it's related to the one i'm thinking of 13:20:52 <mnasiadka> Ok, let's go back to contributor survey data 13:20:59 <mnasiadka> #link https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-07-23-13.00.log.txt 13:21:48 <mnasiadka> key takeaways are that we don't review things on time, but that's unfortunately obvious with 4 active cores which also happen to be the main people pushing new patches 13:22:15 <fungi> well, not that you don't review things quickly, but that contributors reported getting that impression 13:22:15 <mnasiadka> we could try updating the review dashboard with patches that are not older than 2 weeks that are not from these 4 people 13:22:51 <fungi> often the impression is not backed up by reality, because people are more inclined to remember the one change that sat forever and not the other ten that merged right away 13:23:25 <mnasiadka> True, but I also have patches that have been never reviewed if I don't remind other core reviewers to have a look :) 13:23:34 <mnasiadka> So I can't blame people that have that opinion 13:24:06 <frickler> I have patches on my review list for 2 years, so I agree 13:24:19 <fungi> one of the things that's been coming up as a commonality across a lot of teams is that infrequent contributors aren't aware of how the team prioritizes development and review activity, and making that more discoverable/obvious could help put things into perspective by setting clearer expectations 13:24:57 <mnasiadka> We could add the review dashboard links and a link to a place where they can add patches needing review in ether pad - that would make it obvious 13:25:12 <fungi> as well as making it more apparent how prioritization decisions are made, and how newcomers can get involved in those planning discussions 13:25:51 <amir58118> fungi: +1 13:26:12 <mnasiadka> Any volunteer to make this docs change? 13:27:09 <fungi> try to put it somewhere that newer contributors are likely to find, or at least link to it from a place they're going to be referring to for general contribution info 13:27:59 <mnasiadka> I'd say having that in contributor guide should be enough 13:28:01 <mnasiadka> #link https://docs.openstack.org/kolla-ansible/latest/contributor/index.html 13:28:04 <fungi> because the other common challenge i've seen in lots of teams is that stuff like this is documented and maybe even kept up to date, but people who aren't deeply involved in the team don't know about the documentation 13:28:38 <mnasiadka> #link https://docs.openstack.org/kolla-ansible/latest/contributor/code-review.html 13:28:51 <mnasiadka> We even have code review section there 13:28:51 <fungi> yeah, somewhere like that would make sense 13:29:10 <bbezak> ok, I can work on that 13:29:16 <mnasiadka> great 13:29:18 <mnasiadka> thanks bbezak 13:29:33 <bbezak> I recently updated some things there, and by quick look I saw some stuff to change too 13:30:10 <fungi> of course, also try to make sure all maintainers are on the same page when it comes to what's being prioritized and why, though i expect that's likely to be the case already 13:30:41 <frickler> mostly I'd say we agree on having different priorities 13:30:58 <mnasiadka> Other thing was simplifying tests, I think we agreed to do that for a couple of cycles now - it's all planned, but since we're a small team - we're rather trying to keep the project afloat and adapt to changes in OpenStack projects - and we don't really have a lot of time to simplify tests. 13:31:10 <mmalchuk> 'Becoming a core reviewer' didn't work, we didn't get new cores for the last year 13:31:10 <fungi> which is fine so long as those are reflected in the prioritization info that other contributors see 13:31:43 <mnasiadka> Well, for core reviewers we have the Review Priority flag, so probably we should document that 13:32:17 <fungi> mainly try to make sure there's sufficient information discoverable so that when a contributor wants to know why other changes are getting reviewed and not theirs, they have some hope of figuring that out rather than just assuming maintainers are playing favorites 13:33:05 <fungi> people who have a negative experience are, sadly, quick to jump to conspiratorial explanations rather than applying hanlon's razor 13:33:53 <mnasiadka> fungi: well, I would sometimes even prefer these people vent in there or on the ML 13:34:10 <fungi> that's definitely worth documenting too! 13:34:31 <bbezak> hopefully Matrix could make communication somewhat better 13:34:40 <fungi> say "and if you think your change is getting overlooked, reach out in <location> to let us know" 13:34:54 <bbezak> (noting) 13:35:21 <mnasiadka> hmm, Matrix 13:35:30 <fungi> yeah, persistence and time zones make this harder for sure 13:36:17 <fungi> one example i looked at, someone had a change for nova that was getting overlooked, so they finally got up the nerve to ask in #openstack-nova about it... on december 26 when the channel was completely dead 13:36:29 <fungi> then got frustrated when nobody replied or even noticed 13:37:06 <amir58118> bbezak: I agree, Matrix should help. 13:37:16 <fungi> they weren't from a part of the world that observed holidays at that time of year, so it didn't dawn on them that was not a great time to try to reach out and make contact 13:37:17 <mnasiadka> Well, yes - I always love people coming in during my night time and disconnect - so we can't really respond to them. 13:38:32 <mnasiadka> We might think about having a Matrix channel, but I'm afraid these are going to be two disconnected channels (IRC and Matrix) - and we would still be running this meeting here - so I don't know if that would help in anything 13:38:50 <mnasiadka> And I remember frickler is not a fan of Matrix, or at least his IRC client is not 13:39:01 <mmalchuk> we can create a bot to connect them) 13:39:16 <frickler> opendev is planning to do a test with Matrix, let's wait for that to happen first, maybe? 13:39:29 <fungi> #link https://review.opendev.org/954826 Add spec to use Matrix for OpenDev comms 13:39:48 <fungi> though the zuul community does already use a matrix channel, has for years 13:39:50 <bbezak> zuul already is doing that, and it looks nice 13:40:06 <fungi> and the openstack operators community has a channel since a few months now 13:40:07 <bbezak> Maybe we can be second ? :) 13:40:32 <fungi> er, room i mean. matrix calls them rooms, not channels 13:40:32 <bbezak> really? where I can find all opendev zuul channels? 13:40:42 <bbezak> I was struggling to list opendev rooms 13:40:49 <frickler> the operators channel doesn't look like a big success to me 13:40:58 <fungi> #zuul:opendev.org is the zuul community's room. #openstack-ops:opendev.org is the openstack operators 13:41:07 <frickler> afaict they aren't discoverable 13:41:09 <fungi> yeah, it has seen very minimal use so far 13:41:23 <bbezak> imho public channels should be discoverable 13:42:12 <fungi> part of the challenge is that discoverable channels, whether in irc or matrix, are a magnet for spammers. but we're looking at a modbot called mjolnir that could improve that situation for matrix 13:42:46 <opendevreview> Pierre Riteau proposed openstack/kayobe-config-dev master: Synchronise configuration with master https://review.opendev.org/c/openstack/kayobe-config-dev/+/955454 13:42:50 <fungi> spammers will still join discoverable rooms (a lot) but they'll hopefully get banned and their messages hidden fairly quickly 13:43:17 <opendevreview> Merged openstack/kayobe-config master: Synchronise configuration with master https://review.opendev.org/c/openstack/kayobe-config/+/955453 13:43:34 <fungi> fwiw, the mailing lists have that problem too, but i and other ml moderators filter that out so it (usually) doesn't make it through to list subscribers 13:43:55 <frickler> is discoverability a dynamic property? or do we need to decide when creating a room? 13:44:03 <fungi> it's configurable per room 13:44:22 <fungi> it can be adjusted at any time 13:44:51 <fungi> basically it's just adding it to and removing it from the global/searchable room list 13:45:18 <bbezak> I'm fine with closed for start if there are some spam problems 13:45:49 <frickler> also, iiuc, in order to e.g. allow kolla cores to moderate the kolla channel, we'd need to have a dedicated mjolnir instance for that 13:48:43 <mnasiadka> Ok then - does it mean we'd like to do a proof of concept/testing of an additional Matrix channel for OpenStack Kolla? (in addition to the IRC one) - if that's even possible today 13:49:15 <opendevreview> Merged openstack/kolla master: Influxdb: use stable RHEL repo https://review.opendev.org/c/openstack/kolla/+/957680 13:50:14 <fungi> i don't know enough about mjolnir yet, it's possible room-specific behaviors can be decided by people with elevated permissions in those specific rooms but i'd need to read up on it 13:50:37 <frickler> I would prefer to see the opendev test go ahead first, but I won't -2 if others want to test something now 13:51:17 <fungi> if you decide you want one, let me know and i'm happy to do the configuration on the homeserver to create it and add some of you as initial moderators/admins for the room 13:52:06 <mnasiadka> Ok, thanks - we'll think about it and come back. 13:52:33 <mnasiadka> I think we covered some of the contributor issues 13:52:41 <mnasiadka> fungi: Is there something I missed? 13:52:52 <fungi> there's always more, but that was a great discussion. thanks! 13:53:05 <mnasiadka> thank you for gathering that data :) 13:53:16 <mnasiadka> Ok, that covered also Additional agenda I guess 13:53:20 <mnasiadka> #topic Open discussion 13:53:31 <mnasiadka> Anybody anything that was not on the agenda? ;-) 13:53:59 <mmalchuk> own channel in Matrix ?) 13:55:23 <mnasiadka> We just discussed that 13:55:26 <mnasiadka> Anything else? 13:56:57 <mnasiadka> Seems not 13:57:04 <mnasiadka> Thank you all for coming - see you next week :) 13:57:06 <mnasiadka> #endmeeting