Tuesday, 2025-09-09

opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: [doc] Document venv rebuild for python_venv_build role  https://review.opendev.org/c/openstack/openstack-ansible/+/95349300:11
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: [doc] Document venv rebuild for python_venv_build role  https://review.opendev.org/c/openstack/openstack-ansible/+/95349300:11
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: [doc] Document venv rebuild for python_venv_build role  https://review.opendev.org/c/openstack/openstack-ansible/+/95349300:25
f0oHi, this is likely a super silly Q but my Ansible-Fu isnt good enough to figure it out; extra_volume_types is this _additive_ or _replacing_ the volume_type name from the key?10:18
f0ohttps://github.com/openstack/openstack-ansible-os_cinder/blob/eb01602ebcbcf58052665008fdb77022a2597634/defaults/main.yml#L294-L301 << will this create lvm, lvm_high_iops, lvm_low_iops or will it just create lvm_high_iops, lvm_low_iops10:19
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: [doc] Document venv rebuild for python_venv_build role  https://review.opendev.org/c/openstack/openstack-ansible/+/95349310:40
noonedeadpunkf0o: hey10:55
noonedeadpunkit's adding iirc10:55
noonedeadpunkand you can't skip the "default" one so far, unless you just set `cinder_manage_volume_types: false`10:56
noonedeadpunkand skip that alltogether10:56
noonedeadpunkthe reason for that is that openstack ansible collection never had a working module for that10:57
f0oAwesome!11:00
f0oThanks :)11:00
opendevreviewIvan Anfimov proposed openstack/openstack-ansible-lxc_hosts stable/2025.1: Remove COPR repo for python3-lxc  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/96025511:46
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Generate certificates only for API hosts  https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/96025712:12
noonedeadpunkdamiandabrowski: I wonder if we might have the same issue in other places as well ^12:13
damiandabrowskiouh12:37
damiandabrowskinice catch!12:37
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Allow to set conditions on components  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/96026814:13
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Allow to set conditions on components  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/96026814:13
noonedeadpunk#startmeeting openstack_ansible_meeting15:01
opendevmeetMeeting started Tue Sep  9 15:01:11 2025 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
opendevmeetThe meeting name has been set to 'openstack_ansible_meeting'15:01
noonedeadpunk#topic rollcall15:01
noonedeadpunko/15:01
damiandabrowskihi!15:01
* fungi has something for office hours15:01
noonedeadpunk#topic office hours15:04
noonedeadpunkfungi: floor is yours :)15:05
fungiBridging the gap between community and contributing orgs: survey and metric analyses15:06
fungii'll try to be quick, but there's a lot we dug into... for some background on openstack-wide metrics analysis see ildikov's most recent ml post from a couple of months back:15:06
fungi#link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/NTBNI7YIDCWBR6BTPEKVZIODWTVUIOXO/ BtG metrics analysis15:06
fungialso could be worthwhile to revisit her previous post in that thread going over the contributor and maintainer survey results (and anyone who hasn't filled those out for epoxy, please see if you can find a few minutes to do that!)15:06
fungias a follow-up activity, we've been doing some team-specific analyses, focusing on about a dozen teams that had multiple survey responses15:06
fungiwe're early in the process of analyzing these stats with a focus on improving the experience for maintainers and contributors, so for now this is probably a lot of stuff you already know, or at least confirming what you expected15:06
fungia big part of this is establishing a baseline so that we can better gauge whether future attempts at improving have any observable impact, but we aren't at the point where we have much in the way of guidance or recommendations yet15:06
fungiwe have plans to continue with this sort of surveying and metrics analysis over coming release cycles and have already drafted some updates for the flamingo versions; as for the initial results...15:07
fungithe contributor survey had 1 response for openstack ansible and the respondent self-identified as a new contributor who had contributed to at least one other open source project15:07
fungifeedback was polarized (either 1 or 4 out of 5) ranking actionable feedback from reviewers and usefulness of automated testing high, but low scores on timely reviews and basically all aspects of the contributor documentation15:07
fungithe one challenge they listed for contributing was a lack of familiarity with gerrit15:07
fungithe maintainer survey also had only 1 response so far, with more varied scoring (between 1 and 3 out of 5) the highest being for actionable feedback from reviewers and contributor documentation quality, lowest on timely reviewing again15:07
fungithe only contributing challenge reported by that maintainer was with getting review attention, and in reviewing changes for others they struggled with a lack of familiarity in some parts of the project15:07
fungilooking at metrics we gathered from gerrit for the past 5 development cycles, review activity has not been nearly as bad as the survey responses suggest, which i would chalk up to selection bias and negative experience outweighing positive15:07
fungithere was a fairly significant drop in the number of active maintainers and reviewers as well as change volume during the epoxy cycle compared to prior releases, so i don't know whether that might also have anything to do with it15:08
fungiinterestingly, the median time to review nearly doubled from dalmatian to epoxy (bobcat was worse though) while the average (mean) time to review stayed roughly the same15:08
fungion the other hand, the time to merge changes was about the same between the two cycles so the drop in maintainers and reviewers doesn't seem to have impacted that, probably owing to a similar drop in changes proposed15:08
fungiwhile it's hard to draw conclusions from such a small number of responses, some additional focus on revamping or updating the contributor documents, maybe leaning a bit omre into referring out to the general openstack contributor workflow docs might help15:09
fungithough hard to know if that was just a one-off impression in that survey response15:09
noonedeadpunkthere's really a lot to unpack here15:09
fungiand as is probably obvious, trying to claw back some of the maintainer/reviewer activity, though i don't have great suggestions there yet15:10
fungisorry, i know that's a pretty big info dump (i tried to pare it down as much as possible), and i'm happy to answer questions or take feedback either here in the meeting or any time after in #openstack-manila too15:10
fungier, in #openstack-ansible i mean15:10
noonedeadpunkSo lately we indeed have a decreased activity from core team regarding reviews. And my inner feeling that merge time should have increased at least this cycle in an observable way15:11
fungikeep in mind that this was observing the epoxy cycle activity, after the ptg we'll be doing flamingo analyses and surveys15:12
noonedeadpunkHowever when dealing with other projects in OpenStack, I'd estimate review time as above average on the boards still...15:12
fungiyes, it looked that way from the stats15:13
fungislow review response has been just about universal in the survey responses for all teams15:13
noonedeadpunkand yes, this cycle we do have a huge focus on improving and re-working our documentation, which was historically not up to date15:14
fungieven though the stats for most teams suggest it's not all that bad15:14
noonedeadpunkso missing or absent documentation is being recognized as the problem.15:14
noonedeadpunkwhich I guess is imporant, that we also see our own issues and trying to address them15:15
fungithe psychology of it, i think, is that if you have 10 changes that merge quickly and 1 change that takes ages to get reviewer interest, you'll remember the one bad experience while forgetting the 10 good ones15:15
noonedeadpunkwell. it could be as well, that you might have an expectation to get your bugfix merged and released by end of the week15:16
fungithe challenge with this initial round is that looking at these surveys as a stand-alone snapshot in time isn't terribly useful. comparing responses from one cycle to the next will be a lot more indicative of whether things are improving15:16
noonedeadpunkwhile it would take a week-two to  just to land on master and then some time for backports15:16
noonedeadpunkdoes this survey also include contributions in form of bug reports?15:17
noonedeadpunkas how projects are dealing and some kind of stats for bug fixing would be really also interesting15:18
fungicurrently the surveys have focused mostly on people proposing changes for review and maintainers reviewing proposed changes, since this effort came out of the tc's request for getting more openinfra member orgs to get involved in development and/or asking why they aren't15:19
noonedeadpunkwhich is fair. gotcha.15:19
fungialso the current rate of survey responses makes it hard to get any more granular15:19
fungias i said, openstack ansible had a total of two survey responses (one for the contributor survey, one for the maintainer survey)15:20
noonedeadpunkI was just thinking/having an idea of transitioning people reporting bugs to contributors, and why does this happen so rarely15:20
noonedeadpunkfor instance - because of missing knowledge, time, policies, etc15:21
noonedeadpunkjust to spreatch survey audience a bit15:21
fungiso we're mostly trying to contrast and compare the very few survey responses with available contribution and review metrics15:21
noonedeadpunk*streatch15:21
mgariepyon issue with patches  (from what i saw on other opensource contribution) is that quite often trying to ping the correct group/person to get stuff moving it really helpful.15:21
noonedeadpunkwe probably indeed need to update our contributing guidelines and info on how to contact the team with Matrix bridge setup15:22
fungiyeah, some of the member orgs we heard from during early outreach were trying to fix bugs that they reported, but were having trouble getting their proposed fixes merged for various reasons (most of which boiled down to miscommunication both on their part and on the part of reviewers who engaged)15:22
fungithat wasn't for openstack ansible specifically though15:22
fungimgariepy: definitely, one we were prompted to look at for nova sat unreviewed for a while until the proposer asked for help in #openstack-nova... in the middle of the night on december 26th15:24
noonedeadpunkI can recall that one....15:24
fungiand then when they (unsurprisingly for most of us i'm sure) got no response there, they gave up15:24
noonedeadpunkI wonder if adding something from review stats, like average review time, to contributor docs may manage expectations15:25
fungiso knowing who to reach out to and how, but also sometimes when15:25
noonedeadpunkand improve overall feeling of time15:25
noonedeadpunkas more we talk, more it feels that it's unmatched expectations which impact the most15:25
fungiyeah, setting (realistic) expectations in documentation is great and something we're trying to encourage15:26
fungiand i agree, miscommunication around expectations is at the root of most of the specific cases we looked into15:26
fungiespecially around prioritization of reviewing and development areas in projects15:27
noonedeadpunkI think also relatively recently on a cinder meeting there were bunch of driver maintainers who also were not really aware of review process, all asking at the beginning of the meeting for reviews, pretty much making it messy and hard to follow15:27
noonedeadpunkand then appeared there's some etherpad, which took me l,ike 10mins at least to find among prior meeting logs...15:28
fungifrom a contributor perspective, it's things like how to know what the team is prioritizing, how (if possible) to get your work prioritized and/or participate in the prioritization decisions15:28
noonedeadpunkso eh...15:28
noonedeadpunkwe have a long road as a community to be explicit about processes...15:28
fungiabsolutely, this is nothing we're going to fix overnight15:29
fungii'm hoping as a commuinity we can share ideas for what's already working well in some places, make incremental improvements over time, et cetera15:29
noonedeadpunkbut that is really interesting info, and I think we can have couple of actionable items out of it15:29
noonedeadpunkI assume these averages from ML per-project specifically could be taken out of Bitergia relatively easily?15:30
fungias for survey updates, we're adding questions about prioritization to the flamingo round too15:30
funginoonedeadpunk: yes exactly, we pulled them straight from bitergia15:31
fungithough tried to avoid getting into comparing teams to each other since their challenges differ and any comparison would be unfair and possibly divisive15:31
noonedeadpunk++15:32
noonedeadpunkeventually we have another issue about contributing docs15:33
fungithough in cases where teams had particularly positive results we've tried to dig into potential reasons, to see if there are any ideas or workflows that can be transplanted more widely15:33
noonedeadpunkwhich is that we have 2 of them15:33
noonedeadpunk#link https://docs.openstack.org/openstack-ansible/latest/contributors/contributing.html15:33
noonedeadpunkthis one ^was added as part of community goal15:34
noonedeadpunk#link https://docs.openstack.org/openstack-ansible/latest/contributors/contribute.html15:34
noonedeadpunkand this one is historical, containing some extra info (potentially)15:34
fungiprobably worth combining them and redirecting one to the other15:34
fungianyway, that's all i had for now, didn't want to monopolize the entire meeting15:35
noonedeadpunkI think why I didn't do that, as the one from communty goal was having kinda template. And I treated it as a "exclusive" to bring alignment between projects15:36
noonedeadpunkbut I think I need to revise this and indeed merge things together....15:37
fungibeen a while since i read through the goal text, but i thought it was more about making sure some specific details were consistently included by every project, not necessarily limiting what other information could be included there15:37
fungithough also it could link out to other pages as well15:37
noonedeadpunkyeah, will try to figure this out, as it's obviously a thing which brings some confusion at least15:38
noonedeadpunkok, cool15:38
NeilHanlonheya, sorry i am very late :) 15:38
NeilHanlonat the airport right now15:38
fungiwe left you a ton of scrollback to entertain you on your flight15:39
noonedeadpunkheh, no worries :) that's a good excuse NeilHanlon :)15:39
noonedeadpunk++15:39
fungihopefully lots of food for thought, at least for now15:39
NeilHanlon:) was sitting to work on my slides and remembered the time15:39
NeilHanlonspeaking of, does anyone have real-world OpenStack/NVMe performance numbers they're able to share/point at for my reference? 15:40
noonedeadpunkno, not really. 99% of our footprint is ceph15:40
fungimnaser might? i recall him doing a lot of comparative benchmarking for local nvme storage on compute nodes in vexxhost15:41
noonedeadpunkso, epel has promoted python3-lxc for EL10, so now we finally can drop usage of NeilHanlon COPR15:41
noonedeadpunk#link https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/95941515:42
noonedeadpunkand with backport to epoxy, I think we should be able to tag 31.1.0 finally15:42
* noonedeadpunk need to check for other waiting backport15:42
noonedeadpunk#link https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/95897015:43
noonedeadpunkis one would be nice to include though15:44
noonedeadpunkbut I think it should be it15:44
noonedeadpunkon the dark side - there has been no progress with systemd-networkd addition to EPEL15:44
noonedeadpunkdespite there were suggestions for co-maintenance, they were ignored so far15:45
NeilHanlonnoonedeadpunk: well actually.. sorta looking at the intersection of Ceph and NVME, so if you're using nvme backed OSDs (a la BlueStore) i'm interested :D 15:45
NeilHanloni will ping rsc directly about the systemd thing15:45
NeilHanlonsee if I can un-stick that15:45
NeilHanlonalso i guess we should have a couple more packages for rocky 10 that mnasiadka requested -- the openvswitch/ovn and ceph squid repos15:46
NeilHanlon(from centos SIGs)15:46
noonedeadpunk#link https://bugzilla.redhat.com/show_bug.cgi?id=230389215:46
noonedeadpunkwell, while openvswitch/ovn are kinda required - I have not tested if centos sig versions will be working or not15:47
noonedeadpunkbut ceph is known not to work for some time indeed.15:47
NeilHanlonyeah, i dunno the whole state either but I just got asked to provide the -release packages 15:48
noonedeadpunkwe do have nvme osds... will try to check if we have any statistically good performance data...15:48
NeilHanlonappreciate that if you can share! I have some of my own data but being able to back it up would be nice lol15:49
mnasiadkaNeilHanlon: any idea when the compose will land with these extras centos-release-* packages?15:50
NeilHanlontoday or tomorrow i expect15:58
NeilHanlon(mnasiadka)15:58
mnasiadkathanks15:59
noonedeadpunk#endmeeting16:00
opendevmeetMeeting ended Tue Sep  9 16:00:05 2025 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:00
opendevmeetMinutes:        https://meetings.opendev.org/meetings/openstack_ansible_meeting/2025/openstack_ansible_meeting.2025-09-09-15.01.html16:00
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2025/openstack_ansible_meeting.2025-09-09-15.01.txt16:00
opendevmeetLog:            https://meetings.opendev.org/meetings/openstack_ansible_meeting/2025/openstack_ansible_meeting.2025-09-09-15.01.log.html16:00
damiandabrowskismall update from my side: I've started fixing merge conflicts in my osa_hashi_vault patches, should be done tomorrow16:01
noonedeadpunkyeah, we need to get some progress on merging these, and also service patches16:02
damiandabrowskiIn general, I aim to focus on PKI topic this week, but next week I'll be on vacation16:02
*** kleini_ is now known as kleini16:51
mnasiadkaNeilHanlon: mirrors.rockylinux.org has hard time again? ;-)16:53
mnasiadkaoh, just got up16:53
*** kleini_ is now known as kleini17:04
opendevreviewDamian Dąbrowski proposed openstack/ansible-role-pki master: Separate backend-specific logic to improve extensibility  https://review.opendev.org/c/openstack/ansible-role-pki/+/94887817:26
opendevreviewDamian Dąbrowski proposed openstack/ansible-role-pki master: Change the format of 'san' parameter in `pki_certificates` variable  https://review.opendev.org/c/openstack/ansible-role-pki/+/94887917:30
opendevreviewDamian Dąbrowski proposed openstack/ansible-role-pki master: Use ttl instead of not_after in pki_authorities  https://review.opendev.org/c/openstack/ansible-role-pki/+/94888017:30
opendevreviewDamian Dąbrowski proposed openstack/ansible-role-pki master: Add hashi_vault backend  https://review.opendev.org/c/openstack/ansible-role-pki/+/94888118:28
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996518:46
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Fix compatability matrix table layout  https://review.opendev.org/c/openstack/openstack-ansible/+/96030618:53
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996518:58
jrosseroh my sorry i missed the meeting19:07
jrossernoonedeadpunk: ^19:07
jrosseri would not object to the ping at the start of the meeting we had in the past19:07
jrosserNeilHanlon: we run some nvme osd here but depends what you mean by performance19:08
jrosseryou can optimise for throughput or iops which lead you in very different directions19:08
jrosserwe also have some hosts with ceph boot-from-volume plus ceph cache drives for both read and write locally on the host19:09
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996519:09
jrosserfungi: one of the things we have had regarding contributions this cycle is a vast number of tiny docs patches19:10
jrosserand that makes me almost overwhelmed as a reviewer, its requires an extraordianry amount of brain cycles and somehow swamps more normal code review for me19:11
jrosserlikley also made worse by having so many repos in openstack-ansible, making this effect really magnified as a lot of the changes are once per repo19:12
fungijrosser: good to know! are those bot-proposed patches like the glossary sync ones, or coming from specific users?19:13
jrosserspecifc users for these19:13
noonedeadpunkjrosser: can do that - sure19:17
noonedeadpunk(about pings)19:17
jrosserjust got so busy today i totally forgot :(19:17
jrosseralso been feeling not so happy about being able to contribute less at the moment19:19
noonedeadpunkjrosser: do you have a feeling regarding overall doc improvement this cycle?19:19
noonedeadpunkmore thinking if these tons of patches were overall useful or just a noise19:19
jrosseri think that the effort is really worthwhile and the end result is good19:19
jrosserbut i find the reviewing really difficult19:19
noonedeadpunk++19:19
noonedeadpunkagree19:19
jrosserwhich lets say if we were a monorepo might be less bad19:20
jrosserbut thats just how things are19:20
noonedeadpunkto be fair - we payed close to 0 attention to our docs since... xena? or smth?19:20
jrosserindeed :_19:20
jrosser:( even19:20
noonedeadpunkprobably even earlier...19:20
noonedeadpunkbut yeah, I'd love some things to be combined together rather then being proposed separately19:21
noonedeadpunkin tiny iterations19:21
noonedeadpunkfungi: the same one actually19:22
noonedeadpunkbut I don't think this was boted one specifically...19:23
fungii guess if it's an ongoing thing it might be worth having a word with the contributor responsible to see how things could be done more efficiently19:25
noonedeadpunkfungi: it's kinda delicate matter as in general - this is being useful. Just indeed can be optimized. But at the same time I don't want to discourage contributions...19:28
fungiwell, talking to someone about how to improve the efficiency of their contributions shouldn't necessarily be discouraging19:30
noonedeadpunk++19:31
fungibut this is exactly the sort of thing the first contact sig was formed to tackle, and it's now effectively defunct19:31
noonedeadpunknot them being on IRC also makes it harder19:31
noonedeadpunk*them not being on IRC19:31
noonedeadpunkand ML/emails usually is treated as a very formal way of communication, where it could be hard to express intention and overall mood 19:33
noonedeadpunkbut yeah, will reach them out and see :)19:33
opendevreviewDmitriy Chubinidze proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996519:40
opendevreviewDamian Dąbrowski proposed openstack/ansible-role-pki master: Add hashi_vault backend  https://review.opendev.org/c/openstack/ansible-role-pki/+/94888119:58
opendevreviewIvan Anfimov proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996520:23
opendevreviewIvan Anfimov proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996520:26
opendevreviewIvan Anfimov proposed openstack/openstack-ansible master: docs: updated information in the troubleshooting guide  https://review.opendev.org/c/openstack/openstack-ansible/+/95996520:28

Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!