| opendevreview | Brian Rosmaita proposed openstack/python-brick-cinderclient-ext master: Add periodic jobs for brick-cinderclient-ext https://review.opendev.org/c/openstack/python-brick-cinderclient-ext/+/982034 | 02:13 |
|---|---|---|
| opendevreview | Brian Rosmaita proposed openstack/python-brick-cinderclient-ext master: DNM: test proposed periodic-weekly jobs https://review.opendev.org/c/openstack/python-brick-cinderclient-ext/+/982035 | 02:16 |
| *** zseguin_ is now known as zseguin | 06:11 | |
| opendevreview | Victor Chembaev proposed openstack/os-brick master: Multipath hangs on detach https://review.opendev.org/c/openstack/os-brick/+/982067 | 10:24 |
| opendevreview | Artem Goncharov proposed openstack/cinder master: Fix jsonschema parameter https://review.opendev.org/c/openstack/cinder/+/982079 | 12:33 |
| opendevreview | Sanketh Shetty proposed openstack/cinder master: Add support for AutomatedFailOver policy with consistent replication https://review.opendev.org/c/openstack/cinder/+/975880 | 12:59 |
| rosmaita | Cinder meeting in #openstack-cinder at 1400 UTC | 14:01 |
| rosmaita | jungleboyj rosmaita smcginnis tosky whoami-rajat m5z e0ne geguileo eharney jbernard hemna fabiooliveira yuval tobias-urdin adiare happystacker dosaboy hillpd msaravan sp-bmilanov Luzi sfernand simondodsley zaubea nileshthathagar flelain wizardbit agalica lutimura kaisers | 14:01 |
| rosmaita | #startmeeting cinder | 14:01 |
| opendevmeet | Meeting started Wed Mar 25 14:01:55 2026 UTC and is due to finish in 60 minutes. The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| opendevmeet | The meeting name has been set to 'cinder' | 14:01 |
| rosmaita | #topic roll call | 14:02 |
| jbernard | o/ | 14:02 |
| raghavendrat | hi | 14:02 |
| tosky | o/ | 14:02 |
| jbernard | #link https://etherpad.opendev.org/p/cinder-gazpacho-meetings | 14:02 |
| rosmaita | #chair jbernard | 14:02 |
| opendevmeet | Current chairs: jbernard rosmaita | 14:02 |
| jbernard | rosmaita: thanks | 14:02 |
| hvlcchao1 | o/ | 14:02 |
| whoami-rajat | hi | 14:03 |
| VolodymyrBoiko[m] | o/ | 14:04 |
| jbernard | welcome everyone | 14:05 |
| jbernard | this is the final gazpacho meeting, next week will be the opening of hibiscus | 14:05 |
| jbernard | #link https://etherpad.opendev.org/p/cinder-hibiscus-meetings | 14:05 |
| jbernard | we are preparing the final RC | 14:06 |
| jbernard | it is due Thursday | 14:06 |
| jbernard | rosmaita: did you add the powerflex notes? | 14:07 |
| rosmaita | yes | 14:07 |
| jbernard | take it away | 14:07 |
| rosmaita | i think we can't include these in the RC because os-brick is frozen | 14:07 |
| rosmaita | and i think the cinder patch requires the os-brick change | 14:07 |
| jbernard | can we roll them both into rc2? | 14:08 |
| rosmaita | no, can't release a new os-brick | 14:08 |
| jbernard | or post release backport? | 14:08 |
| rosmaita | yes, we can backport them after release | 14:09 |
| rosmaita | those were the only changes proposed for stable/2026.1 | 14:09 |
| rosmaita | so unless someone speaks up now, looks like we will use RC1 for the release | 14:09 |
| rosmaita | unless we need to revert a change | 14:10 |
| rosmaita | which we will discuss later | 14:10 |
| chembervint | hi, I'm for the first time on the meeting in this channel. I've made a small but important patch to os-brick, which is needed to fix cinder-backup with mutipath devices. when it could be reviewd? thanks you in advance! | 14:12 |
| chembervint | https://review.opendev.org/c/openstack/os-brick/+/982067 | 14:12 |
| jbernard | so to be clear, are there any other bugs that you might be working on that haven't been proposed to stable 2026.1 yet? | 14:12 |
| rosmaita | "you" meaning everyone present | 14:13 |
| jbernard | yup | 14:13 |
| jbernard | we can start with topics | 14:13 |
| jbernard | #topic cinderclient gates | 14:14 |
| jbernard | #link https://review.opendev.org/q/topic:%22periodic-cinder%22 | 14:14 |
| rosmaita | you all may remember some unpleasantness just before cinderclient release this cycle | 14:14 |
| rosmaita | to avert that next time, we decided to add periodic jobs to make sure the gates are not blocked | 14:15 |
| rosmaita | so, those are the patches | 14:15 |
| rosmaita | i just need some quick reviews and approval | 14:15 |
| rosmaita | because we will need to revise the zuul.yaml in all cinder projects for the hibiscus supported pythons soon-ish | 14:16 |
| rosmaita | that's all | 14:16 |
| jbernard | will review | 14:17 |
| erlon | \o | 14:17 |
| rosmaita | thanks | 14:17 |
| Anoop_Shukla | \o | 14:18 |
| kaisers | o/ | 14:18 |
| jbernard | #topic unit test job failures due to cinder.tests.unit.volume.drivers.hpe.test_hpe3par.TestHPE3PARDriverBase.test_thread_local_sessions_are_isolated (rosmaita) | 14:18 |
| rosmaita | yeah, so this is what i was talking about when i said maybe we need to revert a change from gazpacho | 14:19 |
| rosmaita | it's an intermittent failure | 14:19 |
| raghavendrat | hi | 14:19 |
| jbernard | how intermittent? | 14:19 |
| rosmaita | i've seen it mostly on python 3.13 | 14:20 |
| jbernard | do you have a sense? | 14:20 |
| rosmaita | well, it's happened on a few patches i was reviewing | 14:20 |
| eharney | i saw it on one of the eventlet removal patches too | 14:20 |
| rosmaita | but it also sometimes passes | 14:20 |
| jbernard | i lean towards reverting | 14:21 |
| rosmaita | eharney: do you remember if it was py313 or another python? | 14:21 |
| eharney | not sure, i can probably look | 14:21 |
| raghavendrat | revert meaning: reverting just that single UT function ? | 14:21 |
| rosmaita | raghavendrat: i'm not sure | 14:21 |
| rosmaita | it's not clear whether it's a test artefact, or whether the functionality is hosed | 14:22 |
| rosmaita | "hosed" == "unreliable" | 14:22 |
| raghavendrat | i just wish to elaborate | 14:22 |
| rosmaita | sure, please go ahead | 14:22 |
| raghavendrat | that UT function was added in: https://review.opendev.org/c/openstack/cinder/+/976596 | 14:23 |
| raghavendrat | this patch was merged on March 09th | 14:23 |
| raghavendrat | or March 10th early morning | 14:24 |
| raghavendrat | this patch is important for HPE; because it addresses hot customer fix. | 14:24 |
| erlon | doenst having a driver creating threads breaks the cinder driver model? | 14:25 |
| raghavendrat | my humble request would be: in case revert has to be done; then is it possible to submit a patch that removes just that UT function please | 14:25 |
| erlon | I haven't reviewed that patch, but I haven't seen that kind of implementation in drivers elsewhere | 14:26 |
| eharney | erlon: the driver is actually creating greenthreads IIRC | 14:26 |
| erlon | Can those continue running after the function call returns to the volume manager? | 14:27 |
| eharney | sorry, it isn't creating them, it's using the ones provided by the cinder, but is a bit different in that it's using (green)thread-local storage | 14:29 |
| raghavendrat | we have performed extensive testing over a period of 1 month; and the functionality is working fine | 14:30 |
| rosmaita | raghavendrat: so you are inclined to think it is a test-only problem? | 14:31 |
| jbernard | where is the flaw in the test? | 14:32 |
| raghavendrat | yes rosmaita: | 14:32 |
| erlon | raghavendrat: our concern here is that this model might be breaking something else on Cinder. It might be just unit tests though, but it might be related to this threads handling that's not usually used in drivers | 14:32 |
| eharney | do we know if the failure looks the same each time? | 14:33 |
| rosmaita | anecdotally, i have seen the mismatch error for line 7228 | 14:34 |
| rosmaita | a few times | 14:34 |
| eharney | i see a few in the build history that look exactly like https://zuul.opendev.org/t/openstack/build/057f9f567ccb4a15b4715d66e973720c | 14:34 |
| rosmaita | but i don't remember whether it's always the same mismatch values or not | 14:34 |
| erlon | raghavendrat: if you spin a unit test, then the unit tests calls a function and this function creates a thread, the test might finish before the function returns | 14:35 |
| erlon | see: | 14:38 |
| erlon | Traceback (most recent call last): | 14:38 |
| erlon | File "/usr/lib/python3.13/threading.py", line 1385, in __del__ | 14:38 |
| erlon | TypeError: 'NoneType' object does not support the context manager protocol | 14:38 |
| erlon | the threading termination code is ending after the unit tests finishes, hence the intermitent behaviour. | 14:38 |
| erlon | I don't think this (having a thread) would interfere with the rest of the cinder workings, but is a practice that should be avoided. I believe that removing only the test (but leaving the code untested?) should be ok | 14:40 |
| rosmaita | ok, so raghavendrat can you file a bug for this issue | 14:41 |
| rosmaita | and then put up a patch to skip test citing the bug | 14:41 |
| raghavendrat | ok, i will submit a launchpad bug | 14:41 |
| rosmaita | and then tell us in cinder channel | 14:41 |
| rosmaita | and then once it merges to master, please backport to stable/2026.1 | 14:42 |
| rosmaita | and then we will merge it into gazpacho and cut a new RC-2 | 14:42 |
| raghavendrat | sure, i will inform | 14:42 |
| rosmaita | ok, great | 14:42 |
| rosmaita | thanks erlon, eharney, and raghavendrat | 14:42 |
| raghavendrat | thank you all | 14:43 |
| jbernard | #topic ogging in drivers (rosmaita) | 14:43 |
| rosmaita | yeah, i just can't keep my mouth shut | 14:43 |
| jbernard | :) | 14:43 |
| rosmaita | this is something i noticed in doing some reviews | 14:43 |
| rosmaita | just want to remind driver maintainers | 14:43 |
| rosmaita | we understand that you want the debug logs to be informative | 14:44 |
| rosmaita | but remember that people will file security bugs if passwords show up in the logs | 14:44 |
| erlon | do you have the links | 14:44 |
| Anoop_Shukla | Are there any specific examples? | 14:44 |
| rosmaita | sure, on the agenda | 14:45 |
| rosmaita | #link https://review.opendev.org/c/openstack/os-brick/+/977447 | 14:45 |
| Anoop_Shukla | Do we have any guidelines on logging for Cinder core and drivers today? | 14:46 |
| rosmaita | key thing is if you are logging a dict of info, use the oslo_utils.strutils.mask_dict_password() function | 14:46 |
| rosmaita | Anoop_Shukla: probably in the driver checklist | 14:46 |
| rosmaita | but Anoop_Shukla that is is good point, if you don't see it somewhere obvious, let us know or put up a docs patch | 14:47 |
| Anoop_Shukla | https://docs.openstack.org/cinder/latest/contributor/new_driver_checklist.html | 14:47 |
| jbernard | yeah, we should have a permalink to point to for that | 14:48 |
| Anoop_Shukla | May be we can improve the checklist and talk about logging do’s and donts. | 14:48 |
| erlon | wouldnt be nice if LOG.debug() already did that for all logs? There will be always cases like that that is hard to tell what's inside some data structures | 14:48 |
| rosmaita | Anoop_Shukla: that would be a good topic for a PTG session, to do a review and brainstorm additions, if you want to propose it | 14:49 |
| Anoop_Shukla | I can take it up. Thanks for the suggestion @rosmaita | 14:49 |
| rosmaita | Anoop_Shukla: ++ | 14:50 |
| Anoop_Shukla | On that topic..do we have the PTG etherpad already available? | 14:50 |
| erlon | It shouldn't be hard to wrap around it, check if the parameters are dictionaries, and mask them. But that is a oslo thing I guess | 14:50 |
| jbernard | https://etherpad.opendev.org/p/cinder-hibiscus-ptg | 14:51 |
| jbernard | Anoop_Shukla: ^ | 14:51 |
| Anoop_Shukla | Great thanks! | 14:51 |
| jbernard | #topic open discussion | 14:51 |
| rosmaita | erlon: might be worth looking into, not sure why it didn't get integrated directly into oslo.logging | 14:52 |
| Anoop_Shukla | One topic that came up sometime back was about guidelines on how backup vendors use Cinder backup workflows.. | 14:53 |
| erlon | rosmaita, what would be the teams to bring it? I mean specifically for the ptg discussions, we might want to take that to a broader public | 14:53 |
| rosmaita | i think there should be some kind of oslo ptg session | 14:54 |
| rosmaita | but i don't know for sure | 14:54 |
| Anoop_Shukla | Today there are no standard guidelines for applications like commvault, Veeam, Rubrik, Veritas | 14:54 |
| Anoop_Shukla | In general these applications use Cinder workflows and Nova to perform their backups | 14:55 |
| Anoop_Shukla | For Cinder drivers there is no way of knowing if its a Cinder Snapshot or a backup workflows | 14:55 |
| Anoop_Shukla | Does oslo have ptg session? | 14:56 |
| rosmaita | well, there's not an oslo-specific etherpad here: https://ptg.opendev.org/etherpads.html | 14:57 |
| raghavendrat | wish to ask for review: we have addressed review comments on below patch. | 14:58 |
| raghavendrat | Earlier there was one +2, but it went away after code was updated to address review comment. | 14:58 |
| Anoop_Shukla | May be I will put it as a topic to brainstorm from Cinder perspective and we can take it from there. | 14:58 |
| raghavendrat | Request to please re-visit & approve (if all is good). Thanks. | 14:58 |
| raghavendrat | HPE: Improved handling of QOS for Alletra MP | 14:59 |
| raghavendrat | https://review.opendev.org/c/openstack/cinder/+/979115 | 14:59 |
| rosmaita | might be worth asking on the mailing list if oslo is going to have a ptg presence | 14:59 |
| rosmaita | raghavendrat: i will re-review | 15:00 |
| raghavendrat | thank you | 15:00 |
| Anoop_Shukla | Review request for the patch: https://review.opendev.org/c/openstack/cinder/+/977911 NetApp CI is successful. I have 1 +1 from agalica - thank you for that :) | 15:01 |
| Anoop_Shukla | I had another question to folks here - Does anyone have any experience migrating from ZUUL to Jenkins for CI? | 15:03 |
| jbernard | i know there is knowledge within the team, but i cannot recall who mentioned it last | 15:04 |
| Anoop_Shukla | Yes..someone did mention in mid-cycle about there experiences.. | 15:04 |
| Anoop_Shukla | Please PM me if anyone can help :) | 15:04 |
| jbernard | i recall jenkins being reported as not-so-bad to setup | 15:04 |
| Anoop_Shukla | We are looking to upgrade ZUUL on our CI, but also weighing in the cost to move to Jenkins and its pros/cons | 15:05 |
| jbernard | last call before we wrap up | 15:08 |
| jbernard | thanks everyone | 15:10 |
| jbernard | #endmeeting | 15:10 |
| opendevmeet | Meeting ended Wed Mar 25 15:10:40 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:10 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/cinder/2026/cinder.2026-03-25-14.01.html | 15:10 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/cinder/2026/cinder.2026-03-25-14.01.txt | 15:10 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/cinder/2026/cinder.2026-03-25-14.01.log.html | 15:10 |
| opendevreview | Eric Harney proposed openstack/cinder master: Enable threads mode in cinder-volume https://review.opendev.org/c/openstack/cinder/+/954650 | 15:16 |
| opendevreview | Eric Harney proposed openstack/cinder master: Replace _add_to_threadpool with native threads https://review.opendev.org/c/openstack/cinder/+/959383 | 15:16 |
| opendevreview | Eric Harney proposed openstack/cinder master: cinder/monkey_patch.py for threading mode https://review.opendev.org/c/openstack/cinder/+/970332 | 15:16 |
| opendevreview | Raghavendra Tilay proposed openstack/cinder master: HPE: Remove UT that fails intermittently https://review.opendev.org/c/openstack/cinder/+/982129 | 16:14 |
| raghavendrat | hi jbernard: rosmaita: as discussed in weekly meeting, created launchpad bug | 16:16 |
| raghavendrat | https://bugs.launchpad.net/cinder/+bug/2146339 | 16:16 |
| raghavendrat | The UT test_thread_local_sessions_are_isolated fails intermittently & causes zuul to fail | 16:16 |
| raghavendrat | and submitted below patch to remove UT function: | 16:16 |
| raghavendrat | https://review.opendev.org/c/openstack/cinder/+/982129 | 16:17 |
| cardoe | hello. just wanted to ask about https://review.opendev.org/c/openstack/cinder/+/962085 it was previously on the review etherpad. | 18:02 |
| jbernard | cardoe: i missed circling back on that one, my appologies, looks okay just needs another +2 | 18:11 |
| opendevreview | Merged openstack/cinder master: HPE: Improved handling of QOS for Alletra MP https://review.opendev.org/c/openstack/cinder/+/979115 | 18:43 |
| opendevreview | Jon Bernard proposed openstack/os-brick master: Remove eventlet from tests and requirements https://review.opendev.org/c/openstack/os-brick/+/973184 | 18:53 |
| jbernard | erlon: did you Niklas' last comment on https://review.opendev.org/c/openstack/cinder/+/484729 ? | 19:01 |
| jbernard | erlon: curious if you have any followup | 19:01 |
| opendevreview | Jon Bernard proposed openstack/cinder master: Add a non-voting job to test eventlet removal https://review.opendev.org/c/openstack/cinder/+/982144 | 19:12 |
| opendevreview | Eric Harney proposed openstack/cinder master: Release notes: move hitachi-vsp-one note https://review.opendev.org/c/openstack/cinder/+/982148 | 19:20 |
| eharney | jbernard: fyi we landed a releasenote in the wrong dir ^ | 19:20 |
| jbernard | eharney: i was just looking at that, i dont see it in my tree... i am losing my mind | 19:23 |
| eharney | jbernard: it's hard to see because it has a leading space | 19:23 |
| eharney | https://opendev.org/openstack/cinder/src/branch/master/%20releasenotes/notes | 19:23 |
| jbernard | ooooh | 19:25 |
| jbernard | wow, i missed that | 19:25 |
| jbernard | thanks | 19:25 |
| eharney | i looked at the original review in gerrit and it's almost invisible there. lol | 19:25 |
| opendevreview | Eric Harney proposed openstack/cinder master: Enable threads mode in cinder-volume https://review.opendev.org/c/openstack/cinder/+/954650 | 19:36 |
| opendevreview | Eric Harney proposed openstack/cinder master: Replace _add_to_threadpool with native threads https://review.opendev.org/c/openstack/cinder/+/959383 | 19:36 |
| opendevreview | Eric Harney proposed openstack/cinder master: cinder/monkey_patch.py for threading mode https://review.opendev.org/c/openstack/cinder/+/970332 | 19:36 |
| opendevreview | Anthony Galica proposed openstack/cinder stable/2024.2: Hitachi: Add support for Hitachi VSP One B20 https://review.opendev.org/c/openstack/cinder/+/982164 | 20:43 |
| agalica | jbernard: oof, I guess that's my error. Need me to do anything about that? | 21:13 |
| agalica | (other than mind my extra spaces next time ~boing~ | 21:14 |
| agalica | oh, looks like Eric already created a patch set | 21:14 |
| agalica | and you already workflowed it | 21:15 |
| agalica | never mind me | 21:15 |
| opendevreview | Merged openstack/cinder master: Release notes: move hitachi-vsp-one note https://review.opendev.org/c/openstack/cinder/+/982148 | 22:41 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!