14:01:47 <jbernard> #startmeeting cinder
14:01:47 <opendevmeet> Meeting started Wed Oct  9 14:01:47 2024 UTC and is due to finish in 60 minutes.  The chair is jbernard. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:47 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:47 <opendevmeet> The meeting name has been set to 'cinder'
14:01:59 <jbernard> #topic roll call
14:02:05 <Sai> o/
14:02:07 <jbernard> o/
14:02:15 <jbernard> #link https://etherpad.opendev.org/p/cinder-epoxy-meetings
14:02:17 <eharney> o7
14:02:18 <jungleboyj> o/
14:02:20 <jbernard> ^ new for epoxy
14:03:11 <whoami-rajat> hi
14:03:26 <ganso> o/
14:05:38 <jbernard> ok
14:05:47 <jbernard> #topic annoucements
14:05:52 <jbernard> welcome everyone
14:06:04 <jbernard> we are about to start the epoxy cycle
14:06:30 <jbernard> the meeting minutes/outline link has been updated
14:06:34 <jbernard> #link https://etherpad.opendev.org/p/cinder-epoxy-meetings
14:07:01 <jbernard> here is the project schedule for this cycle
14:07:04 <jbernard> #link https://releases.openstack.org/epoxy/schedule.html
14:07:32 <jbernard> in 2 weeks we have our virtual PTG
14:07:43 <jbernard> the etherpad for this to collect ideas is
14:07:47 <whoami-rajat> congratulations jbernard for being re-elected as PTL!
14:07:50 <jbernard> #link https://etherpad.opendev.org/p/epoxy-ptg-cinder
14:07:57 <jbernard> whoami-rajat: thanks!
14:08:33 <jbernard> it was a tight race :)
14:08:41 <whoami-rajat> :D
14:08:54 <whoami-rajat> someone on the ML wanted to discuss a new driver and a new interface for drivers
14:09:05 <whoami-rajat> i redirected them to the epoxy PTG etherpad
14:09:16 <jbernard> we have a few cross-project interactions coming up for the ptg
14:09:19 <jbernard> whoami-rajat: perfect
14:09:21 <jungleboyj> jbernard:  Thank you for taking it on again!
14:09:24 <whoami-rajat> i think it would be a good idea to mention the date and time for cinder PTG so people can schedule things accordingly
14:09:36 <whoami-rajat> i mean on the same etherpad
14:09:44 <jungleboyj> ++
14:10:03 <jbernard> will do, i need to get the meta information on there
14:10:18 <whoami-rajat> +1
14:10:20 <jbernard> and also the "room" reservation and bot manipulation stuff
14:10:56 <jbernard> #action jbernard, update ptg pad, foundation ptg room stuff
14:11:58 <jbernard> we've been invited to discuss the horizon feature gap list
14:12:01 <jbernard> at the ptg
14:12:05 <jbernard> #link https://etherpad.opendev.org/p/horizon-feature-gap
14:13:28 <jbernard> with respect to the ptg schedule, we generally have from tuesday to friday
14:13:47 <jbernard> last cycle we met starting at 13:00 UTC
14:13:56 <jbernard> for a couple of hours
14:14:12 <jbernard> and I think friday turned into a bug/code review day
14:14:43 <jbernard> this will similar, I do have an appointment tuesday morning, so I'll either shift the start time slightly or jump on a bit late that day
14:15:46 <jbernard> I'm going to try to have a tentive plan to facilitate a discussion on the removal of eventlet (yes, it's that time)
14:16:19 <jbernard> and I'm working on trying pick up on Pete's backup encryption work from last cycle, so that doesn't get left behind
14:17:31 <jbernard> it's worth noting that Gorka has changed groups within redhat recently, so his availability will be very limited (if more than none) going forward
14:18:13 <jbernard> Gorka has made many significant contributions to the project, his work, knowledge, and guidance will be very much missed
14:18:46 <whoami-rajat> ++
14:18:58 <whoami-rajat> we all will miss him this time
14:19:13 <jbernard> and it also means that we need to keep brick in mind as we're allocating time for patch review and maintaince
14:19:35 <jbernard> (more than usual perhaps) as there is one less pair of eyes there now
14:19:40 <tobias-urdin> is https://etherpad.opendev.org/p/epoxy-ptg-cinder the official ptg pad? https://etherpad.opendev.org/p/oct2024-ptg-cinder is linked on ptg.opendev.org and has some points already added
14:20:04 <jbernard> tobias-urdin: ahh, thank you very much for pointing that out
14:20:27 <jbernard> our working version is https://etherpad.opendev.org/p/epoxy-ptg-cinder (this is the same format as last cycle)
14:20:40 <whoami-rajat> wow, there are a bunch of topics added there
14:20:45 <jbernard> ill update the opendev link and combine the contents
14:21:03 <jbernard> good catch! i was worried we didn't have much to discuss lol
14:21:07 <tobias-urdin> cool, thanks!
14:21:22 * jbernard facepalm
14:22:54 <jbernard> that's all for annoucements
14:23:09 <jbernard> #topic bug discussion
14:23:14 <jbernard> #link https://bugs.launchpad.net/cinder/+bug/2083061
14:23:19 <jbernard> ganso: ^ you're up
14:23:27 <ganso> jbernard: thanks!
14:23:41 <ganso> hello! so we've started discussing that bug in the channel 2 weeks ago
14:23:47 <ganso> and we decided it was better to discuss in a meeting
14:24:14 <ganso> basically we need to evaluate whether it is better to patch the stable branches directly with a 1-liner retry, or backport a bigger patch
14:24:31 <ganso> eharney has more knowledge of the big patch
14:24:56 <ganso> which is this one: https://review.opendev.org/c/openstack/cinder/+/835384
14:25:31 <ganso> it adds a new config option, and someone had mentioned that apparently it has some upgrade impact
14:26:53 <ganso> the big patch fixes more issues than the one I am discussing though. To be honest, I am more interested in the specific issue. If we conclude that the impact is low is may be a better option to backport the big one and fix more issues in the stable branches, that is good as well. I am just worried about the impact
14:26:56 <ganso> and risk
14:27:01 <jbernard> yes, generally we try to avoid backporting things classified as features or containing config changes if possible
14:27:02 <eharney> it's not obvious to me why adding a retry on ImageNotFound works
14:27:36 <ganso> eharney: it is a race condition. When deleting the parent and the child at the same time, the child checks for the parent
14:27:55 <ganso> but the parent gets renamed upon deletion, so there is a window of opportunity where the code doesn't find the parent
14:28:00 <jbernard> do we have a link to the one-liner? is that posted?
14:28:11 <eharney> ah hah, and we don't have a lock or anything to serialize multiple delete operations that could affect each other
14:28:12 <ganso> eharney: if the code is retried, then it finds it
14:28:36 <ganso> eharney: I don't think we can have a good lock for this given it can happen in a HA environment
14:28:48 <eharney> yeah, i wasn't proposing we should have one, better to not
14:28:50 <ganso> eharney: like, 3 cinder-volume services running in parallel processing the deletes at the same time
14:29:08 <ganso> jbernard: no it is not proposed, but it is described in the LP
14:30:13 <ganso> jbernard: it is item (b)
14:30:19 <jbernard> how successful was the proper patch (https://review.opendev.org/c/openstack/cinder/+/835384)? have we seen any regressions or need for followup changes?
14:30:59 <eharney> there is a follow-up to add an option for those who want to stick w/ the old behavior
14:31:32 <eharney> (it fell pretty far back into my "review this eventually" queue but seems fine in concept)
14:32:02 <ganso> eharney: hmmm this complicates things a bit if there are advantages to the old behavior, and a follow-up that we might want to backport as well if we do this one
14:33:08 <eharney> i think we don't really want to support the old behavior, the submitter was concerned about expansion of storage usage etc in certain cases
14:33:27 <jbernard> my concern with a one-liner is that it wouldn't apply to master, it would be something we only carry for stable branches
14:33:57 <ganso> jbernard: yes, that's the big disadvantage
14:34:35 <whoami-rajat> the thing about the one-liner is it only focuses on one of the many issues that eharney has addressed in his patch, even if we go that route, we can have another bug report with another issue that needs patching in stable branches
14:34:54 <whoami-rajat> i feel it is best to address this the right way to avoid future issues in older/stable branches
14:35:29 <whoami-rajat> my only concern with the RBD fix patch is the introduction of the config options, if we have checked that the options have safe default values and won't affect upgrade, I'm up for backporting it
14:36:30 <jbernard> i tend to agree
14:36:41 <jbernard> eharney: what are your thoughts?
14:37:23 <eharney> well the big fix is already in caracal, are we just wanting it in bobcat or also antelope (right before it closes)?
14:37:32 <ganso> jbernard: my understanding from reading the patch is that the config option has a godo default and the upgrade impact is not from the config option. I don't know what the upgrade impact that was raised in the previous discussion is from
14:38:03 <ganso> eharney: upstream backports yes, but I'm interested in backporting it further downstream to yoga
14:38:25 <ganso> eharney: I've tested for conflicts cherry-picking it as far back as yoga, and yoga had some conflicts
14:39:33 <ganso> *only yoga had some conflicts
14:40:02 <eharney> the other upgrade impact is that clone v2 image support should be enabled on the cluster (minimum client compat mimic)
14:40:38 <eharney> it's not a hard requirement but is what we test and advise using
14:41:40 <jbernard> ganso: pre-antelope releases are unmaintained, they will require a different approach
14:42:01 <ganso> eharney: ok, mimic is quite old and we could assume it is quite unlikely someone is using mimic with yoga
14:42:12 <jbernard> ganso: assuming the patch can apply cleanly and there are no other techincal issues
14:43:38 <jbernard> i think we have agreement on backport up to antelope, at the least
14:43:50 <ganso> jbernard: it doesn't apply cleanly to yoga, but we typically don't push all downstream backports to unmaintained branches
14:44:23 <jbernard> ganso: unmaintained is under a different team for review and inclusion
14:45:13 <ganso> jbernard: hmmm maybe I am unfamiliar with the advantages of the unmaintained branches
14:45:37 <ganso> jbernard: AFAIK there isn't CI testing or tags anymore
14:46:19 <jbernard> ganso: that's correct, and the unmaintained group is not the cinder cores - so we cannot review and merge as we do for master and active stable branches
14:47:28 <jbernard> eharney: are you willing to post those backports?
14:47:36 <jbernard> eharney: for B and A
14:48:06 <whoami-rajat> wouldn't it be helpful if a non-core proposed the backport so cores can vote? just a suggestion
14:48:20 <jbernard> oh, that's clever ;)
14:48:32 <ganso> jbernard: yeah. Based on those facts they haven't provided any meaningful value (as far as I am aware) for delivering bugfixes to people still running old versions, such as yoga, so we backport downstream and test it downstream as much as we can
14:48:35 <eharney> not sure, i think we also have to look at devstack-plugin-ceph etc and see if the jobs will test it correctly w/o some changes
14:49:08 <jbernard> ganso: that may be where this one falls as well
14:49:12 <ganso> whoami-rajat: yea I would propose the backport. I am happy to drive it and make sure whatever we agree can be backported as far back. I would be a shame if I end up finding out that it doesn't work in Yoga though
14:49:42 <ganso> s/I would be a shame/it would be a shame
14:49:44 <jbernard> ok, lets get the backports up so that devstack-plugin-ceph can weigh in
14:49:57 <jbernard> and then we can circle back with data and make a final call
14:50:11 <ganso> jbernard: sounds good, thanks!
14:50:32 <jbernard> #action ganso to backport rbd flattening and report back in future meeting
14:51:01 <jbernard> #topic open discussion
14:51:38 <ganso> thanks everyone
14:51:47 <ganso> it was a good discussion to assess the risk
14:51:52 <whoami-rajat> thanks ganso for driving it!
14:51:58 <jbernard> yes, thanks ganso
14:55:00 <eharney> re: rbd i think we also have a change open somewhere to enable trash purge scheduling in CI which is needed for optimal testing
14:55:29 <jbernard> that's a good point
14:55:37 <eharney> https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/919539   (it needs a re-work)
14:55:56 <jbernard> ill summarize this all in the etherpad
14:56:01 <jbernard> thanks eharney
14:58:00 <jbernard> last call
14:59:05 <jbernard> ok, thanks everyone!
14:59:07 <jbernard> #endmeeting