Wednesday, 2022-06-15

opendevreviewYuehuiLei proposed openstack/cinderlib master: Update python testing as per zed cycle teting runtime
opendevreviewPeter Penchev proposed openstack/cinder master: Send the correct location URI to the Glance v2 API
opendevreviewGorka Eguileor proposed openstack/cinder master: Revert "Cleanup code duplication in cinder.cmd.backup module"
opendevreviewGorka Eguileor proposed openstack/cinder master: Doc: Mention Skip Level upgrades
*** tosky_ is now known as tosky09:42
Roamer`at the May 25th meeting I brought up an issue with Cinder refusing to clone a volume containing an image because (due to historical implementation details) it was reported as being in a different pool. I believe there was a consensus there that this would be handled best by adding a new driver capability; does "clone_across_pools" sound good as a capability name? ...maybe I should have asked 09:48
Roamer`here before I registered :)09:48
*** dviroel|afk is now known as dviroel11:28
*** tosky_ is now known as tosky11:55
*** tosky__ is now known as tosky13:17
*** tosky is now known as Guest216913:29
*** tosky__ is now known as tosky13:29
opendevreviewPeter Penchev proposed openstack/cinder master: Send the correct location URI to the Glance v2 API
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool: drop _attach_volume() and _detach_volume()
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool driver: remove the obsolete backup_volume()
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool: create_cloned_volume() improvements
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool: really detach after copy to/from image.
*** tosky__ is now known as tosky13:58
whoami-rajatCinder meeting in #openstack-meeting-alt at 1400 UTC13:58
whoami-rajatjungleboyj rosmaita smcginnis tosky whoami-rajat m5z e0ne geguileo eharney walshh_ jbernard sfernand enriquetaso hemna fabiooliveira yuval tobias-urdin13:58
opendevreviewMerged openstack/os-brick master: Fix Unit Test failures due to host's ANA support
opendevreviewSofia Enriquez proposed openstack/os-brick stable/yoga: Fix encryption symlink issues
opendevreviewJean Pierre Roquesalane proposed openstack/cinder master: WIP: Add Cinder NFS driver for Dell EMC PowerStore
HappyStacker1Hey cinder community14:54
HappyStacker1Can someone explain to me why this patch doesn't make any progress? We need a workflow +114:54
enriquetasoHappyStacker1 reviewed!14:57
HappyStacker1we have some other in the same case15:01
HappyStacker1like this one :
HappyStacker1and this one
HappyStacker1and the last one for now
HappyStacker1all need workflow +115:02
enriquetasoOK, I'll review after the cinder bug meeting HappyStacker1 15:03
HappyStacker1swwet! thanks enriquetaso15:03
enriquetaso#startmeeting cinder_bs15:03
opendevmeetMeeting started Wed Jun 15 15:03:56 2022 UTC and is due to finish in 60 minutes.  The chair is enriquetaso. Information about MeetBot at
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:03
opendevmeetThe meeting name has been set to 'cinder_bs'15:03
enriquetasoWelcome back to the Cinder Bug Meeting.15:04
rosmaitaHappyStacker1: can't tell who you are from your nick, have you been reviewing patches yourself?15:04
enriquetasogood question :P15:04
rosmaita(i mean other peoples' patches)15:04
enriquetasoWe have a few bugs today, so let us get started.15:04
enriquetasoFull list of bugs:15:04
HappyStacker1thats' jp from dell rosmalta15:04
rosmaitaok, thanks15:05
enriquetaso#topic [NFS/NetApp] No feedback from Nova when extending attached Volumes15:05
enriquetasoIn summary, n-cpu is able to extend an attached NFS volume by itself, but c-vol is preventing this from happening when it tries to perform `qemu-img resize`. 15:05
enriquetasoThis seems to be a problem not just for the NFS driver, since drivers using os-brick may fail, and Nova also handles extending the LUKS structure of attached encrypted volumes, which may also fail.15:06
enriquetasoThis bug affects Quobyte, NetApp Cinder NFS drivers as far as I know15:06
enriquetasoNo fix proposed to master yet. 15:06
rosmaita"Lucio Seki, who first worked on [2] and [3], collected some thoughts on this in an etherpad [4] together with other potential issues of the changes."15:07
rosmaitai thought this sounded familiar15:07
enriquetasoyes, Lucio work on this as well 15:08
enriquetasoEtherpad link 15:09
enriquetasoThe original bug is #link
enriquetasoAs far as I understood the new bug is for a related issue but we are going to track it on 197829415:10
enriquetasoMaybe kgube would like to add something to the bug report :) 15:11
enriquetasoCurrently, the solution we have for this is to disallow extension of attached volumes for NFS & Quobyte drivers15:12
enriquetasoOK, moving on15:13
rosmaitayeah, that was a short term "fix"15:13
enriquetasodo you think the second bug should be close as duplicated? rosmaita 15:13
rosmaitanot sure15:14
enriquetasoOK, i'll re review it and double check but looks like a separate bug 15:14
rosmaitai agree15:15
enriquetaso#topic Fails to extend in-use (non LUKS v1) encrypted volumes15:15
enriquetasoSummary: Patch fixing bug #1861071 resolved the issue of extending LUKS v1 volumes when nova connects them via libvirt instead of through os-brick, but nova side still fails to extend in-use volumes when they don't go through libvirt (i.e., LUKS v2).15:15
enriquetasogeguileo, worked on the nova part of the bug that appears as Fix Released.15:15
enriquetasonow we have a cinder patch for this 15:16
enriquetasoFix proposed to master15:16
enriquetaso#link 15:16
enriquetasoOK, feel free to review ^15:18
enriquetasomoving on 15:18
enriquetaso#topic Old-style Glance location URI sent when uploading a volume to an image 15:18
*** dviroel is now known as dviroel|lunch15:18
enriquetasowhoami-rajat, discussed it with the reporter 15:19
enriquetasoFix proposed to master15:19
enriquetasoSo, feel free to review it as well ^15:20
rosmaitathat looks ok to me, i thought glance did a lazy-migrate of old-style URIs, but maybe that was before you could have multiple cinder glance_stores15:21
enriquetasogood to know15:22
rosmaitathe only thing i want to check is what kind of validation we have around the store_id in a volume-type extra-specs15:22
enriquetasomm, not sure about that15:22
enriquetasomaybe we need some coverage there 15:22
rosmaitai think we have something, just can't remember what15:23
rosmaitai will look into it and leave a comment on the patch15:23
enriquetasothank you rosmaita !15:23
enriquetasoOK, last two bugs:15:23
enriquetaso#topic cinder-backup context.message_action is None on errors15:23
enriquetasoNot sure about this one 15:23
enriquetasoIf a driver raises an exception another exception is thrown from because AFAICT the action is None. (also checked above for setting action_id.15:24
enriquetasolooks like valid and the reporter added some logs to the case 15:25
enriquetasoI'll try to reproduce and try the possible solutions15:26
enriquetaso#action (enriquetaso): try the fixes for ^15:26
enriquetasoOK, last one! I'm running out of time15:26
enriquetaso#topic [IBM Storwize] Optimize SSH calls in create replicated volume15:27
enriquetaso#link 15:27
enriquetasoNo fix proposed to master yet but probably IBM will proposed a fix :)15:27
enriquetasoOK, that's all I have for today's meeting15:27
enriquetasothanks rosmaita15:28
enriquetasothanks everyone15:28
opendevmeetMeeting ended Wed Jun 15 15:28:24 2022 UTC.  Information about MeetBot at . (v 0.1.4)15:28
opendevmeetMinutes (text):
rosmaitathanks enriquetaso!15:28
whoami-rajatenriquetaso, cool, thanks for taking a look15:48
*** dviroel|lunch is now known as dviroel16:15
opendevreviewGorka Eguileor proposed openstack/cinderlib master: Update Ceph version on CI job
opendevreviewMerged openstack/cinder master: PowerMax Driver - Improve error handling around deletes
opendevreviewGorka Eguileor proposed openstack/cinder master: Fix cinderlib RBD keyring issue
dansmithgeguileo: do you know about performance.json 18:34
geguileodansmith: no, no idea18:35
dansmithfrom your "reduce memory footprint" run18:35
geguileoooooooooooh, nice!!!18:36
geguileodansmith: I did a bunch of local test runs to ensure that this was actually better18:36
geguileobut it's great to be able to reference something at the gate18:36
dansmithgeguileo: we've been generating that for a few weeks now,18:36
dansmithso you can compare that run to other runs with opensearch18:37
dansmithwe're ideally going to get that graphed at some point, but at the moment it's just .. that18:37
dansmithI have a little tool locally which you can use to get this:
dansmithi.e. the top offenders on memory18:38
dansmithtakes an url so you can just run it against jobs pretty easy.. --mem is the only thing it does right now, but that's the important one18:39
geguileodansmith: thanks!!18:42
HappyStacker1enriquetaso if you can take some time to review our patches, it'd be great! thanks in advance18:45
*** dviroel is now known as dviroel|afk18:47
opendevreviewJean Pierre Roquesalane proposed openstack/cinder master: PowerMax Driver -- Add support for Unisphere for PowerMax 10.0
geguileodansmith: mmmm, I don't know if it's my patch's fault, but n-cpu consumes a lot more memory on that run...18:51
dansmithoh yeah ~660mb, that's a lot18:52
dansmiththings vary per service quite a bit from run to run, but that's way higher than I've seen18:52
geguileoit was 390 on another run without that patch18:53
dansmithmaybe a couple rechecks are in order18:53
geguileoI'll investigate it tomorrow18:53
dansmithso, there's one more thing:18:53
dansmiththis adds something that compares two files and tries to highlight surges18:53
dansmithit's a lot harder for even countable things than I was expecting, not sure where all the variability comes from18:53
dansmithhowever, you could use that there to compare multiple runs maybe18:54
geguileodansmith: from what I've been seeing the variability is caused by memory fragmentation18:55
geguileoboth in the malloc arenas as well as in the Python internal arenas18:56
dansmithin memory yeah, I mean countable things like api requests and db accesses18:56
dansmithgeguileo: we also cycle workers on some of the services, so we'll inflate, then cycle and drop a bunch and the memory at the end of a tempest run depends on how many times we did that18:56
dansmithuwsgi has a max_requests sort of thing I think18:56
geguileoyeah, in my tests for cinder-backup the high water mark memory issue gets fixed18:57
dansmithyeah that's awesome because that sucker was big :)18:57
geguileoso I did 10 concurrent processes each doing 200 backup then restore cycles18:57
dansmithpresumably c-bak must have been reading decent-sized chunks of data into memory to even cause that though right?18:58
geguileoand without the patch the high water memory mark was 2GB (it would not go down from there even idle)18:58
geguileowith the patch it went down immediately to 140MB18:58
dansmithyeah, that's awesome18:58
geguileodansmith: yes, reading chunks and they don't get immediately garbage collected18:58
dansmithnow if you can just avoid making nova huge :P18:59
geguileorofl rofl18:59
dansmithright, okay18:59
geguileoI'll look into it tomorrow18:59
dansmithcool, thanks for doing this18:59
dansmithnot enough people care about our waistline :)18:59
geguileobecause there's a second run on that patch going on now, so I'll be able to compare both18:59
geguileothanks for all the good tools you've just given me19:00
dansmithnp :)19:00
enriquetasorosmaita, geguileo feel free to review this backport :)20:34
enriquetasowe need it in wallaby 20:34
opendevreviewSofia Enriquez proposed openstack/cinder master: RBD: Flattening of child volumes during deletion
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool: really detach after copy to/from image.
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool driver: remove the obsolete backup_volume()
opendevreviewPeter Penchev proposed openstack/cinder master: StorPool: create_cloned_volume() improvements

Generated by 2.17.3 by Marius Gedminas - find it at!