Thursday, 2022-04-07

opendevreviewMounika Sreeram proposed openstack/cinder master: [SVf] Delete/Resize issue in reverse replication
*** bhagyashris_ is now known as bhagyashris05:52
rubasovwhoami-rajat: good morning, I'm back today, when you have time, please let me know how you would like to continue yesterday's debugging07:35
whoami-rajatrubasov, good morning, sure we can continue07:36
whoami-rajatnot sure where we left though07:36
whoami-rajatIIRC you're facing issues when saving multiple images?07:37
rubasovthat with two parallel image saves we still seem to have someting similar as in bug
rubasovthough the error message in my reproduction is a bit different, this: cinderclient.exceptions.BadRequest: Invalid volume: Volume e5ad5acd-c20f-4c9e-a6c5-0771af3056ac status must be available or in-use or downloading to reserve, but the current status is reserved. (HTTP 400)07:38
rubasovwe tested too, that parallel image creates do not trigger the same error07:39
rubasovthe glance image saved is backed by a multiattach volume type07:40
whoami-rajatstrange as we also have a retry mechanism for this case07:41
whoami-rajatdo you see the above errors multiple times in your g-api logs? ^^07:41
rubasovand we tried adding 'cinder_volume_type = lvmdriver-1-multiattach' to glance-api.conf but that does not seemed to have changed the behavior07:41
whoami-rajatrubasov, IIUC, after changing the conf file, you created a new image and trying to download THAT NEW IMAGE in parallel right?07:42
whoami-rajator are you trying that on an old image? (before the conf changes)07:42
rubasovI did not create a new image, but I'll try that now07:46
rubasovby the way I just double checked and the 'status must be available ...' error message is present only once in the logs, and then a few seconds later I have this: FileNotFoundError: [Errno 2] No such file or directory: '/opt/stack/data/glance/cache/incomplete/88d4a9ac-1529-4f3d-8b18-a9eef8a93155' -> '/opt/stack/data/glance/cache/88d4a9ac-1529-4f3d-8b18-a9eef8a93155'07:47
rubasovbut this was still with the old image07:47
whoami-rajatrubasov, sure, I assume you've the config option already changed, just create a new image and try downloading that in parallel?07:47
rubasovtesting it...07:48
whoami-rajatrubasov, hmm, that seems to be a glance specific issue, i think we should continue the discussion in glance channel07:48
whoami-rajatsadly i don't see the usual people in glance channel...07:49
lpetruthi, we'd need someone to take a look over this multipath resize bug fix: 07:52
rubasovI double checked that I left yesterday the cinder_volume_type set to a multiattach type, uploaded a new image (openstack image create --public --disk-format qcow2 --file ~/cirros-0.5.2-x86_64-disk.img image0) and downloading this image twice in parallel, one image save still fails, while having the same error message in glance logs: status must be available or in-use or downloading to reserve, 07:57
rubasovbut the current status is reserved07:57
whoami-rajatrubasov, can you do ``cinder list --all`` and and check if the volume containing the image exists? or just dump the output in a paste
rubasovwhoami-rajat: I believe the volume got created properly, please see here:
whoami-rajatrubasov, yep it did, I would require the glance logs for further debugging as I'm not sure what should be the reason, I tested this code with simultaneous request to create 10 volumes from same image and it worked fine08:13
whoami-rajatif you can provide logs for this particular request ID08:13
rubasovsure, will collect and share08:15
rubasovwhoami-rajat: 127 kilobytes proved too big for pastebin, so I added the logs as an attachment in LP here:
rubasovalso I'm doing one more thing to double check my bug reproduction: in the vm I'm using I previously built a stable/victoria devstack and now I'm throwing this vm away and building a fresh one from master to make sure devstack/pip could not leave some victoria dependencies behind09:28
rubasovwhoami-rajat: I built a master devstack in a fresh vm, in which I never built a victoria devstack before and the parallel image save still fails, so I think leftover dependencies cannot be the cause10:22
rubasovregarding your previous question about whether I created the image before or after the glance change, I realized that it may be relevant that I always tested with an image whose volume I retyped to a multiattach type for example: openstack volume set --type lvmdriver-1-multiattach "$( openstack volume list --all --user glance --name image-$( openstack image show cirros-0.5.2-x86_64-disk -f value 10:24
rubasov-c id ) -f value -c ID )"10:24
rubasovI'll be away for a bit , be back around 13:00 utc10:37
whoami-rajatrubasov, oh, the image volume is not supposed to be retyped... you just set the conf option in glance and it automatically creates the right multiattach type volume10:39
whoami-rajatto store the image10:39
rubasovI mean I retyped it before I knew about the glance config option, but not after I uploaded new image with the changed glance config10:40
rubasovsorry for the confusion10:40
whoami-rajatah ok10:41
opendevreviewRajat Dhasmana proposed openstack/cinder master: Update docstring for copy_image_to_volume method
opendevreviewRajat Dhasmana proposed openstack/cinder master: Add missing copy_image_to_encrypted_volume in interface
opendevreviewRajat Dhasmana proposed openstack/cinder master: Update docstring for copy_volume_to_image
yuvalGuys, how do we test flows related to reboot and abrupt reboot - tempest have something for that?10:57
opendevreviewBrian Rosmaita proposed openstack/cinder master: Add info about code coverage job
rosmaitadc_arun__kv: ^^11:11
*** dviroel|afk is now known as dviroel\11:13
*** dviroel\ is now known as dviroel11:13
opendevreviewHarsh Ailani proposed openstack/cinder master: [SVf]:Fix retype failure for replication volume-type
opendevreviewHarsh Ailani proposed openstack/cinder master: [SVf]:Fix retype failure for replication volume-type
opendevreviewAlexander Malashenko proposed openstack/cinder master: Add check if size exist in model_update and update size in model_update
whoami-rajatCourtesy reminder Cinder PTG happening at
whoami-rajatjungleboyj rosmaita smcginnis tosky whoami-rajat m5z e0ne geguileo eharney walshh_ jbernard sfernand enriquetaso hemna fabiooliveira yuval12:58
toskycoming on while13:22
bingyanh__hi dear all can help to take a look at this cherry pick:
*** dviroel is now known as dviroel|ruck14:14
whoami-rajatrosmaita we are discussing Update on Image Encryption 14:37
rosmaitawhoami-rajat: ty, looks like i will miss it14:37
whoami-rajatok, np14:37
ade_leewhoami-rajat, hey - thanks for putting fips topic on for tomorrow morning -- I do have a session planned with the manila folks on friday 13:55-14:25 UTC, can we move my topic to earlier in the day? (maybe second in the day)14:49
ade_leeor later .. 14:50
ade_leejust concerned about overlap14:50
whoami-rajatade_lee, sure, will ask Gorka (as he has the first topic) if he's fine moving it for later14:51
ade_leewhoami-rajat, thanks14:52
ade_leewhoami-rajat, I anticipate we won't take more than 15 mins most likely14:53
whoami-rajatade_lee, cool, that helps with covering rest of the topics14:53
ade_leewhoami-rajat, of course, I don't know how verbose people are :)14:54
whoami-rajatVERY! but let's see14:55
*** dviroel|ruck is now known as dviroel|ruck|lunch15:28
sean-k-mooneylooking at
sean-k-mooneythe host specific info could just be filtered based on project-admin vs project member15:40
sean-k-mooneyfor example  "os-vol-host-attr:host": null,15:40
sean-k-mooneyyou could also remove the host info at a later micorversion and optionally expose it via a system admin api if required15:41
sean-k-mooneywhat was the acul concern that cinder had with this api with regards to rbac15:41
sean-k-mooneywe have similar things in nova for servers for the host its resides on which are admin only15:42
sean-k-mooneyand will onely be shown if you are a project_admin going forward15:42
sean-k-mooneysince we redefiend project admin to be an operator acting on a project scoped resoures15:43
sean-k-mooneyrosmaita: ^ that was for you by the way but not urgent just wondering what the context was15:48
toskywhoami-rajat: are we in the cinder or in the glance room now?15:52
whoami-rajattosky, in cinder room15:52
toskyoh, I see now, thanks15:52
whoami-rajatshould've cleared that before ...15:52
whoami-rajatglance cinder cross project happening in cinder meeting room at 1600 UTC jungleboyj rosmaita smcginnis tosky whoami-rajat m5z e0ne geguileo eharney walshh_ jbernard sfernand enriquetaso hemna fabiooliveira yuval15:56
whoami-rajatmeeting room:
whoami-rajatjust doing it so people are aware ^^15:56
*** dviroel|ruck|lunch is now known as dviroel|ruck16:13
opendevreviewDataCore Software proposed openstack/cinder master: Reintroduce DataCore driver
opendevreviewFrancesco Pantano proposed openstack/devstack-plugin-ceph master: Deploy with cephadm
geguileorosmaita: thank you for the survey and the results, they were helpful to reach a decission on today's PTG  :-)17:05
rosmaitageguileo: yw!17:06
*** dviroel|ruck is now known as dviroel|afk20:39

Generated by 2.17.3 by Marius Gedminas - find it at!