Tuesday, 2022-07-19

*** hemna7 is now known as hemna00:53
opendevreviewRajat Dhasmana proposed openstack/cinder master: NFS Backup: Remove leftover dirs  https://review.opendev.org/c/openstack/cinder/+/80460306:54
*** tkajinam is now known as Guest543507:14
*** Guest5435 is now known as tkajinam07:36
opendevreviewHarsh Ailani proposed openstack/cinder master: [SVf]: Incorrect portset value during driver intialization  https://review.opendev.org/c/openstack/cinder/+/85031010:03
whoami-rajatgeguileo, left a few questions on https://review.opendev.org/c/openstack/os-brick/+/83605910:17
whoami-rajatIIUC that's the first patch in the nvmeof chain, so we need to merge that first10:17
opendevreviewGorka Eguileor proposed openstack/os-brick master: LUKS: Support extending host attached volumes  https://review.opendev.org/c/openstack/os-brick/+/83605910:41
opendevreviewGorka Eguileor proposed openstack/os-brick master: NVMe-oF: Consolidate code paths  https://review.opendev.org/c/openstack/os-brick/+/83606010:41
opendevreviewGorka Eguileor proposed openstack/os-brick master: NVMe-oF: Disconnect subsystems  https://review.opendev.org/c/openstack/os-brick/+/83606210:41
opendevreviewGorka Eguileor proposed openstack/os-brick master: Support shared_targets tristate value  https://review.opendev.org/c/openstack/os-brick/+/83606310:41
geguileowhoami-rajat: you are correct10:41
geguileowhoami-rajat: I have reworded those 2 log messages to use the cryptsetup naming convention10:42
whoami-rajatgeguileo, great, will take a look10:58
*** dviroel|out is now known as dviroel11:27
opendevreviewTushar Trambak Gite proposed openstack/cinder master: Configure a storage node in cinder  https://review.opendev.org/c/openstack/cinder/+/82099411:49
sfv880Hello! I would like to fix a few bugs in the Infinidat driver and would like to ask your opinion on the best way to do this. For example, I could open several independent pull requests and then bump the driver version with a separate request. Or bump the driver version in the first pull request and make all subsequent changes dependent. Please tell me - what is the best way to do it? Thank you!12:38
rosmaitasfv880: probably best not to worry about versioning at this point, can depend on the severity of the bug and whether the fix will be backported ... please make sure you file a bug in https://launchpad.net/cinder for each of the bugs and include a "Closes-bug: #123456" in the commit message on your patch12:52
rosmaitaneed a review on https://review.opendev.org/c/openstack/cinderlib/+/849432/ to unblock cinderlib gate13:03
rosmaitae0ne eharney geguileo hemna jungleboyj smcginnis enriquetaso ^^13:03
yuvalhey, I see some new failures in our third party ci, only happens on multiattach tests: I get the following error 13:25
yuvalDetails: {'code': 400, 'message': "Invalid input for field/attribute server. Value: {'name': 'tempest-VolumeMultiattachTests-server-1068649218', 'imageRef': '47ba6e9c-0cd1-4e7c-bb96-2b780d326b22', 'flavorRef': '0', 'key_name': 'tempest-VolumeMultiattachTests-1885608089', 'security_groups': [{'name': 'tempest-secgroup-smoke-416482459'}]}. 'networks' is a required property"}13:25
yuvalwhich networks is missing? all other tests in tempest and the cinder plugin temps is passing13:25
hemnais there a way to disable incremental backups?13:36
hemnaforce a full backup every time?13:36
opendevreviewMerged openstack/cinder stable/xena: PowerMax Driver - Retype fix for replication  https://review.opendev.org/c/openstack/cinder/+/82174413:45
enriquetasohemna, There is no way to disable it. Let me know if I am wrong, but as far as I know, the Cinder client always creates a full backup unless you use the optional --incremental arguments.13:46
hemnayah unfortunately, thats what I was seeing as well.13:49
geguileohemna: what would be the use case for that?14:43
hemnavmware sucks14:43
hemnaunfortunately it's a combination of issues related to vmware requiring stream optimized images 14:44
hemnabasically we don't see the raw disk at backup time because it's compressed, so the chunking can be all out of order and different sizes between backups14:44
hemnaso the easiest thing is for us to disable incrementals entirely14:44
hemnato ensure that the backups are consistent/sane14:45
hemnarelated to this review https://review.opendev.org/c/openstack/cinder/+/831439/414:45
hemnawe only get to see the compressed image at backup time, so the chunks can shrink and grow abnormally compared to chunks from a raw disk.14:46
hemnareally needs to be a custom vmware backup driver written by vmware, but vmware doesn't really support openstack at all nowdays.14:47
opendevreviewAlexander Deiter proposed openstack/cinder master: Fix Infinidat driver to use SSL options  https://review.opendev.org/c/openstack/cinder/+/84984614:48
geguileohemna: then it seems like the chunked driver needs some improvements, right?14:55
hemnayah 14:55
geguileowhat I don't get is why we see it as compressed in backup14:55
hemnabut also would be nice if we could disable incrementals w/o hacking the code14:55
geguileobut don't see it as compressed if we are doing a create volume from image...14:55
hemnathis is vmware backend, the images are streamoptimized, re: compressed14:56
geguileoso cinder never uses dd, right?14:56
geguileothen generic live migration would not work either, right?14:57
geguileosorry, generic offline migration14:57
geguileothe one that copies data in Cinder14:57
hemnathat happens in vcenter itself14:58
hemnacinder doesn't touch data during migrations14:58
geguileonot even for offline?14:59
hemnathis is vmware.  everything is fuct14:59
hemnaand different14:59
geguileoI think I'm getting the idea14:59
opendevreviewAlexander Deiter proposed openstack/cinder master: Fix Infinidat driver to return all iSCSI portals  https://review.opendev.org/c/openstack/cinder/+/84902215:00
opendevreviewMerged openstack/cinderlib master: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/84943215:00
hemnawe've tried to come up with ways of fixing this, but every option is risky for corrupting the chunks15:01
geguileohemna: how would they get corrumpted?15:01
hemnawell, we thought of uncompressing the stream and then adding zeros in to fill and recompressing15:02
hemnait just gets ugly from there15:02
geguileohemna: oh! making os-brick return an IOWrapper that did all that?15:03
hemnaheh, in our case we aren't even using os-brick15:03
hemnait's a modified chunked driver that uses an http connection to esxi15:04
geguileoouch, ouch15:04
hemnavmware's support in openstack is not really usable at much of any scale without what we've changed15:04
hemnavmware is not cloud software15:05
hemnaso I was thinking of adding a config option to disable incremental backups.   If that's on, then have cinder throw an exception.   since that isn't part of the current API, I was going to silently force the incremental to False in the API here:  https://github.com/sapcc/cinder/blob/stable/train-m3/cinder/backup/api.py#L24915:07
hemna(that's our fork of cinder)15:07
geguileohemna: wouldn't your users be surprised to request incremental and get a non incremental backup?15:08
hemnapossibly.   not sure they would even know really15:09
hemnaas it stands today, I'm not sure backups are really working well after the primary backup15:09
geguileohemna: From upstream perspective I'm against adding a flag to disable this15:11
geguileoI think we could allow os-brick connectors to somehow mark this particular behavior on the dict they return on connect_volume15:11
geguileoand then the backup driver to act on it15:11
hemnawhat's the problem of allowing a config option?15:12
hemnaespecially if they don't work :)15:12
geguileothat we can have multiple backends, one that works and another that doesn't15:13
hemnacouldn't it be specified on the backend ?15:13
geguileoso if os_brick return in connect_volume return {'path': tmp_file_path, 'last_modified': last_modified, 'varying_size': True}15:14
hemnaI wasn't aware you could configure more than 1 backup service in the cinder.conf at the same time15:15
geguileothen the chunked_backup driver may be able to act on it and: ignore the incremental request and do full backup, then set the DB to the right value, and create a user message explaining what just happened15:15
geguileonot more than 1 backup, more than 1 backend15:15
geguileomore than 1 volume backend15:15
geguileowhich is the actual problem, iiuc15:15
hemnaah ok, that makes more sense15:16
hemnathe vmware driver gives the backup service a vmdk file, which is compressed and there isn't a reliable way to decompress it to do the chunking15:17
geguileoand I don't know how this is not a problem when using the NFS backend with compressed qcow215:17
hemnait probably is15:17
hemnahence that review15:17
geguileoso we would need an automatic solution15:17
geguileohemna: but I'm not sure that that patch would actually fix things15:18
geguileobecause like you said, the size changes15:18
geguileoso a later file could be smaller15:18
hemnachunking on compressed data doesn't really make sense, as the chunks can change quite a lot.  you could end up with more or less chunks depending on where in the raw disk was changed since last backup15:18
geguileohemna: yeah, and it would need to be careful with the size15:19
hemnaI'm not entirely sure that review 'fixes' it either, but what it is alluding to in the change makes sense15:19
geguileoI'd have to check the behavior and check how that patch fixes things15:19
geguileohemna: afaik the fix would also require changes to the restore (though I could be wrong)15:20
hemnaI think as long as the chunk chain is correct, the restore should be ok15:21
geguileohemna: maybe, I'd have to check the restore code15:24
opendevreviewMerged openstack/cinderlib master: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/84940915:28
hemnait's been interesting to see how many assumptions and limitations that cinder proper has in place that I've had to violate to make it work in a large scale environment15:31
*** akekane_ is now known as abhishekk15:37
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/yoga: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/85039315:45
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/yoga: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85039415:46
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/xena: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/85039515:48
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/xena: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85039615:48
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/wallaby: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/85039915:51
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/wallaby: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85040015:51
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/victoria: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/85040115:53
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/victoria: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85040215:53
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/ussuri: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/85040315:54
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/ussuri: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85040415:55
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/train: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85040515:57
whoami-rajattosky, ^ that's a looooong backport chain15:58
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/ussuri: Migrate jobs from CentOS 8 to CentOS 8 Stream  https://review.opendev.org/c/openstack/cinderlib/+/82442116:33
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/ussuri: Don't use yum to install ceph  https://review.opendev.org/c/openstack/cinderlib/+/85040316:33
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/ussuri: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85040416:33
opendevreviewLuigi Toscano proposed openstack/cinderlib stable/train: Update deprecated zuul syntax  https://review.opendev.org/c/openstack/cinderlib/+/85040516:34
opendevreviewSimon Dodsley proposed openstack/cinder master: Add Pure Storage NVMe-RoCE driver  https://review.opendev.org/c/openstack/cinder/+/79987118:58
opendevreviewSofia Enriquez proposed openstack/cinder master: Implement copy encrypted image in NFS generic driver  https://review.opendev.org/c/openstack/cinder/+/74915519:19
*** dviroel is now known as dviroel|dinner22:22
*** dviroel|dinner is now known as dviroel|out23:20

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!