Monday, 2018-02-05

*** s-shiono has joined #openstack-cinder00:06
*** yangyape_ has quit IRC00:33
*** dave-mcc_ has joined #openstack-cinder00:33
*** dave-mccowan has quit IRC00:35
*** wanghao has joined #openstack-cinder00:38
*** threestrands has joined #openstack-cinder00:43
*** threestrands has quit IRC00:43
*** threestrands has joined #openstack-cinder00:43
*** hoangcx has joined #openstack-cinder00:48
*** edmondsw has joined #openstack-cinder00:52
*** edmondsw has quit IRC00:56
openstackgerritwanghao proposed openstack/cinder master: Transfer snapshots with volumes  https://review.openstack.org/53356401:02
*** zhaochao has joined #openstack-cinder01:04
*** BrinZhang has joined #openstack-cinder01:12
*** dalgaaf has quit IRC01:18
*** dalgaaf has joined #openstack-cinder01:18
*** yangyapeng has joined #openstack-cinder01:20
*** zhurong has joined #openstack-cinder01:30
*** namnh has joined #openstack-cinder01:44
*** moshele has joined #openstack-cinder01:45
*** wanghao_ has joined #openstack-cinder01:50
*** wanghao_ has quit IRC01:51
*** wanghao has quit IRC01:51
*** wanghao has joined #openstack-cinder01:52
*** mtreinish has quit IRC01:58
*** mtreinish has joined #openstack-cinder01:59
*** threestrands has quit IRC02:01
*** moshele has quit IRC02:05
*** daidv has joined #openstack-cinder02:06
*** itlinux has joined #openstack-cinder02:07
*** threestrands has joined #openstack-cinder02:16
*** threestrands has quit IRC02:16
*** threestrands has joined #openstack-cinder02:16
*** itlinux has quit IRC02:37
*** edmondsw has joined #openstack-cinder02:40
*** jmlowe has quit IRC02:43
*** zhurong has quit IRC02:43
*** edmondsw has quit IRC02:45
*** jmlowe has joined #openstack-cinder02:45
openstackgerritMaoyangLiu proposed openstack/cinder master: modify the http link to the https link  https://review.openstack.org/54070303:03
*** zhenguo has joined #openstack-cinder03:21
*** dave-mcc_ has quit IRC03:32
*** gcb has joined #openstack-cinder03:37
*** itlinux has joined #openstack-cinder03:47
*** psachin has joined #openstack-cinder04:07
*** threestrands has quit IRC04:07
*** itlinux has quit IRC04:12
*** abhishekk has joined #openstack-cinder04:13
*** threestrands has joined #openstack-cinder04:19
*** threestrands has quit IRC04:19
*** threestrands has joined #openstack-cinder04:19
*** edmondsw has joined #openstack-cinder04:28
*** edmondsw has quit IRC04:33
*** threestrands has quit IRC04:36
*** threestrands has joined #openstack-cinder04:37
*** pgadiya has joined #openstack-cinder05:03
*** threestrands has quit IRC05:07
*** zhenguo has quit IRC05:31
*** zhurong has joined #openstack-cinder05:38
*** threestrands has joined #openstack-cinder05:42
*** links has joined #openstack-cinder05:46
*** threestrands has quit IRC05:47
*** threestrands has joined #openstack-cinder05:47
*** links has quit IRC05:49
*** links has joined #openstack-cinder05:51
*** lkuchlan has joined #openstack-cinder05:52
*** lkuchlan_ has joined #openstack-cinder05:52
*** gkadam has joined #openstack-cinder05:52
openstackgerritwanghao proposed openstack/cinder master: Transfer snapshots with volumes  https://review.openstack.org/53356405:53
*** links has quit IRC05:59
*** links has joined #openstack-cinder06:00
*** jmlowe has quit IRC06:04
*** threestrands_ has joined #openstack-cinder06:04
*** threestrands_ has joined #openstack-cinder06:04
*** jmlowe has joined #openstack-cinder06:05
*** threestrands has quit IRC06:06
openstackgerritwanghao proposed openstack/cinder master: Transfer snapshots with volumes  https://review.openstack.org/53356406:07
*** lkuchlan_ has quit IRC06:09
*** lkuchlan has quit IRC06:09
*** links has quit IRC06:14
*** links has joined #openstack-cinder06:16
*** edmondsw has joined #openstack-cinder06:17
*** jmlowe has quit IRC06:18
*** jmlowe has joined #openstack-cinder06:20
*** edmondsw has quit IRC06:21
openstackgerritOpenStack Proposal Bot proposed openstack/cinder master: Imported Translations from Zanata  https://review.openstack.org/54072406:24
openstackgerritwanghao proposed openstack/cinder master: Move doc of report backend state to cli  https://review.openstack.org/54072506:26
*** jmlowe has quit IRC06:32
*** jmlowe has joined #openstack-cinder06:33
*** e0ne has joined #openstack-cinder06:38
*** e0ne has quit IRC06:45
*** jmlowe has quit IRC06:47
*** moshele has joined #openstack-cinder06:48
*** jmlowe has joined #openstack-cinder06:48
openstackgerritwanghao proposed openstack/cinder master: Move doc of report backend state to cli  https://review.openstack.org/54072507:03
*** pcaruana has joined #openstack-cinder07:04
*** jmlowe has quit IRC07:04
*** josecastroleon has joined #openstack-cinder07:06
*** jmlowe has joined #openstack-cinder07:07
*** lkuchlan_ has joined #openstack-cinder07:07
*** e0ne has joined #openstack-cinder07:09
openstackgerritwanghao proposed openstack/cinder master: Add missing MV history for 3.50  https://review.openstack.org/54073907:12
*** adisky__ has joined #openstack-cinder07:12
*** alexchadin has joined #openstack-cinder07:12
openstackgerritMaoyangLiu proposed openstack/cinder master: modify the http link to the https link  https://review.openstack.org/54070307:12
*** e0ne_ has joined #openstack-cinder07:13
*** e0ne has quit IRC07:14
*** junboli has joined #openstack-cinder07:14
*** e0ne_ has quit IRC07:17
*** rcernin has quit IRC07:18
*** lpetrut has joined #openstack-cinder07:19
*** e0ne has joined #openstack-cinder07:29
*** e0ne has quit IRC07:29
*** AlexeyAbashkin has joined #openstack-cinder07:41
*** alexchadin has quit IRC07:42
*** alexchadin has joined #openstack-cinder07:43
*** armaan has joined #openstack-cinder07:44
*** AlexeyAbashkin has quit IRC07:45
*** AlexeyAbashkin has joined #openstack-cinder07:50
*** andreas_s has joined #openstack-cinder07:54
*** moshele has quit IRC08:00
*** moshele has joined #openstack-cinder08:02
*** edmondsw has joined #openstack-cinder08:05
*** alexchadin has quit IRC08:08
*** edmondsw has quit IRC08:09
*** andreas_s has quit IRC08:10
*** andreas_s has joined #openstack-cinder08:15
*** hoonetorg has quit IRC08:15
*** s-shiono has quit IRC08:17
*** andreas_s has quit IRC08:25
*** hoonetorg has joined #openstack-cinder08:28
*** tesseract has joined #openstack-cinder08:28
*** andreas_s has joined #openstack-cinder08:30
*** andreas_s has quit IRC08:30
*** andreas_s has joined #openstack-cinder08:30
*** e0ne has joined #openstack-cinder08:43
*** ircuser-1 has joined #openstack-cinder08:49
*** BrinZhang has quit IRC08:52
*** BrinZhang has joined #openstack-cinder08:53
*** threestrands_ has quit IRC08:54
*** jmlowe has quit IRC08:59
*** jmlowe has joined #openstack-cinder09:01
*** moshele has quit IRC09:05
*** wanghao has quit IRC09:06
*** wanghao has joined #openstack-cinder09:06
*** wanghao has quit IRC09:07
*** wanghao has joined #openstack-cinder09:07
*** moshele has joined #openstack-cinder09:07
*** wanghao has quit IRC09:07
*** wanghao has joined #openstack-cinder09:08
*** wanghao has quit IRC09:08
*** wanghao has joined #openstack-cinder09:09
*** wanghao has quit IRC09:09
*** wanghao has joined #openstack-cinder09:09
*** wanghao has quit IRC09:10
*** wanghao has joined #openstack-cinder09:10
*** wanghao has quit IRC09:10
*** wanghao has joined #openstack-cinder09:11
*** wanghao has quit IRC09:11
*** wanghao has joined #openstack-cinder09:12
*** wanghao has quit IRC09:12
*** wanghao has joined #openstack-cinder09:12
*** wanghao has quit IRC09:13
*** wanghao has joined #openstack-cinder09:13
*** wanghao has quit IRC09:13
*** wanghao has joined #openstack-cinder09:14
*** wanghao has quit IRC09:14
*** wanghao has joined #openstack-cinder09:15
*** wanghao has quit IRC09:15
*** wanghao has joined #openstack-cinder09:15
*** wanghao has quit IRC09:16
*** wanghao has joined #openstack-cinder09:16
openstackgerritMaoyangLiu proposed openstack/cinder master: fixs the outdated url to the latest url  https://review.openstack.org/54078109:21
*** Dinesh_Bhor has joined #openstack-cinder09:24
openstackgerritDou Rui Yuan proposed openstack/cinder master: Correct lvm_type default value in config doc  https://review.openstack.org/54078709:29
*** Dinesh_Bhor has quit IRC09:32
*** alexchadin has joined #openstack-cinder09:35
*** alexchadin has quit IRC09:39
*** alexchadin has joined #openstack-cinder09:40
*** junboli has quit IRC09:43
openstackgerritAseel Awwad proposed openstack/cinder master: Prevent race condition in RBD tmeporary directory creation  https://review.openstack.org/54063709:50
*** jmlowe has quit IRC10:09
*** jmlowe has joined #openstack-cinder10:10
*** stakeda has quit IRC10:26
*** namnh has quit IRC10:33
*** lkuchlan_ has quit IRC10:56
*** yangyapeng has quit IRC10:56
*** yangyapeng has joined #openstack-cinder10:57
*** jmlowe has quit IRC11:02
openstackgerritXianjin Shao proposed openstack/cinder master: The preconditions of snapshot status when you delete it  https://review.openstack.org/54082111:03
*** jmlowe has joined #openstack-cinder11:04
*** andreas_s has quit IRC11:16
*** ganso has joined #openstack-cinder11:16
*** andreas_s has joined #openstack-cinder11:17
*** BrinZhang has quit IRC11:17
*** BrinZhang has joined #openstack-cinder11:17
*** jmlowe has quit IRC11:20
*** jmlowe has joined #openstack-cinder11:24
*** andreas_s has quit IRC11:26
*** andreas_s has joined #openstack-cinder11:31
*** armaan has quit IRC11:32
*** abhishekk has quit IRC11:38
*** jmlowe has quit IRC11:39
*** jmlowe has joined #openstack-cinder11:40
*** andreas_s has quit IRC11:40
*** edmondsw has joined #openstack-cinder11:41
*** alexchadin has quit IRC11:44
*** alexchadin has joined #openstack-cinder11:45
*** edmondsw has quit IRC11:45
*** andreas_s has joined #openstack-cinder11:46
*** lkuchlan_ has joined #openstack-cinder11:48
*** stvnoyes has quit IRC11:49
*** pgadiya has quit IRC11:51
*** armaan has joined #openstack-cinder11:51
*** tpsilva has joined #openstack-cinder11:52
*** andreas_s has quit IRC11:55
*** andreas_s has joined #openstack-cinder11:56
*** stvnoyes has joined #openstack-cinder11:57
*** moshele has quit IRC11:59
*** erlon has joined #openstack-cinder12:00
*** pcaruana has quit IRC12:02
*** jmlowe has quit IRC12:04
*** andreas_s has quit IRC12:04
*** andreas_s has joined #openstack-cinder12:04
*** jmlowe has joined #openstack-cinder12:05
openstackgerritAseel Awwad proposed openstack/cinder master: Prevent race condition in RBD tmeporary directory creation  https://review.openstack.org/54063712:12
*** jmlowe has quit IRC12:15
*** jmlowe has joined #openstack-cinder12:25
*** yangyapeng has quit IRC12:29
*** gcb has quit IRC12:30
*** yangyapeng has joined #openstack-cinder12:33
*** yangyapeng has quit IRC12:33
*** jmlowe has quit IRC12:36
*** psachin has quit IRC12:56
*** josecastroleon1 has joined #openstack-cinder13:00
*** josecastroleon has quit IRC13:00
*** arnewiebalck_ has joined #openstack-cinder13:01
*** lkuchlan_ has quit IRC13:01
*** alexchadin has quit IRC13:03
*** alexchadin has joined #openstack-cinder13:04
*** abishop has joined #openstack-cinder13:05
*** jmlowe has joined #openstack-cinder13:08
*** edmondsw has joined #openstack-cinder13:13
erlongeguileo, is this related to that topic someone mentioned on a Cinder meeting some time ago?13:29
erlonRBD driver taking too much system resources13:29
geguileoerlon: what is?13:29
erlongeguileo, https://bugs.launchpad.net/cinder/+bug/170410613:29
openstackLaunchpad bug 1704106 in Cinder "Gathering of thin provisioning stats breaks c-vol" [Undecided,Confirmed]13:29
geguileoerlon: I fixed that a while ago13:30
erlongeguileo, recall the version?13:30
erlon"Refreshing this issue. It is still happening in latest stable/newton."13:30
geguileoerlon: the issue they have now is that if your backend is not fast enough you have problems13:30
geguileoerlon: and for that I have to write a caching mechanism13:31
erlongeguileo, hmm, is it that bacd to relay on Cinders provisioned_capacity_gb?13:31
erlon*rely13:31
*** liverpooler has joined #openstack-cinder13:31
*** lhx_ has joined #openstack-cinder13:32
geguileoerlon: there are people sharing the pool with other things beside Cinder13:32
erlongeguileo, facepalm13:32
geguileoerlon: and they cannot change it now13:32
geguileoerlon: that's the case that was mentioned in the meeting13:32
erlonhmm13:32
*** yangyapeng has joined #openstack-cinder13:35
*** liverpooler has quit IRC13:36
*** edmondsw has quit IRC13:36
*** liverpooler has joined #openstack-cinder13:36
*** edmondsw has joined #openstack-cinder13:36
*** yangyapeng has quit IRC13:39
*** moshele has joined #openstack-cinder13:41
*** edmondsw has quit IRC13:41
*** edmondsw has joined #openstack-cinder13:44
*** armaan has quit IRC13:48
*** pchavva has joined #openstack-cinder13:48
*** armaan has joined #openstack-cinder13:48
*** dustins has joined #openstack-cinder13:49
*** zhangbailin_ has joined #openstack-cinder13:50
*** edmondsw has quit IRC13:50
*** edmondsw has joined #openstack-cinder13:51
*** BrinZhang has quit IRC13:53
*** arnewiebalck_ has quit IRC13:54
*** edmondsw has quit IRC13:55
*** edmondsw has joined #openstack-cinder13:59
*** lkuchlan_ has joined #openstack-cinder14:01
*** edmondsw has quit IRC14:01
*** edmondsw has joined #openstack-cinder14:01
*** zhangbailin_ has quit IRC14:06
bhagyashrisHi all, can anyone please tell me the list of backend which are supporting the multi-attach functionality. I mean to say when I create the volume using the default lvm backend (driver-lvm ) by giving the multiattach flag value as true (cinder --debug create --allow-multiattach --name test666 1) then it's not creating the volume successfully and goes into the error state14:07
*** dave-mccowan has joined #openstack-cinder14:09
bhagyashrisso which drivers/backend are supporting fo same14:10
ildikovbhagyashris: if you're deploying from latest master then it's supposed to work with LVM14:10
*** arnewiebalck_ has joined #openstack-cinder14:10
ildikovbhagyashris: also we're deprecating the usage of the flag for volume create14:11
ildikovbhagyashris: the new way to create a multi-attach volume is to create a multi-attach volume_type and create the volume with that14:13
bhagyashrisildikov: ok, thank you :) just one query i mean i am not aware about the last part creating th mulit-attach volume_type14:17
bhagyashrisand hen create volume14:17
ildikovbhagyashris: https://docs.openstack.org/cinder/latest/admin/blockstorage-volume-multiattach.html14:18
bhagyashrisildikov: i have just pull the latest master and create volume using command cinder --debug create --allow-multiattach --name test666 1 but getting some error at c-sch side14:18
ildikovbhagyashris: what is the error?14:19
bhagyashrisFeb 05 19:50:48 openstack-VirtualBox cinder-scheduler[14795]: ERROR cinder.scheduler.flows.create_volume [None req-1a8882da-4161-4664-b1c5-c6e6828605b6 admin None] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available14:20
bhagyashrisildikov: ^^^14:21
openstackgerritMerged openstack/cinder master: Imported Translations from Zanata  https://review.openstack.org/54072414:22
openstackgerritMerged openstack/cinder master: Correct lvm_type default value in config doc  https://review.openstack.org/54078714:23
openstackgerritMerged openstack/cinder master: Add missing MV history for 3.50  https://review.openstack.org/54073914:23
bhagyashrisildikov: i have gone through the doc you share and tried to create the volume that way but still getting same eeror as share above so IMO LVM is not supporting the multi-attach14:25
ildikovbhagyashris: we are testing with LVM in Tempest and it works14:27
ildikovbhagyashris: are you having the client installed from the latest source as well?14:28
bhagyashrisildikov: one thing i have observed in doc is they have mentioned "You’ll need to ensure you have a backend device that reports support of the multiattach capability, otherwise scheduling will fail on create" so How we can found out that which drivers are supporting multi-attach functionality14:28
bhagyashrisildikov: checking...14:28
*** david-lyle has quit IRC14:30
bhagyashrisildikov: just for reference checking this bug https://bugs.launchpad.net/cinder/+bug/1729201 and it is in progress14:32
openstackLaunchpad bug 1729201 in Cinder "cinder volume driver-lvm not support multiattach" [Undecided,In progress] - Assigned to cloud-liang (lianghao)14:32
*** dklyle has quit IRC14:32
ildikovbhagyashris: we turned it off in the capabilities until we have the full implementation ready for multi-attach14:34
ildikovbhagyashris: and we turn it back on for drivers that support it, here's lvm: https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/lvm.py#L27914:34
bhagyashrisildikov: tried with latest client as well but still get same error14:34
ildikovbhagyashris: can you create the volume without multiattach?14:35
bhagyashrisildikov: yes14:36
ildikovbhagyashris: one more thing, I think you will also need to use the microversion for multi-attach in the request which is 3.5014:36
ildikovbhagyashris: so add this to the create call --os-volume-api-version 3.5014:38
*** dave-mcc_ has joined #openstack-cinder14:39
*** mriedem has joined #openstack-cinder14:39
*** moshele has quit IRC14:40
ildikovbhagyashris: we also added a new policy to multi-attach which is admin_or_owner, but I believe that wouldn't be a scheduler error14:41
*** dave-mccowan has quit IRC14:42
bhagyashrisildikov: yeah but even if we use the microversion 3.50 it will give me the error as the  capabilities are turn off right?14:42
ildikovbhagyashris: for lvm it's turned on14:44
ildikovbhagyashris: you can find it here in the code: https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/lvm.py#L27914:44
bhagyashrisildikov: ohh i thought it's off14:44
ildikovbhagyashris: if it's False in your code locally, that's a problem, then you're not running the latest code14:44
bhagyashrisildikov: ok14:45
*** gcb has joined #openstack-cinder14:46
ildikovbhagyashris: it was turned off, we turned it back on two weeks ago :)14:47
*** arnewiebalck_ has quit IRC14:52
*** josecastroleon1 has quit IRC14:53
*** armaan has quit IRC14:58
*** armaan has joined #openstack-cinder14:59
*** r-daneel has joined #openstack-cinder15:03
*** mchlumsky has joined #openstack-cinder15:04
bhagyashrisildikov: ok thank you :) for all your inputs15:07
*** josecastroleon has joined #openstack-cinder15:09
*** alexchadin has quit IRC15:14
*** lkuchlan_ has quit IRC15:24
*** lpetrut has quit IRC15:28
*** yangyapeng has joined #openstack-cinder15:34
swamireddy_hi - any one used "volume_clear=shred" option? volume_clear=zero is working...but with "shred" has some issue?15:35
*** yangyapeng has quit IRC15:38
smcginnisswamireddy_: I didn't think we had shred anymore.15:38
smcginnisswamireddy_: https://review.openstack.org/#/c/396755/15:39
swamireddy_smcginnis: openstack docs says that..cinder supports zero and shred option for data cleaning15:41
swamireddy_smcginnis: https://docs.openstack.org/security-guide/block-storage/volume_wiping.html15:41
*** david-lyle has joined #openstack-cinder15:42
*** lbragstad_ is now known as lbragstad15:43
smcginnisswamireddy_: Looks like that never made it to the openstack-manuals folks back then.15:43
*** KeithMnemonic has joined #openstack-cinder15:45
ildikovbhagyashris: I'm on my way to the airport, but will check it in my local env once I landed15:45
swamireddy_smcginnis: thanks for clarification15:47
*** gcb has quit IRC15:47
*** armaan has quit IRC15:49
smcginnisswamireddy_: https://review.openstack.org/#/c/540900/15:54
*** pandy has joined #openstack-cinder15:58
pandyHi15:58
*** markstur has joined #openstack-cinder15:59
pandyI have tried to cinder incremental backup, where full backup  is fine, incremental backup is throwing status error, where "failing reason: dict has no attribute name"15:59
*** lhx__ has joined #openstack-cinder16:00
e0nepandy: hi. what version do you use? please, share traceback what you have16:01
pandyi am using ocata-devstack16:02
pandye0ne: ..16:02
*** lhx_ has quit IRC16:02
pandye0ne: here launhpad details https://bugs.launchpad.net/cinder/+bug/174740216:04
openstackLaunchpad bug 1747402 in Cinder "cinder incremental backup fails with ceph "dict' object has no attribute 'name'""" [Undecided,New]16:04
*** pchavva has quit IRC16:05
*** mchlumsky has quit IRC16:05
e0nepandy: looks like we need this https://review.openstack.org/#/c/511230/ patch to get it working in ocata :(16:06
pandyAlso whether cinder incremental backup need parent reference, as like trove, but in cinder back up create could not use like that16:06
pandye0ne: can we try this in pike or which devstack version ?16:08
e0nepandy: it should be fixed in pike16:08
*** mchlumsky has joined #openstack-cinder16:10
pandye0ne: thanks, also would like to know if ceph is backend, do i want to mention --incremental parameter ?16:10
e0nepandy: yes, by default we've got incremental=false16:14
pandye0ne: okay, now am cloing pike and will try cinder incremental.16:16
*** pchavva has joined #openstack-cinder16:20
*** mayur_ind has joined #openstack-cinder16:24
mayur_indHi folks,16:24
mayur_indhere is implementation of Cinder driver over ISCSI for Veritas Access:--16:24
mayur_indhttps://review.openstack.org/#/c/539230/16:24
*** AlexeyAbashkin has quit IRC16:34
*** AlexeyAbashkin has joined #openstack-cinder16:34
*** andreas_s has quit IRC16:36
*** andreas_s has joined #openstack-cinder16:37
*** links has quit IRC16:37
*** itlinux has joined #openstack-cinder16:37
*** itlinux has quit IRC16:40
*** lhx__ has quit IRC16:41
pandye0ne: I have tried in pike, still getting same error16:42
*** moshele has joined #openstack-cinder16:42
pandye0ne: getting same as "fail_reason | 'dict' object has no attribute 'name'"16:43
*** zhaochao has quit IRC16:45
*** andreas_s has quit IRC16:46
*** moshele has quit IRC16:47
*** gkadam has quit IRC16:47
*** e0ne has quit IRC16:48
*** itlinux has joined #openstack-cinder16:48
*** edmondsw has quit IRC16:50
pandyhas anyone did cinder incremental backup16:51
*** edmondsw has joined #openstack-cinder16:56
openstackgerritGorka Eguileor proposed openstack/cinder master: Support multiple processes on Cinder Backup  https://review.openstack.org/54092916:56
pandyjdillaman: I saw your comment #1681895, its not related to my cinder incremental issues.16:58
*** edmondsw has quit IRC17:00
*** moshele has joined #openstack-cinder17:00
*** moshele has quit IRC17:04
*** AlexeyAbashkin has quit IRC17:05
*** crushil has joined #openstack-cinder17:06
*** moshele has joined #openstack-cinder17:12
*** AlexeyAbashkin has joined #openstack-cinder17:17
*** mayur_ind has quit IRC17:18
*** AlexeyAbashkin has quit IRC17:21
*** moshele has quit IRC17:24
*** moshele has joined #openstack-cinder17:27
*** yangyapeng has joined #openstack-cinder17:28
chandankumarsmcginnis: Hello17:28
chandankumarsmcginnis: i need some help on this patch https://review.openstack.org/#/c/531206/ to make it passing, please have a look.17:29
smcginnischandankumar: Hmm, access denied errors in the backup service. I haven't seen that before.17:32
*** yangyapeng has quit IRC17:32
smcginnisLooks like that's the cause for all of them. Denied trying to talk to swift.17:33
*** moshele has quit IRC17:37
*** crushil has quit IRC17:40
pandyhave taken cinder backup with ceph as backend and also ran cinder backup again, it performed incremental backup1, incremental backup2 ut in parameters " has_dependent_backups=false" and also not able delete "incremental backup1" saying has dependecy.. how to delete it17:48
smcginnisYou will need to delete backup2 before you can delete backup1.17:49
*** e0ne has joined #openstack-cinder17:53
mriedemyikes http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-c-api.txt.gz#_Feb_05_15_52_28_21393317:54
mriedemrpc call timeout on initialize_connection17:54
mriedemonly thing i see in c-vol is this http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-c-vol.txt.gz#_Feb_05_15_51_28_19991917:55
mriedemhttp://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-c-vol.txt.gz#_Feb_05_15_52_26_61094717:57
mriedemreturned: 0 in 58.416s17:57
mriedemCMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvchange -a y --yes -K stack-volumes-lvmdriver-1/volume-b41782e2-df34-4ed8-b720-c957b2949e7b" returned: 0 in 58.416s17:57
*** josecastroleon has quit IRC17:57
smcginnisTook almost a minute to activate a volume? That's odd.17:59
pandye0ne: in pike i can able to do cinder incremental backup, where ceph is backend. But in incremental backup, its properties is showing "has_dependent_backup=false" , pasted output here paste.openstack.org/show/662554/18:00
smcginnispandy: Right, there aren't any dependent backups on that.18:01
*** david-lyle has quit IRC18:01
*** jose-phillips has joined #openstack-cinder18:01
pandysmcginnis: whether its incremental or fullbackup, in backup-show its showing "incremental=true" ? Also in restoration process, whether "cinder backup-restore --volume <original vol id> <latest_incremental_backup_id> is enough to restore ?18:04
*** crose has joined #openstack-cinder18:05
*** yangyapeng has joined #openstack-cinder18:05
mriedemi'm not sure why nova is calling os-intialize_connection here anyway, we shouldn't call that anymore on master18:05
ildikovmriedem: that was my first thought too... :/18:06
smcginnispandy: Where do you see a full backup setting incremental=True?18:06
smcginnispandy: is_incremental is false in your paste: http://paste.openstack.org/show/662554/18:07
pandysmcginnis: if its not dependent backup, then its taking full backup ? how it referring original data and restoring,  Also is it possible to restore volume using "cinder backup restore <first_full_backup_vol_ID> <latest_incremental_backup_ID> " because assume in case i lost original volume, with fullbackup and last backup should able to restore like disaster recovery18:07
smcginnispandy: It has dependent backups, because the second incremental one is dependent on it.18:09
jgriffithmriedem: ildikov is it possible that the config and api settings aren't quite all up to speed somewhere in there?18:09
pandysmcginnis: I am seeing incremental=True while doing "cinder backup-create <volume_ID>  --incremental, if i am not passing --incremental "Incremental =False" do we need to pass --incremental in ceph as backend type ?18:09
*** yangyapeng has quit IRC18:10
ildikovjgriffith: we force the multi-attach tests to use the 2.60 Nova microversion otherwise they would fail on the Nova side18:10
mriedemjgriffith: the job is running on master so it should all be up to date18:10
mriedemthis isn't a multiattach test18:10
smcginnispandy: Sorry, I don't understand what you are asking.18:10
mriedemit's just a regular old volume attach test18:10
ildikovmriedem: sorry, didn't look up the exact test just saw the job and assumed... :)18:11
mriedemthe attach starts in the compute logs here http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_00015818:11
smcginnispandy: As far as restore goes, I believe it's just "openstack volume backup restore 905062f3-53d9-44ed-ba50-4eabc1c04662 1ad556f3-6190-4658-9bb3-8ec72eca6ead"18:11
mriedemthe volume returned has no attachments in it http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_17141618:11
*** josecastroleon has joined #openstack-cinder18:12
mriedemin n-api this is where we create an attachment on the volume http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-api.txt.gz#_Feb_05_15_52_27_84555218:13
ildikovmriedem: the new flow in Cinder internally uses initialize_connection though even if it's not the os-initialize_connection call anymore18:13
jgriffithildikov: yeah; but that call in the first log is hitting the API server18:14
ildikovjgriffith: so answering your question, I have no idea :)18:14
mriedemildikov: the error in the n-cpu logs says it's failing to initialize the connectoin, which is only from nova's cinder api initialize_connection client call18:14
jgriffithildikov: unless I'm not following (which is quite possible), the log shows it being called from Nova18:14
mriedemPOST attachments returns a 202?18:15
jgriffithat first I thought maybe the exception was just bubbling up; but it looks like the extension was actually called on the API server?18:15
pandysmcginnis: As you said "As far as restore goes, I believe it's just "openstack volume backup restore 905062f3-53d9-44ed-ba50-4eabc1c04662 1ad556f3-6190-4658-9bb3-8ec72eca6ead"" here is my question, assume original volume is deleted "1ad556f3-6190-4658-9bb3-8ec72eca6ead" can't we restore in that case, like with first backup id, it was fullbackup i think so ?18:15
ildikovjgriffith: mriedem's first link looks like an attachment_update call to me18:15
mriedemREQ: curl -g -i -X POST http://198.72.124.149/volume/v3/98b826c754214cdeae569d147336c6fc/attachments18:16
mriedemattachment_update would be a PUT18:16
jgriffithildikov: I agree; but can't figure out this:  http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-c-api.txt.gz#_Feb_05_15_52_28_22450618:16
mriedemjgriffith: that's n-cpu calling os-intialize_connection on the volume actions api18:16
mriedemwhich should only happen for an old style attachment,18:17
mriedemwhich we shouldn't be doing in this job, and i can see the n-api creating an attachment record on that volume18:17
jgriffithmriedem: right... am I incorrect in thinking that *that* is the problem here?18:17
mriedemi think it is a problem yes18:17
mriedemattachment_update would likely maybe also timeout eventually calling lvchange too,18:17
mriedemsince attachment_update is also an rpc blocking call18:18
smcginnispandy: You can restore to a different volume than the backup was created from.18:18
jgriffithyeah, the lvchange thing is another problem18:18
jgriffithwe see those every year or so it seems :(18:18
smcginnisjgriffith: Or every 6 months. :/18:18
jgriffithsmcginnis: 6/12 close enough :)18:19
smcginnisSeems like it was more recent than that.18:19
jgriffithhaha, nooo; I meant you say 6, I say 12.... close enough :)18:20
jgriffithsmcginnis: anyway... bottom line is lv activate has been a problem since day 1 and can be quite a few different things18:20
jgriffithsmcginnis: you know that 1 month is like 2 griffith-months18:21
smcginnisHah!18:22
ildikovjgriffith: like this branding while hoping for a Griffith-First-Class on my next flight :)18:23
jgriffithLOL18:23
ildikovmriedem: jgriffith: ok, so we still need to figure out why initialize_connection is called, right?18:24
ildikovif not, then I guess I should just call it a (Mon)day... :)18:25
jgriffithildikov: yeah, I think so18:26
ildikovshoot; then I go back to read the logs...18:26
mriedemso we got this attachment from cinder in the api RESP BODY: {"attachment": {"status": "reserved", "detached_at": "", "connection_info": {}, "attached_at": "", "attach_mode": null, "instance": "197bb6f9-3410-4fdb-a2a7-88bf97c78e34", "volume_id": "28d1715e-5adc-408b-aaac-bf524fcbd54a", "id": "af2dd524-a945-47d8-b9f0-4b54ab0115f0"}}18:27
jgriffithhttp://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_43005718:27
mriedemwe should be storing that on the bdm18:27
mriedemwhich goes to the compute for the attach18:27
ildikovyep18:29
ildikovjgriffith: hmmm, I don't recall ever seeing that before18:30
mriedemi have an idea of what's going on18:31
pandysmcginnis: here i have pasted steps trying to do, http://paste.openstack.org/show/662564/, I would like to know whether second or third volume backup is right ?18:31
*** itlinux has quit IRC18:31
jgriffithmriedem: it looks like somethign fails bulding the bdm to me?18:32
*** moshele has joined #openstack-cinder18:32
jgriffithand we don't catch it properly18:32
jgriffithso then when we check that on the update we *think* it's an old style attach18:32
* jgriffith wishes he could type better18:32
jgriffithor even more gooder18:32
mriedemif the bdm.create() or bdm.save() failed, we'd get an exception18:32
mriedemnot just ignore it18:32
mriedemnova-api rpc calls to nova-compute reserve_block_device_name,18:33
mriedemwhich creates the bdm record18:33
smcginnispandy: It depends what you want to restore. If you just want the full backup restored from step 2, just restore that. If you want the data included in the incremental, then restore that one.18:33
mriedemwhich is returned to the api18:33
*** wN has quit IRC18:33
ildikovthere's an attachment_update call with a response 500: http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_18688418:33
ildikovI might read it wrong though, unsure at this point18:34
mriedemhmm yeah, wth18:35
pandysmcginnis: I have created new volume "test" and did restore of "third" ( latest_incremental_backup) to "test" volume, whether it will have complete backup,  I mean both incremental and full ?18:36
jgriffithildikov: I guess it's better when you point to it :)18:36
ildikovjgriffith: it was my second attempt :)18:36
smcginnispandy: Yes, it needs to write it all back to be able to restore.18:36
jgriffithI'm confused by all of that in there18:36
jgriffithwhich is why it seems like the block-device method is honked18:37
ildikovthen I have no idea what this is: http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_18688418:37
mriedemso it looks like it's doing a boot from volume here, i'm totally confused18:37
mriedem_prep_block_device only gets called from boot from volume (or unshelve)18:38
*** moshele has quit IRC18:38
ildikovthis might as well be the error from the Cinder internal initialize_connection: http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_22105118:38
ildikovand this one tries to call os-terminate_connection: http://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/logs/screen-n-cpu.txt.gz#_Feb_05_15_52_28_22469018:38
jgriffithahhh, test_delete_server_while_in_shelved_state?18:39
mriedemno18:40
mriedemhttp://logs.openstack.org/57/537957/2/gate/nova-multiattach/456f61e/job-output.txt.gz#_2018-02-05_16_05_19_98416518:40
mriedemshelve isn't involved18:40
pandysmcginnis: really thanks, I have final questions, 1) For Ceph as backend, do we need to pass "--incremental" parameter or by default it will have incremental, If ceph has the feature, how to verify in ceph ? 2) Can we try this in ocata release by changing "https://github.com/openstack/cinder/commit/3e2a593ae9eb80d9753cd118e59008a77a315c11" or supports only from pike release ?18:41
ildikovhmm, I guess it's time to make myself aware of what this whole device tagging thing is doing...18:41
*** armaan has joined #openstack-cinder18:42
mriedemildikov: it creates 3 volumes, 1 bootable, and then creates a server from those 3 volumes18:42
mriedemwith the bootable volume as the root disk18:42
smcginnispandy: You will need to use --incremental to get an incremental back regardless of backend type.18:42
smcginnispandy: And yes, I think that commit would need to be backported to Ocata.18:43
ildikovmriedem: I remember some bits and pieces in the code on the tagging part of the whole thing, mostly exceptions and not sure at this point whether that can affect anything in the attach flow?18:43
mriedemtags shouldn't have anything to do with it18:43
ildikovwhile I agree with you 100%, I was just wondering what can mess up the use of the old vs new flow18:44
pandysmcginnis: thanks,  i will do those commit changes in my ocata setup and try that18:44
mriedemildikov: i've got an idea18:44
smcginnispandy: Good luck18:44
mriedemildikov: with boot from volume, we won't call bdm.save() here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L377618:45
mriedembecause the bdm isn't created in a cell db yet18:45
pandysmcginnis: someone told, ceph has incremental backup, so no need to mention. So asked. How to verify that "incremental"backup has full data while restoring it with new volume ?18:45
mriedemi'm wondering if we're losing the bdm.attachment_id somewhere along the way18:45
ildikovmriedem: ah, that can happen18:46
ildikovwe don't call save there as the DB record does not exist at that point18:46
pandysmcginnis: any checksum or LSN or SHA256 can be used to track incremental backup ?18:46
jgriffithYou mean like:  18:32  jgriffith| mriedem: it looks like somethign fails bulding the bdm to me?18:47
jgriffithso not bulding it but updating it?18:47
jgriffithThat makes sense; and much more likely that a failed update or something slips through than a failure building it18:47
openstackgerritSean McGinnis proposed openstack/cinder master: Clear metadata when restoring backup with no metadata  https://review.openstack.org/46007218:48
mriedemreq-8007f6e8-2ddf-43c3-9330-c420ccca6361 is the server create request id in the api,18:48
pandysmcginnis: wondering, restoring with latest_incremental_backup_volume with newly created volume will restore all data ?, I felt need to keep full_backup_volume and apply_logs of incremental on top of it while restore ?18:48
mriedemi see the GET calls to get the 3 volumes from cinder,18:48
mriedembut don't see any POST calls to create attachments18:48
ildikovjgriffith: well, when we try to build it it should have the attachment_id, but it gets lost somewhere in the jungle18:49
*** traskat has joined #openstack-cinder18:50
*** crose has quit IRC18:50
mriedemi don't see any POST calls to create attachments for these 3 volumes18:52
mriedemfrom nova-api18:52
ildikovmriedem: I thought we reserve the volume in every case, which means that maybe something went wrong there?18:54
ildikovmriedem: I mean reserve = attachment_create without a connector18:54
mriedemildikov: i suspect we're getting here somehow https://github.com/openstack/nova/blob/master/nova/compute/api.py#L134418:57
mriedemso we don't call reserve or attachment create18:57
mriedemi'll put up a debug patch after lunch18:57
mriedemcould be a bug in nova's service version code18:57
*** david-lyle has joined #openstack-cinder18:57
openstackgerritSean McGinnis proposed openstack/cinder master: Correct attachment create success response code  https://review.openstack.org/54097219:03
*** harlowja has joined #openstack-cinder19:03
openstackgerritGorka Eguileor proposed openstack/cinder master: Improve ChunkedBackupDriver hashlib calls  https://review.openstack.org/54097319:04
openstackgerritSean McGinnis proposed openstack/cinder master: Add python 3.5 in classifier  https://review.openstack.org/47787319:04
pandyMay i know Gorka IRC name please ?19:05
ildikovmriedem: yeah, seems highly likely19:06
*** itlinux has joined #openstack-cinder19:07
*** pandy has quit IRC19:09
*** edmondsw has joined #openstack-cinder19:29
*** e0ne has quit IRC19:31
*** tesseract has quit IRC19:36
*** markstur has quit IRC19:37
*** AlexeyAbashkin has joined #openstack-cinder19:38
melwittdoes anyone know what happened to cinder.tests.tempest.api.volume.test_volume_unicode.CinderUnicodeTest? a recent tempest change https://review.openstack.org/#/c/522707 broke the test in the ceph job on stable/ocata but I don't find that the test exists on master. is the test something we should skip on ocata? log http://logs.openstack.org/80/540680/1/check/legacy-tempest-dsvm-full-devstack-plugin-ceph/465f6e2/job-output.txt.gz#_2018-19:43
melwitt02-04_23_18_37_68155219:43
melwittoops http://logs.openstack.org/80/540680/1/check/legacy-tempest-dsvm-full-devstack-plugin-ceph/465f6e2/job-output.txt.gz#_2018-02-04_23_18_37_68155219:44
mriedemmelwitt: it's probably in the cinder repo19:46
melwittit is back in stable/ocata but on master there's no more tempest/ dir19:46
mriedemhttp://git.openstack.org/cgit/openstack/cinder-tempest-plugin/19:46
mriedemqueens goal was to move tempest tests from in-repo to plugins19:47
melwittoh, thanks, there it is19:47
melwittokay, so it's no longer using the old 'volumes' attribute19:47
*** AlexeyAbashkin has quit IRC19:47
*** arnewiebalck_ has joined #openstack-cinder19:53
*** imacdonn has joined #openstack-cinder19:55
*** edmondsw_ has joined #openstack-cinder19:57
*** edmondsw has quit IRC20:00
*** crose has joined #openstack-cinder20:02
*** wN has joined #openstack-cinder20:02
*** wN has quit IRC20:02
*** wN has joined #openstack-cinder20:02
mriedemildikov: this might be our problem in nova-api https://review.openstack.org/#/c/539755/20:04
*** armaan_ has joined #openstack-cinder20:04
*** armaan has quit IRC20:04
*** yangyapeng has joined #openstack-cinder20:04
*** armaan_ has quit IRC20:04
*** armaan has joined #openstack-cinder20:05
ildikovmriedem: I remember having an issue with that at some point20:05
ildikovmriedem: might not want to comment on it further...20:05
mriedemwe'll see https://review.openstack.org/#/c/540992/20:06
ildikovmriedem: how that didn't fail before?20:06
mriedembecause we fallback to using the old APIs20:06
*** itlinux has quit IRC20:06
mriedemlike initialize connection instead of attachment_update20:06
ildikovmriedem: ah yes and that worked...20:06
mriedemright b/c of rolling upgrade that old flow all works20:07
ildikovmriedem: I'm off for two hours now, no wifi on this flight20:07
mriedemso i think what nova-api has been doing all along is checking the nova-compute service version in cell0, which doesn't have any nova-compute services,20:07
mriedemso it just returns 020:07
mriedemnp, enjoy20:07
ildikovmriedem: what an annoying bug...20:07
ildikovmriedem: haha; wish me survival :)20:08
*** yangyapeng has quit IRC20:08
*** diablo_rojo_ has quit IRC20:13
*** diablo_rojo_ has joined #openstack-cinder20:14
*** e0ne has joined #openstack-cinder20:15
SwansonLittle over a month into the new year, time to take a peak at how my Bitcoin and Ripple are doing. OMG!20:17
*** markstur has joined #openstack-cinder20:23
*** itlinux has joined #openstack-cinder20:23
*** crose has quit IRC20:28
*** dave-mcc_ has quit IRC20:29
erlonjungleboyj, hey, can you give a look on this when you have a chance? https://review.openstack.org/#/c/538989/20:48
jungleboyjSure. I will take a look later today.20:49
*** yangyapeng has joined #openstack-cinder20:54
*** yangyapeng has quit IRC20:58
*** moshele has joined #openstack-cinder21:02
*** traskat has quit IRC21:07
*** moshele has quit IRC21:09
*** dave-mccowan has joined #openstack-cinder21:20
openstackgerritSean McGinnis proposed openstack/cinder master: Correct attachment create success response code  https://review.openstack.org/54097221:26
*** rcernin has joined #openstack-cinder21:26
*** e0ne has quit IRC21:36
*** e0ne has joined #openstack-cinder21:41
*** yangyapeng has joined #openstack-cinder21:44
*** itlinux has quit IRC21:45
*** yangyapeng has quit IRC21:48
*** moshele has joined #openstack-cinder21:54
*** e0ne has quit IRC21:54
*** itlinux has joined #openstack-cinder21:54
ildikovmriedem: back now, did you get any luck with that bug fix patch so far?21:55
*** edmondsw_ is now known as edmondsw21:57
mriedemildikov: i was right that the nova-compute min version check in the api was just returning 021:58
mriedemb/c it was using cell021:58
mriedemi've updated that debug patch with a depends-on to mel's fix21:58
ildikovmriedem: I remember I was suffering with that thing, can't believe I messed it up by the end... :/21:59
*** arnewiebalck_ has quit IRC22:03
*** moshele has quit IRC22:06
*** ganso has quit IRC22:07
*** dave-mccowan has quit IRC22:10
*** idlemind_ has quit IRC22:17
*** idlemind has joined #openstack-cinder22:18
*** yangyapeng has joined #openstack-cinder22:23
*** armaan has quit IRC22:26
*** pchavva has quit IRC22:27
*** armaan has joined #openstack-cinder22:27
*** yangyapeng has quit IRC22:28
*** abishop has quit IRC22:28
*** dobson has quit IRC22:31
*** dustins has quit IRC22:34
*** dobson has joined #openstack-cinder22:34
*** jdurgin has joined #openstack-cinder22:41
*** itlinux has quit IRC22:48
openstackgerritSean McGinnis proposed openstack/cinder master: Make CinderException format errors fatal for tests  https://review.openstack.org/54106122:58
*** edmondsw has quit IRC23:07
*** edmondsw has joined #openstack-cinder23:08
*** tpsilva has quit IRC23:10
*** edmondsw has quit IRC23:12
*** yangyapeng has joined #openstack-cinder23:13
*** liverpooler has quit IRC23:13
*** yangyapeng has quit IRC23:17
*** threestrands has joined #openstack-cinder23:19
*** threestrands has quit IRC23:19
*** threestrands has joined #openstack-cinder23:19
openstackgerritSean McGinnis proposed openstack/cinder master: Adding unit test coverage to flow  https://review.openstack.org/51279623:25

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!