Monday, 2016-05-16

*** The_Ball has quit IRC00:00
*** alexpilotti has joined #openstack-cinder00:11
*** lprice has joined #openstack-cinder00:11
*** alexpilotti has quit IRC00:12
*** cknight has joined #openstack-cinder00:14
*** esker has joined #openstack-cinder00:15
*** esker has quit IRC00:16
*** esker has joined #openstack-cinder00:17
*** chlong has joined #openstack-cinder00:19
*** scottda has quit IRC00:25
*** laughterwym has joined #openstack-cinder00:33
*** laughterwym has quit IRC00:40
*** laughterwym has joined #openstack-cinder00:42
*** takashi has joined #openstack-cinder00:55
*** cheneydc has joined #openstack-cinder00:56
*** iceyao_ has joined #openstack-cinder01:00
*** itlinux has quit IRC01:08
*** itlinux has joined #openstack-cinder01:18
*** itlinux has quit IRC01:23
*** itlinux has joined #openstack-cinder01:25
*** jamielennox is now known as jamielennox|away01:26
*** itlinux has quit IRC01:26
*** jamielennox|away is now known as jamielennox01:28
*** bswartz has quit IRC01:28
*** zhenguo_ has joined #openstack-cinder01:31
*** itlinux has joined #openstack-cinder01:35
*** IlyaG has joined #openstack-cinder01:38
*** IlyaG has quit IRC01:40
*** salv-orlando has joined #openstack-cinder01:43
*** sdague has quit IRC01:47
*** EinstCrazy has joined #openstack-cinder01:48
*** salv-orlando has quit IRC01:51
*** Julien-zte has joined #openstack-cinder01:51
*** IlyaG has joined #openstack-cinder01:52
*** raunak has quit IRC01:53
*** chlong has quit IRC01:56
*** EinstCrazy has quit IRC02:03
*** itlinux has quit IRC02:04
*** EinstCrazy has joined #openstack-cinder02:04
*** itlinux has joined #openstack-cinder02:05
*** cknight has quit IRC02:06
*** itlinux has quit IRC02:10
*** houming has joined #openstack-cinder02:13
*** dave-mccowan has joined #openstack-cinder02:20
*** Lee1092 has joined #openstack-cinder02:25
*** EinstCrazy has quit IRC02:26
*** EinstCrazy has joined #openstack-cinder02:28
*** bswartz has joined #openstack-cinder02:33
*** EinstCrazy has quit IRC02:34
*** EinstCrazy has joined #openstack-cinder02:36
*** raunak has joined #openstack-cinder02:39
*** EinstCra_ has joined #openstack-cinder02:40
*** IlyaG has quit IRC02:41
*** EinstCrazy has quit IRC02:43
openstackgerritzhongjun proposed openstack/cinder-specs: Cinder volume rollback snapshot  https://review.openstack.org/31653702:58
*** yangyapeng has joined #openstack-cinder02:58
*** itlinux has joined #openstack-cinder03:03
*** jamielennox is now known as jamielennox|away03:06
openstackgerritzhongjun proposed openstack/cinder-specs: Cinder volume rollback snapshot  https://review.openstack.org/31654003:09
*** raunak has quit IRC03:11
*** harlowja_at_home has joined #openstack-cinder03:16
*** sgotliv has joined #openstack-cinder03:21
*** raunak has joined #openstack-cinder03:22
*** bardia has joined #openstack-cinder03:22
openstackgerritzhongjun proposed openstack/cinder-specs: Cinder volume rollback snapshot  https://review.openstack.org/31654003:26
*** bardia has quit IRC03:28
*** bardia has joined #openstack-cinder03:28
*** jamielennox|away is now known as jamielennox03:36
*** ducttape_ has joined #openstack-cinder03:38
*** bardia has quit IRC03:40
*** links has joined #openstack-cinder03:49
*** zhongjun_ has quit IRC03:53
*** zhongjun_ has joined #openstack-cinder03:53
openstackgerritzhongjun proposed openstack/cinder-specs: Cinder volume rollback snapshot  https://review.openstack.org/31654003:57
*** IlyaG has joined #openstack-cinder03:59
*** ducttape_ has quit IRC04:02
*** ducttape_ has joined #openstack-cinder04:02
*** ducttape_ has quit IRC04:07
*** Julien-zte has quit IRC04:08
*** IlyaG has quit IRC04:14
*** Poornima has joined #openstack-cinder04:30
*** apoorvad has joined #openstack-cinder04:36
*** salv-orlando has joined #openstack-cinder04:36
*** Julien-zte has joined #openstack-cinder04:44
*** salv-orlando has quit IRC04:45
*** harlowja_at_home has quit IRC04:50
*** apoorvad has quit IRC04:52
*** mtanino has quit IRC04:58
*** laughter_ has joined #openstack-cinder05:11
*** laughterwym has quit IRC05:15
*** lpetrut has joined #openstack-cinder05:20
*** savihou has joined #openstack-cinder05:26
*** suyash has joined #openstack-cinder05:28
*** suyash has quit IRC05:28
*** rcernin has joined #openstack-cinder05:38
*** laughter_ has quit IRC05:43
*** GB21 has joined #openstack-cinder05:43
*** laughterwym has joined #openstack-cinder05:43
*** itzdilip has joined #openstack-cinder05:54
*** liverpooler has joined #openstack-cinder05:55
*** salv-orlando has joined #openstack-cinder06:02
*** salv-orlando has quit IRC06:08
*** Julien-zte has quit IRC06:11
openstackgerritSheel Rana proposed openstack/cinder: Updated volume, version, limit, metadata API-ref  https://review.openstack.org/31658706:11
*** sheel has joined #openstack-cinder06:12
*** lpetrut has quit IRC06:20
*** salv-orlando has joined #openstack-cinder06:25
openstackgerritKushal Wathore proposed openstack/cinder: Add unit tests in cinder component for HPE drivers  https://review.openstack.org/31399206:39
*** GB21 has quit IRC07:03
*** raunak has quit IRC07:16
*** GB21 has joined #openstack-cinder07:19
*** lpetrut has joined #openstack-cinder07:32
*** GB21 has quit IRC07:35
*** aarefiev has quit IRC07:43
*** vnogin has quit IRC07:43
*** vnogin has joined #openstack-cinder07:45
*** laughterwym has quit IRC07:48
*** laughterwym has joined #openstack-cinder07:49
*** GB21 has joined #openstack-cinder07:52
*** kongwei has joined #openstack-cinder07:58
*** laughterwym has quit IRC07:58
*** lennyb has joined #openstack-cinder07:58
*** laughterwym has joined #openstack-cinder07:58
*** laughter_ has joined #openstack-cinder07:59
*** zzzeek has quit IRC08:00
*** iceyao has joined #openstack-cinder08:00
*** iceyao_ has quit IRC08:00
*** zzzeek has joined #openstack-cinder08:00
*** lpetrut has quit IRC08:01
*** laughterwym has quit IRC08:03
*** GB21 has quit IRC08:03
*** salv-orlando has quit IRC08:11
*** markvoelker has joined #openstack-cinder08:15
*** GB21 has joined #openstack-cinder08:16
*** laughterwym has joined #openstack-cinder08:17
*** laughter_ has quit IRC08:20
*** markvoelker has quit IRC08:23
*** markvoelker has joined #openstack-cinder08:24
*** e0ne has joined #openstack-cinder08:25
*** dosaboy has quit IRC08:26
*** yangyapeng has quit IRC08:28
*** yangyapeng has joined #openstack-cinder08:30
*** jistr has joined #openstack-cinder08:30
openstackgerritKushal Wathore proposed openstack/cinder: Add unit tests in cinder component for 3PAR driver  https://review.openstack.org/31537608:31
*** laughter_ has joined #openstack-cinder08:33
*** laughterwym has quit IRC08:36
*** dosaboy has joined #openstack-cinder08:36
*** Poornima has quit IRC08:37
*** partner has joined #openstack-cinder08:41
*** Poornima has joined #openstack-cinder08:47
openstackgerritYatin Kumbhare proposed openstack/cinder: Use check_string_length from oslo_utils  https://review.openstack.org/31535808:49
*** sgotliv_ has joined #openstack-cinder08:49
*** sgotliv has quit IRC08:52
*** laughter_ has quit IRC08:53
*** laughterwym has joined #openstack-cinder08:53
*** yhayashi has quit IRC08:53
*** zzzeek has quit IRC08:56
*** caspinol has joined #openstack-cinder08:56
*** laughterwym has quit IRC08:57
*** zzzeek has joined #openstack-cinder08:58
*** takashi has quit IRC08:59
*** yuriy_n17 has joined #openstack-cinder09:11
*** lpetrut has joined #openstack-cinder09:27
*** mvk has quit IRC09:28
*** iceyao has quit IRC09:32
*** lpetrut1 has joined #openstack-cinder09:33
*** lpetrut has quit IRC09:34
*** lpetrut1 is now known as lpetrut09:34
savihouDuncanT: Thanks for the feedback on https://review.openstack.org/#/c/263026/ , the issue you identified was indeed a rebase issue.09:37
*** caspinol has quit IRC09:38
DuncanTsavihou: Glad to hear it was something simple.09:39
savihouDuncanT: :) , May I bother you with a final review ? just to make sure I didn't missed anything ?09:40
DuncanTYes, I'll get back to it today09:40
partnerhi, an unfortunate finding, nova-compute and glusterfs mounts are under same systemd cgroup so restarting nova for config changes kills all the volume mounts. using centos 7 and kilo, can someone point me towards some fixes or instructions to address the issues, thanks! found a quite fresh similar bug: 153086009:42
openstackbug 1530860 in OpenStack Compute (nova) "Nova service restart disconnects Quobyte volumes on systemd systems" [Undecided,In progress] https://launchpad.net/bugs/1530860 - Assigned to Silvan Kaiser (2-silvan)09:42
*** kongwei has quit IRC09:44
partnerperhaps switching to libgfapi?09:49
savihouDuncanT: Thank you!!09:50
*** caspinol has joined #openstack-cinder09:52
*** mvk has joined #openstack-cinder09:57
*** cheneydc has quit IRC10:00
*** laughterwym has joined #openstack-cinder10:09
openstackgerritPetrut Lucian proposed openstack/os-brick: Ensure that the base connector is platform independent  https://review.openstack.org/31299910:09
*** yangyapeng has quit IRC10:10
openstackgerritMerged openstack/cinder: Failback will enable the host that has been frozen  https://review.openstack.org/31233410:10
*** laughterwym has quit IRC10:13
*** sdague has joined #openstack-cinder10:18
*** GB21 has quit IRC10:20
*** lennyb has quit IRC10:32
*** GB21 has joined #openstack-cinder10:33
*** caspinol has quit IRC10:33
*** salv-orlando has joined #openstack-cinder10:34
*** EinstCra_ has quit IRC10:36
*** salv-orlando has quit IRC10:39
*** caspinol has joined #openstack-cinder10:46
*** salv-orlando has joined #openstack-cinder10:48
*** caspinol has quit IRC10:49
*** gcb has joined #openstack-cinder10:57
*** sbezverk has quit IRC11:04
*** markvoelker has quit IRC11:06
*** avishay has joined #openstack-cinder11:08
*** krtaylor has quit IRC11:10
*** _fortis has quit IRC11:13
*** GB21 has quit IRC11:24
*** krtaylor has joined #openstack-cinder11:24
*** salv-orlando has quit IRC11:27
openstackgerritMatan Sabag proposed openstack/cinder: Manage/unmanage snapshot in ScaleIO driver Add support for manage/unmanage snapshot in the ScaleIO driver.  https://review.openstack.org/31674211:30
*** tpsilva has joined #openstack-cinder11:31
*** salv-orlando has joined #openstack-cinder11:35
*** GB21 has joined #openstack-cinder11:37
*** lprice has quit IRC11:38
*** jwcroppe_ has joined #openstack-cinder11:47
*** jwcropp__ has joined #openstack-cinder11:49
*** jwcroppe has quit IRC11:50
*** jwcroppe_ has quit IRC11:52
*** sdake has joined #openstack-cinder11:55
*** mtanino has joined #openstack-cinder11:57
*** raildo-afk is now known as raildo12:06
*** liverpooler has quit IRC12:07
*** markvoelker has joined #openstack-cinder12:08
*** sdake has quit IRC12:10
*** ducttape_ has joined #openstack-cinder12:11
*** markvoelker has quit IRC12:13
*** yangyapeng has joined #openstack-cinder12:14
*** diogogmt has quit IRC12:15
*** GB21 has quit IRC12:20
*** diogogmt has joined #openstack-cinder12:20
*** caspinol has joined #openstack-cinder12:22
caspinolHello, are user problems allowed in here or is this just a dev channel?12:24
*** ducttape_ has quit IRC12:29
*** diogogmt has quit IRC12:29
caspinolok, so i have problem with volume migration in cinder 7.0.1. In the logs i can see that the migration was scheduled but it never takes place and there is no ERROR or any other type of output in logs after that12:31
caspinolwhen i try migration in Horizon i only get a notification that it failed12:31
caspinolbut again no entry in the logs as to why it failed12:32
*** edmondsw has joined #openstack-cinder12:32
*** diogogmt has joined #openstack-cinder12:34
caspinolwould anybody have any tips on how to troubleshoot it?12:35
caspinolalso all the other operations like creating/deleteing are working without problem12:39
*** julim has joined #openstack-cinder12:40
*** laughterwym has joined #openstack-cinder12:40
dulekcaspinol: Try looking for TRACEBACK instead of ERROR. Also if Horizon informs you of that, then you probably need to look in c-api logs.12:40
*** Poornima has quit IRC12:42
caspinolthanks for reply dulek12:42
caspinolBy traceback you mean looking for actual python exception?12:43
caspinolif yes then there is none unfortunately12:43
*** akerr has joined #openstack-cinder12:43
dulekcaspinol: Hm. Or maybe your request is getting lost between c-sch and c-vol?12:44
caspinolThis issue is really weird as i was able to fix most of issues with my openstack deploynment from the logs12:44
dulekcaspinol: Check out RabbitMQ queues for non-received mesages.12:44
caspinolbut this one is really weird one12:44
dulekcaspinol: In logs of which component you see your request last?12:45
dulekcaspinol: cinder-scheduler?12:45
caspinolvery well, let me look at the rabbit stuff12:45
caspinoldulek: let me doublecheck12:45
*** scottda has joined #openstack-cinder12:45
avishaycaspinol: do you have two instances of c-vol?  did you check the logs of both?12:45
*** akerr_ has joined #openstack-cinder12:46
*** laughterwym has quit IRC12:46
*** laughterwym has joined #openstack-cinder12:46
*** timcl has joined #openstack-cinder12:49
*** akerr has quit IRC12:49
caspinolso last INFO log entry is in cinder-api, and its a action post request, followed by a DEBUG in cinder-api "Action-body : { os-migrate-volume..."12:50
caspinoland that is followed by DEBUG in sch-api returning "Filtered [host 'OS-Compute02@lvm#LVM': free_capacity_gb: 7776.49, pools: None, host 'OS-Compute01@lvm#LVM': free_capacity_gb: 7797.49, pools: None] _get_weighted_candidates /usr/lib/python2.7/site-packages/cinder/scheduler/filter_scheduler.py:310 "12:52
caspinoland that is all12:53
*** links has quit IRC12:55
e0nehi. can anybody help me with https://review.openstack.org/#/c/287938/, please? I don't see any errors, only NOT_REGISTERED jobs12:56
e0neoops, wrong channel12:56
*** houming has quit IRC12:58
*** markvoelker has joined #openstack-cinder13:00
akerr_e0ne: i'd just do a recheck.  NOT_REGISTERED means jenkins doesn't know about the required node type for that job.  Usually due to the nodes being redefined and an initial one not having been built by nodepool yet.  Should clear up on its own13:03
e0neakerr_: thanks13:03
akerr_e0ne: if it persists may want to ping infra about it though13:04
*** markvoelker has quit IRC13:04
e0neakerr_: sure, I'll do13:04
*** markvoelker has joined #openstack-cinder13:04
openstackgerritMatan Sabag proposed openstack/cinder: ScaleIO actual volume size model update  https://review.openstack.org/31681113:05
*** porrua has joined #openstack-cinder13:05
*** xyang1 has joined #openstack-cinder13:06
*** akerr_ is now known as akerr13:06
*** mtanino has quit IRC13:08
openstackgerritMatan Sabag proposed openstack/cinder: ScaleIO actual volume size model update  https://review.openstack.org/31681113:08
caspinolavishay: hi, yes i just did actually. And on the c-vol node2 I can see that a new lv is created, the content is dd'ed over from c-vol node1 and then deleted.13:14
caspinolbut does not show any error or reason why it deletes the LV13:14
*** Julien-zte has joined #openstack-cinder13:17
*** dustins has joined #openstack-cinder13:17
*** eharney has quit IRC13:18
scottdasmcginnis: Hi. When was the last time we released a version of cinderclient?13:28
*** markvoelker has quit IRC13:28
*** darosale has joined #openstack-cinder13:29
*** bill_az has joined #openstack-cinder13:31
smcginnisscottda: Just before Mitaka client freeze.13:33
smcginnisscottda: I haven't looked lately, but we probably have enough in there to do another release.13:34
*** diablo_rojo has joined #openstack-cinder13:34
smcginnisscottda: I can get that queued up today.13:34
*** esker has quit IRC13:34
smcginnisI should probably take a look at os-brick as well while I'm at it.13:34
scottdasmcginnis: Thanks. We probably do...I also see the tagged version in devstack is 1.6.013:34
*** cknight has joined #openstack-cinder13:35
scottdaWe are definitely behind in cinderclient release notes. They only go up to 1.4.013:35
*** eharney has joined #openstack-cinder13:35
smcginnisscottda: We have reno now in cinderclient.13:36
smcginnisscottda: Maybe there's a publishing job we need to update, or that page needs to change.13:36
scottdasmcginnis: Yeah, I'm not sure what, but the docs are a bit stale, and release notes in the repo are not current.13:37
scottdahttp://docs.openstack.org/developer/python-cinderclient/#master13:37
smcginnisscottda: Yeah, that should probably point to here now: http://docs.openstack.org/releasenotes/python-cinderclient/mitaka.html13:38
scottdasmcginnis: So that tagging always confuses me, and "git tag -l" just shows the tag numbers. Do you know when 1.6 tag was created? and how to figure that out?13:39
scottdasmcginnis: I see support in the cinderclient for 'delete --cascade' for instance, on master and it was merged in January, but it is not on Devstack cinderclient v 1.613:40
smcginnisscottda: Not sure the best/correct way. I usually have just looked at when I requested the release. :)13:40
smcginnisHmm, that's odd.13:40
smcginnisscottda: It was definitely after January, so that should be in there then.13:40
smcginnisPro tip - don't try to stack on a machine where you're running apache to serve a local apt mirror. :/13:41
smcginnisOr pip mirror13:41
scottdahaha13:42
smcginnisscottda: I won't tell how many times I tried, it failed, and I wondered why apache had stopped.13:42
smcginnisAh Monday's. :)13:42
*** edmondsw has quit IRC13:43
scottdasmcginnis: FYI support for --cascade, git log shows Jan 7 date ccacc338a513:45
*** sheel has quit IRC13:45
smcginnisscottda: Maybe a bug in there that it's not showing up?13:45
scottdasmcginnis: Nope. If I run master, I get support for cascade. But released version 1.6 (on devstack) doesn't have it.13:47
scottdaand it's not on stable/mitaka branch13:48
*** ducttape_ has joined #openstack-cinder13:49
*** cknight has quit IRC13:49
*** mriedem has joined #openstack-cinder13:49
eharneyscottda: try "git tag --contains ccacc338a5"13:51
*** martyturner has joined #openstack-cinder13:51
scottdasmcginnis: That returns nothing for me13:54
*** mtanino has joined #openstack-cinder13:54
eharneyright, so it's not in any release yet13:54
eharneyscottda: the merge commit a330ff is the first commit after 1.613:56
*** martyturner has quit IRC13:56
*** zzzeek has quit IRC13:56
scottdaeharney: OK, thanks.13:57
*** ametts has joined #openstack-cinder13:57
*** zzzeek has joined #openstack-cinder13:58
*** jgregor has joined #openstack-cinder13:59
*** diogogmt has quit IRC14:02
*** eharney has quit IRC14:05
*** dustins has quit IRC14:06
openstackgerritAlex Meade proposed openstack/python-cinderclient: Add pagination support for Messages  https://review.openstack.org/31576914:07
*** sdake has joined #openstack-cinder14:08
*** jungleboyj has joined #openstack-cinder14:09
*** dustins has joined #openstack-cinder14:10
*** cknight has joined #openstack-cinder14:10
*** baumann has joined #openstack-cinder14:11
*** links has joined #openstack-cinder14:15
*** avishay has quit IRC14:20
*** wxy has quit IRC14:20
*** esker has joined #openstack-cinder14:20
*** cbader has joined #openstack-cinder14:20
*** esker has quit IRC14:20
*** eharney has joined #openstack-cinder14:21
*** esker has joined #openstack-cinder14:21
*** markvoelker has joined #openstack-cinder14:25
*** diablo_rojo has quit IRC14:26
*** kfarr has joined #openstack-cinder14:26
*** markvoelker has quit IRC14:30
*** crose has joined #openstack-cinder14:31
*** avishay has joined #openstack-cinder14:34
*** edmondsw has joined #openstack-cinder14:35
*** rajinir has joined #openstack-cinder14:37
*** crose has quit IRC14:38
*** karthikp has joined #openstack-cinder14:39
*** kfarr has quit IRC14:42
*** smcginnis has quit IRC14:42
*** kfarr has joined #openstack-cinder14:43
*** Yogi1 has joined #openstack-cinder14:44
*** salv-orlando has quit IRC14:49
*** lprice has joined #openstack-cinder14:49
*** jdurgin1 has joined #openstack-cinder14:49
*** salv-orlando has joined #openstack-cinder14:50
*** aleksey0 has joined #openstack-cinder14:53
*** merooney has joined #openstack-cinder14:53
openstackgerritEric Harney proposed openstack/cinder: Fix up RPC cleanup process  https://review.openstack.org/31689514:53
*** aleksey0 is now known as alekseyruban14:53
*** merooney has quit IRC14:54
*** diablo_rojo has joined #openstack-cinder14:55
dulekjgriffith: Hi, my teammates are playing with SolidFire backend connected to OS deployment in Kilo version. They use Ceph as Glance backend. Is there a way to prevent double copying when booting from volume?14:55
dulekjgriffith: Right now this is copying image out of Ceph and then into volume on SF backend.14:55
dulekjgriffith: Image cache got in in Liberty, so they cannot use that to speed it up.14:56
*** superdan is now known as dansmith14:56
jgriffithdulek: SolidFire driver has a cache that was added I believe in Kilo14:56
jgriffithdulek: lemme verify :)14:56
dulekjgriffith: That's the one: https://blueprints.launchpad.net/cinder/+spec/implement-solidfire-cloneimage14:56
jgriffithdulek: ahh.. yeah14:56
jgriffithdulek: I thought you meant the "general" one14:56
jgriffithdulek: so I think I have a Kilo backport somewhere14:57
jgriffithdulek: that's the only option really14:57
e0nedulek: AFAIK, this issue is related not only to SolidFire driver14:57
dulekjgriffith: The general one implemented by patrickeast?14:57
jgriffithe0ne: correct14:57
*** liverpooler has joined #openstack-cinder14:57
jgriffithdulek: yes, that's the general impl, but I do have a backport of the SolidFire version that we did for some customers14:58
*** Julien-zte has quit IRC14:58
dulekjgriffith: Wait, so that's not working in Kilo: https://review.openstack.org/#/c/142859/16/cinder/volume/drivers/solidfire.py ?14:58
*** smcginnis has joined #openstack-cinder14:58
*** ChanServ sets mode: +o smcginnis14:58
jgriffithdulek: Oh, sorry14:59
*** merooney has joined #openstack-cinder14:59
jgriffithdulek: Kilo yes14:59
*** Julien-zte has joined #openstack-cinder14:59
jgriffithdulek: backports for Juno :)14:59
*** arch-nemesis has joined #openstack-cinder15:00
*** diogogmt has joined #openstack-cinder15:00
jgriffithdulek: the SolidFire specific one was added in Kilo and yes, that should work15:00
*** esker has quit IRC15:00
dulekjgriffith: Okay, I'll point guys to that snippet so they can review configuration and codepaths to see if images are cached.15:00
dulekjgriffith: Thanks!15:00
jgriffithdulek: sure, let me know if they run into problems15:00
*** julim has quit IRC15:01
*** itlinux has quit IRC15:02
*** julim has joined #openstack-cinder15:03
*** bluex has quit IRC15:03
*** Julien-zte has quit IRC15:04
alekseyrubanHi guys. I encountered an issue when tried to run tempest test_shelve_volume_backed_instance test. The first run works fine but after that other runs hang for a while and I get an error in c-vol screen. Some uncought exception in privsep client channel. Can you help me?15:05
*** mmotiani_ has quit IRC15:05
*** ntpttr- has joined #openstack-cinder15:05
alekseyrubanhe exception is:15:05
alekseyrubanException in thread privsep_reader:15:06
alekseyrubanTraceback (most recent call last):15:06
alekseyruban  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner15:06
alekseyruban    self.run()15:06
alekseyruban  File "/usr/lib/python2.7/threading.py", line 763, in run15:06
alekseyruban    self.__target(*self.__args, **self.__kwargs)15:06
alekseyruban  File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/comm.py", line 127, in _reader_main15:06
alekseyruban    for msg in reader:15:06
alekseyruban  File "/usr/local/lib/python2.7/dist-packages/six.py", line 558, in next15:06
alekseyruban    return type(self).__next__(self)15:06
alekseyruban  File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/comm.py", line 78, in next15:06
alekseyruban    buf = self.readsock.recv(4096)15:06
alekseyruban  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 349, in recv15:06
alekseyruban    return self._recv_loop(self.fd.recv, bufsize, flags)15:06
*** bluex has joined #openstack-cinder15:06
*** avishay has quit IRC15:07
alekseyrubanAlso volume status becomes to be downloading15:08
*** crose has joined #openstack-cinder15:08
*** gouthamr has joined #openstack-cinder15:09
jgriffithalekseyruban: sadly I haven't quite figured out privsep yet15:09
jgriffithalekseyruban: I think hemna was working with Gus on that?15:09
jgriffithalekseyruban: he may have some insight, otherwise I'd have to try and duplicate the failure15:10
jgriffithalekseyruban: are you using the LVM driver?15:10
alekseyrubanNexenta iSCSI for NexentaStor415:10
*** mmotiani_ has joined #openstack-cinder15:10
alekseyrubanIt strated to happen a month ago15:10
*** merooney has quit IRC15:11
*** haplo37 has joined #openstack-cinder15:11
jgriffithalekseyruban: sounds like maybe you had a rootwrap cmd that didn't get copied over?15:11
*** lennyb has joined #openstack-cinder15:11
jgriffithalekseyruban: or doesn't work quite right any more?15:11
*** buhman has joined #openstack-cinder15:11
alekseyrubanjgriffith: we didn't touch the driver. It stopped working. Also out nedge driver has the same problem15:12
jgriffithalekseyruban: right, that's what I'm saying... one of the changes to do the conversion may have left your driver behind15:13
jgriffithalekseyruban: see what I mean?15:13
jgriffithalekseyruban: do you have a full trace of the failure?15:13
*** jwcroppe has joined #openstack-cinder15:14
jgriffithalekseyruban: like something here maybe?15:14
jgriffithalekseyruban: I don't know how... but worth looking at15:14
*** Swanson has quit IRC15:14
jgriffithalekseyruban: oh... wati, that's just a comment change15:15
alekseyrubanIt's the full. There is no any exception in the log. And it happens suddenly. It can be working some time and then fell down15:15
alekseyrubanIt's the full. There is no any exception in the log. And it happens suddenly. It can be working some time and then falls down15:15
jgriffithalekseyruban: hmm... that's rather disappointing15:15
jgriffithalekseyruban: might have better luck pinging the privsep experts... maybe in oslo?15:15
jgriffithalekseyruban: hemna is still a good starting point as I think his brick code is where that's getting picked up15:16
*** jwcropp__ has quit IRC15:16
*** jwcroppe_ has joined #openstack-cinder15:17
*** Swanson has joined #openstack-cinder15:17
*** eantyshev has joined #openstack-cinder15:17
*** itlinux has joined #openstack-cinder15:18
*** avishay has joined #openstack-cinder15:19
*** jwcroppe has quit IRC15:19
alekseyrubanjgriffith: thanks. I'll try oslo15:20
*** alekseyruban has quit IRC15:21
*** xinli has joined #openstack-cinder15:24
*** karthikp has quit IRC15:28
*** iceyao has joined #openstack-cinder15:30
*** savihou has quit IRC15:30
*** kamsz has joined #openstack-cinder15:31
*** esker has joined #openstack-cinder15:31
*** esker has quit IRC15:31
*** esker has joined #openstack-cinder15:32
*** leeantho has joined #openstack-cinder15:33
*** harlowja_at_home has joined #openstack-cinder15:34
*** lcurtis has joined #openstack-cinder15:36
*** kmartin has joined #openstack-cinder15:38
yuriy_n17jgriffith: Hi! Very sorry for disturbing. Please review https://review.openstack.org/#/c/314482/ whenever you have free time.15:40
openstackgerritOpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements  https://review.openstack.org/31634315:43
*** dustins_ has joined #openstack-cinder15:44
*** esker has quit IRC15:44
jgriffithyuriy_n17: that looks great15:44
jgriffithyuriy_n17: might be worth adding a unit test15:44
jgriffithyuriy_n17: would you mind doing that?15:45
jgriffithI'll +2/A it if you do15:45
jgriffithyuriy_n17: meh15:45
jgriffithyuriy_n17: never mind15:45
*** dustins has quit IRC15:46
*** dillaman has joined #openstack-cinder15:46
yuriy_n17jgriffith: Thank you ever so much. Sorry for disturbing.15:47
openstackgerritOpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements  https://review.openstack.org/31634315:55
*** sheel has joined #openstack-cinder15:56
*** fernnest has joined #openstack-cinder15:57
*** fernnest_ has joined #openstack-cinder15:57
sheeljungleboyj: hi15:58
jungleboyjsheel: Hi.15:58
*** jwcroppe_ has quit IRC15:59
sheeljungleboyj:  regarding backporting of https://review.openstack.org/#/c/316353/115:59
*** jwcroppe has joined #openstack-cinder15:59
sheeljungleboyj: actually we want to remove xml API from API ref in newton as same is removed from cinder API..15:59
sheeljungleboyj:  but it is not possible to remove this from newton untill unless we backport API-ref in mitaka15:59
sheeljungleboyj: so plan was like16:00
sheel1. Implement it with xml in newton.16:00
sheel2. Backport to mitaka.16:00
sheel3. Remove XML api from Newton.16:00
openstackgerritOpenStack Proposal Bot proposed openstack/os-brick: Updated from global requirements  https://review.openstack.org/31695316:00
*** adrianofr has joined #openstack-cinder16:01
*** rlrossit has joined #openstack-cinder16:02
*** jwcroppe_ has joined #openstack-cinder16:02
*** julim has quit IRC16:04
*** jwcroppe has quit IRC16:04
*** savihou has joined #openstack-cinder16:05
*** julim has joined #openstack-cinder16:06
*** earlephilhower has quit IRC16:06
sheeljungleboyj: are you  convinced?  ;)16:07
jungleboyjsmcginnis: ^^^ Have you been involved in the work that sheel is talking about?16:08
*** timcl has quit IRC16:08
*** raunak has joined #openstack-cinder16:09
jgriffithsheel: did you see this?16:09
jgriffithhttp://lists.openstack.org/pipermail/openstack-dev/2016-May/095077.html16:09
sheeljgriffith: yep..I am going to update accordingly16:09
jgriffithsheel: k, cool16:09
sheeljgriffith:  sean dague update me about it16:09
jgriffithsheel: ahh... excellent16:09
sheeljgriffith: thanks16:10
sheeldulek: hey.. you back..16:10
jgriffithsheel: hehe... ok, I see the conversation in the comments :)16:10
sheeljgriffith:  ;)16:11
sheeldulek:  congratulations... good luck for new role..16:11
sheeljgriffith:  still a lot of things to clear from API-ref16:12
sheeljgriffith: too big thing to cover..16:12
jgriffithsheel: yeah... honestly you could almost consider the lack of api-ref as a bug :)16:12
sheeljgriffith: targetting one resource (all APIs) a day..16:12
jgriffithsheel: that's pretty ambitious16:12
sheeljgriffith: hope i will get support from reviewers...(I always get :))16:13
* sheel finds it quite boring to implement API-ref...16:14
sheelneed motivation to complete it...  :)16:14
mtaninobut it will help admin and users, I hope!16:14
sheelmtanino:  yep for sure...16:15
* sheel mtanino: you can refer one at https://review.openstack.org/#/c/316587/16:15
*** salv-orl_ has joined #openstack-cinder16:15
*** Simonsen has joined #openstack-cinder16:16
*** laughterwym has quit IRC16:16
jungleboyjsheel: Ok, I guess I was not aware that this would require a backport to Mitaka.  I will pull my -2 off and stay out of the way.16:16
openstackgerritKarthik Prabhu Vinod proposed openstack/cinder: Add ServiceTopic & ServiceDisabledReason Enum  https://review.openstack.org/31317916:16
*** laughterwym has joined #openstack-cinder16:17
mtaninosheel: wow, large patch...16:17
sheeljungleboyj:  yes please...but please review once i am done with cleaning16:17
sheeljungleboyj: i will remind you ...16:17
*** jwcroppe has joined #openstack-cinder16:17
sheelmtanino:  yes... I have other 2000 LOC in my vm which is cleaned for current API ref... going to come in next 2 days... ;)16:18
*** salv-orlando has quit IRC16:18
*** jwcroppe_ has quit IRC16:21
*** laughterwym has quit IRC16:21
*** julim has quit IRC16:23
openstackgerritAdriano Freires Rosso proposed openstack/cinder: HNAS drivers refactoring  https://review.openstack.org/28546916:25
openstackgerritAdriano Freires Rosso proposed openstack/cinder: Deprecating XML config file  https://review.openstack.org/29407616:25
*** e0ne has quit IRC16:25
*** lpetrut has quit IRC16:27
*** merooney has joined #openstack-cinder16:30
*** Apoorva has joined #openstack-cinder16:30
*** darosale has quit IRC16:31
*** julim has joined #openstack-cinder16:34
*** caspinol has quit IRC16:35
*** lprice has quit IRC16:35
*** savihou has quit IRC16:40
*** IlyaG has joined #openstack-cinder16:41
*** savihou has joined #openstack-cinder16:41
*** The_Ball has joined #openstack-cinder16:45
eharneyguitarzan: do you have a full log from https://bugs.launchpad.net/cinder/+bug/1582306 ?16:46
openstackLaunchpad bug 1582306 in Cinder "test_retype_volume_migration_failed "Notifier count incorrect"" [Undecided,New]16:46
*** merooney has quit IRC16:47
*** links has quit IRC16:47
*** timcl has joined #openstack-cinder16:48
xinlixyang1: two items 1) the patch # 307451 https://review.openstack.org/#/c/307451, the Flashsystem driver owner reviewed code and also Flashsystem CI passed. 2) for 268283, I fixed partial code, but I have one question, does cinder bd maintain the cgsnapshot status or the driver code will update it aftter the delete_cgsnapshots() completes?16:49
duleksheel: Thanks! :)16:53
*** merooney has joined #openstack-cinder16:53
*** harlowja_at_home has quit IRC16:53
*** IlyaG has quit IRC16:54
*** jgregor has quit IRC16:55
openstackgerritKendall Nelson proposed openstack/cinder: Add SnapshotStatus enum field  https://review.openstack.org/28482516:56
openstackgerritEric Harney proposed openstack/cinder: Tests: Use unique volume IDs in backup tests  https://review.openstack.org/31699316:57
*** baumann has quit IRC16:57
*** cknight has quit IRC16:58
*** cknight has joined #openstack-cinder16:58
xyang1xinli: driver return status and manager will update db based on what driver returns16:59
*** diablo_rojo1 has joined #openstack-cinder17:00
*** jistr has quit IRC17:02
*** alkhodos has joined #openstack-cinder17:04
*** jungleboyj has quit IRC17:05
*** cknight1 has joined #openstack-cinder17:05
*** diablo_rojo has quit IRC17:05
*** merooney has quit IRC17:05
*** cknight has quit IRC17:07
alkhodosHi everyone. Does anyone else have issues with creating iSCSI volumes from images or maybe know why this happens? When I try to do so it can either create successfully or just hang in "downloading" state forever, and it seems completely random.17:07
smcginnisalkhodos: There was someone here earlier having privsep issues with Nexenta,17:08
smcginnisalkhodos: Sounds kind of similar.17:08
smcginnisalkhodos: Do you see any traceback errors in the logs?17:08
alkhodos@smcginnis: I am from Nexenta too :) I only see traceback in warnings, but they are there in successful creations too17:09
smcginnisalkhodos: Yep, I knew, just wasn't sure if you were actually hitting the same thing. :)17:10
smcginnisalkhodos: Sounds different.17:10
alkhodos@smcginnis: well it is related to privsep17:10
*** jdurgin1 has quit IRC17:10
alkhodos@smcginnis: but I just can't even think of how to debug this anymore17:10
alkhodos@smcginnis: since it can go 5 times success and then 5 times hang17:11
smcginnisalkhodos: Do you know where things are hanging?17:11
openstackgerritKendall Nelson proposed openstack/cinder: Add VolumeAttachStatus Enum  https://review.openstack.org/29088517:11
*** diablo_rojo1 has quit IRC17:11
*** avishay has quit IRC17:12
*** david-lyle_ has quit IRC17:12
*** david-lyle has joined #openstack-cinder17:13
alkhodoshere is what I have in log http://paste.openstack.org/show/497246/ After this it just stops responding.17:14
*** esker has joined #openstack-cinder17:14
*** esker has quit IRC17:14
ildikovthingee: hi17:14
*** esker has joined #openstack-cinder17:14
ildikovthingee: sorry, I gave it up on Friday by the time you pinged17:15
ildikovthingee: my question is basically that which channel to use for the Cinder-Nova meeting we're having for a short while now, I got lost in the mail thread17:15
smcginnisalkhodos: Hmm, looks like it successfully reads and returns the contents of /etc/isci/iniatorname.iscsi17:16
hemnahey what's up17:20
smcginnisalkhodos: Check the memory consumption next time it hangs.17:20
hemnasmcginnis, alkhodos attach issues ?17:20
smcginnishemna: Occasionally stuck with "Downloading" status.17:20
alkhodoshemna: not really, attach goes fine17:21
smcginnishemna: Am I missing something? This looks like infinite recursion: https://github.com/openstack/os-brick/blob/6d71d55caf50d56ee23f314c1fa1e3fed1626c4d/os_brick/initiator/connector.py#L14717:21
hemnaheh no17:21
hemnathe connector is the class17:22
hemnanot the function in the file called connector.py17:22
hemnahttps://github.com/openstack/os-brick/blob/6d71d55caf50d56ee23f314c1fa1e3fed1626c4d/os_brick/initiator/connector.py#L14117:22
smcginnisAh, I see.17:22
hemnawe give every connector class a chance now to report any initiator information17:22
hemnaalkhodos, are you using an unreleased os-brick ?17:23
hemnafrom master ?17:23
guitarzaneharney: that one isn't me, sorry17:23
alkhodoshemna: it's the same on master and stable/mitaka17:24
hemnathe latest released 1.3.0 os-brick doesn't have privsep yet17:24
*** dims_ has joined #openstack-cinder17:25
alkhodoshemna: where can I check os-brick ver?17:26
*** dims has quit IRC17:26
hemnapip freeze |grep os-brick17:27
eharneyguitarzan: whoops, i knew that :)17:27
*** jungleboyj has joined #openstack-cinder17:28
guitarzanyou've got a patch anyway :)17:29
*** enriquetaso has joined #openstack-cinder17:29
alkhodos$ pip freeze |grep os-brick -e git://git.openstack.org/openstack/os-brick.git@934cdef509ca324b14f4ea5269524e48e924ce09#egg=os_brick17:29
xinligeguileo: I fixed all the issue which you mentioned in code review patch # 268283, but I stll have one question regarding the CG snapshot status update. The link: https://review.openstack.org/#/c/268283/6/cinder/volume/drivers/ibm/gpfs.py17:30
hemnaalkhodos, ok that's a non released version17:30
hemnaprobably from master is my guess17:30
alkhodoshemna: turns out I had this in my local.conf; LIBS_FROM_GIT=os-brick17:31
*** lprice has joined #openstack-cinder17:31
alkhodoshemna: don't remember why already...17:31
alkhodoshemna: probably the reason17:31
hemnawell17:31
hemnaso....17:31
*** suyash has joined #openstack-cinder17:31
hemnait would still be nice to find out where you are hanging17:31
hemnaif it is a bug in brick, we need to find it :)17:31
hemnaso you can remove the existing install17:32
hemnaand install from pip17:32
hemnaand see if the issue goes away17:32
hemnaif it does, then install from git again...17:32
hemnaplease :)17:32
alkhodoshemna: I'm already in progress of installing fresh devstack without that line :)17:32
*** esker has quit IRC17:33
*** mvk has quit IRC17:33
smcginnisalkhodos: Good to have it there for testing against os-brick before it's released. But probably don't want to run that way otherwise. ;)17:33
*** baumann has joined #openstack-cinder17:33
smcginnisSo it does look there's an issue there.17:33
alkhodoshemna: @smcginnis: thanks for your help, will update with my results :)17:33
hemnaok cool17:33
smcginnisWe probably do need to figure out what's going on before we release a new os-brick.17:34
*** jungleboyj has quit IRC17:34
hemnaif the volume has attached17:34
hemnathen brick was done17:34
hemnaalkhodos, can you post the log in pastebin?17:34
hemnaI'd like to see them17:34
alkhodoshemna: ye sure, which logs do u need?17:34
alkhodoshemna: just the c-vol?17:35
hemnac-vol17:35
hemnathis was from a copy image to volume ?17:35
hemnaif so, then all the logs are in c-vol17:36
alkhodoshemna: yes, copy image to vol. Here is the full log from start of create till it hangs http://paste.openstack.org/show/497250/17:38
hemnathanks17:39
hemnaok17:39
hemnaso those privesep WARNINGS aren't really warnings17:39
hemnaI think those should be changed to DEBUG17:39
hemnaWARNING oslo.privsep.daemon [-] privsep log: CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.005s17:40
hemna0 return = success17:40
hemnafwiw, that doesn't look like the complete log17:41
hemnathat's only the front side of the attach process17:41
hemnaattach process is basically this:17:41
hemna1) collect initiator information17:41
hemna2) call driver to attach (initialize_connection)17:41
hemna3) discover volume attachment17:41
hemnathat log only got to the end of part 117:41
*** merooney has joined #openstack-cinder17:42
alkhodoshemna: here is what goes after that http://paste.openstack.org/show/497252/17:42
alkhodosjust the status checks17:43
hemnahrmm17:44
hemnathat's not good17:45
hemnawe should see it trying other connectors17:46
hemnahttps://github.com/openstack/os-brick/blob/master/os_brick/initiator/connector.py#L14517:46
*** mriedem is now known as mriedem_afk17:46
alkhodoshemna: that is what I can't understand, sometimes it does and sometime it just stops here17:47
*** rcernin has quit IRC17:47
*** diablo_rojo has joined #openstack-cinder17:51
*** e0ne has joined #openstack-cinder17:52
hemnaalkhodos, do you still have the VM with that setup ?17:54
openstackgerritVasanthi Thirumalai proposed openstack/cinder: Violin Memory ISCSI storage for 7000 series AFA  https://review.openstack.org/26979617:55
alkhodoshemna: yes17:55
hemnaalkhodos, https://gist.github.com/2f74696645f6d2cea9c34570af9c45a318:00
hemnaapply that and re-run the test18:00
hemnasee if you can reproduce it18:00
hemnathat should output some more debugging along the process of building the connector.18:01
hemnawe'll see if it gets out of that function or hangs.18:01
*** bardia has joined #openstack-cinder18:02
*** rlrossit has quit IRC18:03
*** dustins_ has quit IRC18:05
*** kfarr has quit IRC18:05
*** dustins has joined #openstack-cinder18:05
*** julim has quit IRC18:07
*** dustins has quit IRC18:07
*** dustins has joined #openstack-cinder18:08
*** julim has joined #openstack-cinder18:08
*** delattec has joined #openstack-cinder18:09
*** yangyapeng has quit IRC18:10
*** cdelatte has quit IRC18:12
*** merooney has quit IRC18:13
*** jungleboyj has joined #openstack-cinder18:13
*** delatte has joined #openstack-cinder18:14
*** IlyaG has joined #openstack-cinder18:16
*** laughterwym has joined #openstack-cinder18:17
*** delattec has quit IRC18:17
*** merooney has joined #openstack-cinder18:20
*** aleksey2 has joined #openstack-cinder18:21
*** aleksey2 has quit IRC18:21
*** aleksey4 has joined #openstack-cinder18:22
*** laughterwym has quit IRC18:22
*** aleksey4 is now known as alekseyruban18:22
*** savihou has quit IRC18:26
*** kfarr has joined #openstack-cinder18:26
*** e0ne has quit IRC18:28
*** e0ne has joined #openstack-cinder18:31
*** diablo_rojo has quit IRC18:32
*** diablo_rojo has joined #openstack-cinder18:32
*** baumann1 has joined #openstack-cinder18:33
*** baumann has quit IRC18:33
*** baumann has joined #openstack-cinder18:33
alkhodoshemna: funny thing - I'm not able to reproduce the issue after I added additional debugging18:34
hemnahrmm18:34
hemnacomment out the debugging ?18:35
hemnawth18:35
*** e0ne has quit IRC18:35
*** IlyaG has quit IRC18:36
*** baumann1 has quit IRC18:37
*** jgregor has joined #openstack-cinder18:39
*** lpetrut has joined #openstack-cinder18:40
*** jgregor has quit IRC18:40
*** merooney has quit IRC18:40
*** krtaylor has quit IRC18:44
*** mriedem_afk is now known as mriedem18:44
*** yangyapeng has joined #openstack-cinder18:46
*** merooney has joined #openstack-cinder18:47
openstackgerritWalter A. Boring IV (hemna) proposed openstack/os-brick: Add support for processutils.execute  https://review.openstack.org/30920118:48
*** iceyao has quit IRC18:49
*** kamsz has quit IRC18:54
*** kamsz has joined #openstack-cinder18:54
*** angela-s has joined #openstack-cinder18:58
*** edmondsw has quit IRC19:02
alkhodoshemna: http://paste.openstack.org/show/497257/19:06
*** salv-orl_ has quit IRC19:07
openstackgerritEric Harney proposed openstack/cinder: Tests: Use unique volume IDs in backup tests  https://review.openstack.org/31699319:09
openstackgerritEric Harney proposed openstack/cinder: Tests: Use mock for notifier tests  https://review.openstack.org/31705519:09
openstackgerritKendall Nelson proposed openstack/os-brick: WIP: Splitting Out Connectors from connector.py :WIP  https://review.openstack.org/30797419:13
xinlixyang1: I reviewed your code review comments. yes, I didn't modify the code in flashSystem FC, the component owner still insists that I should add the version updated in order to make consistent. this is his answer: Yes, we have to make all history on the same page.19:14
xinliBest Wishes,19:14
xinliEdwin Wang 王杨鸣19:14
*** merooney has quit IRC19:16
openstackgerritHelen Walsh proposed openstack/cinder: EMC VMAX - locking SG for concurrent threads  https://review.openstack.org/31705719:17
*** arch-nemesis has quit IRC19:17
*** merooney has joined #openstack-cinder19:20
*** merooney has quit IRC19:20
*** raunak has quit IRC19:21
*** merooney has joined #openstack-cinder19:21
*** caspinol has joined #openstack-cinder19:22
openstackgerritHelen Walsh proposed openstack/cinder: EMC VMAX - iSCSI Multipath support  https://review.openstack.org/31706019:23
openstackgerritAngela Smith proposed openstack/cinder: Fix Brcd zone driver initiator zone update  https://review.openstack.org/31706119:24
*** lpetrut has quit IRC19:24
openstackgerritSean McGinnis proposed openstack/cinder: Add sample config file to cinder docs  https://review.openstack.org/31422019:27
*** diablo_rojo has quit IRC19:29
*** Simonsen has quit IRC19:30
*** caspinol has quit IRC19:30
*** Lee1092 has quit IRC19:32
*** Simonsen has joined #openstack-cinder19:34
openstackgerritMerged openstack/os-brick: Updated from global requirements  https://review.openstack.org/31695319:36
*** e0ne has joined #openstack-cinder19:38
hemnaalkhodos, still looks like it's stuck ?19:38
*** Simonsen_ has joined #openstack-cinder19:39
openstackgerritSean McGinnis proposed openstack/cinder: Add driver interface checks  https://review.openstack.org/29714019:39
*** e0ne has quit IRC19:40
*** Simonsen has quit IRC19:41
openstackgerritXinli Guan proposed openstack/cinder: IBM FlashSystem: Cleanup host resource leaking  https://review.openstack.org/30745119:42
*** rcernin has joined #openstack-cinder19:43
*** edmondsw has joined #openstack-cinder19:44
*** sheel has quit IRC19:45
*** kfarr has quit IRC19:48
*** kfarr has joined #openstack-cinder19:53
*** jgregor has joined #openstack-cinder19:54
*** caspinol has joined #openstack-cinder19:55
*** merooney has quit IRC19:58
*** karthikp has joined #openstack-cinder20:04
openstackgerritEric Harney proposed openstack/cinder: Tests: Don't attempt connection for Coho driver init  https://review.openstack.org/31707120:05
*** diablo_rojo has joined #openstack-cinder20:05
*** raunak has joined #openstack-cinder20:07
*** krtaylor has joined #openstack-cinder20:08
xyang1xinli: hi, just stating that this is for iscsi in history sounds good to me20:12
*** cknight1 has quit IRC20:12
*** caspinol has quit IRC20:13
*** xingyang has joined #openstack-cinder20:16
*** salv-orlando has joined #openstack-cinder20:17
xinlismcginnis: I uploaded a PatchSet for IBM FlashSystem, but Gerrit throws error message that HGST Solution CI syaing the code could not be megred. https://review.openstack.org/#/c/307451/ Do you have any suggestion to debug?20:17
smcginnisxinli: That is an issue with the HGST CI system. You don't need to worry about that.20:18
*** caspinol has joined #openstack-cinder20:21
*** avishay has joined #openstack-cinder20:22
xingyangsmcginnis, jungleboyj, hemna: we returned size in model_update in create_volume in scaleio driver for the 8GB issue discussed at the summit: https://review.openstack.org/#/c/316811/20:24
xingyangsmcginnis, jungleboyj, hemna: but it could not pass CI. There is a check in the API that blocks creating a volume from snapshot if volume size is smaller than source: https://github.com/openstack/cinder/blob/master/cinder/volume/flows/api/create_volume.py#L14620:25
smcginnisxingyang: SHoot, didn't think of that.20:25
jungleboyjOOps.20:25
smcginnisSo maybe no easy fix on that one. :/20:25
xingyangsmcginnis, jungleboyj: more complicated than we thought:(20:26
jungleboyjxingyang: :-(20:26
smcginnisI think we'll have to add the mechanism for drivers to report back their size allocation units, then on things like requesting a volume from snapshot we can see that even though they are requesting 1 they will get 8.20:26
eharneywouldn't returning back a different size cause the quotas to be broken anyway?20:27
xingyangsmcginnis: tempest will try to create 1GB volumes, so even with that we still could not pass tempest20:27
openstackgerritEric Harney proposed openstack/cinder: Move CG unit tests to test_cg.py  https://review.openstack.org/31573620:28
xingyangeharney: this was the temp solution we agreed at the summit20:28
openstackgerritAngela Smith proposed openstack/cinder: Fix Brcd zone driver initiator zone update  https://review.openstack.org/31706120:28
eharneyxingyang: it may be a broken temp solution20:28
*** mriedem is now known as mriedem_away20:28
smcginnisxingyang: Does tempest check the returned size?20:28
xingyangsmcginnis: no20:28
eharneyjust because people agreed doesn't make it all correct20:28
*** caspinol has quit IRC20:29
smcginnis:)20:29
xingyangsmcginnis: I think if you don’t pass a size, 1 is the default:)20:29
smcginnisxingyang: Yep20:29
smcginnisxingyang: So we would definitely have to build in awareness to API calls and defaults that allocation unit would need to be taken into account.20:30
*** ericksonsantos has quit IRC20:30
xingyangsmcginnis: right20:30
*** krtaylor has quit IRC20:32
*** raildo is now known as raildo-afk20:33
*** raildo-afk is now known as raildo20:33
jungleboyjsmcginnis: ++20:33
*** avishay has quit IRC20:33
*** ericksonsantos has joined #openstack-cinder20:34
*** sdake has quit IRC20:37
*** sdake has joined #openstack-cinder20:37
*** savihou has joined #openstack-cinder20:40
*** xingyang has quit IRC20:41
*** xingyang has joined #openstack-cinder20:41
*** ericksonsantos has quit IRC20:41
*** liverpooler has quit IRC20:42
*** ericksonsantos has joined #openstack-cinder20:42
*** Simonsen_ has quit IRC20:46
*** mriedem_away is now known as mriedem20:46
*** liverpooler has joined #openstack-cinder20:46
eharneyfwiw the summit etherpad seems to say "Not a good idea to hack around this." w.r.t. quota issues.20:47
jungleboyjeharney: Confirmed.  :-)20:47
xingyangeharney: yes, we didn’t have a solution for the quota issue20:48
xingyangeharney: we asked whether we could return model_update as a partial solution and people were ok at the summit20:48
xingyangeharney: it turns out this does not work either20:48
xingyangeharney: so unless someone has another idea, we’ll wait for the complete solution20:49
*** CIA has quit IRC20:51
*** Yogi1 has quit IRC20:51
*** angela-s has quit IRC20:52
*** porrua has quit IRC20:52
*** gouthamr has quit IRC20:54
*** julim has quit IRC20:55
*** eharney has quit IRC20:57
openstackgerritMerged openstack/cinder: VMware: Reduce volume creation time  https://review.openstack.org/29693421:06
*** timcl has quit IRC21:08
*** xingyang has quit IRC21:10
*** haplo37 has quit IRC21:11
*** diablo_rojo has quit IRC21:11
*** xingyang has joined #openstack-cinder21:11
alkhodoshemna: sorry, was afk. Yes, after a bunch of successful creations, started to hang again. Btw using stable brick helped21:12
*** savihou has quit IRC21:12
*** savihou has joined #openstack-cinder21:12
*** sgotliv_ has quit IRC21:13
*** savihou has quit IRC21:14
*** savihou has joined #openstack-cinder21:15
*** jungleboyj has quit IRC21:16
*** savihou has quit IRC21:19
*** caspinol has joined #openstack-cinder21:21
*** jgregor has quit IRC21:22
*** raildo is now known as raildo-afk21:22
*** enriquetaso has quit IRC21:29
*** crose has quit IRC21:29
hemnaos-brick 1.3.0 fix the problem ?21:30
*** gouthamr has joined #openstack-cinder21:32
*** baumann has left #openstack-cinder21:32
*** gouthamr_ has joined #openstack-cinder21:33
jgriffithxingyang: smcginnis just FYI, that's a long running *bug* that's been debated forever (create different size from snap) ask guitarzan about it :)21:34
jgriffithxingyang: smcginnis one thing you can look at is resize on your own in the driver after the call21:34
*** xingyang has quit IRC21:34
jgriffithxingyang: smcginnis but the quota thing is going to be the thorn in your side21:35
hemnayah we puked over that on Friday at the summit21:35
hemnaupdating the quota might fail...sadness21:35
*** merooney has joined #openstack-cinder21:36
*** merooney has quit IRC21:36
*** xingyang has joined #openstack-cinder21:36
*** gouthamr has quit IRC21:37
xingyangjgriffith: hi, the problem we have is scaleio can only create volume of size that is in 8GB multiples. so we try to address it temporarily by returning the real size in create volume21:39
hemnawhy would the create from snap fail?21:40
hemnaI'm slow today21:40
xingyangjgriffith: but then it could not pass tempest because tempest tries to create a volume of 1GB from a snapshot of 8GB21:40
hemnabecause the tempest test is hard coded at 1G ?21:41
hemnaand doesn't use the snapshot size ?21:41
xingyanghemna: so after we return the round up size of 8, the db is updated, a snapshot is created as 8 , but tempest always tries to create 1 GB volume from snap21:41
hemnacan't we change the tempest test to use the size of the snapshot ?21:41
hemnaunless we want to catch this particular failure in the test that's running.21:42
xingyanghemna: it fails in api verfication21:42
*** enriquetaso has joined #openstack-cinder21:42
xingyanghemna: we check in api to see if volume size is not smaller than the source snapshot size21:42
hemnayah I get that21:43
hemnawhich is a valid failure21:43
xingyanghemna: so you said we should change the tempest test?21:43
hemnawhat I'm suggesting is, does it make sense to change the tempest test itself21:43
hemnato create a volume with a size that is from the snapshot21:43
hemnainstead of a hard coded 1G value ?21:43
patrickeastseems like it makes the most sense21:43
xingyanghemna: sure, we could try that21:44
hemnaunless, the purpose of the test is to intentionally find this case :P21:44
*** karthikp has quit IRC21:44
patrickeasthaha21:44
xingyanghemna: :)21:44
hemnaxingyang, do you have the url to the review w/ the failure ?21:46
hemnawhich tempest test pukes ?21:46
*** xingyang has quit IRC21:47
*** xingyang has joined #openstack-cinder21:48
xingyanghttps://review.openstack.org/#/c/316811/221:48
*** kfarr has quit IRC21:49
xingyanghemna: Details: {u'code': 400, u'message': u"Invalid input received: Volume size '1'GB cannot be smaller than the snapshot size 8GB. They must be >= original21:49
hemnatempest/scenario/test_volume_boot_pattern.py21:50
*** dustins has quit IRC21:50
xingyanghemna: if I’m not mistaken, the test does not pass in a size.  by default the size is 1 if not passed in21:51
*** mriedem has quit IRC21:52
hemnayah it's not using a size at all21:52
hemnahttps://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L10321:52
hemnahttps://github.com/openstack/tempest/blob/master/tempest/scenario/manager.py#L271-L27221:53
hemnaso it only adds size if it's passed in.21:53
hemnapython-cinderclient requires a size on create21:55
xingyanghemna: there is a default size of 1 here: https://github.com/openstack/tempest/blob/master/tempest/services/volume/base/base_volumes_client.py#L3021:55
xingyanghemna: https://github.com/openstack/tempest/blob/master/tempest/services/volume/base/base_volumes_client.py#L8021:56
hemnaah default_volume_size21:56
hemnaso....21:56
hemnaI don't think it's unreasonable in this case to pull the size out of the snapshot21:57
hemnaand pass it21:57
*** markvoelker has joined #openstack-cinder21:57
*** edmondsw has quit IRC21:58
hemnahttps://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L10321:58
*** xingyang has quit IRC21:58
jgriffithhemna: You mean the parent volume's size?21:58
hemnahave to pull the snapshot and then get the size21:59
jgriffithhemna: snapshots don't have a "size"21:59
hemnasure21:59
*** xingyang has joined #openstack-cinder21:59
xingyanghemna: sure. I think there are probably a few other tests too that have the same issues.21:59
jgriffithxingyang: +121:59
jgriffithxingyang: I think there are some far reaching side effects if you do that22:00
hemnathe error suggests that the snapshot does have a size22:00
hemnathe wording at least22:00
*** jwcroppe has quit IRC22:00
jgriffithhemna: https://github.com/openstack/cinder/blob/master/cinder/db/sqlalchemy/models.py#L45222:00
hemnahttps://github.com/openstack/cinder/blob/master/cinder/db/sqlalchemy/models.py#L47222:00
hemnathat one ?22:01
*** markvoelker has quit IRC22:01
jgriffithhemna: so should I just paste the same link for a 3'rd time :)22:01
*** markvoelker has joined #openstack-cinder22:01
hemnaI must be missing something22:01
hemnasorry22:01
jgriffithhemna: that's the parent volume size22:02
hemnathat snapshot object shows it has a volume_size col.22:02
xingyangjgriffith: so the volume_size is 8, but tempest tries to create a 1GB volume from that snapshot22:02
jgriffithhemna: the size of the volume you took a snapshot of at the time22:02
hemnayah yah22:02
jgriffithxingyang: yeah, I hear ya22:02
hemnaok cool. :)22:02
jgriffithI wouldn't change tempest22:02
jgriffithI'd change Cinder22:02
jgriffithrather than use the default size of 1G for create from snap.. it should use volume_size of snap22:03
hemnashouldn't the snapshot.volume_size be 8 in this case?22:03
jgriffithhemna: yes, and I think xingyang just mentioned that it is22:03
jgriffithxingyang: correct?22:03
hemnajgriffith, well tempest is basically hard coding that size to 1g22:03
jgriffithxingyang: if it's not then there's another crappy bug to be fixed22:03
hemnaby using the tempest volumes_client default_volume_size22:03
hemnawhich is a tempest'ism22:03
xingyangjgriffith: I think tempest uses 1 by default22:03
jgriffithno, we're talking about create from snapshot22:04
jgriffiththat's different22:04
hemnahttps://github.com/openstack/tempest/blob/master/tempest/services/volume/base/base_volumes_client.py#L3322:04
hemnajgriffith, correct, we are talking about create from snap22:04
hemnathat eventually boils down to a create_volume call22:04
jgriffithlet me try this again....22:04
hemnawith the tempest client22:04
hemnawhich defaults to https://github.com/openstack/tempest/blob/master/tempest/services/volume/base/base_volumes_client.py#L3322:04
jgriffithso there are two things that cause trouble here...22:05
jgriffith1. we use the same create flow for create from snap22:05
jgriffithso yes, it uses that default value22:05
jgriffith2. Cinder then gets the call and says "hey, I can't do this... the snap needs 8Gig to be restored to"22:05
jgriffithWhat I'm saying is that Cinder, should be intelligent enough to figure out how to gracefully resolve that22:06
jgriffithIMO it should take either None for size in that case OR take >= volume_size22:06
hemnabut anyone can ask cinder to create a volume from a snap, and provide an incorrect size22:07
hemnawhich is a valid failure22:07
*** xingyang has quit IRC22:07
hemnawhich I believe is the case here with the tempest test.22:07
jgriffithhemna: yep, you're correct22:07
hemnaso None for a size would be better than a default of 122:07
jgriffiththe intent of that test is actually that "everything" is just always 1G22:07
jgriffithhemna: so yes, that's where I was going.  Modify Cinder to accept none for create --snapshot-id22:08
jgriffithwhich means, whatever it's supposed to be based on volume_size in the snapshot object22:08
jgriffithTHEN22:08
hemnawouldn't that require us to change the tempest default_volume_size to None then ?22:08
hemnafor this to pass ?22:08
jgriffithmodify create-snapshot in tempest appropriately22:08
jgriffithBUT!!!22:08
hemnaah yah22:08
hemnaok22:08
jgriffithwhat I'd like to know is how you're passing anything in Tempest now22:08
*** xingyang has joined #openstack-cinder22:09
jgriffithbecause you're obviously lying to Tempest when you say "create 1"22:09
jgriffithYou're creating an 8G volume22:09
hemnawell, scaleio driver can't do a 1g Volume22:09
jgriffithhemna: I'm aware22:09
hemnait's doing a model update after create_volume22:09
jgriffithhemna: and qutoas mean nothing :(22:10
alkhodoshemna: yes, using released brick solved the issue22:10
hemnaalkhodos, ok...boo! and yeah! at the same time.22:10
jgriffithall I'm saying is I'm not a fan of just "lieing" about things22:10
hemnaand by lying, you mean the model update?22:10
jgriffithbut I guess that's up to Scale-IO support to deal with22:10
alkhodoshemna: still I couldn22:10
alkhodoscouldn't figure out why it failed22:11
hemnaalkhodos, does that fail every create volume from image call ?22:11
jgriffithjust tell your users they have to create things in 8Gig increments22:11
jgriffithotherwise it fails22:11
alkhodoshemna: not every, that's my biggest concern22:11
jgriffithOR make a conf option for Cinder if you must22:11
jgriffithand have it do it at the API layer22:11
xingyangjgriffith: but tempest doesn’t create 8GB, it create 1GB:)  we try to pass the tempest test22:11
alkhodoshemna: sometimes they fail, sometimes they succeed. The exact same cmd22:12
jgriffithbut my point is this work of hacking up all these little corners of the Cinder code is nothing but trouble22:12
jgriffithxingyang: yes... so what i'm saying is change Cinder22:12
jgriffithxingyang: in the case of Cinder and ScaleIO set var that says "minimum size = 8"22:12
hemnaxingyang, yah but tempest is 'dumb' in the respect that it thinks it should default to 1G and there is never a difference between what it asks cinder to create and what cinder creates.22:12
jgriffiththen every time a call comes in to create at API layer it checks that22:13
jgriffithand does a moulus on it22:13
jgriffithrounds it up or whatever22:13
jgriffiththen just passes the "correct size" all the way through the flow22:13
hemnajgriffith, a customer scheduler filter maybe ?22:13
hemnacustom22:13
jgriffithrather than this crazy monkeying around that's guaranteed to come back and bight someone in the butt some day22:13
jgriffithhemna: well, not even that22:14
hemnaone of my suggestions in the summit was to just fail in the driver22:14
hemnaif it wasn't a mod 822:14
jgriffithhemna: I mean literally set a default-size in the config and adjust before it even gets to cinder.volume.api22:14
xingyangjgriffith: we still need to change the default_volume_size in tempest from 1 to None?22:14
*** salv-orl_ has joined #openstack-cinder22:14
jgriffithxingyang: not really :)22:14
jgriffithxingyang: because if you have this set on your config in your ci system it gets the call... says "nope, 1 isn't valid... bump it up"22:15
jgriffithxingyang: here, let me show you... gimmie a minute22:15
xingyangjgriffith: thanks!22:15
jgriffithof course I don't know where the hell to change it... V1, V2 or V322:16
jgriffithsigh22:16
xingyang:)22:16
xingyangI assume we can only change in v3 now22:17
*** salv-orlando has quit IRC22:18
Swanson"bight"?22:18
jgriffithhttps://gist.github.com/j-griffith/3d89e31a49cc5686586f1369c6aa545122:19
jgriffithxingyang: that only works if you make that a global setting22:19
xingyangjgriffith: sure22:20
jgriffithxingyang: in which case as hemna pointed out you can have a filter.... fail if it's not mod 822:20
jgriffithxingyang: but in either case, for snapshots, it's easy to just add the same auto-set/adjust in the file I just referenced for snapshots22:21
xingyangjgriffith: so if we do a mod 8 there instead of min size, what do you think?22:22
xingyangjgriffith: a formula in cinder.conf22:22
*** ducttape_ has quit IRC22:22
jgriffithhttps://gist.github.com/j-griffith/0755dcda222473e810ac16e8d49c636722:23
jgriffithxingyang: so yeah, a min value in cinder.conf is what i was thinking.  That second gist solves the snapshot problem IMO22:24
jgriffithand that could be something like "auto_correct_create_from_snapshot_size=8"  or something longer maybe :)22:24
jgriffithxingyang: that would preserve the test, make overriding things explicit so there's no magic22:25
xingyangjgriffith: sure, this sounds good22:25
jgriffithxingyang: would make quotas just *work* the way they're supposed to, and negate any funny business with model updates etc22:25
jgriffithxingyang: it's not as clean on a regular create, I'd have to think about that a bit (because of multi-backend)22:26
xingyangjgriffith: so for snapshot, we can do a mod 8, but you don’t suggest mod 8 in the first place?22:26
jgriffithxingyang: yeah, so that's the problem in the first case, multi-backend22:26
jgriffithor maybe you say "DONT RUN SCALEIO IN  MULTI_BACKEND CONFIG"22:26
jgriffithwait, no22:27
jgriffithcan't do that :)22:27
xingyangjgriffith: ?22:27
xingyangjgriffith: which part?22:27
jgriffithnothing, thinking out loud :)22:27
*** tpsilva has quit IRC22:27
jgriffithso create-from-snapshot or create-from-volume I think is pretty safe and that suggestion works well22:27
jgriffithfor create-raw it's tricky though22:27
jgriffithsay they have an LVM backend and a ScaleIO backend.....22:28
jgriffithWe shouldn't force the LVM volumes to 8Gig22:28
xingyangjgriffith: that’s true22:28
jgriffithand we don't know where the create is going until it's too late22:28
jgriffithYou could deal with it via types, but that's not 100%22:29
xingyangjgriffith: if we can have the config work for each backend, then we can do it?22:29
jgriffithxingyang: how would you do that?22:29
jgriffithI don't think that's possible at this layer22:29
jgriffithUnless you require the type.. in which case that's cool22:29
xingyangjgriffith: I mean if the config is at per backend level, so that will be different for each backend?22:29
jgriffithand to be clear, type with a volume_backend_name in it22:29
jgriffithxingyang: no, you can't do that at the API22:30
jgriffithxingyang: you don't necessarily know which backend it's going to be scheduled on22:30
*** Julien-zte has joined #openstack-cinder22:30
jgriffithyou don't have enough info yet22:30
xingyangjgriffith: you are right22:30
jgriffithxingyang: so I think I mentioned before... it kinda sucks but;22:31
jgriffithwhat you should probably do is set a stats report from your driver that says "min-vol-size=8"22:31
jgriffithand then the filter scheduler as is should disqualify you for anything that isn't a multipe of 8Gig22:31
jgriffithIMHO that's the only really correct way to do this22:32
xingyangjgriffith: but that will not help us pass the tempest test.  we still need the change you suggested in API22:32
jgriffithYou do that in conjunction with the v2/volumes.py changes I mentioned above for snaps and clones22:32
jgriffithyeah, that's a tough one :)22:33
jgriffithI'm saying you do both of those things... I can't fix the default size in tempest, you could certainly make that configurable in your CI system... in fact I think it already is22:33
*** cknight has joined #openstack-cinder22:34
xingyangjgriffith: we’ll try what you suggested in your git.  that should get us pass tempest and quota should be right too22:34
jgriffithxingyang: yeah, do something like that for snapshots and clones22:34
xingyangjgriffith: is the default size in tempest configurable?  I didn’t know about that, I’ll check22:35
jgriffithxingyang: I am not sure if it is right now or not, BUT that's much more pallatable than "lieing" about things or disabling the test IMHO22:35
jgriffithxingyang: and I think it's relatively easy to do22:36
jgriffithxingyang: and YES it's already configurable in tempest22:37
xingyangjgriffith: Is it?22:37
xingyangjgriffith: default_volume_size?22:37
jgriffithxingyang: which solves your problem anyway22:37
jgriffithxingyang: https://github.com/openstack/tempest/blob/master/tempest/clients.py#L57022:37
*** lprice has quit IRC22:37
xingyangjgriffith: awesome!  that will be the quickest fix:)22:38
jgriffithxingyang: yeah, and the most correct :)22:38
jgriffithxingyang: and it means you don't have to "lie" :)22:38
xingyangjgriffith: we’ll change that in tempest.conf.  thanks!22:38
jgriffithxingyang: lemme know how it goes22:38
*** ajnouri has quit IRC22:38
xingyangjgriffith: sure:)22:39
*** ametts has quit IRC22:41
*** xingyang has quit IRC22:41
*** sdague has quit IRC22:50
*** xinli has quit IRC22:52
*** kongwei has joined #openstack-cinder22:56
*** ajnouri has joined #openstack-cinder22:56
*** kongwei has quit IRC23:00
*** xingyang has joined #openstack-cinder23:01
*** xingyang has quit IRC23:02
*** cknight has quit IRC23:05
*** markvoelker has quit IRC23:06
*** rcernin has quit IRC23:08
*** xyang1 has quit IRC23:11
*** bardia has quit IRC23:16
*** bardia has joined #openstack-cinder23:17
*** caspinol has quit IRC23:17
*** laughterwym has joined #openstack-cinder23:18
*** laughterwym has quit IRC23:22
*** IlyaG has joined #openstack-cinder23:25
*** lcurtis has quit IRC23:25
*** diogogmt has quit IRC23:29
*** IlyaG has quit IRC23:31
*** adrianofr has quit IRC23:32
*** jamielennox is now known as jamielennox|away23:51
*** yhayashi has joined #openstack-cinder23:51
*** yhayashi has quit IRC23:53
*** Julien-zte has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!