| *** markvoelker has quit IRC | 00:01 | |
| *** asselin_ has joined #openstack-cinder | 00:02 | |
| *** yangyapeng has quit IRC | 00:05 | |
| *** yamada-h has joined #openstack-cinder | 00:05 | |
| *** asselin_ has quit IRC | 00:07 | |
| *** tyrola has joined #openstack-cinder | 00:07 | |
| *** cknight has joined #openstack-cinder | 00:08 | |
| *** yamada-h has quit IRC | 00:09 | |
| *** tyrola has quit IRC | 00:13 | |
| *** rcernin has quit IRC | 00:13 | |
| *** laughterwym has joined #openstack-cinder | 00:13 | |
| *** laughterwym has quit IRC | 00:19 | |
| *** yhayashi has joined #openstack-cinder | 00:20 | |
| *** laughterwym has joined #openstack-cinder | 00:25 | |
| *** chlong has joined #openstack-cinder | 00:36 | |
| *** v12aml has quit IRC | 00:39 | |
| *** baater has joined #openstack-cinder | 00:41 | |
| baater | hi does someone know how to use zabbix plugin for ceph monitoring ? i did as instructed on the github, but no luck to see. | 00:51 |
|---|---|---|
| *** baater has quit IRC | 00:51 | |
| *** baater has joined #openstack-cinder | 00:52 | |
| *** baater has quit IRC | 00:53 | |
| *** cheneydc has joined #openstack-cinder | 00:59 | |
| *** v12aml has joined #openstack-cinder | 01:01 | |
| *** iceyao has joined #openstack-cinder | 01:03 | |
| *** mriedem has quit IRC | 01:04 | |
| *** yangyapeng has joined #openstack-cinder | 01:11 | |
| *** baater has joined #openstack-cinder | 01:12 | |
| *** baater has quit IRC | 01:14 | |
| *** baater has joined #openstack-cinder | 01:15 | |
| *** baater has quit IRC | 01:16 | |
| *** zhonghua-lee has joined #openstack-cinder | 01:20 | |
| *** cheneydc has quit IRC | 01:23 | |
| *** cheneydc has joined #openstack-cinder | 01:25 | |
| *** EinstCrazy has joined #openstack-cinder | 01:37 | |
| *** jwcroppe has quit IRC | 01:38 | |
| *** jwcroppe has joined #openstack-cinder | 01:39 | |
| *** Lee1092 has joined #openstack-cinder | 01:58 | |
| *** markvoelker has joined #openstack-cinder | 01:58 | |
| *** EinstCrazy has quit IRC | 02:01 | |
| *** markvoelker has quit IRC | 02:02 | |
| openstackgerrit | wuyuting proposed openstack/cinder: Fix infortrend driver bug: cloned with src vol size https://review.openstack.org/326792 | 02:03 |
| *** EinstCrazy has joined #openstack-cinder | 02:04 | |
| *** yangyape_ has joined #openstack-cinder | 02:04 | |
| *** yangyapeng has quit IRC | 02:05 | |
| *** Poornima has joined #openstack-cinder | 02:11 | |
| *** yangyapeng has joined #openstack-cinder | 02:12 | |
| *** raunak has joined #openstack-cinder | 02:15 | |
| *** yangyape_ has quit IRC | 02:16 | |
| *** yangyape_ has joined #openstack-cinder | 02:26 | |
| *** yangyap__ has joined #openstack-cinder | 02:28 | |
| *** yangyape_ has quit IRC | 02:28 | |
| *** yangyapeng has quit IRC | 02:29 | |
| openstackgerrit | huanan proposed openstack/cinder: Huawei: Fixed problems caused by password expiration https://review.openstack.org/305633 | 02:39 |
| *** jwcroppe_ has joined #openstack-cinder | 02:41 | |
| *** jwcroppe has quit IRC | 02:44 | |
| *** yangyapeng has joined #openstack-cinder | 02:45 | |
| *** winston-d_ has joined #openstack-cinder | 02:47 | |
| *** yangyap__ has quit IRC | 02:47 | |
| *** sgotliv__ has joined #openstack-cinder | 03:00 | |
| *** houming has joined #openstack-cinder | 03:01 | |
| *** houming has quit IRC | 03:01 | |
| *** jwcroppe_ has quit IRC | 03:03 | |
| *** ducttape_ has joined #openstack-cinder | 03:07 | |
| *** jwcroppe has joined #openstack-cinder | 03:08 | |
| *** laughterwym has quit IRC | 03:09 | |
| *** raunak has quit IRC | 03:09 | |
| *** gouthamr has quit IRC | 03:12 | |
| *** gcb has quit IRC | 03:12 | |
| *** jwcroppe has quit IRC | 03:13 | |
| *** karthikp_ has quit IRC | 03:15 | |
| *** ducttape_ has quit IRC | 03:23 | |
| *** yangyape_ has joined #openstack-cinder | 03:28 | |
| *** yangyapeng has quit IRC | 03:28 | |
| *** ducttape_ has joined #openstack-cinder | 03:32 | |
| *** raunak has joined #openstack-cinder | 03:32 | |
| *** EinstCrazy has quit IRC | 03:36 | |
| *** EinstCrazy has joined #openstack-cinder | 03:37 | |
| *** ducttape_ has quit IRC | 03:42 | |
| *** laughterwym has joined #openstack-cinder | 03:45 | |
| *** laughterwym has quit IRC | 03:47 | |
| *** laughterwym has joined #openstack-cinder | 03:51 | |
| *** laughterwym has joined #openstack-cinder | 03:52 | |
| *** laughter_ has joined #openstack-cinder | 03:53 | |
| *** cheneydc1 has joined #openstack-cinder | 03:53 | |
| *** cheneydc has quit IRC | 03:55 | |
| *** cheneydc1 is now known as cheneydc | 03:55 | |
| *** laughterwym has quit IRC | 03:56 | |
| *** EinstCrazy has quit IRC | 03:57 | |
| *** EinstCrazy has joined #openstack-cinder | 03:57 | |
| *** jungleboyj has joined #openstack-cinder | 03:58 | |
| *** markvoelker has joined #openstack-cinder | 03:59 | |
| *** markvoelker has quit IRC | 04:03 | |
| openstackgerrit | wuyuting proposed openstack/cinder: Fix xio driver bug: cloned with src vol size https://review.openstack.org/328828 | 04:06 |
| *** cknight has quit IRC | 04:10 | |
| openstackgerrit | wuyuting proposed openstack/cinder: Fix xio driver bug: cloned with src vol size https://review.openstack.org/328828 | 04:10 |
| *** lpetrut has joined #openstack-cinder | 04:19 | |
| openstackgerrit | xing-yang proposed openstack/python-cinderclient: WIP: Add generic volume groups https://review.openstack.org/322627 | 04:24 |
| openstackgerrit | xing-yang proposed openstack/python-cinderclient: Add group types and group specs https://review.openstack.org/320157 | 04:24 |
| *** links has joined #openstack-cinder | 04:26 | |
| openstackgerrit | xing-yang proposed openstack/cinder: Add group type and group specs https://review.openstack.org/320165 | 04:27 |
| openstackgerrit | xing-yang proposed openstack/cinder: WIP: Add group snapshots https://review.openstack.org/328052 | 04:27 |
| openstackgerrit | xing-yang proposed openstack/cinder: Add generic volume groups https://review.openstack.org/322459 | 04:27 |
| *** raunak has quit IRC | 04:28 | |
| *** M00nr41n has left #openstack-cinder | 04:28 | |
| *** raunak has joined #openstack-cinder | 04:35 | |
| *** raunak has quit IRC | 04:40 | |
| *** yamada-h has joined #openstack-cinder | 04:42 | |
| *** ducttape_ has joined #openstack-cinder | 04:43 | |
| *** ducttape_ has quit IRC | 04:49 | |
| *** pgadiya has joined #openstack-cinder | 04:50 | |
| *** ircuser-1 has joined #openstack-cinder | 04:50 | |
| *** asselin_ has joined #openstack-cinder | 04:51 | |
| *** houming has joined #openstack-cinder | 04:58 | |
| *** houming has quit IRC | 04:58 | |
| *** liverpooler has joined #openstack-cinder | 05:05 | |
| *** lpetrut has quit IRC | 05:10 | |
| *** avishay has joined #openstack-cinder | 05:14 | |
| *** salv-orlando has joined #openstack-cinder | 05:21 | |
| *** salv-orl_ has quit IRC | 05:21 | |
| *** dave-mccowan has quit IRC | 05:27 | |
| *** salv-orl_ has joined #openstack-cinder | 05:29 | |
| *** laughterwym has joined #openstack-cinder | 05:29 | |
| *** salv-orlando has quit IRC | 05:31 | |
| *** laughter_ has quit IRC | 05:32 | |
| *** raunak has joined #openstack-cinder | 05:41 | |
| *** savihou has joined #openstack-cinder | 05:43 | |
| *** Amit___ has joined #openstack-cinder | 05:43 | |
| openstackgerrit | wuyuting proposed openstack/cinder: Fix xio driver bug: cloned with src vol size https://review.openstack.org/328828 | 05:47 |
| openstackgerrit | xing-yang proposed openstack/cinder: Add group type and group specs https://review.openstack.org/320165 | 05:50 |
| *** yangyapeng has joined #openstack-cinder | 05:51 | |
| *** yangyape_ has quit IRC | 05:55 | |
| openstackgerrit | Ankit Agrawal proposed openstack/cinder: Cleanup volume_type_projects https://review.openstack.org/328852 | 05:56 |
| *** laughterwym has quit IRC | 05:58 | |
| *** laughterwym has joined #openstack-cinder | 05:58 | |
| *** markvoelker has joined #openstack-cinder | 05:59 | |
| *** chlong has quit IRC | 06:03 | |
| *** markvoelker has quit IRC | 06:04 | |
| *** bhavik has joined #openstack-cinder | 06:04 | |
| *** geguileo_phone has joined #openstack-cinder | 06:06 | |
| *** karthikp_ has joined #openstack-cinder | 06:12 | |
| *** rcernin has joined #openstack-cinder | 06:14 | |
| *** chlong has joined #openstack-cinder | 06:15 | |
| *** karthikp_ has quit IRC | 06:18 | |
| *** rcernin has quit IRC | 06:18 | |
| *** rcernin has joined #openstack-cinder | 06:19 | |
| *** raunak has quit IRC | 06:21 | |
| *** gcb has joined #openstack-cinder | 06:22 | |
| *** arecknag has joined #openstack-cinder | 06:24 | |
| openstackgerrit | LisaLi proposed openstack/cinder: Create encrypted volumes from images https://review.openstack.org/216567 | 06:25 |
| *** flip217 is now known as flip214 | 06:42 | |
| *** flip214 has quit IRC | 06:42 | |
| *** flip214 has joined #openstack-cinder | 06:42 | |
| *** ducttape_ has joined #openstack-cinder | 06:45 | |
| *** jungleboyj has quit IRC | 06:48 | |
| *** jungleboyj has joined #openstack-cinder | 06:49 | |
| *** ducttape_ has quit IRC | 06:49 | |
| *** e0ne has joined #openstack-cinder | 06:53 | |
| *** EinstCrazy has quit IRC | 06:53 | |
| *** yamada-h has quit IRC | 06:55 | |
| *** EinstCrazy has joined #openstack-cinder | 06:56 | |
| *** geguileo_phone has quit IRC | 06:58 | |
| *** yamada-h has joined #openstack-cinder | 06:59 | |
| *** tesseract has joined #openstack-cinder | 07:00 | |
| *** salv-orl_ has quit IRC | 07:07 | |
| *** salv-orlando has joined #openstack-cinder | 07:08 | |
| *** chlong has quit IRC | 07:15 | |
| *** bbhavsar_ has joined #openstack-cinder | 07:16 | |
| *** geguileo_phone has joined #openstack-cinder | 07:16 | |
| *** sbezverk_ has joined #openstack-cinder | 07:16 | |
| *** sbezverk has quit IRC | 07:17 | |
| *** nkrinner has joined #openstack-cinder | 07:19 | |
| *** yangyapeng has quit IRC | 07:22 | |
| *** Poornima has quit IRC | 07:23 | |
| *** yangyapeng has joined #openstack-cinder | 07:26 | |
| *** geguileo_phone has quit IRC | 07:27 | |
| *** geguileo_phone has joined #openstack-cinder | 07:31 | |
| openstackgerrit | LisaLi proposed openstack/cinder: Handle quota exceed exception https://review.openstack.org/281666 | 07:32 |
| *** e0ne has quit IRC | 07:37 | |
| *** Poornima has joined #openstack-cinder | 07:40 | |
| *** geguileo_mobile has joined #openstack-cinder | 07:43 | |
| openstackgerrit | Peter Wang proposed openstack/cinder: VNX: New Cinder driver in Newton https://review.openstack.org/312427 | 07:44 |
| *** geguileo_phone has quit IRC | 07:47 | |
| *** geguileo_mobile has quit IRC | 07:50 | |
| *** geguileo_phone has joined #openstack-cinder | 07:52 | |
| *** geguileo_mobile has joined #openstack-cinder | 07:53 | |
| *** e0ne has joined #openstack-cinder | 07:55 | |
| openstackgerrit | Peter Wang proposed openstack/cinder: VNX: New Cinder driver in Newton https://review.openstack.org/312427 | 07:55 |
| *** geguileo_phone has quit IRC | 07:56 | |
| *** e0ne has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** laughterwym has quit IRC | 08:00 | |
| *** permalac has joined #openstack-cinder | 08:00 | |
| *** markvoelker has joined #openstack-cinder | 08:00 | |
| *** laughterwym has joined #openstack-cinder | 08:00 | |
| *** zzzeek has joined #openstack-cinder | 08:00 | |
| *** laughter_ has joined #openstack-cinder | 08:01 | |
| *** laughter_ has quit IRC | 08:02 | |
| *** laughter_ has joined #openstack-cinder | 08:02 | |
| openstackgerrit | ningwei proposed openstack/cinder: Added backend driver for huawei FusionStorage. https://review.openstack.org/328887 | 08:03 |
| *** laughterwym has quit IRC | 08:04 | |
| *** avishay has quit IRC | 08:04 | |
| *** markvoelker has quit IRC | 08:05 | |
| *** geguileo_mobile has quit IRC | 08:12 | |
| *** adisky has joined #openstack-cinder | 08:15 | |
| *** yamada-h has quit IRC | 08:16 | |
| *** geguileo_phone has joined #openstack-cinder | 08:16 | |
| openstackgerrit | Ankit Agrawal proposed openstack/cinder: Cleanup volume_type_projects https://review.openstack.org/328852 | 08:16 |
| *** tyrola has joined #openstack-cinder | 08:17 | |
| openstackgerrit | KongWei proposed openstack/cinder: Add ZTE Block Storage Driver https://review.openstack.org/312355 | 08:17 |
| *** avishay has joined #openstack-cinder | 08:19 | |
| *** laughter_ has quit IRC | 08:20 | |
| *** laughterwym has joined #openstack-cinder | 08:20 | |
| *** yamada-h has joined #openstack-cinder | 08:22 | |
| *** laughterwym has quit IRC | 08:23 | |
| *** laughterwym has joined #openstack-cinder | 08:23 | |
| *** salv-orlando has quit IRC | 08:23 | |
| *** salv-orlando has joined #openstack-cinder | 08:24 | |
| *** yangyape_ has joined #openstack-cinder | 08:24 | |
| *** yangyapeng has quit IRC | 08:24 | |
| *** yamada-h has quit IRC | 08:26 | |
| *** geguileo_phone has quit IRC | 08:27 | |
| *** yamada-h has joined #openstack-cinder | 08:34 | |
| *** lpetrut has joined #openstack-cinder | 08:38 | |
| *** avishay has quit IRC | 08:39 | |
| *** geguileo_phone has joined #openstack-cinder | 08:43 | |
| *** laughterwym has quit IRC | 08:44 | |
| *** laughterwym has joined #openstack-cinder | 08:44 | |
| *** sgotliv_ has joined #openstack-cinder | 08:44 | |
| *** ducttape_ has joined #openstack-cinder | 08:46 | |
| *** sgotliv__ has quit IRC | 08:47 | |
| *** laughter_ has joined #openstack-cinder | 08:47 | |
| *** geguileo_phone has quit IRC | 08:48 | |
| *** laughterwym has quit IRC | 08:49 | |
| *** laughterwym has joined #openstack-cinder | 08:49 | |
| *** geguileo_phone has joined #openstack-cinder | 08:49 | |
| *** ducttape_ has quit IRC | 08:50 | |
| *** laughter_ has quit IRC | 08:52 | |
| *** akapil has joined #openstack-cinder | 08:54 | |
| *** laughterwym has quit IRC | 08:54 | |
| Amit___ | Hello Team , is it possible to use cinder-migrate command to move an existing volume to other openstack node | 08:56 |
| Amit___ | and then import it to a given instance | 08:56 |
| *** bbhavsar_ has quit IRC | 09:04 | |
| *** geguileo_phone has quit IRC | 09:05 | |
| *** iceyao has quit IRC | 09:07 | |
| *** iceyao has joined #openstack-cinder | 09:07 | |
| openstackgerrit | huanan proposed openstack/cinder: Huawei: Fixed problems caused by password expiration https://review.openstack.org/305633 | 09:08 |
| *** geguileo_phone has joined #openstack-cinder | 09:15 | |
| DuncanT | Amit___: You don't need to move it to import it to an instance - every cinder volume should work on any instance in the AZ (sometimes all AZs) | 09:18 |
| *** e0ne has joined #openstack-cinder | 09:22 | |
| *** mvk_ has quit IRC | 09:24 | |
| Amit___ | Thanks DuncanT for your reply , just an elaboration on my scenario | 09:25 |
| Amit___ | I have an Openstack Node , where I am provisioning block device to Instances via NFS share. So, on the NFS share it looks like a block device created by cinder is in the form of a file. Now, I have one more openstack node , where I am sharing the same NFS share via Cinder. What I am trying to find out is, if the same file on the NFS share, that is acting as block device to instance on one openstack node, can it imported as | 09:25 |
| DuncanT | Amit___: That isn't a complete sentence.... | 09:27 |
| DuncanT | Amit___: to use the cinder NFS driver, you need the share to be accessible to all compute nodes. The same volume cannot be attached to more than one instance at a time, but you can certainly detach it from an instance on one host and attach it to an instance on another | 09:27 |
| Amit___ | Actually , I have two different host of openstack (installed as "all in one node"). There is a file created in NFS share as "volume-1505f786-7474-486d-9bd5-b04d8687548a" is it possible to import is file as block device to other instance on different openstack node | 09:32 |
| Amit___ | the nfs share is accessible on both openstack nodes I have configured it in cinder | 09:32 |
| DuncanT | Possible? Yes. A good idea? No. | 09:32 |
| DuncanT | If you're looking to do a one time move of the volume from one instance to the other, you can do unmanage on one cloud then manage on another. Having it as a live volume on two clouds at once will likely lead to pain however | 09:33 |
| DuncanT | For that scenario, you'd do better to use a common keystone and cinder between the two clouds | 09:34 |
| DuncanT | Is there a good reason you've got two all-in-one nodes? There are very few cases where that is the right thing to do | 09:34 |
| Amit___ | actually I am working POC kind of stuff, as there volumes are stored in external storage device | 09:36 |
| Amit___ | So , in this case If I loose the first cloud the second would act as DR | 09:36 |
| Amit___ | So , spun up new instance and attach the volume that resides on NFS | 09:37 |
| Amit___ | Can the command "cinder upload-to-image" help me in this case | 09:39 |
| *** geguileo_phone has quit IRC | 09:41 | |
| *** geguileo_phone has joined #openstack-cinder | 09:44 | |
| *** mvk_ has joined #openstack-cinder | 09:51 | |
| *** geguileo_mobile has joined #openstack-cinder | 09:57 | |
| *** geguileo_phone2 has joined #openstack-cinder | 09:57 | |
| *** iceyao has quit IRC | 09:58 | |
| *** iceyao has joined #openstack-cinder | 09:58 | |
| *** iceyao has quit IRC | 09:59 | |
| *** geguileo_phone has quit IRC | 09:59 | |
| *** iceyao has joined #openstack-cinder | 09:59 | |
| *** cheneydc has quit IRC | 10:00 | |
| *** geguileo_mobile has quit IRC | 10:01 | |
| openstackgerrit | wuyuting proposed openstack/cinder: Fix xio driver bug: cloned with src vol size https://review.openstack.org/328828 | 10:05 |
| *** geguileo_phone has joined #openstack-cinder | 10:06 | |
| *** geguileo_mobile has joined #openstack-cinder | 10:08 | |
| *** geguileo_phone2 has quit IRC | 10:09 | |
| *** bhavik has quit IRC | 10:10 | |
| *** geguileo_phone has quit IRC | 10:11 | |
| *** avishay has joined #openstack-cinder | 10:18 | |
| *** yamada-h has quit IRC | 10:24 | |
| *** akapil has quit IRC | 10:37 | |
| *** akapil has joined #openstack-cinder | 10:37 | |
| *** iceyao has quit IRC | 10:41 | |
| *** ducttape_ has joined #openstack-cinder | 10:46 | |
| *** EinstCrazy has quit IRC | 10:49 | |
| *** ducttape_ has quit IRC | 10:51 | |
| *** yuriy_n17 has joined #openstack-cinder | 10:51 | |
| *** alyson_ has joined #openstack-cinder | 10:56 | |
| dulek | e0ne: Hi, if you're reviewing o.vo patches - can you take a look on https://review.openstack.org/#/c/307075 ? | 10:56 |
| dulek | e0ne: That's the one blocking whole chain and a few other patches. | 10:56 |
| e0ne | dulek: hi. looking on it | 10:57 |
| *** cdelatte has joined #openstack-cinder | 11:02 | |
| *** ociuhandu has joined #openstack-cinder | 11:03 | |
| dulek | e0ne: Thanks, finally we're getting that in. :) | 11:03 |
| e0ne | dulek: you're welcome:_ | 11:04 |
| e0ne | :) | 11:04 |
| *** cdelatte has quit IRC | 11:05 | |
| *** salv-orlando has quit IRC | 11:07 | |
| *** salv-orlando has joined #openstack-cinder | 11:08 | |
| *** Amit___ has quit IRC | 11:10 | |
| *** yamada-h has joined #openstack-cinder | 11:25 | |
| *** sgotliv_ has quit IRC | 11:25 | |
| *** sgotliv has joined #openstack-cinder | 11:26 | |
| *** bhavik has joined #openstack-cinder | 11:26 | |
| *** salv-orl_ has joined #openstack-cinder | 11:28 | |
| *** yamada-h has quit IRC | 11:29 | |
| *** salv-orlando has quit IRC | 11:32 | |
| openstackgerrit | Ivan Kolodyazhny proposed openstack/cinder: Remove duplicated fake drivers https://review.openstack.org/328958 | 11:33 |
| *** ducttape_ has joined #openstack-cinder | 11:38 | |
| *** rraja has joined #openstack-cinder | 11:39 | |
| *** timcl has joined #openstack-cinder | 11:42 | |
| *** sdague has joined #openstack-cinder | 11:49 | |
| *** avishay has quit IRC | 11:50 | |
| *** akapil has quit IRC | 11:54 | |
| *** ducttape_ has quit IRC | 11:55 | |
| *** laughterwym has joined #openstack-cinder | 12:00 | |
| *** raildo-afk is now known as raildo | 12:01 | |
| *** markvoelker has joined #openstack-cinder | 12:01 | |
| *** laughter_ has joined #openstack-cinder | 12:01 | |
| *** mvk_ has quit IRC | 12:03 | |
| *** laughterwym has quit IRC | 12:05 | |
| *** yangyape_ has quit IRC | 12:05 | |
| *** markvoelker has quit IRC | 12:06 | |
| *** akapil has joined #openstack-cinder | 12:07 | |
| *** pauloewerton has joined #openstack-cinder | 12:08 | |
| *** timcl has quit IRC | 12:10 | |
| *** akapil_ has joined #openstack-cinder | 12:10 | |
| *** liverpooler has quit IRC | 12:11 | |
| *** mtanino has joined #openstack-cinder | 12:11 | |
| *** akapil has quit IRC | 12:14 | |
| *** e0ne has quit IRC | 12:15 | |
| *** e0ne has joined #openstack-cinder | 12:15 | |
| *** sdague has quit IRC | 12:16 | |
| *** jwcroppe has joined #openstack-cinder | 12:16 | |
| *** sdague has joined #openstack-cinder | 12:16 | |
| *** zhenguo_ has quit IRC | 12:21 | |
| openstackgerrit | Merged openstack/cinder: Use manifest to backport OVOs during upgrades https://review.openstack.org/307075 | 12:22 |
| openstackgerrit | Merged openstack/cinder: Remove remotable from OVOs https://review.openstack.org/312929 | 12:24 |
| openstackgerrit | Merged openstack/cinder: Move OVO methods from CinderObject to Persistent https://review.openstack.org/312930 | 12:24 |
| *** akerr has joined #openstack-cinder | 12:25 | |
| *** markvoelker has joined #openstack-cinder | 12:25 | |
| *** yhayashi has quit IRC | 12:26 | |
| *** akapil_ has quit IRC | 12:28 | |
| *** akapil has joined #openstack-cinder | 12:28 | |
| *** pgadiya has quit IRC | 12:30 | |
| *** scottda has joined #openstack-cinder | 12:32 | |
| *** tpsilva has joined #openstack-cinder | 12:32 | |
| *** julim has joined #openstack-cinder | 12:42 | |
| *** e0ne has quit IRC | 12:48 | |
| *** mvk has joined #openstack-cinder | 12:49 | |
| *** e0ne has joined #openstack-cinder | 12:49 | |
| *** EinstCrazy has joined #openstack-cinder | 12:51 | |
| *** Julien-zte has joined #openstack-cinder | 12:51 | |
| openstackgerrit | huanan proposed openstack/cinder: Huawei: Fixed problems caused by password expiration https://review.openstack.org/305633 | 12:54 |
| *** tyrola has quit IRC | 12:54 | |
| *** timcl has joined #openstack-cinder | 12:55 | |
| *** EinstCrazy has quit IRC | 12:55 | |
| *** xyang1 has joined #openstack-cinder | 12:55 | |
| *** chlong has joined #openstack-cinder | 12:58 | |
| *** iceyao has joined #openstack-cinder | 13:00 | |
| *** dave-mccowan has joined #openstack-cinder | 13:03 | |
| *** sheel has joined #openstack-cinder | 13:05 | |
| *** iceyao_ has joined #openstack-cinder | 13:07 | |
| *** eharney has joined #openstack-cinder | 13:07 | |
| *** iceyao has quit IRC | 13:09 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/cinder: Import auth_uri from different module https://review.openstack.org/328995 | 13:10 |
| *** lprice has joined #openstack-cinder | 13:13 | |
| *** edmondsw has joined #openstack-cinder | 13:15 | |
| *** gouthamr has joined #openstack-cinder | 13:15 | |
| *** lprice1 has joined #openstack-cinder | 13:15 | |
| *** jwcroppe_ has joined #openstack-cinder | 13:16 | |
| *** jwcropp__ has joined #openstack-cinder | 13:17 | |
| *** lprice has quit IRC | 13:17 | |
| *** jwcroppe has quit IRC | 13:18 | |
| openstackgerrit | Lucian Petrut proposed openstack/cinder: Windows SMBFS: fix VHD/x resize https://review.openstack.org/329000 | 13:18 |
| *** dustins has joined #openstack-cinder | 13:19 | |
| *** jwcroppe_ has quit IRC | 13:20 | |
| *** liverpooler has joined #openstack-cinder | 13:25 | |
| *** diogogmt has quit IRC | 13:26 | |
| *** ducttape_ has joined #openstack-cinder | 13:28 | |
| *** mriedem has joined #openstack-cinder | 13:31 | |
| *** cknight has joined #openstack-cinder | 13:32 | |
| *** mvk has quit IRC | 13:32 | |
| *** permalac has quit IRC | 13:37 | |
| *** xyang has joined #openstack-cinder | 13:37 | |
| *** bhavik has quit IRC | 13:38 | |
| *** diogogmt has joined #openstack-cinder | 13:39 | |
| openstackgerrit | Kedar Vidvans proposed openstack/cinder: Backup and restore broken in zfssaiscsi driver https://review.openstack.org/328465 | 13:40 |
| *** arecknag has quit IRC | 13:42 | |
| openstackgerrit | Ivan Kolodyazhny proposed openstack/os-brick: WIP. Local attach feature in RBD connector https://review.openstack.org/328297 | 13:45 |
| scottda | xyang: I'd like to cherry-pick that model_update for temp volume patch into stable/mitaka. Is that OK with you? | 13:45 |
| xyang | scottda: I’ve already done that | 13:46 |
| xyang | scotta: here: https://review.openstack.org/#/c/328067/ | 13:46 |
| scottda | xyang: haha..I thought you might have. Thanks! | 13:47 |
| xyang | scottda: welcome | 13:47 |
| *** bhavik has joined #openstack-cinder | 13:50 | |
| swamireddy | sheel: hi | 13:52 |
| *** diogogmt has quit IRC | 13:53 | |
| *** links has quit IRC | 13:54 | |
| *** Poornima has quit IRC | 13:54 | |
| *** mvk has joined #openstack-cinder | 13:56 | |
| *** eharney has quit IRC | 13:58 | |
| *** mriedem1 has joined #openstack-cinder | 13:59 | |
| openstackgerrit | Ivan Kolodyazhny proposed openstack/os-brick: WIP. Local attach feature in RBD connector https://review.openstack.org/328297 | 13:59 |
| *** akapil_ has joined #openstack-cinder | 14:00 | |
| *** diogogmt has joined #openstack-cinder | 14:01 | |
| *** mriedem has quit IRC | 14:02 | |
| *** akapil has quit IRC | 14:04 | |
| *** e0ne has quit IRC | 14:04 | |
| *** r-daneel has joined #openstack-cinder | 14:05 | |
| *** e0ne has joined #openstack-cinder | 14:05 | |
| *** mriedem1 is now known as mriedem | 14:06 | |
| *** avishay has joined #openstack-cinder | 14:08 | |
| *** krtaylor has quit IRC | 14:09 | |
| *** akapil_ has quit IRC | 14:10 | |
| *** akapil has joined #openstack-cinder | 14:12 | |
| *** diogogmt has quit IRC | 14:12 | |
| *** eharney has joined #openstack-cinder | 14:13 | |
| *** xyang has quit IRC | 14:18 | |
| *** xyang has joined #openstack-cinder | 14:19 | |
| *** akapil has quit IRC | 14:19 | |
| *** akapil has joined #openstack-cinder | 14:19 | |
| *** permalac has joined #openstack-cinder | 14:21 | |
| *** jungleboyj has quit IRC | 14:21 | |
| *** xyang1 has quit IRC | 14:22 | |
| *** andrearosa has quit IRC | 14:24 | |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Switch type extra_specs to DictOfNullableStrings https://review.openstack.org/325248 | 14:24 |
| *** xyang1 has joined #openstack-cinder | 14:25 | |
| *** timcl has quit IRC | 14:25 | |
| *** enriquetaso has joined #openstack-cinder | 14:25 | |
| sheel | swamireddy: pong | 14:29 |
| *** ebalduf_ has joined #openstack-cinder | 14:31 | |
| *** ducttape_ has quit IRC | 14:33 | |
| *** kfarr has joined #openstack-cinder | 14:34 | |
| *** ducttape_ has joined #openstack-cinder | 14:37 | |
| *** EinstCrazy has joined #openstack-cinder | 14:39 | |
| *** geguileo_mobile has quit IRC | 14:43 | |
| *** akapil has quit IRC | 14:44 | |
| openstackgerrit | Michal Dulko proposed openstack/cinder: OVO request_spec in migrate/manage/retype calls https://review.openstack.org/252458 | 14:45 |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Switch request_spec in create volume calls to ovo https://review.openstack.org/246600 | 14:45 |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Switch request_spec_list in create CG to OVO https://review.openstack.org/253033 | 14:45 |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Convert filter_properties to OVO (create, retype) https://review.openstack.org/257305 | 14:45 |
| *** akapil has joined #openstack-cinder | 14:46 | |
| openstackgerrit | Alex O'Rourke proposed openstack/cinder: 3PAR: Fix delete volume when online clone https://review.openstack.org/327840 | 14:48 |
| *** timcl has joined #openstack-cinder | 14:48 | |
| *** porrua has joined #openstack-cinder | 14:48 | |
| *** ebalduf_ has quit IRC | 14:50 | |
| *** akapil has quit IRC | 14:51 | |
| *** xinli has joined #openstack-cinder | 14:52 | |
| *** mriedem1 has joined #openstack-cinder | 14:53 | |
| *** mriedem has quit IRC | 14:55 | |
| *** suyash has joined #openstack-cinder | 14:57 | |
| *** permalac has quit IRC | 14:58 | |
| scottda | ildikov: I updated the date for today's meeting of cinder-nova-api-changes. Would you like to update the agenda? | 14:58 |
| *** julim has quit IRC | 14:59 | |
| *** akapil has joined #openstack-cinder | 15:00 | |
| *** julim has joined #openstack-cinder | 15:00 | |
| *** diogogmt has joined #openstack-cinder | 15:02 | |
| *** geguileo_phone has joined #openstack-cinder | 15:02 | |
| *** akapil has quit IRC | 15:02 | |
| Swanson | Some love for this one? https://review.openstack.org/#/c/325506/ | 15:03 |
| Swanson | smcginnis ^^ | 15:03 |
| *** jwcroppe has joined #openstack-cinder | 15:07 | |
| *** avishay has quit IRC | 15:10 | |
| *** jwcropp__ has quit IRC | 15:10 | |
| *** akerr has quit IRC | 15:10 | |
| *** akerr has joined #openstack-cinder | 15:10 | |
| *** akapil has joined #openstack-cinder | 15:12 | |
| *** permalac has joined #openstack-cinder | 15:13 | |
| *** geguileo_phone has quit IRC | 15:15 | |
| *** david-lyle has joined #openstack-cinder | 15:15 | |
| Swanson | eharney, thanks! | 15:16 |
| openstackgerrit | Sean McGinnis proposed openstack/cinder: Add test type clarification to devref https://review.openstack.org/327806 | 15:16 |
| *** ajnouri has quit IRC | 15:16 | |
| *** krtaylor has joined #openstack-cinder | 15:17 | |
| *** alyson_ has quit IRC | 15:18 | |
| *** akapil has quit IRC | 15:18 | |
| *** jwcroppe_ has joined #openstack-cinder | 15:20 | |
| *** adisky has quit IRC | 15:20 | |
| *** lpetrut has quit IRC | 15:20 | |
| *** yangyapeng has joined #openstack-cinder | 15:21 | |
| *** IanGovett has joined #openstack-cinder | 15:21 | |
| openstackgerrit | Sean McGinnis proposed openstack/cinder: Add driver interface checks https://review.openstack.org/297140 | 15:22 |
| *** mriedem1 is now known as mriedem | 15:22 | |
| *** lennyb has quit IRC | 15:22 | |
| *** jwcroppe has quit IRC | 15:23 | |
| xinli | dulek: after someone merging the code, I try to rebase the patch 251989, but all the test cases failed now, need some help. | 15:23 |
| *** lennyb has joined #openstack-cinder | 15:25 | |
| Swanson | Is showing the rebase changes between versions a bug or something gerrit does to be helpful? | 15:26 |
| *** geguileo_phone has joined #openstack-cinder | 15:26 | |
| *** karthikp_ has joined #openstack-cinder | 15:26 | |
| e0ne | jbernard: hi. just fyi, I've found workagound for that issue with multipath: I've added /dev/rbd to ignore list | 15:27 |
| *** mtanino has quit IRC | 15:28 | |
| xinli | Swanson: do you ask me the question? The code conflicts in cinder.tests.unit.objects.test_objects. Patch adds two additional line for 'QualityOfServiceSpecs': '1.0-7f4611b0efff1ac5eb43eb01a2bb3f36', | 15:29 |
| xinli | 'QualityOfServiceSpecsList': '1.0-a9a0fa26933e9a6e6a290c7b0aededd1', | 15:29 |
| Swanson | xinli, No, just a generic complaint tossed into the void. | 15:29 |
| xinli | swanson: after I adding these two lines in rebase, the test tox -e py27 cinder.tests.unit.objects.test_objects failed with the error message: | 15:30 |
| openstackgerrit | Eric Harney proposed openstack/cinder: Tests: Use mock for notifier tests https://review.openstack.org/317055 | 15:30 |
| *** garthb has joined #openstack-cinder | 15:30 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder: Deleting volume metadata keys with a single request https://review.openstack.org/328399 | 15:31 |
| xinli | swanson:File "cinder/objects/volume_type.py", line 121, in VolumeTypeList | 15:31 |
| xinli | @base.remotable_classmethod | 15:31 |
| xinli | AttributeError: 'module' object has no attribute 'remotable_classmethod' | 15:31 |
| xinli | The test run didn't actually run any tests | 15:31 |
| smcginnis | xinli: The objects have changed, so you now get a different hash. You will be able to see the new hash value from the test failure output. | 15:31 |
| e0ne | smcginnis: morning, Sean. do we have any workflow how to request you new os-brick release? | 15:31 |
| smcginnis | xinli: You can use that to update the tests for the new versions. | 15:31 |
| e0ne | smcginnis: or, may be, have we some schedule? | 15:31 |
| xinli | swanson: sorry. | 15:31 |
| openstackgerrit | Alex O'Rourke proposed openstack/cinder: 3PAR: Fix delete volume when online clone https://review.openstack.org/327840 | 15:31 |
| smcginnis | e0ne: They get requested from the releases team. We just released 1.4.0 early last week IIRC. | 15:32 |
| smcginnis | There's a problem publishing the py34 version though. | 15:32 |
| smcginnis | Infra is aware of it. | 15:32 |
| e0ne | smcginnis: when do we plan to make new release? | 15:32 |
| smcginnis | e0ne: Probably not for a while now since we just did it. Or did something important get added in the last few days? | 15:34 |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder: Deleting volume metadata keys with a single request https://review.openstack.org/328399 | 15:35 |
| *** harlowja_at_home has joined #openstack-cinder | 15:35 | |
| e0ne | smcginnis: I'm interesting in getting released few of my patches | 15:35 |
| smcginnis | e0ne: Did they just get merged? | 15:36 |
| e0ne | smcginnis: to unblock 'local attach' feature and some rbd refactor | 15:36 |
| e0ne | smcginnis: only one | 15:36 |
| e0ne | smcginnis: there is no hurry me get them merged asap | 15:36 |
| *** xinli has quit IRC | 15:37 | |
| *** xinli has joined #openstack-cinder | 15:37 | |
| e0ne | smcginnis: I'm fine to wait for N-2 milestone. maybe, we'll get more usefull thing in os-brick untils that | 15:37 |
| *** jdillaman has joined #openstack-cinder | 15:38 | |
| smcginnis | e0ne: Are you talking about the processutils.execute patch? | 15:39 |
| e0ne | smcginnis: yes | 15:39 |
| *** tesseract has quit IRC | 15:39 | |
| smcginnis | e0ne: I see, it was in there but the Jenkins merge hadn't gone through. | 15:40 |
| smcginnis | Honestly not clear how that exactly works. | 15:40 |
| smcginnis | But we can do a release whenever we need to. If we can wait a bit for some other things I think that would be good. | 15:40 |
| *** leeantho has joined #openstack-cinder | 15:40 | |
| e0ne | smcginnis: sure, we can wait | 15:41 |
| *** bhavik has quit IRC | 15:41 | |
| smcginnis | e0ne: OK, cool. Just let me know if you think we are due and haven't seen anything from me. Thanks! | 15:41 |
| e0ne | smcginnis: I'll do, thanks! | 15:41 |
| *** leeantho_ has joined #openstack-cinder | 15:42 | |
| *** ducttape_ has quit IRC | 15:42 | |
| openstackgerrit | Eric Harney proposed openstack/cinder: Tests: Remove notification_driver config https://review.openstack.org/329064 | 15:43 |
| *** ociuhandu has quit IRC | 15:45 | |
| *** leeantho has quit IRC | 15:45 | |
| *** ducttape_ has joined #openstack-cinder | 15:45 | |
| *** xyang has quit IRC | 15:46 | |
| *** xyang has joined #openstack-cinder | 15:46 | |
| *** IanGovett has quit IRC | 15:47 | |
| *** manjeets has left #openstack-cinder | 15:47 | |
| *** xyang has quit IRC | 15:48 | |
| *** xyang has joined #openstack-cinder | 15:52 | |
| *** Apoorva has joined #openstack-cinder | 15:52 | |
| *** iceyao_ has quit IRC | 15:53 | |
| mriedem | smcginnis: hemna: fyi pretty high failure rates in multinode jobs with os-brick 1.4.0 https://bugs.launchpad.net/os-brick/+bug/1592043 | 15:55 |
| openstack | Launchpad bug 1592043 in os-brick "os-brick 1.4.0 increases volume setup failure rates" [Undecided,Confirmed] | 15:55 |
| *** xyang has quit IRC | 15:56 | |
| *** xyang has joined #openstack-cinder | 15:57 | |
| xinli | smcginnis: I fixed the code in test_objects.py with latest hash values, but the test still failed, where can I get the correct has value? | 15:58 |
| *** rcernin has quit IRC | 15:59 | |
| *** geguileo_phone has quit IRC | 16:04 | |
| *** EinstCrazy has quit IRC | 16:04 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder: Deleting volume metadata keys with a single request https://review.openstack.org/328399 | 16:06 |
| *** jseiler_ has joined #openstack-cinder | 16:09 | |
| *** suyash has quit IRC | 16:11 | |
| *** raunak has joined #openstack-cinder | 16:12 | |
| *** jseiler has quit IRC | 16:13 | |
| *** mtanino has joined #openstack-cinder | 16:13 | |
| *** laughter_ has quit IRC | 16:14 | |
| *** laughterwym has joined #openstack-cinder | 16:16 | |
| *** laughterwym has quit IRC | 16:17 | |
| *** laughterwym has joined #openstack-cinder | 16:17 | |
| *** laughterwym has quit IRC | 16:17 | |
| mriedem | actually it's probably this change https://github.com/openstack/os-brick/commit/dbf77fba1061cb4e93b3db5f8117d6ccc689f702 | 16:18 |
| *** ihrachys has joined #openstack-cinder | 16:18 | |
| *** laughterwym has joined #openstack-cinder | 16:18 | |
| mriedem | sdague: ^ since that depends on devstack changes | 16:18 |
| mriedem | some of those deps are probably not in mitaka | 16:18 |
| *** IanGovett has joined #openstack-cinder | 16:18 | |
| mriedem | yar https://review.openstack.org/#/c/277696/ | 16:19 |
| *** Julien-zte has quit IRC | 16:19 | |
| *** IanGovett has quit IRC | 16:20 | |
| mriedem | there are a few actually | 16:20 |
| ihrachys | mriedem: are you aware of http://logs.openstack.org/04/328304/8/check/gate-grenade-dsvm-neutron-multinode/fa6539e/logs/new/screen-n-cpu.txt.gz?level=TRACE#_2016-06-13_14_19_13_747 ? | 16:22 |
| mriedem | ihrachys: heh, yeah, see above | 16:22 |
| mriedem | https://bugs.launchpad.net/os-brick/+bug/1592043 | 16:22 |
| openstack | Launchpad bug 1592043 in os-brick "os-brick 1.4.0 increases volume setup failure rates" [Undecided,Confirmed] | 16:22 |
| *** laughterwym has quit IRC | 16:22 | |
| ihrachys | oh ok | 16:22 |
| openstackgerrit | Merged openstack/cinder: Dell SC: Retry unhandled exception REST Gets https://review.openstack.org/325506 | 16:23 |
| ihrachys | it got us to 70% failure rate in multinode grenade for neutron :-| | 16:23 |
| mriedem | ihrachys: yeah, we should probably blacklist os-brick 1.4.0 in g-r | 16:23 |
| mriedem | ihrachys: or, we need to backport some config changes to stable/mitaka | 16:23 |
| mriedem | ihrachys: see https://review.openstack.org/#/c/277224/ | 16:23 |
| smcginnis | mriedem: Catching up. What is the dependency on devstack changes? | 16:24 |
| mriedem | smcginnis: see the depends-on in that change ^ | 16:24 |
| mriedem | note to self, we should probably run grenade on os-brick changes | 16:24 |
| mriedem | but i'm not sure how many libraries run grenade jobs | 16:25 |
| sdague | mriedem: not many, I think oslo.messaging might | 16:25 |
| *** bardia has joined #openstack-cinder | 16:25 | |
| sdague | it's mostly not a big deal | 16:25 |
| *** salv-orl_ has quit IRC | 16:25 | |
| sdague | mriedem: so... why isn't it an all the time failure? | 16:25 |
| *** salv-orlando has joined #openstack-cinder | 16:25 | |
| smcginnis | sdague: That's what I'm still trying to understand. | 16:26 |
| smcginnis | Not entirely sure I'm up to speed on things here. What did we miss? | 16:26 |
| sdague | that kind of fail rate is usually 2 requests at the same time tripping over something | 16:26 |
| sdague | smcginnis: well, this is exactly the sweet spot for bad failure rates, just low enough that it's easy to land, but high enough that it ruins everything | 16:27 |
| smcginnis | So... privsep bug? I thought it provided execution isolation. | 16:27 |
| smcginnis | Unless what it is executing on. | 16:27 |
| openstackgerrit | Ivan Kolodyazhny proposed openstack/os-brick: Local attach feature in RBD connector https://review.openstack.org/328297 | 16:28 |
| hemna | whats up | 16:28 |
| sdague | how does daemon mode work with privsep? | 16:28 |
| *** suyash has joined #openstack-cinder | 16:29 | |
| smcginnis | hemna: Lot's of failures in grenade pointing to os-brick. | 16:29 |
| ihrachys | mriedem: I don't think config changes backport is the right thing to do. since the failure is legit: it shows that we did bad job providing compatibility that could be expected as per supports-upgrade governance tag | 16:29 |
| hemna | hrmm | 16:29 |
| *** e0ne has quit IRC | 16:29 | |
| sdague | is it possible there is a race setting up the daemon if 2 requests hit at the same time | 16:29 |
| mriedem | ihrachys: yeah i agree | 16:29 |
| sdague | because - "raise FailedToDropPrivileges(msg)" is odd | 16:29 |
| smcginnis | Don't support gus is around? | 16:30 |
| ihrachys | mriedem: if I understand correctly, unless we revert the breaking change, we will need to blacklist all releases of the library until Newton. which I guess is not something os-brick folks would like either. :) | 16:30 |
| smcginnis | *suppose | 16:30 |
| hemna | oh we pushed 1.4.0 w/ privsep | 16:30 |
| hemna | ok | 16:30 |
| *** rcernin has joined #openstack-cinder | 16:31 | |
| smcginnis | hemna: So much for "trivial" | 16:31 |
| hemna | FailedTorDropPrivileges | 16:32 |
| hemna | wtf | 16:32 |
| hemna | so privsep itself puked | 16:33 |
| *** nikeshm has joined #openstack-cinder | 16:34 | |
| hemna | https://github.com/openstack/oslo.privsep/blob/master/oslo_privsep/daemon.py#L283-L290 | 16:35 |
| *** rcernin has quit IRC | 16:35 | |
| smcginnis | This looks like a privsep issue to me, but we will probably need to block 1.4.0 until it gets sorted. | 16:36 |
| hemna | looks like it's trying to read the iscsi initiator file | 16:36 |
| hemna | every failure is doing that | 16:36 |
| *** rcernin has joined #openstack-cinder | 16:36 | |
| hemna | initiator = iscsi.get_initiator() | 16:36 |
| hemna | https://github.com/openstack/os-brick/blob/master/os_brick/initiator/connector.py#L1079-L1084 | 16:37 |
| hemna | every failure is that one afaik | 16:38 |
| *** bardia has quit IRC | 16:38 | |
| hemna | so that stack dump is the 'client' side of the privsep call | 16:40 |
| hemna | we need to see the daemon side | 16:40 |
| ildikov | scottda: hey, thanks for fixing the meeting date | 16:40 |
| smcginnis | Blowing up cat'ing a file? | 16:40 |
| hemna | smcginnis, yup | 16:40 |
| hemna | that's all that is | 16:40 |
| ildikov | scottda: I briefly updated the agenda as we never follow it anyway, but I totally forgot the date... :) | 16:41 |
| hemna | is cat /etc/iscsi/initiatorname.iscsi | 16:41 |
| hemna | no reason that should fail | 16:41 |
| hemna | unless the daemon is dead ? | 16:41 |
| *** savihou has quit IRC | 16:41 | |
| *** Swanson has quit IRC | 16:43 | |
| sdague | hemna: what controls the daemon? | 16:43 |
| sdague | this is post upgrade | 16:43 |
| *** Swanson has joined #openstack-cinder | 16:43 | |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/new/screen-n-cpu.txt.gz?#_2016-06-13_13_21_27_738 | 16:43 |
| hemna | almost not help there afaik | 16:44 |
| sdague | hemna: what is the daemon process called? | 16:44 |
| hemna | good question | 16:44 |
| hemna | I'm trying to find the startup process | 16:45 |
| hemna | sdague, https://review.openstack.org/#/c/277670/ | 16:46 |
| *** ociuhandu has joined #openstack-cinder | 16:46 | |
| hemna | https://review.openstack.org/#/c/277224/21/os_brick/privileged/__init__.py | 16:46 |
| sdague | hemna: well that's definitely not in the filters - http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/etc/nova/rootwrap.d/compute.filters.txt.gz | 16:47 |
| mriedem | yeah because those filters are mitaka | 16:47 |
| mriedem | and https://review.openstack.org/#/c/277670/5/etc/nova/rootwrap.d/compute.filters landed in newton | 16:47 |
| sdague | mar 23? | 16:48 |
| sdague | yeh, I guess we branched before that | 16:48 |
| mriedem | we intentionally left the privsep stuff out of mitaka | 16:48 |
| sdague | so there needs to be a newton release note, plus an upgrade script | 16:48 |
| sdague | though.... I still don't understand how it works any of the time | 16:48 |
| *** avishay has joined #openstack-cinder | 16:49 | |
| sdague | that's the party I'm not really comfortable with, nothing explains the passing on the single node case | 16:49 |
| mriedem | sdague: this job passed, | 16:49 |
| mriedem | in subnode-2 os-brick is the mitaka version http://logs.openstack.org/06/329006/1/check/gate-grenade-dsvm-neutron-multinode/6c28824/logs/subnode-2/pip2-freeze.txt.gz | 16:50 |
| mriedem | so i'm guessing it passed there because the server create request landed on the mitaka node? | 16:50 |
| hemna | https://github.com/openstack/nova/commit/18129874cdc745b05b35d61b1c1979566453ca13 | 16:50 |
| hemna | that looks like Feb 8 is when it got in ? | 16:50 |
| hemna | should still be there | 16:50 |
| sdague | in the single node case, here is how things look - http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/new/screen-n-cpu.txt.gz#_2016-06-13_13_09_31_773 | 16:50 |
| hemna | why isn't it in the grenade tests filter ? | 16:51 |
| mriedem | hemna: that's the same as https://review.openstack.org/#/c/277670/ above, merged 3/23 | 16:51 |
| * hemna is confused | 16:51 | |
| sdague | that's the same patch as above with failure | 16:51 |
| hemna | Spawned new privsep daemon via rootwrap | 16:52 |
| hemna | that looks like it worked | 16:52 |
| *** rraja has quit IRC | 16:52 | |
| sdague | hemna: right... which is why I'm curious why it fails only in the multinode case | 16:53 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/etc/nova/rootwrap.d/compute.filters.txt.gz | 16:54 |
| hemna | I don't see the filter in there either | 16:54 |
| sdague | hemna: right it isn't | 16:55 |
| hemna | cinder's filter has it though | 16:55 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/etc/cinder/rootwrap.d/volume.filters.txt.gz | 16:55 |
| mriedem | hemna: i think that's because cinder merged that change in mitaka | 16:55 |
| mriedem | this one https://review.openstack.org/#/c/280030/ | 16:56 |
| hemna | so do the grenade jobs not update/upgrade the filters files ? | 16:56 |
| *** dobson has quit IRC | 16:56 | |
| sdague | hemna: no... that's a fundamental principle of theory of upgrade | 16:57 |
| sdague | new code with old config | 16:57 |
| sdague | so that CD works | 16:57 |
| hemna | hmm ok | 16:57 |
| hemna | so yah that'll never work then | 16:57 |
| sdague | it's always been that way | 16:57 |
| hemna | heh | 16:57 |
| sdague | hemna: except, it does, some times apparently | 16:58 |
| sdague | which is massively confusing | 16:58 |
| hemna | so confused | 16:58 |
| *** smcginnis has quit IRC | 16:58 | |
| *** smcginnis has joined #openstack-cinder | 16:58 | |
| *** harlowja_at_home has quit IRC | 16:59 | |
| sdague | https://github.com/openstack-dev/grenade#theory-of-upgrade - that's been the status quo since we introduced grenade in grizzly | 16:59 |
| hemna | that's fine. I'm not sure why we haven't seen other major failures then like this | 17:00 |
| hemna | we've added other rootwrap needed things before | 17:00 |
| hemna | kinda the same scenario, this time is just confused by privsep | 17:00 |
| hemna | why aren't we seeing rootwrap bitching about not allowing the daemon to start ? | 17:01 |
| sdague | hemna: right, I don't know | 17:05 |
| sdague | the fact that this works some times, and not others, is the part that concerns me | 17:05 |
| hemna | so it looks like in the successful runs, the first time the helper is called is when the daemon starts | 17:05 |
| *** ihrachys has quit IRC | 17:06 | |
| sdague | and I feel like we need a real answer on that, otherwise just adding rootwrap rules potentially papers over something serious here. And it's security code, which is even more concerning. | 17:06 |
| hemna | yah, I'm digging into it | 17:06 |
| hemna | https://github.com/openstack/oslo.privsep/blob/master/oslo_privsep/daemon.py#L489 | 17:08 |
| hemna | so that's where the daemon side is supposed to start | 17:08 |
| hemna | brb | 17:10 |
| *** gluytium has quit IRC | 17:15 | |
| *** xyang has quit IRC | 17:17 | |
| *** darosale has joined #openstack-cinder | 17:18 | |
| mriedem | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/new/screen-n-cpu.txt.gz#_2016-06-13_13_09_32_245 | 17:21 |
| mriedem | privsep process running with uid/gid: 0/0 | 17:21 |
| mriedem | privsep daemon running as pid 9057 | 17:21 |
| mriedem | i don't see that in http://logs.openstack.org/71/327971/5/gate/gate-grenade-dsvm-neutron-multinode/f2690e3/logs/new/screen-n-cpu.txt.gz | 17:22 |
| *** xyang has joined #openstack-cinder | 17:22 | |
| *** xyang has quit IRC | 17:24 | |
| hemna | wait, so it's not even using rootwrap no? | 17:27 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/new/screen-n-cpu.txt.gz#_2016-06-13_13_09_31_773 | 17:27 |
| hemna | sudo privsep-helper | 17:27 |
| *** xyang has joined #openstack-cinder | 17:27 | |
| hemna | is that not installed on multinode ? | 17:27 |
| *** julim has quit IRC | 17:28 | |
| hemna | https://github.com/openstack-dev/devstack/blob/5248283e0df0dda9a1bfba4c618167894d717fb0/lib/nova#L484 | 17:28 |
| *** salv-orl_ has joined #openstack-cinder | 17:29 | |
| *** xyang has quit IRC | 17:30 | |
| *** salv-orl_ has quit IRC | 17:32 | |
| *** salv-orlando has quit IRC | 17:32 | |
| *** xyang has joined #openstack-cinder | 17:32 | |
| *** salv-orlando has joined #openstack-cinder | 17:32 | |
| *** bardia has joined #openstack-cinder | 17:38 | |
| mriedem | hemna: i'm not sure about the internals of grenade, but i'm assuming it overwrites the config from master with the existing n-1 config | 17:39 |
| mriedem | so https://github.com/openstack-dev/devstack/blob/5248283e0df0dda9a1bfba4c618167894d717fb0/lib/nova#L484 wouldn't be in there | 17:39 |
| mriedem | sdague: can you confirm that? ^ | 17:39 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/syslog.txt.gz#_Jun_13_13_09_32 | 17:40 |
| *** wN has quit IRC | 17:42 | |
| *** xyang has quit IRC | 17:42 | |
| *** wN has joined #openstack-cinder | 17:44 | |
| *** wN has quit IRC | 17:44 | |
| *** wN has joined #openstack-cinder | 17:44 | |
| *** xyang has joined #openstack-cinder | 17:45 | |
| *** xyang has left #openstack-cinder | 17:45 | |
| *** tongli has joined #openstack-cinder | 17:47 | |
| xinli | smcginnis: I merged the code and rerun the patch 251989, and got error message which is not caused by has value , any idea how to debug: http://paste.openstack.org/show/515790/ | 17:47 |
| *** mtanino has quit IRC | 17:47 | |
| *** erlon has joined #openstack-cinder | 17:49 | |
| *** kfarr has quit IRC | 17:54 | |
| *** dobson has joined #openstack-cinder | 17:58 | |
| xinli | team, I hit one issue when I merge the test_objects.py and rerun the code for patch 251989, the error message is here. Does any one could help me to debug? http://paste.openstack.org/show/515790/, the patch 251989: https://review.openstack.org/#/c/251989/ | 17:58 |
| *** lpetrut has joined #openstack-cinder | 17:59 | |
| openstackgerrit | Adriano Freires Rosso proposed openstack/cinder: HNAS drivers refactoring https://review.openstack.org/285469 | 18:00 |
| *** Apoorva_ has joined #openstack-cinder | 18:04 | |
| *** mtanino has joined #openstack-cinder | 18:05 | |
| *** ebalduf has quit IRC | 18:06 | |
| sdague | create_nova_conf is not run on new side | 18:06 |
| sdague | it's not overriding configs, it's just not regenerating them | 18:06 |
| *** ChanServ sets mode: +o smcginnis | 18:07 | |
| *** Apoorva has quit IRC | 18:07 | |
| *** gluytium has joined #openstack-cinder | 18:09 | |
| *** sdake_ has joined #openstack-cinder | 18:23 | |
| *** kfarr has joined #openstack-cinder | 18:43 | |
| *** karthikp_ has quit IRC | 18:47 | |
| *** mvk has quit IRC | 18:51 | |
| *** karthikp_ has joined #openstack-cinder | 18:56 | |
| *** The_Ball has quit IRC | 18:59 | |
| *** salv-orlando has quit IRC | 19:04 | |
| *** salv-orlando has joined #openstack-cinder | 19:05 | |
| *** avishay has quit IRC | 19:05 | |
| *** e0ne has joined #openstack-cinder | 19:11 | |
| *** akapil has joined #openstack-cinder | 19:19 | |
| *** The_Ball has joined #openstack-cinder | 19:22 | |
| *** akapil has quit IRC | 19:24 | |
| *** salv-orlando has quit IRC | 19:31 | |
| *** salv-orlando has joined #openstack-cinder | 19:31 | |
| *** ihrachys has joined #openstack-cinder | 19:35 | |
| *** sheel has quit IRC | 19:35 | |
| *** gluytium has quit IRC | 19:40 | |
| *** e0ne has quit IRC | 19:48 | |
| hemna | sdague, so this issue ( http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/new/screen-n-cpu.txt.gz?#_2016-06-13_13_22_29_717 ) seems to be related to the sudoers file, but we don't capture that in our logs | 19:53 |
| sdague | hemna: we do | 19:54 |
| sdague | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/ | 19:54 |
| sdague | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/sudoers.txt.gz | 19:54 |
| sdague | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/sudoers.d/ | 19:54 |
| hemna | oh! | 19:54 |
| hemna | I didn't see it at the top level | 19:54 |
| sdague | yeh, it's a little weird that it's there and not in the etc dir | 19:54 |
| hemna | I looked in the etc dir and didn't see it, so I assumed.. | 19:55 |
| hemna | I'm trying to reproduce the issue on a local vagrant box | 19:56 |
| hemna | but I can't | 19:56 |
| hemna | :( | 19:56 |
| *** timcl has quit IRC | 19:56 | |
| hemna | I've nuked the nova.conf entry | 19:56 |
| hemna | and the rootwrap filter | 19:56 |
| sdague | hemna: right, well, it only seems to expose in the multinode config | 19:56 |
| hemna | the daemon still starts fine | 19:56 |
| sdague | which, I have no idea why | 19:57 |
| hemna | it's basically just a sudo privsep-helper | 19:57 |
| hemna | doesn't use nova-rootwrap afaik | 19:57 |
| xinli | xyang1: do you have time to take a look why this error message shows up? http://paste.openstack.org/show/515790/ | 19:58 |
| hemna | only thing I can think of is that it has to be related to the sudo warning | 19:58 |
| hemna | sudo: no tty present and no askpass program specified | 19:58 |
| hemna | so I'm trying to reproduce that | 19:58 |
| *** mvk has joined #openstack-cinder | 20:03 | |
| hemna | sdague, so, subnode-2 is the other n-cpu host ? | 20:03 |
| hemna | sdague, http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/subnode-2/pip2-freeze.txt.gz | 20:03 |
| hemna | that has os-brick 1.2.0 on it | 20:03 |
| hemna | and no privsep | 20:03 |
| sdague | hemna: right, that's mitaka | 20:04 |
| sdague | this is a rolling upgrade | 20:04 |
| sdague | so a worker still at mitaka | 20:04 |
| sdague | and the controller/worker node at master | 20:05 |
| hemna | and the attach is happening against the controller host, not the mitaka host I presume ? | 20:05 |
| hemna | hrmm yah I guess so | 20:05 |
| hemna | the mitaka host has os-brick attach calls happening that those are working there. | 20:06 |
| *** sdake_ is now known as sdake | 20:06 | |
| *** zul has quit IRC | 20:06 | |
| sdague | right | 20:06 |
| xinli | geguileor: I hit one issue when rebase the code for patch 251989, the error message shows: http://paste.openstack.org/show/515790/ | 20:09 |
| xinli | geguileor: The code merging just in hash values in test_objects, but the test failed in loading the code Volume_Type.py, any idea to debug? | 20:10 |
| *** akerr has quit IRC | 20:12 | |
| *** chlong has quit IRC | 20:12 | |
| *** Lee1092 has quit IRC | 20:13 | |
| *** tongli has quit IRC | 20:16 | |
| *** porrua has quit IRC | 20:25 | |
| *** kfarr has quit IRC | 20:26 | |
| _alastor_ | Hey folks. I'm attempting to update the docs for the Datera driver. I've cloned openstack-doc-tools and I'm running `./autohelp-wrapper update cinder`, but I get a config option error when I do so | 20:27 |
| _alastor_ | Here's the stacktrace: http://pastebin.com/Cn3A4mJk | 20:28 |
| _alastor_ | The virtualenv appears to be created and the necessary requirements installed | 20:28 |
| *** asselin has joined #openstack-cinder | 20:31 | |
| hemna | sdague, ok I figured it out | 20:33 |
| sdague | hemna: oh... I'm listening | 20:34 |
| hemna | I think | 20:34 |
| hemna | the sudoers setup on that vm is restrictive | 20:34 |
| jgriffith | _alastor_: you using tox? | 20:34 |
| hemna | and there is no config for the stack user to run the privsep-helper with sudo privileges | 20:34 |
| jgriffith | _alastor_: which command? | 20:34 |
| sdague | hemna: why does it work on the single node grenade? | 20:35 |
| sdague | they have the same sudoers files from what I can see | 20:35 |
| hemna | I was able to reproduce it by hacking my vagrant sudoers config files to be the same as that devstack run that has the failure | 20:35 |
| hemna | sdague, http://paste.openstack.org/show/515822/ | 20:35 |
| _alastor_ | jgriffith: Building the docs works out fine with `tox -e build -- config-reference`, but I need to generate the config tables to go with the changes in openstack-manuals | 20:36 |
| hemna | sdague, http://paste.openstack.org/show/515823/ | 20:36 |
| sdague | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/ | 20:36 |
| _alastor_ | jgriffith: and that's the part that fails. It clones cinder, sets up the virtualenv, then fails to import cinder.manager | 20:36 |
| hemna | sdague, haven't gotten to the single node yet | 20:36 |
| hemna | I just discovered how to reproduce the failure | 20:36 |
| sdague | hemna: ok | 20:37 |
| jgriffith | _alastor_: hmm... afraid I haven't tried that, and don't know much about it | 20:37 |
| jgriffith | _alastor_: I just use things like 'tox -e docs' etc | 20:37 |
| jgriffith | _alastor_: assuming you can't get it to work on master either? | 20:38 |
| jgriffith | _alastor_: ie without your modifications? | 20:38 |
| _alastor_ | jgriffith: I'm doing it on upstream master without changes | 20:38 |
| jgriffith | _alastor_: I was afraid you'd say that :( | 20:38 |
| hemna | sdague, http://paste.openstack.org/show/515824/ | 20:39 |
| _alastor_ | jgriffith: So...have you changed any config documenation recently? | 20:39 |
| _alastor_ | jgriffith: what's your process? | 20:39 |
| sdague | I wonder if there is some other difference in the base images we're not capturing related to sudo | 20:39 |
| jgriffith | _alastor_: I have not | 20:39 |
| _alastor_ | jgriffith: darn | 20:39 |
| jgriffith | _alastor_: but I wonder if anybody has... and if it's related to Cinder's new*ish* config generation stuff | 20:40 |
| jgriffith | _alastor_: I'd check git for the last person to add something and ping them :) | 20:40 |
| jgriffith | _alastor_: I'm not seeing much added around config options in the history at first glance | 20:42 |
| hemna | sdague, if the stack user is in the sudo group, it will get priveleges to run that command and will work. | 20:42 |
| sdague | so we think the stack user isn't added during multinode? | 20:43 |
| sdague | that's a viable theory | 20:43 |
| _alastor_ | jgriffith: When I build config-reference it inserts old config information, which is why I'm going through this | 20:43 |
| sdague | that should be in the logs, I think | 20:43 |
| hemna | I don't see /etc/group in our log dump though | 20:43 |
| jgriffith | _alastor_: sounds like you've discovered a problem :( Last official update I seem to find for cidner.conf reference is Liberty | 20:43 |
| _alastor_ | jgriffith: and old config information makes the SE's very unhappy :P | 20:43 |
| jgriffith | _alastor_: but I could be looking in the wrong places | 20:43 |
| jgriffith | _alastor_: pish... they can't just *know* :) | 20:44 |
| jgriffith | _alastor_: wait.. found the mitaka version at least | 20:44 |
| _alastor_ | jgriffith: omniscience is an entry-level SE job requirement | 20:45 |
| jgriffith | _alastor_: It should be, especially if they're expected to sell OpenStack :) | 20:45 |
| jgriffith | _alastor_: so somebody got this to work on the 13'th (today) http://docs.openstack.org/mitaka/config-reference/block-storage/volume-misc.html | 20:46 |
| hemna | sdague, http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/devstack-gate-setup-host.txt.gz#_2016-06-13_12_33_40_541 | 20:46 |
| hemna | there we go | 20:46 |
| *** ihrachys has quit IRC | 20:47 | |
| sdague | that's in multinode as well - http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/devstack-gate-setup-host.txt.gz#_2016-06-13_12_33_14_355 | 20:47 |
| *** vgridnev has joined #openstack-cinder | 20:48 | |
| hemna | arg | 20:48 |
| hemna | ok...hrmm. | 20:48 |
| hemna | I noticed the 50_stach_sh file isn't in our log/sudoers.d dir | 20:48 |
| sdague | hmmm | 20:48 |
| smcginnis | _alastor_: You're trying to update sample config docs? | 20:49 |
| hemna | so I'm not sure where it's going, or if it's just not captured | 20:49 |
| sdague | hemna: yeh, it's deleted later in the process | 20:49 |
| smcginnis | _alastor_: Or the actual documentation? | 20:49 |
| *** julim has joined #openstack-cinder | 20:50 | |
| sdague | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/console.html#_2016-06-13_13_19_52_833 | 20:50 |
| *** ihrachys has joined #openstack-cinder | 20:50 | |
| *** cknight has quit IRC | 20:50 | |
| sdague | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/console.html#_2016-06-13_13_10_08_606 | 20:50 |
| _alastor_ | smcginnis: I'm trying to update the actual documentation for the driver. Right now there isn't any in recent releases | 20:51 |
| sdague | it's not supposed to exist during testing | 20:51 |
| smcginnis | _alastor_: You just need to update the openstack-manuals repo. | 20:51 |
| sdague | what it sounds like, is there is a very specific sudoers addition required for privsep to operate | 20:51 |
| smcginnis | _alastor_: Tables of driver specific options are generated on the fly and added in there. | 20:51 |
| _alastor_ | smcginnis: Updating that is fine, but when it pulls in the config information, it's out of date | 20:51 |
| sdague | which isn't getting added | 20:51 |
| sdague | and happens to work in some configs | 20:52 |
| hemna | sdague, so it's there during the tests on single host runs? | 20:52 |
| sdague | no | 20:52 |
| sdague | it's removed before tests | 20:52 |
| smcginnis | _alastor_: Hmm, may need to file a openstack-manuals bug on that if it's not pulling the latest info. | 20:52 |
| hemna | ok was just confused by the log "grenade.sh completed in 1575 seconds." | 20:53 |
| hemna | I guess that's the setup/upgrade prior to running the tests | 20:53 |
| _alastor_ | smcginnis: Ok, I was trying to force generate the config options by following these instructions: https://github.com/openstack/openstack-doc-tools/tree/master/autogenerate_config_docs | 20:53 |
| sdague | right | 20:53 |
| sdague | we hand off back to tempest at that point, devstack-gate runs tempest after that | 20:53 |
| hemna | ok, sorry for my lack of understanding grenade :) | 20:53 |
| _alastor_ | smcginnis: but that fails to import cinder.manager | 20:53 |
| smcginnis | _alastor_: Hm, never had to do that myself. | 20:54 |
| *** crose has joined #openstack-cinder | 20:54 | |
| smcginnis | Odd, a few of the tables updated just a few days ago. | 20:55 |
| hemna | I presume it restarts sudo after it removes that file | 20:55 |
| *** vgridnev has quit IRC | 20:55 | |
| smcginnis | _alastor_: Looks like it might actually be a manual job. Maybe someone on that team just needs to get around to kicking it off. Not really sure. | 20:56 |
| *** pauloewerton has quit IRC | 20:57 | |
| _alastor_ | smcginnis: It's certainly not been as straightforward as I was hoping :( | 20:57 |
| *** crose has quit IRC | 20:57 | |
| jgriffith | hemna: sudo reads sudoers file dynamically | 20:58 |
| hemna | ok not sure why ubuntu has a 'sudo' service to restart then | 20:58 |
| hemna | odd | 20:58 |
| hemna | ubuntuism mabye | 20:58 |
| xinli | smcginnis: sean, I checked the error message regarding the patch 251989, it seems not caused by hash value, the error message is here: http://paste.openstack.org/show/515790/ any idea to debug? | 20:58 |
| jgriffith | hemna: ubuntu/debian have an sbin to restart, but that's not to read in the sudoers file usually | 20:59 |
| *** gouthamr has quit IRC | 20:59 | |
| hemna | I'm just grasping at straws here. | 20:59 |
| hemna | the problem is defintely sudo related | 20:59 |
| *** ericksonsantos has quit IRC | 20:59 | |
| hemna | I can't figure out how it works in single host grenade | 20:59 |
| jgriffith | hemna: yeah | 20:59 |
| hemna | would be nice to have /etc/group along w/ the logs | 21:00 |
| jgriffith | hemna: sorry, I haven't really been following so I don't know what you're actually working on etc | 21:00 |
| hemna | https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L681-L685 | 21:00 |
| jgriffith | hemna: ok... what about that? | 21:00 |
| hemna | jgriffith, os-brick 1.4.0 got released | 21:00 |
| hemna | and it's causing failures in multinode grenade | 21:01 |
| hemna | and the failure is related to not having sudo privs | 21:01 |
| hemna | jgriffith, http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/new/screen-n-cpu.txt.gz?#_2016-06-13_13_22_29_684 | 21:01 |
| jgriffith | hemna: so that's kinda different, using a sudoers.d directory... or having ansible remove it | 21:01 |
| hemna | privsep log: sudo: no tty present and no askpass program specified | 21:01 |
| hemna | because the stack user doesn't have a specific priv to execute 'sudo privsep-helper ....' | 21:02 |
| hemna | but it works in single host grenade | 21:02 |
| jgriffith | hemna: hmm... I've had those with Ansible in the past... not quite sure I remember the root of the issue though | 21:02 |
| *** raildo is now known as raildo-afk | 21:02 | |
| jgriffith | hemna: I don't know anything about what was done with privsep, so I'm of little help to you | 21:02 |
| hemna | so I thought it could be the /etc/sudoers.d/50_stack_sh file | 21:02 |
| jgriffith | hemna: the multi-node seems to indicate to me that it's a host-->host perms problem | 21:03 |
| hemna | which openly gives the stack user the ability to sudo <anything> | 21:03 |
| jgriffith | hemna: obviously :) | 21:03 |
| *** raunak has quit IRC | 21:03 | |
| jgriffith | hemna: that line you reverenced removes that FWIW | 21:03 |
| hemna | but that ansible magic looks like it gets removed before tempest starts its run | 21:03 |
| smcginnis | xinli: Looks like a problem with that class decorator. | 21:03 |
| smcginnis | xinli: Or "base" is not imported right. | 21:04 |
| *** eharney has quit IRC | 21:04 | |
| smcginnis | xinli: Do you have a link to the review? | 21:04 |
| jgriffith | hemna: yeah, no magic... "state=absent" is explicitly removing it | 21:04 |
| hemna | ok | 21:04 |
| hemna | I'm not familiar with ansible | 21:04 |
| *** raunak has joined #openstack-cinder | 21:04 | |
| * hemna has a sad... | 21:04 | |
| jgriffith | I don't know what DEVSTACK_GATE_REMOVE_STACK_SUDO var is about | 21:04 |
| jgriffith | ansible is pretty easy, until you have to debug it :) | 21:05 |
| xinli | smcginnis: yes, this is the review:https://review.openstack.org/#/c/251989 | 21:05 |
| jgriffith | anyway... sorry, I don't have much to offer. Would be happy to have a look later but kinda behind (as usual) right now | 21:05 |
| hemna | jgriffith, thanks | 21:06 |
| hemna | I'm hacking away and think I'm close........famous last words. | 21:06 |
| smcginnis | xinli: Those line numbers don't match up. Do you have an updated version locally that you haven't pushed up yet? | 21:07 |
| xinli | smcginnis: which lines? in Gerrit or the error message? | 21:09 |
| smcginnis | xinli: The lines in your paste traceback and the lines in the corresponding file in your patch. | 21:10 |
| xinli | smcginnis: the only one I have not push up is test_object.py I need to merge. | 21:10 |
| smcginnis | xinli: Something lese has changed in https://review.openstack.org/#/c/251989/19/cinder/objects/volume_type.py | 21:11 |
| *** ihrachys has quit IRC | 21:14 | |
| xinli | smcginnis: I made double check, my code is the same sa the Gerrit. | 21:14 |
| *** ihrachys has joined #openstack-cinder | 21:14 | |
| *** akapil has joined #openstack-cinder | 21:15 | |
| smcginnis | xinli: This is blowing up on line 121, so something is different. http://paste.openstack.org/show/515790/ | 21:15 |
| xinli | smcginnis: this is line 121: http://paste.openstack.org/show/515833/ | 21:18 |
| xinli | I cut and paste in the paste. | 21:18 |
| xinli | which is the decorator: @base.remotable_classmethod | 21:18 |
| smcginnis | xinli: That's what I would expect. That does not match your latest patch https://review.openstack.org/#/c/251989/19/cinder/objects/volume_type.py | 21:19 |
| openstackgerrit | Szymon Wróblewski proposed openstack/cinder: Start/Stop coordinator with Volume service https://review.openstack.org/263313 | 21:21 |
| *** dustins has quit IRC | 21:22 | |
| openstackgerrit | Merged openstack/os-brick: Add fast8 to quickly test pep8 changes https://review.openstack.org/327964 | 21:22 |
| xinli | smcginnis: how does 't it happen? | 21:25 |
| hemna | sdague, so maybe I'm crazy, but it looks like the grenade.sh script that removes sudo happens after the privsep daemon starts | 21:27 |
| hemna | on the single node test case | 21:27 |
| hemna | 2016-06-13 13:09:32.243 daemon is started | 21:27 |
| hemna | 2016-06-13 13:10:08.609 50_stack_sh is removed | 21:27 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/logs/new/screen-n-cpu.txt.gz#_2016-06-13_13_09_32_243 | 21:27 |
| *** vgridnev has joined #openstack-cinder | 21:27 | |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm/0c44cda/console.html#_2016-06-13_13_10_08_609 | 21:27 |
| openstackgerrit | Szymon Wróblewski proposed openstack/cinder: Update initialize_connection to use versionedobjects https://review.openstack.org/238894 | 21:28 |
| *** gouthamr has joined #openstack-cinder | 21:28 | |
| *** sgotliv has quit IRC | 21:30 | |
| xinli | smcginnis: does someone modify this code volume_type.py/ | 21:31 |
| *** julim has quit IRC | 21:33 | |
| smcginnis | xinli: I think this is your problem: https://github.com/openstack/cinder/commit/504b466295f3d97307216095f450ff8421796319 | 21:34 |
| smcginnis | xinli: The traceback tells you exactly what the problem is. There is no such decorator. | 21:34 |
| smcginnis | hemna: Is it different order in the multinode test case? | 21:36 |
| hemna | smcginnis, in the multinode failure, the 50_stack_sh is removed prior to the first volume attach call (that induces os-brick to start privsep) | 21:37 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/console.html#_2016-06-13_13_19_52_833 | 21:37 |
| smcginnis | hemna: So just a test setup issue? | 21:37 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/new/screen-n-cpu.txt.gz?#_2016-06-13_13_21_27_769 | 21:37 |
| hemna | well, the single node grenade should fail | 21:37 |
| hemna | all things being equal | 21:37 |
| hemna | I think we have to add a new line in the nova-rootwrap sudoers file. | 21:38 |
| hemna | http://logs.openstack.org/83/328983/1/check/gate-grenade-dsvm-multinode/e600477/logs/sudoers.d/nova-rootwrap.txt.gz | 21:38 |
| hemna | if we add | 21:38 |
| hemna | stack ALL=(root) NOPASSWD: /usr/local/bin/privsep-helper | 21:39 |
| hemna | to that line | 21:39 |
| hemna | it will work | 21:39 |
| smcginnis | hemna: Backported to Mitaka now too? Isn't that part of the problem? Or only on Newton? | 21:40 |
| smcginnis | hemna: Seems right that it would need to be in the sudoers thoguh. | 21:40 |
| smcginnis | *though | 21:40 |
| hemna | I think that's what has to happen | 21:40 |
| hemna | but, that goes against the grenade guidelines | 21:40 |
| hemna | so.... | 21:40 |
| hemna | all grenade tests are supposed to work w/o changing any config files | 21:41 |
| hemna | so, honestly, I don't know how nova ever adds changes to rootwrap filter files. | 21:41 |
| hemna | or cinder | 21:41 |
| hemna | or any project for that matter. | 21:41 |
| *** edleafe has joined #openstack-cinder | 21:41 | |
| smcginnis | I'm still not clear why this is grenade only. Isn't that needed regardless? | 21:41 |
| hemna | so new installs get a rootwrap filter | 21:42 |
| hemna | and a nova.conf | 21:42 |
| hemna | change | 21:42 |
| smcginnis | hemna: Oh, so grenade will run the new code with the old config? | 21:43 |
| hemna | yes | 21:44 |
| smcginnis | hemna: Yeah, how would that ever work for rootwrap changes? | 21:44 |
| hemna | and grenade multinode upgrades the controller and leaves the remote n-cpu hosts the older version | 21:44 |
| hemna | smcginnis, that part I have no idea | 21:44 |
| smcginnis | :/ | 21:44 |
| * hemna is so confused | 21:44 | |
| hemna | but what else is new | 21:44 |
| smcginnis | hemna: Well, you're not alone. | 21:45 |
| *** lpetrut has quit IRC | 21:46 | |
| hemna | smcginnis, http://logs.openstack.org/33/253033/28/check/gate-tempest-dsvm-full-lio/c90da23/logs/etc/nova/nova.conf.txt.gz | 21:47 |
| hemna | so devstack injects the privsep_osbrick section | 21:47 |
| hemna | and that overrides what privsep client code (that starts the daemon up) defaults | 21:47 |
| hemna | to use rootwrap to start the daemon | 21:47 |
| hemna | which is why we don't see this in normal tests | 21:48 |
| *** rcernin has quit IRC | 21:48 | |
| hemna | grenade doesn't have that | 21:48 |
| hemna | so unless the stack user has sudo privs to run the privsep-helper command, it will vomit | 21:48 |
| hemna | which it does in multinode grenade | 21:48 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements https://review.openstack.org/328738 | 21:49 |
| _alastor_ | smcginnis: It seems like most of the driver config-table updates are done by gpocentek. I've ping-ed him and I'll update you with the process | 21:50 |
| *** akapil has quit IRC | 21:50 | |
| hemna | so | 21:51 |
| hemna | I think I could be on to it | 21:51 |
| hemna | smcginnis, I just picked another random cinder patch | 21:51 |
| hemna | http://logs.openstack.org/33/253033/28/check/gate-grenade-dsvm/09d1572/console.html#_2016-06-13_15_10_35_935 | 21:51 |
| hemna | 15:10:35 is when the 50_stack_sh is removed | 21:51 |
| smcginnis | _alastor_: Thanks! | 21:51 |
| hemna | http://logs.openstack.org/33/253033/28/check/gate-grenade-dsvm/09d1572/logs/new/screen-n-cpu.txt.gz#_2016-06-13_15_10_13_853 | 21:51 |
| hemna | 15:10:13 is the first volume attach (daemon gets started) | 21:52 |
| hemna | so...I think single host grenade is getting lucky. | 21:52 |
| hemna | sdague, ^^ | 21:52 |
| hemna | this isn't good :( | 21:53 |
| smcginnis | hemna: Does seem to have a timing element to it. | 21:54 |
| hemna | I'm seeing the same thing in every gate-grenade-dsvm that's passing | 21:56 |
| hemna | the first os-brick call is made before the 50_stack_sh is removed. | 21:56 |
| hemna | so, when we fix that | 21:56 |
| hemna | every gate-grenade-dsvm will fail | 21:56 |
| smcginnis | Would be good to get input from sdague and mriedem. | 21:56 |
| hemna | every patch. | 21:56 |
| hemna | the panic sets in.... | 21:56 |
| smcginnis | Hopefully gus is around and has some thoughts on this as well. | 21:58 |
| hemna | so, the workaround for this failure | 21:58 |
| hemna | is to have a nova.conf change (that already exists in devstack) | 21:58 |
| hemna | that updates the helper_command | 21:58 |
| *** Julien-zte has joined #openstack-cinder | 21:58 | |
| hemna | but that also violates the grenade workflow | 21:58 |
| *** ihrachys has quit IRC | 22:00 | |
| hemna | ok, so I'm stuck | 22:01 |
| hemna | I don't know what to do at this point. | 22:01 |
| hemna | 1) there is a grenade.sh issue with timing that incorrectly makes single host grenade tests work....by luck. sometimes within seconds. | 22:02 |
| *** ducttape_ has quit IRC | 22:02 | |
| hemna | 2) the way the privsep-helper is being launched outside of rootwrap by default will always make grenade fail because | 22:02 |
| hemna | 3) we aren't allowed to touch nova.conf in grenade tests. | 22:02 |
| hemna | so, multinode grenade just happened to find a valid bug | 22:03 |
| jgriffith | hemna: shouldn't you just support *both* for N+1? | 22:03 |
| jgriffith | hemna: ie both rootwrap and privsep? | 22:04 |
| smcginnis | jgriffith: That may be the route we need to go just to get privsep established. | 22:04 |
| hemna | I think that's possible | 22:04 |
| jgriffith | smcginnis: I'm not sure how else you'd do it and maintain back compat and rolling upgrades etc | 22:04 |
| hemna | but, it would require us changing os-brick's default execution to rootwrap from privsep | 22:05 |
| jgriffith | smcginnis: hemna you could just write a shim to handle it | 22:05 |
| *** ihrachys has joined #openstack-cinder | 22:05 | |
| *** hoonetorg has quit IRC | 22:05 | |
| hemna | and then push a nova patch that forces privsep | 22:05 |
| jgriffith | hemna: not sure about that | 22:05 |
| jgriffith | hemna: wondering if you could implement a shim | 22:05 |
| smcginnis | I got some flak for not having privsep in M. Nova doesn't want to have to do any more rootwrap filter updates. But I don't see anther way around it. | 22:06 |
| jgriffith | hemna: look at it from the other way... instead of trying to keep privsep from breaking rootwrap... maybe try to fake out rootwrap to use privsep settings | 22:06 |
| jgriffith | hemna: impl | 22:06 |
| jgriffith | hemna: know what I mean? | 22:06 |
| hemna | yah | 22:06 |
| smcginnis | Process execution drivers? | 22:06 |
| hemna | but w/o changing rootwrap filters | 22:06 |
| jgriffith | hemna: in other words a shim to make privsep backwards compatable | 22:06 |
| hemna | I'm not sure how to do it | 22:06 |
| *** edmondsw has quit IRC | 22:07 | |
| jgriffith | smcginnis: well.. that's the other thing I'm kinda confused. I should admittedly look at this all more | 22:07 |
| smcginnis | I'm not sure if privsep could be backwards compatible. | 22:07 |
| hemna | jgriffith, it took me a bit to wrap my head around it | 22:07 |
| jgriffith | smcginnis: but... IIRC os-brick and lots of other code has a driver type architecture for the priv stuff | 22:07 |
| smcginnis | This problem is kind of fundamental to how it needs to run, if I understand it right. | 22:08 |
| jgriffith | smcginnis: ie the executor I thought was pluggable | 22:08 |
| *** hoonetorg has joined #openstack-cinder | 22:08 | |
| smcginnis | SO a shim that can be aware of whether privsep can be used, and if not fail back to using rootwrap as the executor. | 22:09 |
| hemna | maybe it's possible to detect if the helper_command conf setting is there | 22:10 |
| hemna | if it is, then privsep can be used. | 22:10 |
| hemna | else, use rootwrap | 22:10 |
| smcginnis | Piece of cake. What could wrong? :) | 22:10 |
| hemna | https://github.com/openstack/os-brick/blob/master/os_brick/privileged/__init__.py#L18 | 22:11 |
| hemna | that builds the privsep context | 22:11 |
| hemna | and sets the config section name | 22:11 |
| hemna | internally that context looks for that section to populate the helper_command | 22:11 |
| hemna | I think I can try and read the https://github.com/openstack/oslo.privsep/blob/master/oslo_privsep/priv_context.py#L83 | 22:12 |
| hemna | cfg_section attribute | 22:12 |
| hemna | if it's None | 22:12 |
| hemna | then the helper_command isn't set | 22:13 |
| hemna | and then the privsep-helper will be run w/o privs | 22:13 |
| hemna | and fail during grenade | 22:13 |
| hemna | so, if it's None, then force os_brick to use rootwrap instead. | 22:13 |
| hemna | phew | 22:13 |
| hemna | it might work | 22:13 |
| jgriffith | hemna: sounds like a plan | 22:14 |
| smcginnis | So we would need to make sure to make any necessary updates to rootwrap filters as well. | 22:14 |
| smcginnis | And nova too. | 22:14 |
| smcginnis | sdague: ^^ | 22:14 |
| hemna | smcginnis, I think they are all in there...... | 22:14 |
| * hemna hopes | 22:14 | |
| smcginnis | And we can't really switch everything over to "native" privsep usage like we would eventually like to. | 22:14 |
| smcginnis | But at least then we would be in good shape to move forward in Ocata. | 22:15 |
| hemna | well if my patch works, then when nova has the privsep_osbrick section in place with the helper_command specified, it should just work...... | 22:15 |
| smcginnis | hemna: Yeah, we should be good for now. Unless something important gets added yet in this cycle. | 22:15 |
| smcginnis | hemna: It would just be an upgrade issue. | 22:16 |
| hemna | yah | 22:16 |
| hemna | the devstack scripts inject that section in nova.conf and cinder.conf today | 22:16 |
| hemna | also need to eventually add the sudoers.d/nova-rootwrap change too | 22:17 |
| hemna | or add a sudoers.d/privsep | 22:17 |
| hemna | or whatever | 22:17 |
| openstackgerrit | Szymon Wróblewski proposed openstack/cinder: Replace locks in volume manager https://review.openstack.org/185646 | 22:17 |
| *** ihrachys_ has joined #openstack-cinder | 22:18 | |
| *** ihrachys has quit IRC | 22:20 | |
| *** chlong has joined #openstack-cinder | 22:21 | |
| openstackgerrit | Szymon Wróblewski proposed openstack/cinder: Replace locks in volume manager https://review.openstack.org/185646 | 22:21 |
| *** xinli has quit IRC | 22:22 | |
| *** darosale has quit IRC | 22:22 | |
| _alastor_ | smcginnis: I've filed the bug against openstack-manuals regarding the failure in openstack-doc-tools. Here's the bug: https://bugs.launchpad.net/openstack-manuals/+bug/1592178. Still waiting on a response from gpocentek | 22:23 |
| openstack | Launchpad bug 1592178 in openstack-manuals "NoSuchOptError openstack-doc-tools with Cinder" [Undecided,New] | 22:23 |
| smcginnis | _alastor_: Cool, thanks. | 22:23 |
| *** ihrachys_ has quit IRC | 22:25 | |
| smcginnis | _alastor_: At a minimum, it's hiding the actual error by doing "raise e" instead of "raise". | 22:25 |
| _alastor_ | smcginnis: agreed. I'll make sure to mention it in the bug | 22:26 |
| openstackgerrit | Szymon Wróblewski proposed openstack/cinder: Replace locks in remotefs backend driver https://review.openstack.org/270240 | 22:26 |
| *** ihrachys has joined #openstack-cinder | 22:27 | |
| openstackgerrit | Szymon Wróblewski proposed openstack/cinder: Replace locks in remotefs backend driver https://review.openstack.org/270240 | 22:28 |
| *** mriedem is now known as mriedem_away | 22:29 | |
| *** karthikp_ has quit IRC | 22:31 | |
| *** ihrachys has quit IRC | 22:33 | |
| *** akshai has joined #openstack-cinder | 22:36 | |
| *** ihrachys has joined #openstack-cinder | 22:39 | |
| *** akshai has quit IRC | 22:45 | |
| *** dave-mccowan has quit IRC | 22:47 | |
| *** ihrachys has quit IRC | 22:48 | |
| *** salv-orlando has quit IRC | 22:51 | |
| *** ihrachys has joined #openstack-cinder | 22:52 | |
| *** rhagarty__ has quit IRC | 22:52 | |
| *** dave-mccowan has joined #openstack-cinder | 22:52 | |
| *** rhagarty has joined #openstack-cinder | 22:52 | |
| *** salv-orlando has joined #openstack-cinder | 22:52 | |
| *** ducttape_ has joined #openstack-cinder | 23:00 | |
| *** ihrachys has quit IRC | 23:01 | |
| *** ihrachys has joined #openstack-cinder | 23:01 | |
| *** hemna is now known as hemnafk | 23:04 | |
| *** xyang1 has quit IRC | 23:05 | |
| *** bardia has quit IRC | 23:10 | |
| *** akshai has joined #openstack-cinder | 23:13 | |
| *** ihrachys has quit IRC | 23:17 | |
| *** Julien-zte has quit IRC | 23:21 | |
| *** ihrachys has joined #openstack-cinder | 23:22 | |
| *** akshai_ has joined #openstack-cinder | 23:24 | |
| *** akshai has quit IRC | 23:24 | |
| *** tpsilva has quit IRC | 23:27 | |
| *** salv-orl_ has joined #openstack-cinder | 23:28 | |
| *** salv-orlando has quit IRC | 23:31 | |
| *** ihrachys has quit IRC | 23:33 | |
| *** r-daneel has quit IRC | 23:33 | |
| *** ihrachys has joined #openstack-cinder | 23:33 | |
| *** gouthamr_ has joined #openstack-cinder | 23:38 | |
| *** sdague has quit IRC | 23:38 | |
| *** gouthamr has quit IRC | 23:42 | |
| *** akshai_ has quit IRC | 23:42 | |
| *** ihrachys has quit IRC | 23:47 | |
| *** ihrachys has joined #openstack-cinder | 23:47 | |
| *** leeantho_ has quit IRC | 23:51 | |
| *** ihrachys has quit IRC | 23:55 | |
| *** yhayashi has joined #openstack-cinder | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!