| *** pv_ has quit IRC | 00:02 | |
| *** laughterwym has joined #openstack-cinder | 00:03 | |
| *** tsekiyam_ has joined #openstack-cinder | 00:12 | |
| *** IlyaG_ has quit IRC | 00:14 | |
| *** DericHorn-HP has joined #openstack-cinder | 00:15 | |
| *** mtanino has quit IRC | 00:15 | |
| *** DericHorn-HP has quit IRC | 00:16 | |
| *** tsekiyama has quit IRC | 00:16 | |
| *** tsekiyam_ has quit IRC | 00:17 | |
| *** logan2 has quit IRC | 00:19 | |
| *** chlong has joined #openstack-cinder | 00:21 | |
| *** logan2 has joined #openstack-cinder | 00:22 | |
| *** julim has quit IRC | 00:24 | |
| *** cebruns has quit IRC | 00:27 | |
| *** cebruns has joined #openstack-cinder | 00:28 | |
| *** anish has quit IRC | 00:29 | |
| *** jasondotstar has joined #openstack-cinder | 00:30 | |
| *** dims has joined #openstack-cinder | 00:32 | |
| *** leeantho has quit IRC | 00:33 | |
| *** annashen has joined #openstack-cinder | 00:34 | |
| *** breitz has quit IRC | 00:34 | |
| *** anish has joined #openstack-cinder | 00:35 | |
| *** dims_ has joined #openstack-cinder | 00:35 | |
| *** breitz has joined #openstack-cinder | 00:37 | |
| *** nikeshm has joined #openstack-cinder | 00:37 | |
| *** annashen has quit IRC | 00:38 | |
| *** dims has quit IRC | 00:39 | |
| *** IlyaG has joined #openstack-cinder | 00:46 | |
| *** zhenguo has joined #openstack-cinder | 00:47 | |
| *** IanGovett has quit IRC | 00:48 | |
| *** _cjones_ has quit IRC | 00:49 | |
| *** _cjones_ has joined #openstack-cinder | 00:50 | |
| *** Yogi1 has joined #openstack-cinder | 00:51 | |
| *** _cjones_ has quit IRC | 00:55 | |
| *** dannywilson has quit IRC | 00:55 | |
| *** annegentle has joined #openstack-cinder | 01:01 | |
| *** boris-42 has quit IRC | 01:02 | |
| *** Rockyg has joined #openstack-cinder | 01:03 | |
| *** tellesnobrega_ has joined #openstack-cinder | 01:05 | |
| *** IlyaG has quit IRC | 01:09 | |
| *** lixiaoy1 has joined #openstack-cinder | 01:10 | |
| *** r-daneel has quit IRC | 01:11 | |
| *** liuxg has joined #openstack-cinder | 01:13 | |
| *** IlyaG has joined #openstack-cinder | 01:14 | |
| *** bswartz has quit IRC | 01:14 | |
| *** Yogi1 has quit IRC | 01:17 | |
| *** changbl has joined #openstack-cinder | 01:19 | |
| *** davechen has joined #openstack-cinder | 01:20 | |
| openstackgerrit | Teruaki Ishizaki proposed openstack/cinder: sheepdog: change create and delete operation https://review.openstack.org/197393 | 01:20 |
|---|---|---|
| *** Apoorva has quit IRC | 01:22 | |
| *** dannywilson has joined #openstack-cinder | 01:23 | |
| *** patrickeast has quit IRC | 01:24 | |
| *** liuxg has quit IRC | 01:25 | |
| *** liuxg has joined #openstack-cinder | 01:25 | |
| *** Rockyg has quit IRC | 01:28 | |
| *** asselin_ has joined #openstack-cinder | 01:33 | |
| *** annashen has joined #openstack-cinder | 01:35 | |
| *** IlyaG has quit IRC | 01:36 | |
| *** tellesnobrega_ has quit IRC | 01:38 | |
| *** annashen has quit IRC | 01:39 | |
| *** Lee1092 has joined #openstack-cinder | 01:40 | |
| *** liuxg has quit IRC | 01:42 | |
| *** annegentle has quit IRC | 01:42 | |
| *** liuxg has joined #openstack-cinder | 01:43 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: set/unset volume image metadata https://review.openstack.org/147726 | 01:44 |
| *** liuxg has quit IRC | 01:47 | |
| *** heyun has joined #openstack-cinder | 01:48 | |
| *** liuxg has joined #openstack-cinder | 01:48 | |
| *** liuxg has quit IRC | 01:57 | |
| *** jwcroppe has quit IRC | 02:02 | |
| *** jwcroppe has joined #openstack-cinder | 02:02 | |
| openstackgerrit | Dave Chen proposed openstack/python-cinderclient: Add commands for modifying image metadata https://review.openstack.org/147077 | 02:05 |
| *** IlyaG has joined #openstack-cinder | 02:05 | |
| *** jwcroppe has quit IRC | 02:07 | |
| *** jasondotstar has quit IRC | 02:07 | |
| *** annegentle has joined #openstack-cinder | 02:08 | |
| *** Longgeek_ has joined #openstack-cinder | 02:10 | |
| *** Longgeek has quit IRC | 02:12 | |
| *** coolsvap|away has quit IRC | 02:13 | |
| *** haomaiwa_ has joined #openstack-cinder | 02:14 | |
| *** coolsvap|away has joined #openstack-cinder | 02:15 | |
| *** mriedem has quit IRC | 02:16 | |
| *** mutoulbj has joined #openstack-cinder | 02:18 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: Role based properties protection https://review.openstack.org/160090 | 02:20 |
| openstackgerrit | wanghao proposed openstack/cinder-specs: Support query volume filter by glance metadata https://review.openstack.org/174778 | 02:23 |
| openstackgerrit | Dave Chen proposed openstack/cinder: Policies based properties protection https://review.openstack.org/160145 | 02:23 |
| *** xyang1 has quit IRC | 02:23 | |
| *** tobe has joined #openstack-cinder | 02:29 | |
| *** mutoulbj_ has joined #openstack-cinder | 02:30 | |
| *** mutoulbj has quit IRC | 02:34 | |
| *** annashen has joined #openstack-cinder | 02:36 | |
| *** haomaiwa_ has quit IRC | 02:39 | |
| *** Longgeek_ has quit IRC | 02:39 | |
| *** haomaiw__ has joined #openstack-cinder | 02:40 | |
| *** Longgeek has joined #openstack-cinder | 02:41 | |
| *** annashen has quit IRC | 02:41 | |
| *** winston-d_afk has quit IRC | 02:49 | |
| *** winston-d has joined #openstack-cinder | 02:50 | |
| *** annegentle has quit IRC | 02:53 | |
| openstackgerrit | Rakesh H S proposed openstack/cinder: Return 404 if volume type encryption is not found https://review.openstack.org/194677 | 02:58 |
| *** annegentle has joined #openstack-cinder | 03:04 | |
| *** haomaiw__ has quit IRC | 03:09 | |
| *** haomaiwang has joined #openstack-cinder | 03:10 | |
| *** liuxg has joined #openstack-cinder | 03:11 | |
| *** Longgeek has quit IRC | 03:14 | |
| openstackgerrit | LisaLi proposed openstack/cinder: Replace missed basestring by six for python3 compatability https://review.openstack.org/197405 | 03:16 |
| *** liuxg has quit IRC | 03:16 | |
| *** mutoulbj_ has quit IRC | 03:18 | |
| openstackgerrit | wanghao proposed openstack/cinder: query volume detail support volume_glance_metadata https://review.openstack.org/147738 | 03:18 |
| *** mutoulbj has joined #openstack-cinder | 03:21 | |
| *** mutoulbj has quit IRC | 03:23 | |
| *** garthb_ has joined #openstack-cinder | 03:24 | |
| *** garthb has joined #openstack-cinder | 03:24 | |
| *** breitz has quit IRC | 03:31 | |
| *** annegentle has quit IRC | 03:31 | |
| *** mutoulbj has joined #openstack-cinder | 03:32 | |
| *** bill_az has quit IRC | 03:34 | |
| *** yamada-h has joined #openstack-cinder | 03:36 | |
| *** annashen has joined #openstack-cinder | 03:37 | |
| *** annegentle has joined #openstack-cinder | 03:38 | |
| *** harlowja_at_home has joined #openstack-cinder | 03:40 | |
| *** annashen has quit IRC | 03:41 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: Role based properties protection https://review.openstack.org/160090 | 03:44 |
| openstackgerrit | Dave Chen proposed openstack/cinder: Policies based properties protection https://review.openstack.org/160145 | 03:44 |
| *** Longgeek has joined #openstack-cinder | 03:44 | |
| *** hemna has joined #openstack-cinder | 03:45 | |
| *** BharatK has joined #openstack-cinder | 03:46 | |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Refactor Huawei Volume driver https://review.openstack.org/188240 | 03:48 |
| *** annegentle has quit IRC | 03:48 | |
| *** Longgeek has quit IRC | 03:49 | |
| *** Liu has joined #openstack-cinder | 03:51 | |
| *** Liu has quit IRC | 03:53 | |
| *** hemna has quit IRC | 03:53 | |
| *** MentalRay has joined #openstack-cinder | 03:56 | |
| *** heyun1 has joined #openstack-cinder | 03:56 | |
| *** heyun has quit IRC | 03:57 | |
| *** jwcroppe has joined #openstack-cinder | 04:00 | |
| *** dims_ has quit IRC | 04:03 | |
| *** davechen has quit IRC | 04:07 | |
| *** davechen has joined #openstack-cinder | 04:08 | |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Support SmartX for Huawei volume driver https://review.openstack.org/188732 | 04:13 |
| *** coolsvap|away is now known as coolsvap | 04:14 | |
| *** harlowja_at_home has quit IRC | 04:15 | |
| *** davechen1 has joined #openstack-cinder | 04:17 | |
| *** davechen has quit IRC | 04:18 | |
| *** hemna has joined #openstack-cinder | 04:20 | |
| *** davechen has joined #openstack-cinder | 04:20 | |
| *** hemna has quit IRC | 04:21 | |
| *** dannywilson has quit IRC | 04:21 | |
| *** dannywilson has joined #openstack-cinder | 04:21 | |
| *** davechen1 has quit IRC | 04:22 | |
| *** morganfainberg is now known as caerbannograbbit | 04:24 | |
| *** caerbannograbbit is now known as CaerbannogRabbit | 04:24 | |
| openstackgerrit | Vincent Hou proposed openstack/cinder: Implement the update_migrated_volume for the drivers https://review.openstack.org/180873 | 04:25 |
| *** tobe has quit IRC | 04:26 | |
| *** deepakcs has joined #openstack-cinder | 04:33 | |
| *** Longgeek has joined #openstack-cinder | 04:34 | |
| *** annashen has joined #openstack-cinder | 04:37 | |
| *** avishay has joined #openstack-cinder | 04:40 | |
| *** shyama has joined #openstack-cinder | 04:41 | |
| *** nkrinner has joined #openstack-cinder | 04:41 | |
| *** annashen has quit IRC | 04:42 | |
| *** annegentle has joined #openstack-cinder | 04:49 | |
| *** MentalRay has quit IRC | 04:53 | |
| *** annegentle has quit IRC | 04:55 | |
| *** nihilifer has joined #openstack-cinder | 04:58 | |
| *** skraynev has joined #openstack-cinder | 05:02 | |
| *** chlong has quit IRC | 05:03 | |
| *** Nikolay_St has joined #openstack-cinder | 05:06 | |
| *** dims has joined #openstack-cinder | 05:07 | |
| *** dims has quit IRC | 05:12 | |
| *** chlong has joined #openstack-cinder | 05:21 | |
| *** tobe has joined #openstack-cinder | 05:26 | |
| *** heyun1 has quit IRC | 05:30 | |
| *** heyun has joined #openstack-cinder | 05:31 | |
| *** haomaiwang has quit IRC | 05:33 | |
| *** gaurang has joined #openstack-cinder | 05:36 | |
| *** haomaiwa_ has joined #openstack-cinder | 05:38 | |
| *** annashen has joined #openstack-cinder | 05:39 | |
| *** kaisers1 has quit IRC | 05:43 | |
| *** annashen has quit IRC | 05:43 | |
| *** gaurang1 has joined #openstack-cinder | 05:44 | |
| *** gaurang has quit IRC | 05:45 | |
| *** gaurang1 has left #openstack-cinder | 05:46 | |
| *** gaurang1 has joined #openstack-cinder | 05:46 | |
| *** gaurang1 has left #openstack-cinder | 05:49 | |
| *** jdurgin1 has joined #openstack-cinder | 05:51 | |
| *** gaurangt has joined #openstack-cinder | 05:52 | |
| *** dannywilson has quit IRC | 05:53 | |
| openstackgerrit | Shay Halsband proposed openstack/cinder: XtremIO support for iscsi discovery auth https://review.openstack.org/189358 | 05:54 |
| *** gaurangt has left #openstack-cinder | 05:56 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: Role based properties protection https://review.openstack.org/160090 | 05:56 |
| openstackgerrit | Dave Chen proposed openstack/cinder: set/unset volume image metadata https://review.openstack.org/147726 | 05:56 |
| openstackgerrit | Dave Chen proposed openstack/cinder: Policies based properties protection https://review.openstack.org/160145 | 05:56 |
| *** gaurangt has joined #openstack-cinder | 05:56 | |
| *** jdurgin has quit IRC | 05:57 | |
| *** garthb_ has quit IRC | 05:57 | |
| *** garthb has quit IRC | 05:57 | |
| *** tobe has quit IRC | 05:59 | |
| *** e0ne has joined #openstack-cinder | 05:59 | |
| *** jdurgin1 has quit IRC | 06:00 | |
| *** chlong has quit IRC | 06:01 | |
| *** chlong has joined #openstack-cinder | 06:01 | |
| *** BharatK has quit IRC | 06:02 | |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Add iSCSI multipath support for Huawei driver https://review.openstack.org/188365 | 06:05 |
| *** e0ne has quit IRC | 06:11 | |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Add CHAP support for Huawei driver https://review.openstack.org/188360 | 06:15 |
| *** BharatK has joined #openstack-cinder | 06:17 | |
| *** gaurangt has left #openstack-cinder | 06:18 | |
| *** chlong has quit IRC | 06:19 | |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Add multiple pools support for Huawei driver https://review.openstack.org/188251 | 06:23 |
| *** chlong has joined #openstack-cinder | 06:34 | |
| *** skraynev has quit IRC | 06:35 | |
| openstackgerrit | LisaLi proposed openstack/cinder: Fix backup.rpcapi to pass object backup https://review.openstack.org/197442 | 06:39 |
| *** annashen has joined #openstack-cinder | 06:39 | |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Mock socket.gethostbyaddr in test_v7000_fcp https://review.openstack.org/197091 | 06:41 |
| *** annashen has quit IRC | 06:44 | |
| *** agarciam has joined #openstack-cinder | 06:50 | |
| *** annegentle has joined #openstack-cinder | 06:51 | |
| *** tobe has joined #openstack-cinder | 06:53 | |
| *** belmoreira has joined #openstack-cinder | 06:54 | |
| *** skraynev has joined #openstack-cinder | 06:56 | |
| *** annegentle has quit IRC | 06:56 | |
| *** anshul has joined #openstack-cinder | 06:56 | |
| *** ndipanov has quit IRC | 06:57 | |
| *** sgotliv has quit IRC | 06:59 | |
| *** jdurgin has joined #openstack-cinder | 07:07 | |
| *** avishay has quit IRC | 07:12 | |
| *** avishay_ has joined #openstack-cinder | 07:12 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: set/unset volume image metadata https://review.openstack.org/147726 | 07:22 |
| *** chlong has quit IRC | 07:24 | |
| winston-d | davechen: ping | 07:28 |
| *** gaurangt has joined #openstack-cinder | 07:32 | |
| *** zhenguo has quit IRC | 07:32 | |
| *** zhenguo has joined #openstack-cinder | 07:33 | |
| *** annashen has joined #openstack-cinder | 07:40 | |
| *** annashen has quit IRC | 07:45 | |
| *** jistr has joined #openstack-cinder | 07:45 | |
| *** avishay_ has quit IRC | 07:49 | |
| *** avishay_ has joined #openstack-cinder | 07:50 | |
| openstackgerrit | Xi Yang proposed openstack/cinder: Fix getting out-of-date volume operation state issue for VNX https://review.openstack.org/197458 | 07:51 |
| *** e0ne has joined #openstack-cinder | 07:56 | |
| *** jordanP has joined #openstack-cinder | 07:57 | |
| openstackgerrit | Tom Fifield proposed openstack/cinder: Fix typo in solidfire driver option https://review.openstack.org/197459 | 07:59 |
| *** afazekas is now known as __afazekas | 07:59 | |
| openstackgerrit | Anton Arefiev proposed openstack/cinder: Add entry create and cast tasks to manage workflow https://review.openstack.org/139071 | 08:02 |
| openstackgerrit | Anton Arefiev proposed openstack/cinder: Fix lvm manage existing volume https://review.openstack.org/156939 | 08:02 |
| davechen | winston-d: hi, zhiteng | 08:04 |
| davechen | davechen: sorry, just see your message. | 08:04 |
| davechen | wiston-d: sorry, just see your message. | 08:04 |
| davechen | winston-d: still around? | 08:05 |
| winston-d | davechen: hi | 08:05 |
| openstackgerrit | Peter Wang proposed openstack/cinder: Add white list support for target ports in VNX driver https://review.openstack.org/196979 | 08:05 |
| openstackgerrit | Dave Chen proposed openstack/cinder: Role based properties protection https://review.openstack.org/160090 | 08:06 |
| davechen | hi, | 08:06 |
| openstackgerrit | Dave Chen proposed openstack/cinder: Policies based properties protection https://review.openstack.org/160145 | 08:06 |
| davechen | :) | 08:06 |
| winston-d | the enum34 lib isn't part of cinder requirement yet. | 08:06 |
| davechen | winston-d: but it's in global-req, just as you said. | 08:06 |
| winston-d | you will have to sync it from global-requirement before you can use it. otherwise jenkins will puke | 08:06 |
| davechen | winston-d: I am confused, so we cannot rely on global-req? | 08:07 |
| e0ne | winston-d: hi. i answered your questions accroding to https://review.openstack.org/#/c/192683/ | 08:07 |
| davechen | winston-d: how to sync with it, can you show me? | 08:08 |
| winston-d | davechen: in short, you need to explicitly add 'enum34' to cinder/requirements.txt before you can use it | 08:08 |
| davechen | winston-d: got it. | 08:08 |
| davechen | winston-d: will update it shortly. | 08:08 |
| *** bluex-pl has joined #openstack-cinder | 08:09 | |
| davechen | eOne: thanks for your review on the patches about the policy protection. | 08:09 |
| winston-d | e0ne: thx, what about the configurations for two setups? | 08:09 |
| davechen | winston-d: anything else do you found with that patch? | 08:09 |
| davechen | find* | 08:09 |
| winston-d | davechen: i'm good with the rest | 08:10 |
| e0ne | winston-d: i added it to the comment: "I tested it on VM with Ubuntu 14.04.1 LTS, 1 CPU, 1GB RAM, SSD. Rally: commit e4ee07f755f32a7962fb01dc0658e14d7ba622b7 Keystone: commit a70b514ed9aa1ccb4084e85fb7d342daabd337ed Keystone client: the latest version from PyPI." | 08:10 |
| winston-d | e0ne: i saw that. | 08:10 |
| winston-d | that's hardware config and software version. | 08:10 |
| winston-d | i'm asking for detailed keystone configurations | 08:11 |
| davechen | winston-d: And, Sean has already vote +2 on the patch of cinder client. https://review.openstack.org/#/c/147077/ | 08:11 |
| e0ne | winston-d: it's devstack configuration w/o changes | 08:11 |
| winston-d | davechen: i'll wait for your server-side change to land. | 08:11 |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Add CHAP support for Huawei driver https://review.openstack.org/188360 | 08:11 |
| e0ne | winston-d: i just stopped apache and started 'keystone-all' manually | 08:11 |
| davechen | winston-d: okay, many thanks, will ping you when that patch pass the Jenkins. | 08:12 |
| winston-d | e0ne: can you paste keystone.conf and httpd.conf ? | 08:12 |
| e0ne | winston-d: sure, 1min please | 08:13 |
| winston-d | e0ne: thx | 08:13 |
| *** sgotliv has joined #openstack-cinder | 08:13 | |
| davechen | winston-d: quite not understand why Jenkins passed with sync with global-req. | 08:15 |
| davechen | winston-d: anyway, I will add it the requirment.txt | 08:16 |
| openstackgerrit | Liu Xinguo proposed openstack/cinder: Add iSCSI multipath support for Huawei driver https://review.openstack.org/188365 | 08:16 |
| e0ne | winston-d: apache config: http://paste.openstack.org/show/331356/ | 08:18 |
| e0ne | winston-d: keystone config w/o commented lines: http://paste.openstack.org/show/331357/ | 08:20 |
| *** __afazekas is now known as afazekas | 08:20 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: Role based properties protection https://review.openstack.org/160090 | 08:22 |
| openstackgerrit | Dave Chen proposed openstack/cinder: set/unset volume image metadata https://review.openstack.org/147726 | 08:22 |
| openstackgerrit | Dave Chen proposed openstack/cinder: Policies based properties protection https://review.openstack.org/160145 | 08:22 |
| haypo | hemnafk: hi! i'm victor stinner | 08:23 |
| winston-d | e0ne: is the same keystone.conf be used in both tests? | 08:23 |
| e0ne | winston-d: yes | 08:23 |
| winston-d | haypo: 1am for hemnafk, so ... | 08:23 |
| haypo | winston-d: i will wait for him ;) | 08:24 |
| winston-d | e0ne: ok, so for keystone-apache, it would ignore 'admin/public_worker' option, right? | 08:24 |
| e0ne | winston-d: yes, it will use WSGIDaemonProcess settings | 08:25 |
| winston-d | e0ne: and for your apache config, 'keystone-public processes=5 threads=1' means you configured apache to use 5 processes, right? | 08:25 |
| e0ne | yes | 08:25 |
| haypo | hey! os-brick got its (non-voting) py34 check job, cool ;) | 08:26 |
| winston-d | e0ne: but for evenlet option, 'admin_workers = 2' | 08:26 |
| winston-d | 2 vs 5. | 08:26 |
| winston-d | that's not fair game. | 08:26 |
| e0ne | winston-d: my fail | 08:26 |
| winston-d | e0ne: that's why i need detailed config. :) | 08:26 |
| e0ne | winston-d: i'll fix it later today | 08:26 |
| e0ne | winston-d: sure, you're right | 08:26 |
| winston-d | how many vCPUs does your VM have? | 08:26 |
| e0ne | winston-d: i didn't verify that there are different numbers of workers | 08:27 |
| e0ne | 1 CPU | 08:27 |
| winston-d | e0ne: ok. so next time, when you re-do the test, please make sure the workers = processes. and paste the config to gerrit. | 08:28 |
| e0ne | winston-d: thanks for pointing me where i'm wrong! | 08:28 |
| winston-d | e0ne: np, thx for gathering the data. | 08:28 |
| e0ne | winston-d: i'll update my tests results whith configs today before our meeting | 08:29 |
| winston-d | e0ne: ok. sounds good. | 08:29 |
| davechen | winston-d: do you know where Sean based? | 08:29 |
| e0ne | need to head to office. bb in ~1.5 hour | 08:29 |
| winston-d | Sean is in US too, so basically he's sleeping. | 08:29 |
| openstackgerrit | LisaLi proposed openstack/cinder: Fix backup.rpcapi to pass object backup https://review.openstack.org/197442 | 08:32 |
| davechen | damn it, gate-cinder-requirements failed after adding enum34 into req.txt. | 08:34 |
| winston-d | davechen: you didn't do it right. | 08:35 |
| *** mutoulbj_ has joined #openstack-cinder | 08:36 | |
| *** liudong has joined #openstack-cinder | 08:36 | |
| openstackgerrit | LisaLi proposed openstack/cinder: Fix backup.rpcapi to pass object backup https://review.openstack.org/197442 | 08:36 |
| *** e0ne is now known as e0ne_ | 08:36 | |
| davechen | emm, how? | 08:36 |
| *** liudong has left #openstack-cinder | 08:36 | |
| winston-d | davechen: the actual line in global-requirements looks like this: https://github.com/openstack/requirements/blob/master/global-requirements.txt#L36 | 08:37 |
| winston-d | davechen: it must be exactly the same when adding it to cinder. | 08:37 |
| winston-d | davechen: 'sync' | 08:37 |
| *** gaurangt has quit IRC | 08:37 | |
| *** gaurangt has joined #openstack-cinder | 08:38 | |
| *** e0ne_ is now known as e0ne | 08:38 | |
| *** mutoulbj has quit IRC | 08:39 | |
| davechen | winston-d: oh, may local code is out of date. | 08:40 |
| davechen | winston-d: how do you know all of these things. :) | 08:40 |
| *** lpetrut has joined #openstack-cinder | 08:40 | |
| *** e0ne has quit IRC | 08:41 | |
| *** annashen has joined #openstack-cinder | 08:41 | |
| openstackgerrit | Dave Chen proposed openstack/cinder: set/unset volume image metadata https://review.openstack.org/147726 | 08:42 |
| *** avishay_ has quit IRC | 08:45 | |
| *** avishay__ has joined #openstack-cinder | 08:45 | |
| winston-d | davechen: hmm, there is no unit test for db api change and volume.api change | 08:45 |
| *** annashen has quit IRC | 08:46 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder: Checking the existence of volume https://review.openstack.org/175809 | 08:46 |
| *** laughterwym has quit IRC | 08:47 | |
| *** aix has joined #openstack-cinder | 08:48 | |
| *** jasondotstar has joined #openstack-cinder | 08:48 | |
| *** afazekas is now known as __afazekas | 08:50 | |
| openstackgerrit | Victor Stinner proposed openstack/cinder: Port dothill to Python 3 https://review.openstack.org/197049 | 08:51 |
| haypo | smcginnis: hum, i had to rebase https://review.openstack.org/#/c/197049/ to fix a simple conflict in tox.ini. you may approve it again | 08:52 |
| *** liudong has joined #openstack-cinder | 09:08 | |
| *** liudong has left #openstack-cinder | 09:08 | |
| *** dims has joined #openstack-cinder | 09:08 | |
| *** dims has quit IRC | 09:14 | |
| davechen | winston-d: can we address this in a seperated patch? | 09:18 |
| davechen | winston-d: I am worried that testcase will bloat the patch again. | 09:19 |
| *** haomaiwa_ has quit IRC | 09:21 | |
| davechen | winston-d: seems there is no new method added but some refactoring instead. | 09:21 |
| *** afazekas has joined #openstack-cinder | 09:24 | |
| *** yrabl has quit IRC | 09:29 | |
| avishay__ | Anyone here familiar with the LVM driver? Specifically LIO vs TGT? | 09:29 |
| *** yrabl has joined #openstack-cinder | 09:29 | |
| *** avishay__ is now known as avishay | 09:29 | |
| openstackgerrit | wanghao proposed openstack/cinder: Query volume detail support volume_glance_metadata https://review.openstack.org/147738 | 09:29 |
| *** aix has quit IRC | 09:31 | |
| *** yamada-h has quit IRC | 09:31 | |
| *** e0ne has joined #openstack-cinder | 09:35 | |
| *** laughterwym has joined #openstack-cinder | 09:37 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder: Checking the existence of volume https://review.openstack.org/175809 | 09:38 |
| *** ociuhandu has quit IRC | 09:39 | |
| *** annashen has joined #openstack-cinder | 09:42 | |
| *** davechen has left #openstack-cinder | 09:42 | |
| *** annashen has quit IRC | 09:46 | |
| *** vincent_hou has joined #openstack-cinder | 09:47 | |
| vincent_hou | geguileo; Hey, dude. | 09:48 |
| geguileo | vincent_hou: Hi | 09:48 |
| vincent_hou | geguileo: https://review.openstack.org/#/c/180873/69/cinder/volume/drivers/storpool.py I add the explanation at the bottom. | 09:48 |
| geguileo | vincent_hou: Is it about my downvote for the StorPool? | 09:48 |
| vincent_hou | Yes. and my opinion about the rename. | 09:48 |
| geguileo | vincent_hou: So you say that even if that throws an exception if will be Ok? | 09:49 |
| vincent_hou | I think we do not have to throw an exception. | 09:49 |
| geguileo | vincent_hou: No, we don't have to | 09:49 |
| geguileo | vincent_hou: But if the rename raises the exception | 09:49 |
| vincent_hou | But need to handle _name_id well | 09:49 |
| avishay | geguileo: hey | 09:50 |
| geguileo | avishay: Hi | 09:50 |
| geguileo | avishay: Any progress with your tgt? | 09:50 |
| avishay | geguileo: Yes | 09:50 |
| geguileo | avishay: Awesome!! | 09:51 |
| geguileo | avishay: What was it? | 09:51 |
| avishay | geguileo: The first problem is that the centos package installs some default conf file that places iscsi_helper=lioadm in the [DEFAULT] section | 09:51 |
| geguileo | avishay: That's why I asked you to look at all configurations ;-) | 09:52 |
| vincent_hou | I will drop off now. Any concerns, just leave your comment there. | 09:52 |
| geguileo | avishay: in /usr/share | 09:52 |
| vincent_hou | See you, both. | 09:52 |
| geguileo | vincent_hou: My concern is what I said | 09:52 |
| geguileo | vincent_hou: If the rename raises an exception | 09:52 |
| geguileo | vincent_hou: We should catch it | 09:52 |
| avishay | geguileo: I added iscsi_helper = tgtadm to the backend section, and I think maybe sometimes it was using LIO and sometimes TGT | 09:52 |
| geguileo | avishay: You need to have DEFAULT set to tgtadm and backend set to tgtadm as well | 09:53 |
| avishay | geguileo: Further, I see that scsi-target-utils is no longer included in RH7 | 09:53 |
| geguileo | avishay: Or just remove it from both and it will use tgt by default | 09:53 |
| *** lixiaoy1 has quit IRC | 09:53 | |
| avishay | geguileo: Well that's broken - it's a driver setting, not a general setting | 09:53 |
| vincent_hou | geguileo: Ah, I see your point. | 09:54 |
| geguileo | avishay: That's how Openstack configuration works | 09:54 |
| geguileo | avishay: Defaults are not inherited :( | 09:54 |
| vincent_hou | Sorry, I missed a little. | 09:54 |
| geguileo | vincent_hou: Sometimes I don't explain myself very well, sorry XD | 09:54 |
| avishay | geguileo: So basically if I want something supported in RH I need to use the LIO driver, which is broken and has no CI | 09:54 |
| vincent_hou | geguileo: Yes, I am with you geguileo | 09:54 |
| geguileo | avishay: No, no, tgt should work as well | 09:55 |
| avishay | geguileo: tgt is not supported in RH7 | 09:55 |
| geguileo | avishay: There's already a patch upstream to fix broken LIO | 09:55 |
| avishay | geguileo: it was moved to EPEL | 09:55 |
| geguileo | avishay: And I'll start working on a CI pretty soon | 09:55 |
| vincent_hou | C U. | 09:55 |
| *** vincent_hou has quit IRC | 09:55 | |
| avishay | geguileo: i won't run anything without a CI - LIO should have been removed | 09:56 |
| geguileo | avishay: LIO is not a driver | 09:56 |
| avishay | geguileo: LVM+LIO is a driver | 09:56 |
| geguileo | avishay: The driver is iSCSI ;-) | 09:56 |
| avishay | geguileo: you know that's BS as well as I do | 09:57 |
| openstackgerrit | wanghao proposed openstack/cinder: Incremental backup improvements for L https://review.openstack.org/188328 | 09:57 |
| geguileo | avishay: It's not complete BS, but I see your point | 09:57 |
| geguileo | avishay: That's why I'll work on the CI :-D | 09:57 |
| avishay | geguileo: it's 100% BS. Just like HP has iSCSI and FC CIs, LVM needs CIs for all target types. | 09:58 |
| *** aix has joined #openstack-cinder | 09:58 | |
| geguileo | avishay: iSCSI and FC are protocols | 09:58 |
| *** dims has joined #openstack-cinder | 09:59 | |
| geguileo | Anyway, no reason to argue | 09:59 |
| geguileo | Since we both agree there should be a CI :) | 09:59 |
| geguileo | I'll keep you posted on the progress | 09:59 |
| avishay | geguileo: I will suggest in the Cinder meeting that all non-tgt backends should be removed until they have a CI | 09:59 |
| avishay | geguileo: Semantic loopholes are not interesting to me when I waste 2-3 days of work on a broken driver | 10:00 |
| avishay | geguileo: Sorry I'm a bit ticked off, but RH really dropped the ball on this one | 10:01 |
| *** BharatK has quit IRC | 10:01 | |
| avishay | geguileo: Nothing personal | 10:01 |
| geguileo | avishay: I know it's not personal, don't worry | 10:01 |
| geguileo | avishay: And the problem you had is not specific with LIO | 10:01 |
| e0ne | winston-d: are you around? | 10:01 |
| geguileo | avishay: The problem is with oslo configuration | 10:01 |
| *** yamada-h has joined #openstack-cinder | 10:01 | |
| geguileo | avishay: Since backends don't inherit from default value the iscsi_helper | 10:02 |
| avishay | geguileo: that's the latest problem when i tried moving away from LIO | 10:02 |
| geguileo | avishay: Oh, ok | 10:02 |
| avishay | geguileo: LIO is better than TGT in my opinion, and i would have been happy to use it if the driver worked and was tested | 10:02 |
| *** wanghao has left #openstack-cinder | 10:03 | |
| *** mutoulbj_ has quit IRC | 10:03 | |
| geguileo | avishay: Well, now you are making me feel bad, because I'm the one who broke it in Juno :( | 10:03 |
| sgotliv | avishay, so I guess we have to work to make it working | 10:03 |
| avishay | geguileo: You don't have to feel bad, it happens, and that's what CI is for | 10:03 |
| *** wanghao has joined #openstack-cinder | 10:03 | |
| *** wanghao has left #openstack-cinder | 10:03 | |
| *** wanghao has joined #openstack-cinder | 10:04 | |
| avishay | geguileo: I think jgriffith had some issues with CI, where LIO behaved differently on RH and Ubuntu, but I don't know the details | 10:04 |
| geguileo | avishay: I guess I'll find out once I start working on it :) | 10:05 |
| avishay | geguileo: anyway, i hope you can get it running soon | 10:05 |
| avishay | geguileo: thank you | 10:05 |
| geguileo | avishay: He mentioned that the main problem is that Ubuntu does not support LIO, you need some kernel changes and whatnot | 10:05 |
| avishay | geguileo: could be, like i said, i don't know the details | 10:06 |
| geguileo | avishay: I'll figure it out, no problem | 10:06 |
| *** yamada-h has quit IRC | 10:06 | |
| avishay | geguileo: you have a fix for LIO now? tested somehow? | 10:07 |
| geguileo | avishay: The patch is waiting review: https://review.openstack.org/#/c/197149/ | 10:07 |
| geguileo | avishay: But it's not passing the gates because of an issue with the gate | 10:07 |
| geguileo | avishay: So I had to write a new elastic-recheck fingerprint | 10:08 |
| geguileo | avishay: https://review.openstack.org/#/c/197500/ | 10:08 |
| avishay | geguileo: OK, if I don't make progress with tgt soon I may go back to lio, and I hope to have a CI in the near future | 10:09 |
| e0ne | winston-d: i've updated review requests with new rally results | 10:09 |
| geguileo | avishay: Ok, if you move back to LIO you just need to remove a couple of lines for it to work ;-) | 10:09 |
| avishay | geguileo: thanks a lot | 10:09 |
| avishay | geguileo: i'm worried that it's broken in other places because it's not tested | 10:10 |
| geguileo | avishay: I can tell you that there are some potential issues in Juno that are fixed in Kilo | 10:12 |
| geguileo | avishay: But the most common got fixed: https://github.com/openstack/cinder/commit/d89f493c2dc535b9110d222f3806d1a7fa208618 | 10:12 |
| avishay | geguileo: yes applied that one | 10:12 |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Service object https://review.openstack.org/160417 | 10:18 |
| openstackgerrit | Michal Dulko proposed openstack/cinder: Fix saving tz aware datetimes in Versioned Objects https://review.openstack.org/196071 | 10:18 |
| *** lpetrut has quit IRC | 10:24 | |
| geguileo | dulek: Why did you update a patch to my Versioned object tz fix patch? | 10:29 |
| *** BharatK has joined #openstack-cinder | 10:32 | |
| *** ndipanov has joined #openstack-cinder | 10:36 | |
| *** e0ne is now known as e0ne_ | 10:39 | |
| *** e0ne_ is now known as e0ne | 10:39 | |
| *** DericHorn-HP has joined #openstack-cinder | 10:42 | |
| *** annashen has joined #openstack-cinder | 10:43 | |
| *** annashen has quit IRC | 10:48 | |
| *** sgotliv has quit IRC | 10:52 | |
| *** sgotliv has joined #openstack-cinder | 10:53 | |
| *** mutoulbj has joined #openstack-cinder | 10:53 | |
| *** zhenguo has quit IRC | 10:54 | |
| *** BharatK has quit IRC | 10:57 | |
| *** heyun has quit IRC | 11:02 | |
| *** ociuhandu has joined #openstack-cinder | 11:03 | |
| dulek | geguileo: I needed to rebase it to make my Service object patch dependent on it. | 11:03 |
| dulek | geguileo: Don't worry, votes are still there, because I haven't changed anything. :) | 11:04 |
| dulek | geguileo: Authorship info also seems to be intact. | 11:05 |
| geguileo | dulek: Yes, only LP bug has changed. | 11:06 |
| geguileo | dulek: Not cool updating my patch. ;-) (kidding) | 11:07 |
| dulek | geguileo: Hm? Oh, I see, patch topic. You can update it from Gerrit UI. Basically it doesn't mean anything, LP tracks patches by Closes-Bug flags in commit messages. | 11:08 |
| dulek | geguileo: I've could prevented that, sorry. | 11:08 |
| geguileo | dulek: And now you are the assignee for this bug in LP | 11:09 |
| *** IanGovett has joined #openstack-cinder | 11:09 | |
| *** avishay has quit IRC | 11:10 | |
| *** BharatK has joined #openstack-cinder | 11:10 | |
| dulek | geguileo: Fixed. | 11:11 |
| geguileo | dulek: XD, there was no need to change it | 11:12 |
| dulek | geguileo: Well, statistics count. ;) | 11:12 |
| geguileo | XD | 11:12 |
| *** ankit_ag has joined #openstack-cinder | 11:14 | |
| *** Vikash_cz has joined #openstack-cinder | 11:16 | |
| *** anshul has quit IRC | 11:16 | |
| *** anshul has joined #openstack-cinder | 11:17 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder: Filter cgsnapshots data on the DB side https://review.openstack.org/197163 | 11:18 |
| *** avishay has joined #openstack-cinder | 11:20 | |
| *** yuriy_n17 has joined #openstack-cinder | 11:21 | |
| *** e0ne is now known as e0ne_ | 11:24 | |
| *** e0ne_ is now known as e0ne | 11:24 | |
| *** DericHorn-HP has quit IRC | 11:25 | |
| *** gaurangt has left #openstack-cinder | 11:28 | |
| *** yrabl has quit IRC | 11:28 | |
| *** yrabl has joined #openstack-cinder | 11:29 | |
| *** jasondotstar has quit IRC | 11:35 | |
| *** jasondotstar has joined #openstack-cinder | 11:36 | |
| *** jasondotstar has quit IRC | 11:41 | |
| *** annashen has joined #openstack-cinder | 11:44 | |
| *** tobe has quit IRC | 11:46 | |
| *** avishay has quit IRC | 11:46 | |
| *** annashen has quit IRC | 11:48 | |
| geguileo | dulek: ping - HA | 11:56 |
| *** Longgeek has quit IRC | 11:59 | |
| *** lpetrut has joined #openstack-cinder | 12:05 | |
| *** abhishekk has joined #openstack-cinder | 12:08 | |
| *** ociuhandu has quit IRC | 12:09 | |
| *** markvoelker has quit IRC | 12:14 | |
| *** markvoelker has joined #openstack-cinder | 12:14 | |
| *** eharney has joined #openstack-cinder | 12:14 | |
| *** ociuhandu has joined #openstack-cinder | 12:21 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements https://review.openstack.org/197545 | 12:22 |
| openstackgerrit | Erickson Filipe Guedes dos Santos proposed openstack/cinder: Nested Quota: Set default values to subproject https://review.openstack.org/194406 | 12:25 |
| *** Longgeek has joined #openstack-cinder | 12:25 | |
| *** deepakcs has quit IRC | 12:26 | |
| *** juzuluag has joined #openstack-cinder | 12:26 | |
| *** zigo has quit IRC | 12:28 | |
| *** mutoulbj has quit IRC | 12:28 | |
| *** zigo has joined #openstack-cinder | 12:29 | |
| *** mutoulbj has joined #openstack-cinder | 12:29 | |
| *** BharatK has quit IRC | 12:29 | |
| *** mutoulbj has quit IRC | 12:30 | |
| *** mutoulbj has joined #openstack-cinder | 12:30 | |
| openstackgerrit | Vincent Hou proposed openstack/cinder: Implement the update_migrated_volume for the drivers https://review.openstack.org/180873 | 12:31 |
| *** Longgeek has quit IRC | 12:31 | |
| *** nihilifer has quit IRC | 12:32 | |
| *** edmondsw has joined #openstack-cinder | 12:32 | |
| *** kjnelson has joined #openstack-cinder | 12:34 | |
| *** BharatK has joined #openstack-cinder | 12:35 | |
| *** annegentle has joined #openstack-cinder | 12:37 | |
| *** ankit_ag has quit IRC | 12:38 | |
| *** tellesnobrega_ has joined #openstack-cinder | 12:40 | |
| *** e0ne is now known as e0ne_ | 12:41 | |
| *** e0ne_ is now known as e0ne | 12:42 | |
| *** thangp has joined #openstack-cinder | 12:42 | |
| *** annashen has joined #openstack-cinder | 12:45 | |
| *** tellesnobrega__ has joined #openstack-cinder | 12:47 | |
| *** annegentle has quit IRC | 12:48 | |
| *** annegentle has joined #openstack-cinder | 12:49 | |
| *** tellesnobrega_ has quit IRC | 12:49 | |
| *** annashen has quit IRC | 12:49 | |
| *** Longgeek has joined #openstack-cinder | 12:49 | |
| *** amoturi has joined #openstack-cinder | 12:53 | |
| *** e0ne is now known as e0ne_ | 12:53 | |
| *** Lee1092 has quit IRC | 12:55 | |
| *** tellesnobrega__ has quit IRC | 12:55 | |
| *** Ctina__ has joined #openstack-cinder | 12:56 | |
| *** e0ne_ is now known as e0ne | 12:57 | |
| *** annegentle has quit IRC | 12:57 | |
| *** annegentle has joined #openstack-cinder | 12:58 | |
| *** flip214 has quit IRC | 13:00 | |
| *** kbyrne has quit IRC | 13:03 | |
| *** yamada-h has joined #openstack-cinder | 13:04 | |
| *** yamada-h has quit IRC | 13:08 | |
| winston-d | e0ne: around? | 13:10 |
| winston-d | e0ne: saw the new result. | 13:10 |
| e0ne | winston-d: yes | 13:10 |
| winston-d | e0ne: thx. i remember you said the VM has only one vCPU. | 13:11 |
| e0ne | yes | 13:11 |
| e0ne | kvm instance with 1 cpu | 13:11 |
| winston-d | e0ne: is it possible to do the test on a multi-core VM? 2 is fine, 4 would be ideal | 13:11 |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: RemoteFS: Fix the offline snapshot delete operation https://review.openstack.org/195196 | 13:12 |
| e0ne | winston-d: i'll test with 2 cpu | 13:12 |
| e0ne | winston-d: need to find env to setup 4 cpu per instance | 13:12 |
| *** bluex-pl has quit IRC | 13:13 | |
| winston-d | if we can have 4vCPU VM, we can collect 3 data point for both test cases: 1 worker/1 process; 2 workers/2 processes; 4 workers/4 processes. then we can draw a line for each to see how they scale | 13:14 |
| *** Longgeek has quit IRC | 13:14 | |
| e0ne | winston-d: how many workers should i setup for 2 vCPU? | 13:14 |
| *** Longgeek has joined #openstack-cinder | 13:14 | |
| *** subscope has joined #openstack-cinder | 13:15 | |
| *** bill_az has joined #openstack-cinder | 13:15 | |
| winston-d | if you can find 2vCPU VM, then we can collect 2 data points for each setup: 1worker/1process and 2 workers/2 processes. | 13:15 |
| winston-d | e0ne: what's the difference between 'load duration' and 'full duration'? | 13:17 |
| *** avishay has joined #openstack-cinder | 13:19 | |
| *** ociuhandu has quit IRC | 13:19 | |
| winston-d | e0ne: does keystone have osprofiler support? | 13:19 |
| *** ociuhandu has joined #openstack-cinder | 13:20 | |
| winston-d | e0ne: it'd be interesting to see the latency breakdown | 13:20 |
| e0ne | winston-d: afaik, it doesn't support osprofiler | 13:21 |
| winston-d | e0ne: it's fine. | 13:21 |
| *** dtynan1 has joined #openstack-cinder | 13:22 | |
| *** kbyrne has joined #openstack-cinder | 13:22 | |
| openstackgerrit | Vincent Hou proposed openstack/cinder-specs: Volume migration improvement for Liberty version https://review.openstack.org/186327 | 13:22 |
| winston-d | e0ne: so you new/old data showed that changing # of workers/processes in a single CPU environment doesn't affect keystone peformance, which is expected. | 13:22 |
| *** dtynan has quit IRC | 13:22 | |
| e0ne | winston-d: i think, load duration is duration w/o rally invokes. trying to veryfy it from rally team | 13:22 |
| winston-d | e0ne: but apache sems to have smaller overhead than eventlet, so the data is still valuable. | 13:23 |
| openstackgerrit | Vincent Hou proposed openstack/cinder-specs: Volume migration improvement for Liberty version https://review.openstack.org/186327 | 13:24 |
| winston-d | 2vCPU VM data will be more interesting, it should tell us more. | 13:24 |
| dulek | geguileo: pong, what's up? | 13:25 |
| dulek | geguileo: I need "I survived another meeting sticker". ;) | 13:25 |
| geguileo | dulek: XD | 13:25 |
| geguileo | dulek: That's a good idea, I would love one too ;-) | 13:26 |
| *** coolsvap is now known as coolsvap|away | 13:26 | |
| geguileo | dulek: I have added HA to the meeting's agenda | 13:26 |
| geguileo | dulek: And I didn't know if you wanted to lead as well | 13:26 |
| geguileo | s/as well/too | 13:27 |
| *** ociuhandu has quit IRC | 13:27 | |
| *** mriedem has joined #openstack-cinder | 13:27 | |
| dulek | geguileo: I'll participate today, if you need backup - I can cover. :) | 13:27 |
| geguileo | dulek: Ok | 13:28 |
| geguileo | dulek: The main idea is to get a list of Cinder-Nova issues stopping HA progress | 13:28 |
| dulek | geguileo: To be honest - I'm a little confused on that now. | 13:29 |
| geguileo | dulek: Someone must know about that one, right? | 13:29 |
| e0ne | winston-d: http://e0ne.github.io/keystone/keystone-apache-2cpu.html and http://e0ne.github.io/keystone/keystone-eventlet-2cpu.html | 13:29 |
| e0ne | 2 workers per cpu | 13:29 |
| geguileo | dulek: The other one is locking deletion from resources used for reading (allowing multiple readers) | 13:30 |
| geguileo | dulek: And finally get a fine grained list of jobs and who's going to be doing them | 13:30 |
| dulek | geguileo: Okay, I'm familiar with that one. | 13:30 |
| geguileo | dulek: Because I think we are not making much progress | 13:30 |
| e0ne | winston-d: meeting:(. i'll test with 2 cpu and 1 worker per cpu a bit later | 13:30 |
| dulek | geguileo: And that's needed also. :) | 13:30 |
| *** ociuhandu has joined #openstack-cinder | 13:30 | |
| *** dtynan1 has quit IRC | 13:31 | |
| dulek | geguileo: My perspective on Nova->Cinder is that attach has reserve and detach has begin_detaching. | 13:31 |
| geguileo | dulek: Do you know what point 3 in action items is referring to? https://etherpad.openstack.org/p/cinder-active-active-vol-service-issues | 13:31 |
| dulek | geguileo: Yeah, it's about locking deletion from resources used for reading. | 13:32 |
| dulek | geguileo: We somehow figured it out. ;) | 13:32 |
| geguileo | dulek: Oh, that's "cinder.services" changes? | 13:32 |
| *** avishay has quit IRC | 13:32 | |
| dulek | geguileo: Mhm. | 13:32 |
| dulek | geguileo: The way it's written is hard to understand, that's why this item is there. | 13:33 |
| geguileo | dulek: I thought it was referring to atomic state changes XD | 13:33 |
| *** dtynan has joined #openstack-cinder | 13:33 | |
| geguileo | dulek: We definitely need better naming in action items XD | 13:34 |
| *** avishay has joined #openstack-cinder | 13:34 | |
| dulek | geguileo: It is related. Once atomic state changes are done we won't need these locks (in most cases I think). | 13:34 |
| e0ne | winston-d: also, i could test with concurrency >10. it could be interesting | 13:34 |
| geguileo | dulek: No we'll need some of them, the ones locking resources that are used only for reading | 13:35 |
| avishay | geguileo: I've decided to stick with LIO since I'd rather have a less stable driver than deprecated target, hope to see a CI soon :) | 13:35 |
| geguileo | avishay: Cool | 13:35 |
| dulek | geguileo: Okay, so an example would be creating volume from snapshot - we need to lock on snapshot for reading. | 13:36 |
| geguileo | avishay: We'll discuss it in today's meeting :) | 13:36 |
| avishay | geguileo: Have you seen this error? rtslib.utils.RTSLibError: This _Backstore already exists in configFS | 13:36 |
| geguileo | dulek: Yes, and there could be multiple volumes creating from that one | 13:36 |
| geguileo | avishay: Yes | 13:36 |
| dulek | geguileo: We can probably change state of the snapshot in the db to used_for_volume_creation and allow multiple creations but no deleting | 13:36 |
| geguileo | avishay: I fixed that one in L and backported it to K | 13:36 |
| geguileo | dulek: Then you can only have 1 reader | 13:37 |
| avishay | geguileo: link to kilo backport please? i'd like to apply it to our juno | 13:37 |
| geguileo | dulek: I sent my proposal to the mail list and I refer to it in the agenda | 13:37 |
| geguileo | avishay: 1 sec | 13:37 |
| dulek | geguileo: I think depending on API call you can allow multiple operations or not. But you would need to maintain a list of operations already pending... Okay, this is getting messy. | 13:38 |
| *** abhishekk has quit IRC | 13:38 | |
| winston-d | e0ne: and maybe we can increate the rally iterations to 500 or even 1,000 | 13:38 |
| geguileo | avishay: https://review.openstack.org/#/c/187667/ | 13:38 |
| avishay | geguileo: thanks a lot! | 13:38 |
| geguileo | avishay: You may be also interested on the persistence one | 13:39 |
| avishay | geguileo: sure | 13:39 |
| geguileo | dulek: That's why I added it to today's meeting, because we have to decide how to do it XD | 13:39 |
| dulek | geguileo: Okay, fine for me. | 13:40 |
| *** julim has joined #openstack-cinder | 13:42 | |
| geguileo | avishay: I don't know if the persistence one is in Kilo: https://review.openstack.org/#/c/160428/ | 13:42 |
| geguileo | avishay: And that introduces a bug if you don't have targetcli installed, so there was this other patch later: https://review.openstack.org/#/c/181739/ | 13:43 |
| *** Ctina__ is now known as ctina | 13:43 | |
| geguileo | avishay: Hope it all goes well now :-) | 13:44 |
| avishay | geguileo: awesome thank you! | 13:45 |
| geguileo | avishay: No problem | 13:45 |
| *** annashen has joined #openstack-cinder | 13:45 | |
| *** dtynan has quit IRC | 13:46 | |
| *** dtynan has joined #openstack-cinder | 13:47 | |
| *** jungleboyj has quit IRC | 13:49 | |
| *** annashen has quit IRC | 13:50 | |
| *** ociuhandu has quit IRC | 13:51 | |
| e0ne | winston-d: times 300, concurrency 50 - http://e0ne.github.io/keystone/ | 13:52 |
| *** Vikash_cz has quit IRC | 13:53 | |
| *** ociuhandu has joined #openstack-cinder | 13:53 | |
| winston-d | e0ne: cool. the link text should change, they are all eventlet | 13:56 |
| e0ne | yep. fixed | 13:56 |
| e0ne | times 1000, concurrency 50 - in process | 13:57 |
| *** dims has quit IRC | 13:59 | |
| *** dims has joined #openstack-cinder | 14:00 | |
| *** andrey-mp has joined #openstack-cinder | 14:01 | |
| *** mtanino has joined #openstack-cinder | 14:02 | |
| *** subscope has quit IRC | 14:03 | |
| winston-d | e0ne: thx. the data is already very promising. | 14:03 |
| *** jungleboyj has joined #openstack-cinder | 14:03 | |
| *** avishay has quit IRC | 14:04 | |
| *** avishay has joined #openstack-cinder | 14:05 | |
| *** dalgaaf has joined #openstack-cinder | 14:06 | |
| openstackgerrit | xhzhf proposed openstack/cinder: VMware driver: compatible to vc 6.0 https://review.openstack.org/197599 | 14:07 |
| *** lpetrut has quit IRC | 14:07 | |
| *** e0ne is now known as e0ne_ | 14:08 | |
| *** lcurtis has joined #openstack-cinder | 14:08 | |
| *** lpetrut has joined #openstack-cinder | 14:09 | |
| *** avishay has quit IRC | 14:10 | |
| andrey-mp | DuncanT, hemnafk, please see my answers in review https://review.openstack.org/#/c/193596/ (Avoid race condition at snapshot deletion stage). We were stuck because of this bug. Gating of EC2 project is failed very often... | 14:11 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements https://review.openstack.org/197545 | 14:11 |
| DuncanT | @andrey-mp I'll take a look now | 14:11 |
| DuncanT | @andrey-mp So I still don't see why we need to call snapshot_metadata_get for each snap in the list - that is just broken behaviour | 14:14 |
| *** zhenguo has joined #openstack-cinder | 14:14 | |
| lpetrut | Hi guys. I have a few questions: can we still send blueprints targeting Liberty? | 14:15 |
| DuncanT | @andrey-mp If there's a list codepath that hits this code, that codepath is totally broken and should be fixed | 14:15 |
| andrey-mp | DuncanT, I think that code doesn't know about join in DB layer - so I left snapshot_metadata_get for such cases. But in this case bug can come back... | 14:15 |
| lpetrut | We'd like to support volume backup/migration for the Windows iSCSI and SMB drivers. One step would be adding Windows support in os-brick | 14:16 |
| andrey-mp | So you mean that I should remove snapshot_metadta_get at all and be sure that all queries have join clause? | 14:16 |
| DuncanT | @andrey-mp I'd rather this code was removed - it is dangerous - somebody with many snapshots doing a snapshot list can bring the db to a grinding halt | 14:17 |
| DuncanT | @andrey-mp Ideally you'd only do the join in codepaths that need it, but that is hard to get right, particularly now we've moved to objects, so I think always joining is better than this | 14:18 |
| andrey-mp | In this case I have to rework many unit tests that rely on this feature. is it ok? | 14:19 |
| DuncanT | To me, yes | 14:20 |
| DuncanT | I'm not sure I understand to comment "First change removes getting metadata from DB for each found snapshot. | 14:21 |
| DuncanT | Instead of this it copies metadata from query result. This query | 14:21 |
| DuncanT | is already joined with metadata table." | 14:21 |
| DuncanT | It looks like you're adding a db query, not removing one | 14:22 |
| DuncanT | I might be failing to understand, however | 14:22 |
| andrey-mp | ok, I will rewrite commit message ) It means that first change tries to copy metadata from db object (as in code) | 14:23 |
| DuncanT | Oh, I think I see | 14:24 |
| DuncanT | Ok, sorry, I think this change can probably go in as-is - we just need to do an audit to make sure all of the API list snapshot paths do a joined load | 14:25 |
| *** jungleboyjiPad has joined #openstack-cinder | 14:26 | |
| andrey-mp | Ouch ) Now i'm sure that all snapshot queries have join clause. But snapshot code calls snapshot_metadata_get if metadata is absent in db object. | 14:27 |
| DuncanT | Changed my vote to 0 for now, I'll go read Walt's comments and maybe I'll change it to +2 - certainly nobody should feel like I strongly object anymore, I'd like to see those unit tests cleaned up, but that is future work | 14:28 |
| DuncanT | Most of our unit tests, if written correctly, shouldn't care IMO | 14:29 |
| andrey-mp | most of unit tests now mock snapshot_metadata_get and use fake snapshot db object without metadata. But live cloud uses snapshot db object with metadata. It is confusing... | 14:31 |
| DuncanT | andrey-mp: Could we keep track of whether we did a joined load or not and make it an exception to have metadata in the required fields if you didn't do a joined load? | 14:32 |
| DuncanT | expected_attrs, not required fields | 14:33 |
| DuncanT | (I'm not saying instead of your patch, I'm asking for future work) | 14:34 |
| andrey-mp | hm, it make sense. I will do this. (And rework most unit tests also) | 14:35 |
| andrey-mp | I think that I can do this in this patch. It will be more correct. | 14:35 |
| *** ociuhandu_ has joined #openstack-cinder | 14:37 | |
| *** jwang__ has joined #openstack-cinder | 14:37 | |
| *** serverascode_ has joined #openstack-cinder | 14:39 | |
| *** Longgeek has quit IRC | 14:40 | |
| *** cppforlife__ has joined #openstack-cinder | 14:40 | |
| *** guitarza1 has joined #openstack-cinder | 14:42 | |
| *** thingee_ has joined #openstack-cinder | 14:43 | |
| openstackgerrit | Kendall Nelson proposed openstack/cinder: Enable cinder-manage to delete old services from the database https://review.openstack.org/197613 | 14:44 |
| *** EmilienM_ has joined #openstack-cinder | 14:45 | |
| *** mtreinish_ has joined #openstack-cinder | 14:45 | |
| *** rwall_ has joined #openstack-cinder | 14:45 | |
| openstackgerrit | Victor Stinner proposed openstack/cinder: Fix Python 3 issues in Hitachi HNAS tests https://review.openstack.org/197127 | 14:46 |
| openstackgerrit | Victor Stinner proposed openstack/cinder: Port remotefs driver to Python 3 https://review.openstack.org/197075 | 14:46 |
| *** ociuhandu has quit IRC | 14:46 | |
| *** juzuluag has quit IRC | 14:46 | |
| *** marcusvrn has quit IRC | 14:46 | |
| *** mtreinish has quit IRC | 14:46 | |
| *** Swanson has quit IRC | 14:46 | |
| *** rwall has quit IRC | 14:46 | |
| *** rushiagr_away has quit IRC | 14:46 | |
| *** andymccr has quit IRC | 14:46 | |
| *** git-harry has quit IRC | 14:46 | |
| *** rhe00_ has quit IRC | 14:46 | |
| *** jwang_ has quit IRC | 14:46 | |
| *** EmilienM has quit IRC | 14:46 | |
| *** cppforlife_ has quit IRC | 14:46 | |
| *** rex_lee has quit IRC | 14:46 | |
| *** thingee has quit IRC | 14:46 | |
| *** serverascode has quit IRC | 14:46 | |
| *** guitarzan has quit IRC | 14:46 | |
| *** EmilienM_ is now known as EmilienM | 14:46 | |
| *** juzuluag_ has joined #openstack-cinder | 14:46 | |
| *** strictlyb is now known as sb | 14:46 | |
| *** ociuhandu_ is now known as ociuhandu | 14:46 | |
| *** mtreinish_ is now known as mtreinish | 14:46 | |
| *** rwall_ is now known as rwall | 14:46 | |
| *** annashen has joined #openstack-cinder | 14:46 | |
| *** annegentle has quit IRC | 14:47 | |
| *** nkrinner has quit IRC | 14:47 | |
| *** MentalRay has joined #openstack-cinder | 14:47 | |
| *** rhe00_ has joined #openstack-cinder | 14:48 | |
| *** thingee_ is now known as thingee | 14:48 | |
| *** DericHorn-HP has joined #openstack-cinder | 14:48 | |
| *** sb has quit IRC | 14:51 | |
| *** annashen has quit IRC | 14:51 | |
| *** git-harry has joined #openstack-cinder | 14:51 | |
| *** strictlyb has joined #openstack-cinder | 14:51 | |
| *** cppforlife__ is now known as cppforlife_ | 14:53 | |
| *** marcusvrn has joined #openstack-cinder | 14:54 | |
| *** Swanson has joined #openstack-cinder | 14:55 | |
| *** serverascode_ is now known as serverascode | 14:56 | |
| DuncanT | andrey-mp: That's great, thanks! Sorry to have to cause you more work, but I think it will make things better in the end | 14:56 |
| andrey-mp | sure :) | 14:56 |
| *** subscope has joined #openstack-cinder | 14:57 | |
| *** rmesta has joined #openstack-cinder | 14:58 | |
| jordanP | with the LVM driver, I can't extend a volume that has snapshots. It fails with "Snapshot origin volumes can be resized only while inactive: try lvchange -an" | 15:00 |
| jordanP | is that expected ? | 15:00 |
| *** jordanP has quit IRC | 15:01 | |
| *** xyang1 has joined #openstack-cinder | 15:02 | |
| *** jordanP has joined #openstack-cinder | 15:02 | |
| *** andrey-mp has quit IRC | 15:02 | |
| eharney | jordanP: sounds like a bug | 15:04 |
| jordanP | eharney, will submit a bug report then | 15:04 |
| jordanP | *i will | 15:04 |
| *** rex_lee has joined #openstack-cinder | 15:05 | |
| *** guitarza1 is now known as guitarzan | 15:06 | |
| *** andymccr has joined #openstack-cinder | 15:06 | |
| *** Longgeek has joined #openstack-cinder | 15:07 | |
| *** e0ne_ is now known as e0ne | 15:07 | |
| *** Longgeek has quit IRC | 15:07 | |
| *** rmesta has left #openstack-cinder | 15:08 | |
| *** Longgeek has joined #openstack-cinder | 15:08 | |
| lpetrut | eharney: Hi. I was thinking about adding volume backup support for the RemoteFS based drivers. I think we could easily support backing up volumes having snapshots as well by mounting them, using qemu-nbd for example. | 15:09 |
| *** Apoorva has joined #openstack-cinder | 15:09 | |
| *** belmoreira has quit IRC | 15:10 | |
| *** Longgeek has quit IRC | 15:12 | |
| *** Longgeek has joined #openstack-cinder | 15:14 | |
| *** blmartin has joined #openstack-cinder | 15:15 | |
| jordanP | https://bugs.launchpad.net/cinder/+bug/1470558 | 15:15 |
| openstack | Launchpad bug 1470558 in Cinder "LVM driver: volume with snaps can't be extended" [Undecided,New] | 15:15 |
| DuncanT | lpetrut: Are there still security concerns around qemu-nbd on user-controlled volumes? They've certainly been raised before, though I don't know how substantial they are | 15:15 |
| jordanP | as an ops, I don't like qemu-nbd. I've seen weird stack traces in my kernel.log files when we used to no use libguestfs | 15:16 |
| jordanP | lpetrut, so if you want to go forward with qemu-nbd, I recommend to code a switch to be able to completely disable the call to qemu nbd | 15:17 |
| jordanP | (and loose some feature of course) | 15:17 |
| jordanP | *lose | 15:17 |
| *** liewegas has joined #openstack-cinder | 15:18 | |
| *** e0ne is now known as e0ne_ | 15:19 | |
| *** e0ne_ is now known as e0ne | 15:20 | |
| e0ne | winston-d: results for 1000 runs with concurrency 50 are ready: http://e0ne.github.io/keystone/ | 15:24 |
| *** aarefiev_home has joined #openstack-cinder | 15:24 | |
| *** afazekas has quit IRC | 15:26 | |
| *** IlyaG has quit IRC | 15:26 | |
| openstackgerrit | Victor Stinner proposed openstack/os-brick: Cleanup Python 3 changes https://review.openstack.org/197624 | 15:27 |
| *** CaerbannogRabbit is now known as morganfainberg | 15:30 | |
| haypo | smcginnis: https://review.openstack.org/197068 "Huawei CI shows a failure, but its logs are not accessible." hum, i just schedule a recheck | 15:32 |
| lpetrut | thanks for the heads up guys. DuncanT: you are reffering to the image format autodetection issue, right? Well, we can easily specify the expected format | 15:32 |
| haypo | smcginnis: i wrote patches per driver because of all these Cinder CIs :) | 15:32 |
| smcginnis | haypo: Yeah, don't worry about it. I'm pretty sure it's an issue on their end and not your patch. | 15:32 |
| smcginnis | haypo: I appreciate that. Good to see the individual confirmations from the CIs. When they actually work. :) | 15:33 |
| *** coolsvap|away is now known as coolsvap | 15:33 | |
| haypo | smcginnis: only Jenkins is allowed to vote -1, or some vendor CIs are also allowed to vote -1? | 15:33 |
| smcginnis | haypo: I think one of the VMware CIs has voting, but we've been holding of for stability before letting them all. | 15:33 |
| *** xyang has joined #openstack-cinder | 15:33 | |
| haypo | smcginnis: yeah, it's really hard to get a stable CI :-( i know that | 15:34 |
| smcginnis | haypo: Yeah, I'm in the same boat. Mostly stable, but still not as much as I would like it to be. | 15:34 |
| haypo | i put my py3 patches depending on os-brick into WIP. i'm now working with walter on https://review.openstack.org/#/c/196284/ - i hope that a new version of os-brick with these py3 fixes will be released quickly | 15:35 |
| eharney | lpetrut: there is a patch out now related to backup of qcow2: https://review.openstack.org/#/c/182276/ | 15:36 |
| *** tsekiyama has joined #openstack-cinder | 15:36 | |
| haypo | hemnafk: ping again for https://review.openstack.org/#/c/196284/ ;) | 15:36 |
| smcginnis | haypo: Awesome, was just looking at hemnafk's patch and read your comment. | 15:36 |
| smcginnis | haypo: Would be good to combine them IMO. | 15:36 |
| lpetrut | eharney: thanks for the link, checking it out | 15:37 |
| haypo | smcginnis: i don't like "hijacking" a patch. i hate when someone does that on my patches :) | 15:37 |
| eharney | lpetrut: it depends on some other work which hasn't landed yet so it's not quite ready | 15:37 |
| *** jaypipes has quit IRC | 15:37 | |
| eharney | lpetrut: there were also some concerns about the idea in general which i'm not sure have been sorted out adequately | 15:37 |
| *** Apoorva has quit IRC | 15:39 | |
| e0ne | eharney: just fyi. i'm testing this simple patch http://paste.openstack.org/show/332404/ | 15:40 |
| e0ne | eharney: from the first look - it works. i'm waiting results from the load tests | 15:41 |
| *** liuxg has joined #openstack-cinder | 15:42 | |
| *** dtynan has quit IRC | 15:43 | |
| openstackgerrit | Jon Bernard proposed openstack/cinder: Add support for generic volume migration https://review.openstack.org/187270 | 15:44 |
| *** coolsvap is now known as coolsvap|away | 15:44 | |
| *** dtynan has joined #openstack-cinder | 15:44 | |
| eharney | e0ne: isn't that what we had previously? | 15:47 |
| *** annashen has joined #openstack-cinder | 15:47 | |
| e0ne | eharney: no, i leave tpool to connect to RDB and removed in _connect_to_rados method | 15:48 |
| *** adurbin_ has joined #openstack-cinder | 15:48 | |
| *** laughterwym has quit IRC | 15:48 | |
| e0ne | eharney: to decrease cross-threading communications | 15:48 |
| *** annashen has quit IRC | 15:52 | |
| *** dannywilson has joined #openstack-cinder | 15:52 | |
| *** hemnafk is now known as hemna | 15:55 | |
| hemna | mornin | 15:55 |
| e0ne | hemna: evening:) | 15:55 |
| *** patrickeast has joined #openstack-cinder | 15:55 | |
| *** flip214 has joined #openstack-cinder | 15:57 | |
| *** erlon has joined #openstack-cinder | 15:57 | |
| *** deepakcs has joined #openstack-cinder | 15:58 | |
| hemna | so, I don't want to remove the testenv:py34 section in tox.ini | 15:59 |
| hemna | it causes runs of tox -epy34 to fail | 15:59 |
| *** zongliang has joined #openstack-cinder | 16:00 | |
| openstackgerrit | Alberto Murillo proposed openstack/cinder: Update expected error message from lvs https://review.openstack.org/197338 | 16:01 |
| *** alau2 has joined #openstack-cinder | 16:01 | |
| winston-d | meeting? | 16:02 |
| *** rmesta has joined #openstack-cinder | 16:04 | |
| *** aarefiev_home has quit IRC | 16:04 | |
| *** garthb has joined #openstack-cinder | 16:04 | |
| *** garthb_ has joined #openstack-cinder | 16:04 | |
| smcginnis | hemna: When I did a side-by-side I thought there were a few things haypo had in his patch that you didn't. Not related to tox.ini. | 16:04 |
| smcginnis | hemna: Did you take a look at that? | 16:04 |
| *** lpetrut has quit IRC | 16:04 | |
| *** aarefiev_home has joined #openstack-cinder | 16:05 | |
| *** Vikash_cz has joined #openstack-cinder | 16:05 | |
| *** lpetrut has joined #openstack-cinder | 16:05 | |
| *** lpetrut has quit IRC | 16:05 | |
| *** leeantho has joined #openstack-cinder | 16:06 | |
| hemna | his follow up/dep patch had tox.ini changes (removing the section) and removed the Exception.py changes | 16:06 |
| hemna | I can try removing the exception.py changes and see if the tests pass again | 16:06 |
| hemna | I just -1'd his follow up patch | 16:06 |
| hemna | removing that section breaks running tests | 16:06 |
| smcginnis | hemna: Oh, I didn't see a the follow up patch. | 16:06 |
| *** Longgeek has quit IRC | 16:06 | |
| smcginnis | So we've been whitelisting tests for now. Maybe not needed once it is all compatible? | 16:07 |
| hemna | I think there is some problem with testr | 16:07 |
| *** asselin_ has quit IRC | 16:07 | |
| hemna | it pukes running tox -epy34 standalone w/o that section | 16:07 |
| hemna | nova does it this way, as does cinder | 16:07 |
| hemna | so... | 16:07 |
| *** Longgeek has joined #openstack-cinder | 16:08 | |
| *** jistr has quit IRC | 16:08 | |
| hemna | http://paste.openstack.org/show/332500/ | 16:08 |
| hemna | that's what you get when you remove that section and run the tests | 16:08 |
| hemna | just vomits every time. | 16:08 |
| smcginnis | hemna: Not good. | 16:09 |
| hemna | yah | 16:10 |
| *** Vikash_cz has quit IRC | 16:10 | |
| hemna | and forcing folks to do rm -rf .testrepository && tox -epy34 is not acceptable to me | 16:10 |
| smcginnis | hemna: Definitely. That would be very confusing. | 16:10 |
| *** anshul has quit IRC | 16:11 | |
| *** Longgeek has quit IRC | 16:12 | |
| haypo | hemna: hey. "removing that section breaks running tests" that's strange. i'm unable to reproduce your issue | 16:13 |
| haypo | hemna: "db type could not be determined" ah you missed my comment :) you must run py34 *before* running py27 | 16:14 |
| *** Apoorva has joined #openstack-cinder | 16:14 | |
| haypo | hemna: otherwise, .testrepository is incompatible with python 3. stupid testr bug | 16:14 |
| hemna | I'm running tox -epy34 | 16:14 |
| *** avishay has joined #openstack-cinder | 16:14 | |
| hemna | standalone | 16:14 |
| haypo | hemna: this issue will not affect the check jobs which always start from a fresh checkout | 16:15 |
| hemna | fails every time | 16:15 |
| haypo | hemna: try: rm -rf .testrepository | 16:15 |
| hemna | no | 16:15 |
| hemna | I don't want people to have to do that | 16:15 |
| haypo | hemna: you should write py34,py27 in tox.ini | 16:15 |
| *** DericHorn-HP has quit IRC | 16:15 | |
| hemna | that's not right | 16:15 |
| openstackgerrit | Kyrylo Romanenko proposed openstack/python-cinderclient: Add tests for python-cinderclient https://review.openstack.org/197643 | 16:15 |
| haypo | hemna: people will have not to do that :) | 16:15 |
| smcginnis | Doh! Meeting. | 16:15 |
| hemna | I changed it to py34,py27... | 16:15 |
| haypo | hemna: this issue is a bug in testr, it's unrelated to cinder or os-brick. all projects are affected | 16:15 |
| hemna | same problem | 16:15 |
| hemna | same failure | 16:15 |
| haypo | hemna: first, remove .testrepositry | 16:16 |
| *** DericHorn-HP has joined #openstack-cinder | 16:17 | |
| openstackgerrit | Bharat Kumar Kobagana (BharatK) proposed openstack/cinder: GlusterFS: support extending a volume that has snapshots https://review.openstack.org/197645 | 16:17 |
| hemna | haypo, ok py27 fails now for the same reason | 16:18 |
| *** garthb_ has quit IRC | 16:18 | |
| *** garthb has quit IRC | 16:18 | |
| hemna | lol | 16:18 |
| hemna | ok yah, this doesn't work | 16:18 |
| hemna | http://paste.openstack.org/show/332512/ | 16:18 |
| smcginnis | haypo: Even if it's in the right order, wouldn't existing ones fail? | 16:19 |
| smcginnis | haypo: Everyone would have to know to delete that after pulling down the latest. | 16:19 |
| *** harlowja_at_home has joined #openstack-cinder | 16:20 | |
| haypo | hemna: "ok py27 fails now for the same reason" ah? "No module named gdbm" is a different error | 16:20 |
| haypo | smcginnis: it doesn't break py27, it's just an annoying issue when running tox -e py34 | 16:21 |
| haypo | i'm hitting this issuse since more than one year... | 16:21 |
| haypo | hemna: your python 2.7 doesn't have the gdbm module? | 16:21 |
| haypo | on Fedora 22, "tox -e pep8,py34,py27" works well, and python 2.7 has the gdbm module | 16:22 |
| smcginnis | haypo: But that's my point. If someone already is working on things and has run py27 before, when they pull down the latest code and see python3 support is now available, it won't be obvious to them they need to delete anything before they are able to run it. | 16:22 |
| haypo | gdbm is part of the Python stdlib... | 16:23 |
| haypo | smcginnis: i don't know how to advertise on this bug | 16:23 |
| mriedem | e0ne: http://graphite.openstack.org/render/?width=1430&height=808&_salt=1435767786.683&target=stats.zuul.pipeline.check.job.gate-rally-dsvm-cinder.FAILURE&target=stats.zuul.pipeline.check.job.check-tempest-dsvm-full.FAILURE&from=00%3A00_20150101&until=23%3A59_20150701 | 16:23 |
| haypo | smcginnis: i documented it at https://haypo-notes.readthedocs.org/openstack.html#tox-testr-db-type-could-not-be-determined-error :) | 16:23 |
| smcginnis | haypo: Well, it sounds like the way hemna has it you won't need to. | 16:23 |
| mriedem | that compares the rally failures to the check-tempest-dsvm-full job failures over the last 6 months | 16:23 |
| haypo | smcginnis: again, it's unrelated to os-brick. and it would be a shame to not use testr for this reason :-/ | 16:23 |
| smcginnis | haypo: So if we don't remove that section it can just work. | 16:24 |
| e0ne | mriedem: thanks | 16:24 |
| e0ne | mriedem: now, we need to find why they failed | 16:24 |
| haypo | smcginnis: yes | 16:24 |
| smcginnis | haypo: Can't we still use testr though? | 16:24 |
| mriedem | e0ne: from graphite.openstack.org you can drill into jobs and add stats to graphs via stats - zuul - pipeline - check - job | 16:24 |
| mriedem | then pick jobs | 16:24 |
| mriedem | i used check queue since that's what the gate-rally-dsvm-cinder job is running in right now | 16:24 |
| mriedem | DuncanT: ^ | 16:24 |
| haypo | smcginnis: testr relies on testrepository. testrepository has an annoying bug if you run py27 first | 16:25 |
| haypo | i ported or helped to port +20 projects, all of them are now using testr on python 3 | 16:25 |
| haypo | i'm surprised that it doesn't work on hemna host :-/ | 16:25 |
| e0ne | mriedem: we can't compare fails count w/o analyzing why it fails | 16:26 |
| hemna | I'm trying again | 16:26 |
| *** rushiagr_away has joined #openstack-cinder | 16:26 | |
| DuncanT | mriedem: expanding the jobs list took our my browser :-) I'll try later when there's no meeting. Thanks for the pointer | 16:26 |
| hemna | haypo, rm -rf .tox && rm -rf .testrepository && tox | 16:26 |
| hemna | we'll see if it works | 16:26 |
| haypo | hemna: i guess that you will hit the same error on gdbm | 16:26 |
| haypo | hemna: is it a regular Ubuntu? | 16:26 |
| hemna | 14.04 | 16:26 |
| haypo | ah, it'a a package: http://packages.ubuntu.com/trusty/python/python-gdbm | 16:27 |
| smcginnis | My py3 on 14.04 does not have gdbm. | 16:27 |
| hemna | gah | 16:27 |
| smcginnis | py2 does. | 16:27 |
| hemna | ok py27 puked again | 16:27 |
| hemna | :( | 16:27 |
| *** Vikash_cz has joined #openstack-cinder | 16:27 | |
| haypo | hemna: try: sudo apt-get install -y python-gdbm | 16:28 |
| mriedem | e0ne: well, if the rally job trends about the same with the jenkins job, then you could consider it fairly stable | 16:28 |
| mriedem | e0ne: if there were big mismatches, then it would be unstable | 16:28 |
| mriedem | since jenkins is the baseline | 16:28 |
| mriedem | well, check-tempest-dsvm-full is the baseline | 16:28 |
| e0ne | mriedem: fair enugh | 16:28 |
| hemna | poop: http://paste.openstack.org/show/332537/ | 16:28 |
| mriedem | it looks fairly consistent over 6 months | 16:28 |
| e0ne | s/enuph/enouph | 16:28 |
| haypo | hemna: ah, it's explicitly installed in DevStack for testr: files/apts/general:20:python-gdbm # needed for testr | 16:29 |
| hemna | hrmm | 16:30 |
| hemna | why has it never failed before then | 16:30 |
| * hemna is confused | 16:30 | |
| haypo | smcginnis: gdbm was renamed to dbm.gnu in python 3 ;) | 16:30 |
| hemna | haypo, so I just did another test | 16:30 |
| hemna | I removed py34 from tox.ini | 16:31 |
| hemna | then | 16:31 |
| hemna | rm -rf .testrepository && rm -rf .tox && tox | 16:31 |
| hemna | py27 worked | 16:31 |
| smcginnis | There it is! | 16:31 |
| *** Vikash_cz has quit IRC | 16:31 | |
| haypo | hemna: testr is fun :) | 16:31 |
| hemna | POS | 16:31 |
| haypo | hemna: it looks like depending on the python version, it uses a different dbm format | 16:31 |
| *** tsbot_ has joined #openstack-cinder | 16:31 | |
| haypo | hemna: gdbm is you first run tox -e py34 | 16:31 |
| hemna | brb | 16:32 |
| haypo | ndbm is you first run tox -e py27 | 16:32 |
| haypo | you can check using: file .testrepository/times.dbm | 16:32 |
| haypo | i'm checking again. tox -e py27 first: "Berkeley DB (Hash, version 9, native byte-order)". tox -e py34 first: "GNU dbm 1.x or ndbm database, little endian, 64-bit" | 16:34 |
| haypo | it looks like python 3 is unabled to open "Berkeley DB (Hash, version 9, native byte-order)" | 16:34 |
| *** BharatK has quit IRC | 16:34 | |
| haypo | where python 2 is able to open "GNU dbm 1.x or ndbm database ..." | 16:35 |
| *** vilobhmm has joined #openstack-cinder | 16:35 | |
| *** harlowja_at_home has quit IRC | 16:37 | |
| *** yamada-h has joined #openstack-cinder | 16:37 | |
| *** jwcroppe_ has joined #openstack-cinder | 16:38 | |
| *** ociuhandu has quit IRC | 16:38 | |
| *** jwcroppe has quit IRC | 16:40 | |
| *** yamada-h has quit IRC | 16:42 | |
| hemna | so with the py34 section in place it all just works | 16:43 |
| hemna | every time. | 16:43 |
| hemna | so I'm not removing it. | 16:43 |
| haypo | hemna: i also understand that tests pass with testr (without the py34) section if you remove .testrepository and you install gdbm | 16:44 |
| openstackgerrit | Tom Swanson proposed openstack/cinder: Dell SC: Add support for consistency groups https://review.openstack.org/196818 | 16:44 |
| hemna | if cinder and nova did it w/o the py34 testenv section, then I'd say fine, lets do it. but both nova and cinder do exactly what my patch does. | 16:45 |
| hemna | it's something that can be revisited later | 16:45 |
| haypo | hemna: the final goal is to remove the py34 section | 16:45 |
| hemna | for now, I'd rather get this in, as it just works. | 16:45 |
| *** Longgeek has joined #openstack-cinder | 16:45 | |
| haypo | hemna: i wrote the py34 section of cinder and nova because we cannot port them at once. it will require many weeks of work | 16:45 |
| hemna | so we can get py34 validation in os-brick. | 16:46 |
| haypo | for os-brick, we are very lucky: the whole port can be done in a single commit! | 16:46 |
| *** deepakcs has quit IRC | 16:46 | |
| hemna | lets just do this for now. it works. | 16:46 |
| *** BharatK has joined #openstack-cinder | 16:48 | |
| *** annashen has joined #openstack-cinder | 16:48 | |
| e0ne | winston-d: could you take a look on new prerformance tests after the meeting or tomorrow, please | 16:49 |
| haypo | hemna: no problem. i will probably a following patch to drop the py34 section :) it will tested by Jenkins ;) | 16:50 |
| hemna | haypo, but if it fails locally, then what good is it ? | 16:51 |
| hemna | it simply doesn't work | 16:51 |
| hemna | anyway, I'm over it. | 16:51 |
| *** sgotliv has quit IRC | 16:52 | |
| *** annashen has quit IRC | 16:53 | |
| *** rmesta1 has joined #openstack-cinder | 16:53 | |
| haypo | hemna: you should fix your setup (install gdbm) and it should work | 16:53 |
| haypo | i know how these testr bugs are annoying | 16:53 |
| haypo | but i didn't choose testr :) | 16:53 |
| *** rmesta has quit IRC | 16:54 | |
| *** aix has quit IRC | 16:55 | |
| *** belmoreira has joined #openstack-cinder | 16:55 | |
| *** bswartz has joined #openstack-cinder | 16:56 | |
| e0ne | thingee, jgriffith: one topic left from me in an agenda: Move check-cinderclient-dsvm-functional job from experimental queue to non-voting job | 17:00 |
| e0ne | and others: ^^ | 17:00 |
| e0ne | what do you think about this idea? | 17:00 |
| *** greghaynes has joined #openstack-cinder | 17:00 | |
| smcginnis | e0ne: Seems like a good thing. | 17:00 |
| smcginnis | e0ne: How have the experimental runs gone? | 17:00 |
| geguileo | winston-d: So that wip is not still up for review, right? | 17:00 |
| thingee | so geguileo I'm confused. you asked earlier about those three patches solving ha? | 17:01 |
| thingee | Force iSCSI disconnect after timeout: https://review.openstack.org/#/c/167815/ | 17:01 |
| winston-d | geguileo: sorry, it's not, it's on my local branch | 17:01 |
| geguileo | thingee: No, not solving HA XD XD | 17:01 |
| thingee | Rollback if attach_volume timesout: https://review.openstack.org/#/c/138664/ | 17:01 |
| e0ne | smcginnis: it's successful after my fix was merged :) | 17:01 |
| geguileo | thingee: Helping us solve Cinder/Nova interactions that block our HA work | 17:01 |
| thingee | Detach and terminate conn if Cinder attach fails: https://review.openstack.org/#/c/186742/ | 17:01 |
| thingee | this list is so big | 17:01 |
| geguileo | winston-d: Ok | 17:01 |
| e0ne | smcginnis: tbh, it has only a very and very simple tests | 17:01 |
| *** Longgeek has quit IRC | 17:02 | |
| geguileo | winston-d: And with that patch, what else do we need to do before we can change Cinder? | 17:02 |
| thingee | geguileo, winston-d hemna it would be good to know which I can help push through in nova | 17:02 |
| dulek | hemna: There's a lock on snapshot when creating a volume - so it won't get deleted during the procedure. | 17:02 |
| dulek | hemna: But I don' | 17:02 |
| dulek | hemna: I don't thing it's related to TaskFlow stuff. ;) | 17:02 |
| winston-d | e0ne: hey, i've already read your new numbers, it's constantly showing apache has 50% peformance gain over eventlet, i'm convinced. | 17:02 |
| MentalRay | Someone a POC of Kilo+Ceph up. We have a weird snapshot issue and curious to see if the issue is local to our setup or not. | 17:02 |
| *** rajinir has joined #openstack-cinder | 17:02 | |
| winston-d | thingee: will let you know once my patch is up. | 17:02 |
| *** patrickeast_ has joined #openstack-cinder | 17:02 | |
| *** patrickeast has quit IRC | 17:03 | |
| e0ne | winston-d: thanks for pointing me when i was wrong! | 17:03 |
| e0ne | winston-d: could you remove your -1 now? | 17:03 |
| winston-d | e0ne: sure | 17:03 |
| e0ne | great! | 17:03 |
| thingee | MentalRay: meet jbernard | 17:04 |
| *** jordanP has quit IRC | 17:04 | |
| *** hemna is now known as hemnafk | 17:04 | |
| liuxg | thingee: Huawei CI's problem have been solved now, please check it | 17:04 |
| MentalRay | yes, waiting to see if he can reproduce on his end | 17:04 |
| MentalRay | will wait for him | 17:04 |
| geguileo | So once Nova accepts VolumeBusy, what else do we need to do to be able to do changes to Cinder? | 17:04 |
| *** lpetrut has joined #openstack-cinder | 17:04 | |
| winston-d | geguileo: what change? | 17:05 |
| geguileo | winston-d: Changes like atomic status change | 17:05 |
| geguileo | winston-d: Removing some write locks using status on DB | 17:05 |
| geguileo | winston-d: Try to get some kind of shared locks to allow multiple reading of the same resource while we prevent deletions | 17:06 |
| *** patrickeast has joined #openstack-cinder | 17:06 | |
| geguileo | winston-d: Move drivers to use a DLM | 17:06 |
| winston-d | DuncanT, hemnafk: any idea about geguileo's question? | 17:06 |
| geguileo | Because if we don't know what we need to do for HA to advance it's going to be hard to get there | 17:06 |
| geguileo | (Or if only some people know) | 17:07 |
| winston-d | geguileo: I can't think of any, but DuncanT and hemnafk they know better. | 17:07 |
| geguileo | DuncanT: mentioned that we couldn't simple change Cinder because of the clients | 17:07 |
| *** patrickeast_ has quit IRC | 17:07 | |
| winston-d | plus, it seems we haven't agreed on using DLM to solve the problem, yet, do we? | 17:07 |
| geguileo | winston-d: Ok, then lets call it driver locking | 17:08 |
| geguileo | So we need to: | 17:08 |
| geguileo | 1- Allow Nova to accept VolumeBusy exceptions | 17:08 |
| jgriffith | DuncanT: I get what you're saying "kinda" | 17:08 |
| thingee | liuxg: I already spoke about this a while back on the ML and today's meeting http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-07-01-16.00.log.html#l-25 | 17:08 |
| thingee | liuxg: please stop asking me | 17:08 |
| jgriffith | DuncanT: my first answer was "ok, just spawn a new one" but you also want to pick up tasks it was in the middle of | 17:09 |
| geguileo | 2- Other stuff we don't know about before changing Cinder | 17:09 |
| DuncanT | jgriffith: spawning a new one means knowing for sure that the old one is dead, which means node fencing and pacemaker, which has a whole bunch of issues of its own | 17:09 |
| dulek | jgriffith: Oh, picking tasks is harder. I was able to do that with taskflow persistence patches, but we know whole taskflow story. | 17:10 |
| openstackgerrit | Bharat Kumar Kobagana (BharatK) proposed openstack/cinder: GlusterFS: Using 'fallocate' instead of 'dd' to create raw volumes https://review.openstack.org/164527 | 17:10 |
| geguileo | 3- Change locks for resource writting (possible using status and atomic state changes) | 17:10 |
| *** vilobhmm has quit IRC | 17:10 | |
| *** annashen has joined #openstack-cinder | 17:10 | |
| thingee | liuxg: if this not for new drivers, I will be check improvements to existing drivers when I get the chance. | 17:10 |
| geguileo | 4- Shared locks for reading and preventing deletion while there's readers | 17:10 |
| *** vilobhmm has joined #openstack-cinder | 17:10 | |
| geguileo | 5- Drivers locks | 17:10 |
| jgriffith | dulek: :) | 17:10 |
| DuncanT | geguileo: Doing locking without changing the API contract is way harder than I thought... changign the contract with Nova is much easier than changing it with tenant facing APIs | 17:10 |
| dulek | DuncanT, jgriffith: Oh, that's right, it needs to fence it - we don't know if it isn't running and talking with backend. | 17:10 |
| jgriffith | dulek: DuncanT You could start simple.... | 17:11 |
| geguileo | DuncanT: Any insights on how we can work on those APIs? | 17:11 |
| jgriffith | dulek: DuncanT turns "hosts" into a list.. if manager can't reach one, try the other | 17:11 |
| jgriffith | dulek: DuncanT that would at least solve DuncanT 's bad port last night | 17:11 |
| dulek | jgriffith: Still - you need to distributely lock. | 17:12 |
| jgriffith | dulek: DuncanT in progress task may have failed still of course, but at least user coudl ecover | 17:12 |
| jgriffith | dulek: I dont' think you do | 17:12 |
| dulek | jgriffith: For example RemoteFS patches cannot do snapshot in parallel. | 17:12 |
| jgriffith | dulek: primary/secondary | 17:12 |
| dulek | jgriffith: RemoteFS drivers. | 17:12 |
| DuncanT | jgriffith: you still need to know the the primary has stopped | 17:12 |
| dulek | jgriffith: So you need to fence primary on fail. | 17:12 |
| jgriffith | secondary takes over IFF primary isn't able to be contacted | 17:12 |
| dulek | jgriffith: You need to be 100% sure. | 17:12 |
| jgriffith | DuncanT: yes, and that's simple | 17:12 |
| jgriffith | DuncanT: Scheduler can't communicate with it when it tries to send a request | 17:13 |
| DuncanT | jgriffith: 'can't be contacted' isn't enough, you need to fence (pull the power, worse case) | 17:13 |
| dulek | jgriffith: But you don't know if link died or whole server. | 17:13 |
| liuxg | thingee: thanks, this is not for new drivers, just for existing drivers | 17:13 |
| jgriffith | DuncanT: We can disagree all day :) | 17:13 |
| DuncanT | jgriffith: The scheduler does fire and forget | 17:13 |
| dulek | jgriffith: So maybe server is alive and able to contact backend - and messing with it. | 17:13 |
| jgriffith | dulek: but you can't solve that anyway | 17:13 |
| *** aarefiev_home has quit IRC | 17:13 | |
| thingee | liuxg: ok great. I will remove the -2's after I have time to review the CIs... | 17:13 |
| jgriffith | dulek: that's my point | 17:13 |
| jgriffith | about "in progress" things | 17:14 |
| jgriffith | at least at first pass.. who cares | 17:14 |
| DuncanT | jgriffith: So start a snap of a big volume, link goes down, another snap request for that volume comes in, goes to the second c-vol, boom, corruption | 17:14 |
| dulek | jgriffith: Ah, okay, I see. Link dies, distributed lock drops, you need fencing anyway. | 17:14 |
| jgriffith | you fail those tasks, but user can re-issue and clean up | 17:14 |
| liuxg | thingee: Thanks thingee :) | 17:14 |
| jgriffith | DuncanT: no corruption | 17:14 |
| jgriffith | DuncanT: new snap, new UUID | 17:14 |
| *** liuxg has quit IRC | 17:15 | |
| DuncanT | jgriffith: Not if it is file backed... | 17:15 |
| openstackgerrit | Bharat Kumar Kobagana (BharatK) proposed openstack/cinder: RemoteFS: Reporting configured reserved_percentage in _update_volume_stats https://review.openstack.org/182286 | 17:15 |
| jgriffith | DuncanT: sounds like a problem with the choice of backends? | 17:15 |
| avishay | can we please not not not not not add distributed locks? the solution is to make things simpler, not more complex. | 17:15 |
| jgriffith | DuncanT: or at least impl of snaps in the backend | 17:15 |
| jgriffith | avishay: +1 | 17:15 |
| *** IlyaG has joined #openstack-cinder | 17:15 | |
| DuncanT | jgriffith: We have the backends we have | 17:15 |
| avishay | backends should have the requirement of idempotent operations, then life becomes simpler | 17:16 |
| jgriffith | DuncanT: fair enough I guess, but maybe you could fix the impl in their snaps in the Cinder driver that's public? | 17:16 |
| dulek | avishay: Well, using tooz isn't hard actually. But you probably meant fact it's there. | 17:16 |
| DuncanT | jgriffith: I'm at least trying to push the locks down into just the drivers that need them... they can be cleaned up case by case then if anybody really cares | 17:16 |
| jgriffith | DuncanT: which driver? I can look and see if I understand better | 17:16 |
| geguileo | avishay: We need distributed locks | 17:16 |
| avishay | dulek: yes, it's another thing to administer, it's another thing that can fail | 17:16 |
| geguileo | avishay: Be it using DB fields as locks | 17:17 |
| geguileo | avishay: Or an actual DLM | 17:17 |
| avishay | geguileo: i am not convinced | 17:17 |
| jgriffith | DuncanT: I'm not all that familiar with f-based backends, but I can look through the driver | 17:17 |
| DuncanT | jgriffith: e.g. gluster | 17:17 |
| dulek | avishay: But it may default to using good old file locks. Configure it if you want it. | 17:17 |
| geguileo | avishay: Then let me ask you | 17:17 |
| avishay | i'm not convinced that cinder needs ANY locks | 17:17 |
| geguileo | avishay: How do you prevent some node from deleting the snapshot you are using to create a volume? | 17:17 |
| jgriffith | DuncanT: dulek geguileo So is all of this just about solving issues with Gluster? | 17:18 |
| avishay | geguileo: garbage collect snapshots/volumes offline, and only when there is no current operation on them | 17:18 |
| dulek | jgriffith: Here's how to reproduce problems with running two c-vol A/A on gluster: https://gist.github.com/bluex-pl/ba50e5bd75a866c09e64 | 17:18 |
| jgriffith | or more generally file-based backends? | 17:18 |
| DuncanT | jgriffith: They seem to be the worst ones | 17:18 |
| jgriffith | dulek: DuncanT so I'm realizing this is mostly a Gluster discussion now | 17:18 |
| geguileo | jgriffith: Gluster? | 17:18 |
| DuncanT | jgriffith: I seem to remember some backends don't like it either, but I guess they can poll in the driver | 17:18 |
| geguileo | jgriffith: For me it's about getting Cinder to a point were I'm not ashamed of the races we have | 17:19 |
| jgriffith | dulek: thanks for the link | 17:19 |
| DuncanT | jgriffith: File based backends have the worst problems with races for sure | 17:19 |
| *** e0ne is now known as e0ne_ | 17:19 | |
| avishay | there are more complex systems that don't have any locks, just requires some more thought and creativity | 17:19 |
| *** patrickeast has quit IRC | 17:19 | |
| winston-d | avishay: +100 for idempotent | 17:19 |
| dulek | jgriffith: I think all RemoteFS drivers share the issues. | 17:19 |
| jgriffith | DuncanT: well... I guess I just don't understand how the corruption happens | 17:19 |
| jgriffith | DuncanT: "most" backends you say 'create snapshot' it creates a snapshot and it's independent | 17:19 |
| geguileo | avishay: And how do you know there is no operations on them? | 17:20 |
| jgriffith | I'm not trying to think about resuming tasks etc | 17:20 |
| dulek | jgriffith: But hemna once told that FC drivers suffer similar issues if I recall correctly. | 17:20 |
| avishay | novel idea - maybe block storage management shouldn't be implemented on file systems | 17:20 |
| *** e0ne_ is now known as e0ne | 17:20 | |
| DuncanT | avishay: That boat has sailed | 17:20 |
| winston-d | dulek: they said using taskflow isn't hard | 17:20 |
| eharney | cinder isn't block storage management. it's block storage for instances. | 17:20 |
| *** Yogi1 has joined #openstack-cinder | 17:20 | |
| jgriffith | dulek: data corruption? | 17:20 |
| avishay | eharney: i am working on a solidfire driver for manila as we speak BTW | 17:20 |
| avishay | :) | 17:20 |
| avishay | geguileo: you can check if there is a volume has the snapshot as a source and is in "creating" state | 17:20 |
| jgriffith | DuncanT: dulek hemnafk so let me see if I got this right.... | 17:21 |
| dulek | winston-d: :D It isn't. ;) | 17:21 |
| *** lpetrut has quit IRC | 17:21 | |
| geguileo | avishay: So you block the whole table to checkk all volumes at the same time? | 17:21 |
| jgriffith | DuncanT: dulek hemnafk what you're saying is that you have some backend devices that can't do things like more than one call to a single resource at a time? | 17:21 |
| DuncanT | jgriffith: Yes | 17:21 |
| jgriffith | DuncanT: dulek hemnafk without corrupting "something" | 17:21 |
| jgriffith | DuncanT: I have NO sympathy | 17:21 |
| *** patrickeast has joined #openstack-cinder | 17:21 | |
| jgriffith | that's stupid | 17:22 |
| jgriffith | and they don't belong in a cloud | 17:22 |
| jgriffith | even LVM can handle that | 17:22 |
| avishay | geguileo: why block the whole table? once a snapshot is in "deleted" state you can't create from it. so the amount of volumes being created from it after it is deleted should go down with time. | 17:22 |
| jgriffith | which everybody likes to talk badly about :) | 17:22 |
| avishay | geguileo: just scan periodically and delete when the number is 0 | 17:22 |
| *** jaypipes has joined #openstack-cinder | 17:22 | |
| geguileo | avishay: Because you need to check no volume is in creating state | 17:22 |
| jgriffith | DuncanT: you inparticular over the years have often said "I don't care, your device doesn't belong in a cloud then" | 17:22 |
| avishay | geguileo: no, you need to check no volume with that particular source is in creating state | 17:23 |
| geguileo | avishay: And lock the whole table so no one starts using it | 17:23 |
| jgriffith | I don't know why a devices inability to be "used" in a modern data center becomes a Cinder problem to work around | 17:23 |
| avishay | geguileo: nobody can create a volume from a deleted source | 17:23 |
| jgriffith | kinda like lipstick on a pig... it's still a pig :) | 17:23 |
| jgriffith | fix it in the driver IMHO | 17:23 |
| geguileo | avishay: Oh, so you have volumes being created from a deleted source? | 17:24 |
| *** dannywilson has quit IRC | 17:24 | |
| winston-d | dulek: you have been creative to fix 'allocated_capacity' bug with taskflow foo, you think it's simple? | 17:24 |
| DuncanT | jgriffith: We can, I think, with a bit of effort, push all of the mutual exclusion down into the drivers | 17:24 |
| * jgriffith now understands why some people put locks everywhere :) | 17:24 | |
| geguileo | avishay: So we'll have now a limbo status? | 17:24 |
| jgriffith | DuncanT: frankly that's where it belongs don't you think? | 17:24 |
| DuncanT | avishay: Delayed operations cause chaos | 17:24 |
| avishay | geguileo: i have no idea what you mean | 17:24 |
| dulek | winston-d: There will be a functionality now which makes it easier. :) | 17:25 |
| geguileo | avishay: They are being used, therefore they exist | 17:25 |
| DuncanT | avishay: Sorry, meant to say 'quota chaos' | 17:25 |
| avishay | DuncanT: no, delayed operations are the best way of garbage collection | 17:25 |
| geguileo | avishay: But they are deleted | 17:25 |
| geguileo | avishay: And cannot be used in the meantime for new operations | 17:25 |
| avishay | OK, let's start again | 17:25 |
| geguileo | avishay: Even though there are current operations that can used them | 17:25 |
| DuncanT | avishay: No, they aren't. We tried that. | 17:25 |
| geguileo | avishay: So any operation that would do a retry would probably not be able to find the resource | 17:26 |
| eharney | so we just had some discussion in the meeting about NFS snapshots | 17:26 |
| DuncanT | jgriffith: I agree we should push this down into the drivers. Coming up with a sane & safe mechanism to do so is proving tricky | 17:26 |
| avishay | I have snapshot A, and create volumes 1, 2, and 3 from it. While they are being created, i delete snapshot A. now i have a STATELESS background process that says "let's find all the deleted snaphots that i can delete right now" | 17:26 |
| eharney | we aren't going to implement that and then have folks yelling about it not belonging in the cloud afterward, right? | 17:26 |
| avishay | and then it finds snapshot A and says "oh, this has 3 volumes being created from it, i can't delete it now" | 17:27 |
| dulek | jgriffith: Apart from drivers locking on for example snapshot when creating a volume is needed. But that may be solved by atomic state changes. | 17:27 |
| DuncanT | avishay: But it is still messing up your quota | 17:27 |
| avishay | and then after 20 minutes it finds snapshot A again and says "oh boy, no volumes being created from snapshot A, this is the best day of my life - delete" | 17:27 |
| dulek | jgriffith: Only problematic thing would be when c-vol will die during such creation - who will drop the status of a snapshot back to normal? | 17:27 |
| DuncanT | avishay: You get users who do long chains of snap & delete and have to squash, which causes I/O spikes | 17:27 |
| avishay | DuncanT: no, the quota gets update as soon as the delete happens | 17:27 |
| DuncanT | avishay: The API delete? But the space is still in use (potentially indefinitely) on your backend | 17:28 |
| avishay | no, it will be deleted once active operations on it finish | 17:28 |
| *** yuriy_n17 has quit IRC | 17:28 | |
| jgriffith | You guys are all crazy :) | 17:29 |
| dulek | :D | 17:29 |
| avishay | OK fine, add distributed locks, add states in the DB, just write the entire volume to the DB and be done with it | 17:29 |
| geguileo | jgriffith: +1 | 17:29 |
| *** e0ne is now known as e0ne_ | 17:29 | |
| DuncanT | Sigh. | 17:29 |
| winston-d | avishay: mysql vol driver for cinder? :) | 17:29 |
| jgriffith | DuncanT: sorry :) | 17:30 |
| jgriffith | winston-d: stop that right now! | 17:30 |
| jgriffith | :) | 17:30 |
| avishay | winston-d: yes, why not? | 17:30 |
| DuncanT | So we kick most of the drivers out of cinder and rewrite it to just support a couple of backends, because engineering is hard? | 17:30 |
| avishay | winston-d: make one for manilla too :) | 17:30 |
| jgriffith | DuncanT: that's hardly what anybody is saying here | 17:30 |
| jgriffith | DuncanT: I'm certainly not saying that | 17:30 |
| *** IlyaG has quit IRC | 17:30 | |
| geguileo | DuncanT: XD +1 | 17:30 |
| DuncanT | jgriffith: I kind of is what you said | 17:30 |
| eharney | i kinda hear what DuncanT said when people start going on about things not belonging in the cloud | 17:31 |
| geguileo | DuncanT: Sounds like a plan ;-) | 17:31 |
| jgriffith | DuncanT: just a sec, phone call | 17:31 |
| DuncanT | jgriffith: I actually think there might be a space for a 'cinder++' blue sky codebase, but that is totally different to fixing the problems in the current one | 17:31 |
| winston-d | you know what, i remember the room when Nova people talked about adding cell to solve scaling problem. | 17:31 |
| DuncanT | cells still suck... | 17:32 |
| jgriffith | DuncanT: you keep throwing out all sorts of things so I have no idea what to think about | 17:32 |
| *** e0ne_ is now known as e0ne | 17:32 | |
| winston-d | some people said, if it has scaling issue, let's fix it, not work around it. | 17:32 |
| DuncanT | jgriffith: Saying 'that doesn't belong in the cloud' to 30% of out drivers is, to my mind, rather unhelpful at this point | 17:33 |
| jgriffith | winston-d: +1 | 17:33 |
| avishay | DuncanT: I would say 95% :-P | 17:33 |
| *** BharatK has quit IRC | 17:33 | |
| winston-d | question to us now is: is introducing DLM fixing the problem or just working around it? | 17:34 |
| DuncanT | avishay: I'd suggest you guys would be better writing an api-compatible re-write, but still.... | 17:34 |
| jgriffith | DuncanT: avishay Really? | 17:34 |
| avishay | but seriously, i think with atomic state changes, and much fewer states, we can get a recoverable system with no distributed locks | 17:34 |
| DuncanT | atomic state changes *are* a form of distributed lock | 17:34 |
| jgriffith | We have that many drivers that you can't send two API calls to at the same time without them falling over? | 17:34 |
| dulek | jgriffith: I honestly don't understand why it shouldn't be an option in Cinder if it may solve some of the problems people have. It's completely optional the way we're designing it. | 17:34 |
| avishay | DuncanT: i would love to, but then i have no drivers | 17:34 |
| jgriffith | dulek: which? | 17:34 |
| *** dtynan has quit IRC | 17:34 | |
| DuncanT | jgriffith: All of the (many) file based ones.... that's a third of the drivers | 17:35 |
| jgriffith | DuncanT: I'm so confused | 17:35 |
| *** dtynan has joined #openstack-cinder | 17:35 | |
| jgriffith | but ok | 17:35 |
| DuncanT | jgriffith: A third of our drivers or so are file based I think | 17:35 |
| avishay | well we can say that those are not HA-ready, and tough cookies | 17:35 |
| dulek | jgriffith: DuncanT has problems. ;) I'm not running any setup. | 17:35 |
| *** e0ne has quit IRC | 17:35 | |
| jgriffith | DuncanT: I guess if you and dulek implement tooz or something that would be cool | 17:35 |
| DuncanT | jgriffith: My math might be a bit out, but it is not a small number | 17:35 |
| winston-d | dulek: nova cell is optional, but people get tricked into using it and hard to get away from it. | 17:36 |
| jgriffith | DuncanT: well when you have 60 to choose from I don't know about 33% | 17:36 |
| jgriffith | let alone avishay 95% :) | 17:36 |
| jgriffith | regardless... | 17:36 |
| DuncanT | jgriffith: I'd love to push all the locks down into just the drivers that need them... I considered proposing we just push the current file locks down into *all* the driver, then the ones that don't need them can remove them | 17:36 |
| avishay | DuncanT: do you really want to administer tooz as well? | 17:36 |
| dulek | winston-d: Hm, probably clear documentation on that would be helpful... | 17:36 |
| jgriffith | I obviously don't "get it" because I don't see the problem in LVM or any of the other things that I focus on | 17:36 |
| jgriffith | so I should leave it to you and others that seem to have a really good handle on the problem | 17:37 |
| avishay | i think we can remove that ONE lock from the manager. if a driver has its own locks, then the documentation says it can't be run active-active. end of story. | 17:37 |
| DuncanT | avishay: I already have to administer rabbit, mysql, backends, horizon, load ballancers, rate limiters... adding redis isn't going to make it that much harder | 17:37 |
| avishay | DuncanT: in that case, i have a few ideas to make it more complex and fragile :) | 17:38 |
| DuncanT | jgriffith: Or at least wait until we have code before you complain... I think you'll be not too unhappy with the end result, just hard to get there | 17:38 |
| *** belmoreira has quit IRC | 17:38 | |
| avishay | OK I'm done for tonight, bye all, was a pleasure :) | 17:38 |
| DuncanT | avishay: Adding DLM might make things less fragile, not more. Atomic ops in the DB are painful to the DB... you can't do reader/writer with 'update where' as far as I can tell | 17:39 |
| dulek | avishay: bye! :) | 17:39 |
| *** lcurtis has quit IRC | 17:39 | |
| DuncanT | Fuck, I've just snapped the power lead on my laptop :-( | 17:39 |
| avishay | DuncanT: maybe SQL was the wrong choice for a distributed management service then ;) | 17:39 |
| dulek | DuncanT: You cannot with tooz in current state tooz. ;) | 17:39 |
| geguileo | avishay: Bye | 17:39 |
| avishay | adios! | 17:39 |
| *** avishay has quit IRC | 17:40 | |
| jungleboyj | DuncanT: Yikes. That sucks! | 17:40 |
| winston-d | DuncanT: really? unless you already have someone who's familiar with redis or other nosql stuff, otherwise, getting someone one to operate one new stuff they don't familiar with prove to be ... | 17:40 |
| jgriffith | DuncanT: that's fair... but FTR I wasn't "complaining" I was just asking to better understand the issue | 17:40 |
| jgriffith | which I "kinda" do now | 17:40 |
| dulek | winston-d: Well, tooz can use even MySQL as locking backend. It's not too reliable, but it's a possibility. ;) | 17:41 |
| *** ndipanov has quit IRC | 17:41 | |
| winston-d | dulek: can you quantatively measure 'how not too reliable' it is? | 17:41 |
| geguileo | And tooz is only a thin abstraction layer | 17:41 |
| DuncanT | dulek: I don't actually care if we use tooz or not... if it gives us what we need, great, if not we can do something else | 17:41 |
| geguileo | DuncanT: +1 | 17:42 |
| dulek | DuncanT: Right, I'm getting locked on library, which isn't needed here. | 17:42 |
| greghaynes | Yes - do not use the sql DB for distributed locks - this really breaks down into deployment-specific cases even and it super hard to make it work at all | 17:42 |
| *** flaper87 has joined #openstack-cinder | 17:43 | |
| DuncanT | Right, it's time for me to bale. I'll try to write up some of the discussion from tonight, there's been some food for thought. | 17:43 |
| *** marcusvrn1 has joined #openstack-cinder | 17:43 | |
| greghaynes | (e.g. galera vs single node mariadb, etc) | 17:43 |
| DuncanT | greghaynes: Having it as an option is nice for POC deploys | 17:43 |
| greghaynes | Sure, so if you want that kind of flexibility then definitely use tooz, but there should be a big red flag of "this is not for prod" | 17:43 |
| DuncanT | greghaynes: Particularly if you can do it with 'update... where' - that has to work on all backends | 17:44 |
| *** flaper87 has left #openstack-cinder | 17:44 | |
| greghaynes | Yes, youre going to turn all your reads into writes and poll on failures ;) | 17:44 |
| DuncanT | greghaynes: 'This will suck for performance above 4 nodes' is a perfectly fair warning label to put on it, sure | 17:44 |
| dulek | https://aphyr.com/posts/315-call-me-maybe-rabbitmq - you may even use RabbitMQ. ;) | 17:44 |
| greghaynes | exactly | 17:45 |
| xyang | DuncanT: hi, can you take a look of the non disruptive backup patch again? https://review.openstack.org/#/c/193937/ Addressed your comments. | 17:45 |
| *** thangp is now known as thangp_afk | 17:45 | |
| DuncanT | xyang: Sure, it'll be a while though, I need to clear my head | 17:45 |
| xyang | DuncanT: sure, thanks | 17:45 |
| DuncanT | xyang: If it isn't done by tomorrow, please shout at me | 17:46 |
| xyang | DuncanT: ok:) | 17:46 |
| *** marcusvrn has quit IRC | 17:46 | |
| *** lcurtis has joined #openstack-cinder | 17:46 | |
| *** vilobhmm has quit IRC | 17:47 | |
| *** winston-d is now known as winston-d_zZZ | 17:48 | |
| *** rwsu has quit IRC | 17:50 | |
| *** hemnafk is now known as hemna | 17:53 | |
| hemna | jgriffith, I don't believe I ever said that. | 17:53 |
| jgriffith | hemna: yeah, I dunno... was just going off what DuncanT and others said | 17:53 |
| hemna | jgriffith, we removed all of our locks on purpose because our backends can do more than one operation | 17:53 |
| jgriffith | hemna: yeah, I'm not the one who said that | 17:54 |
| jgriffith | hemna: scroll back and read, don't blame me :) | 17:54 |
| hemna | but the problem is that you can't run multiple c-vol servicing the same backend right now in cinder because of the local file locks in the manager. | 17:54 |
| hemna | "jgriffith> DuncanT: dulek hemnafk what you're saying is that you have some backend devices that can't do things like more than one call to a single resource at a time?" | 17:54 |
| hemna | I was replying to that one. | 17:54 |
| *** rwsu has joined #openstack-cinder | 17:54 | |
| jgriffith | hemna: and scroll back | 17:54 |
| hemna | regardless | 17:54 |
| jgriffith | hemna: they invoked your name saying "this is true for FC devices as well according to hemna " | 17:55 |
| hemna | removing the local file locks in the volume manager is a good goal to have IMHO | 17:55 |
| *** dtynan has quit IRC | 17:55 | |
| *** DericHorn-HP has quit IRC | 17:55 | |
| hemna | it's not my/nor Cinder's problem if a particular vendor's driver has local file locks. | 17:55 |
| *** dtynan has joined #openstack-cinder | 17:55 | |
| hemna | but Cinder itself shouldn't have them IMHO | 17:56 |
| jgriffith | hemna: I'd agree with that probably | 17:56 |
| hemna | and if Cinder needs locks, they shouldn't be local file locks, but that's a separate discussion I thinks. | 17:56 |
| *** lpetrut has joined #openstack-cinder | 17:56 | |
| jbernard | mtanino: heya, re live migration | 17:56 |
| hemna | so, we can do something in the mean time to help remove those local file locks in the volume manager | 17:56 |
| jbernard | mtanino: i need to decouple terminate connection from remove export | 17:57 |
| hemna | by 1) getting Nova to expect and handle VolumeIsBusy exceptions coming back from Cinder. | 17:57 |
| jbernard | mtanino: so that they can be called individually from the context of a migration | 17:57 |
| jbernard | mtanino: else, live migration will suffer | 17:57 |
| hemna | 2) put 'ing' checks in each of the cinder API calls, and return VolumeIsBusy. | 17:57 |
| jbernard | mtanino: is that accurate? | 17:57 |
| *** IlyaG has joined #openstack-cinder | 17:58 | |
| jgriffith | hemna: yes, and IMHO those things you list solves at least 80% of the problem areas for us I think | 17:58 |
| hemna | so whenever 'HA' discussions come up, I always point to that first | 17:58 |
| jgriffith | hemna: but as I mentioned to DuncanT and others, I don't fully understand the challenges they have in their envs | 17:59 |
| hemna | and folks get bogged down in locking mechanism talks seems pointless to me. | 17:59 |
| hemna | tooz vs. whatever vs. sucksslesslocks | 17:59 |
| *** Yogi1 has quit IRC | 18:00 | |
| jgriffith | hemna: we should write a sucksslesslocks lib | 18:02 |
| hemna | :) | 18:02 |
| *** lpetrut has quit IRC | 18:03 | |
| *** DericHorn-HP has joined #openstack-cinder | 18:03 | |
| *** delatte has quit IRC | 18:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements https://review.openstack.org/197545 | 18:05 |
| *** cdelatte has joined #openstack-cinder | 18:05 | |
| jgriffith | hemna: thingee BTW, I traced down the issue with my CI system... | 18:05 |
| jgriffith | hemna: thingee it turns out that "somewhere" port 3260 is being blocked on that entire rack | 18:05 |
| *** cdelatte has quit IRC | 18:06 | |
| thingee | jgriffith: glad to hear you figured it out! | 18:06 |
| *** cdelatte has joined #openstack-cinder | 18:06 | |
| jgriffith | thingee: yeah... now if I could just figure out exactly what was done and by whom | 18:06 |
| jgriffith | thingee: in the meantime I'm building a new cloud in a different lab :) | 18:06 |
| jgriffith | thingee: It's a bummer, because I just hit 275 consecutive succesful runs :( | 18:07 |
| jgriffith | now I have to start over | 18:07 |
| thingee | haha | 18:07 |
| thingee | a new cinder ci record probably! | 18:07 |
| thingee | setting the bar high there for us :D | 18:08 |
| jgriffith | well to be fair that doesn't count anything that Jenkins failed | 18:08 |
| mtanino | jbernard: hi | 18:09 |
| mtanino | jbernard: Correct "so that they can be called individually from the context of a migration, else, live migration will suffer" | 18:10 |
| xyang | jgriffith: hi, here is a fix to the VNX unit test problem you ran into earlier: https://review.openstack.org/#/c/194573/ can you take a look? | 18:11 |
| openstackgerrit | Marcus V R Nascimento proposed openstack/cinder: Fix HBSD horcm driver with oslo.concurrency 2.1.0 https://review.openstack.org/197686 | 18:12 |
| *** zongliang has quit IRC | 18:12 | |
| jgriffith | xyang: oh.. awesome! | 18:13 |
| jgriffith | yes | 18:13 |
| xyang | jgriffith: thanks | 18:13 |
| jgriffith | xyang: I'd just +2/A it but I will get scolded (again) so just a +2 | 18:14 |
| xyang | jgriffith: sure:) thanks! | 18:14 |
| *** thangp_afk is now known as thangp | 18:14 | |
| jgriffith | OMG IR SO DUMB | 18:16 |
| openstackgerrit | Marcus V R Nascimento proposed openstack/cinder: Fix HBSD horcm driver with oslo.concurrency 2.1.0 https://review.openstack.org/197686 | 18:19 |
| hemna | jgriffith, doh, well that's a good find then. those are hard to find. | 18:19 |
| jgriffith | hemna: annoying | 18:20 |
| jgriffith | hemna: I realized after seeing 12 in a row that it was def my env | 18:20 |
| *** jungleboyj has quit IRC | 18:22 | |
| *** thangp has quit IRC | 18:23 | |
| *** e0ne has joined #openstack-cinder | 18:23 | |
| *** e0ne is now known as e0ne_ | 18:23 | |
| *** e0ne_ is now known as e0ne | 18:23 | |
| *** e0ne has quit IRC | 18:23 | |
| *** e0ne has joined #openstack-cinder | 18:24 | |
| *** thangp has joined #openstack-cinder | 18:28 | |
| *** Yogi1 has joined #openstack-cinder | 18:30 | |
| dulek | hemna: Sorry for this FC-drivers locks confusion, I've either misunderstood you or haven't remembered correctly. | 18:30 |
| hemna | no worries. | 18:30 |
| hemna | I'm not worried about it. I'd just like to see some more folks working on known problems instead of adding new features. | 18:31 |
| *** Nikolay_St has quit IRC | 18:31 | |
| *** dannywilson has joined #openstack-cinder | 18:33 | |
| *** Nikolay_St has joined #openstack-cinder | 18:33 | |
| *** e0ne is now known as e0ne_ | 18:34 | |
| *** e0ne_ is now known as e0ne | 18:37 | |
| *** Lee1092 has joined #openstack-cinder | 18:37 | |
| openstackgerrit | xing-yang proposed openstack/cinder: EMC VMAX Modify CG https://review.openstack.org/194829 | 18:37 |
| *** xyang has quit IRC | 18:37 | |
| *** Rockyg has joined #openstack-cinder | 18:41 | |
| *** juzuluag_ has quit IRC | 18:42 | |
| *** xyang has joined #openstack-cinder | 18:43 | |
| *** mutoulbj has quit IRC | 18:46 | |
| e0ne | eharney: tested rbd patch with cuncorrency and high load: it looks good | 18:49 |
| e0ne | eharney: i'll make review request to get some feedback from you and community | 18:49 |
| *** julim has quit IRC | 18:51 | |
| *** Yogi1 has quit IRC | 18:52 | |
| *** hodos has joined #openstack-cinder | 18:52 | |
| *** rajinir has quit IRC | 18:52 | |
| *** rajinir has joined #openstack-cinder | 18:53 | |
| *** dtynan has quit IRC | 19:00 | |
| *** dtynan has joined #openstack-cinder | 19:01 | |
| *** rajinir has quit IRC | 19:02 | |
| *** rajinir has joined #openstack-cinder | 19:03 | |
| eharney | e0ne: ok i'll check it out | 19:04 |
| e0ne | eharney: https://docs.python.org/2/library/threading.html#importing-in-threaded-code - the second item | 19:04 |
| openstackgerrit | Patrick East proposed openstack/cinder: WIP generic image cache https://review.openstack.org/195795 | 19:07 |
| openstackgerrit | Ivan Kolodyazhny proposed openstack/cinder: Fix block eventlet threads on rbd calls https://review.openstack.org/197710 | 19:07 |
| patrickeast | jgriffith: hey, not sure if you had time to mess with that image cache poc code or not, but i’ve got it mostly working pretty well now… i did however realize i overlooked something important, it only works for public images | 19:08 |
| patrickeast | jgriffith: i’m trying to figure out a nice way to have it work for non-public ones… but the only ways i’m seeing involve some ninja moves with the contexts and/or cinder database :( | 19:09 |
| *** erlon has quit IRC | 19:10 | |
| *** DericHorn-HP has quit IRC | 19:11 | |
| mriedem | tbarron: xyang: fyi https://review.openstack.org/#/c/197713/ | 19:12 |
| mriedem | thingee: jgriffith: DuncanT: hemna: ^ | 19:13 |
| mriedem | per the cinder meeting today on encrypted volume drivers | 19:13 |
| *** boris-42 has joined #openstack-cinder | 19:13 | |
| *** reed_ has joined #openstack-cinder | 19:15 | |
| *** blmartin has quit IRC | 19:16 | |
| *** garthb has joined #openstack-cinder | 19:20 | |
| *** garthb_ has joined #openstack-cinder | 19:20 | |
| * morganfainberg waves at the cinder-type people cause... it's that time of day to wander through IRC channels. | 19:22 | |
| *** zhenguo has quit IRC | 19:23 | |
| *** DericHorn-HP has joined #openstack-cinder | 19:24 | |
| ericksonsantos | morganfainberg, o/ | 19:24 |
| *** rajinir has quit IRC | 19:25 | |
| *** thangp has quit IRC | 19:26 | |
| *** rajinir has joined #openstack-cinder | 19:26 | |
| *** IlyaG has quit IRC | 19:28 | |
| *** annashen has quit IRC | 19:29 | |
| *** Apoorva has quit IRC | 19:29 | |
| *** dansmith has quit IRC | 19:30 | |
| *** Apoorva has joined #openstack-cinder | 19:30 | |
| *** annashen has joined #openstack-cinder | 19:30 | |
| *** dansmith has joined #openstack-cinder | 19:30 | |
| *** rajinir has quit IRC | 19:32 | |
| jbernard | mtanino: got it, thank you. i should have an update soon | 19:33 |
| *** rajinir has joined #openstack-cinder | 19:33 | |
| mtanino | jbernard: Sure. thank you for your understanding. | 19:33 |
| *** lpetrut has joined #openstack-cinder | 19:34 | |
| *** annashen has quit IRC | 19:34 | |
| *** Rockyg has quit IRC | 19:35 | |
| *** ociuhandu has joined #openstack-cinder | 19:36 | |
| jgriffith | patrickeast: hey | 19:36 |
| jgriffith | patrickeast: I had some things I wanted to talk to you about on that | 19:36 |
| jgriffith | patrickeast: go a minute now? | 19:37 |
| *** e0ne has quit IRC | 19:38 | |
| thingee | jgriffith: hey did you get my message yesterday about helping me with the scheduler part of the capability spec? | 19:38 |
| jgriffith | thingee: I did not | 19:38 |
| ericksonsantos | Hi | 19:39 |
| ericksonsantos | I'm new to openstack and I've been helping vilobh with nested quota driver for a while. | 19:39 |
| thingee | jgriffith: since you have that more figured out, would you mind updating my spec with that part? I'm on my phone so I can't pull the link at the moment. | 19:39 |
| ericksonsantos | this may be a dumb question, but... | 19:39 |
| jgriffith | thingee: haha | 19:39 |
| jgriffith | sure | 19:39 |
| jgriffith | I may not get to it til a bit later | 19:39 |
| ericksonsantos | does someone know when to use 'cinder quota-class-update' instead of 'cinder quota-update'? | 19:39 |
| *** yamada-h has joined #openstack-cinder | 19:39 | |
| jgriffith | but I can have it by lunch tomorrow at the latest if that's ok? | 19:40 |
| thingee | That's fine. That'll be faster than me going through revisions. | 19:40 |
| openstackgerrit | Tom Swanson proposed openstack/cinder: Dell SC: Add support for consistency groups https://review.openstack.org/196818 | 19:40 |
| thingee | jgriffith: appreciate it | 19:40 |
| *** nkrinner has joined #openstack-cinder | 19:40 | |
| jgriffith | thingee: no problem | 19:41 |
| ericksonsantos | on cinderclient | 19:42 |
| *** yamada-h has quit IRC | 19:44 | |
| *** mutoulbj has joined #openstack-cinder | 19:47 | |
| patrickeast | jgriffith: hey, sry was afk | 19:47 |
| jgriffith | patrickeast: hey | 19:48 |
| jgriffith | patrickeast: so here's the thing... IIUC | 19:48 |
| *** ctina_ has joined #openstack-cinder | 19:48 | |
| jgriffith | patrickeast: this still requires the bulk of the impl in the driver via "clone_image" | 19:48 |
| jgriffith | patrickeast: but adds a whole mess of things on top to try and keep records of things right? | 19:49 |
| jgriffith | patrickeast: so what if we just steppe back and made it super simple.... | 19:49 |
| patrickeast | jgriffith: uhh no, it shouldn’t need clone_image at all | 19:49 |
| patrickeast | jgriffith: it only runs this if clone_image didn’t do anything | 19:49 |
| jgriffith | patrickeast: ummm... well then I absolutely can't get it to work :) | 19:49 |
| patrickeast | jgriffith: haha, so i was wondering how it would play with your drivers cache stuff… its very possible i broke that | 19:50 |
| patrickeast | jgriffith: because you do implement clone_image | 19:50 |
| jgriffith | patrickeast: well, I turned my stuff off | 19:50 |
| patrickeast | jgriffith: it works great on mine which doesn’t implement it | 19:50 |
| patrickeast | jgriffith: ah ok, something else then | 19:50 |
| jgriffith | so it is a noop | 19:50 |
| patrickeast | jgriffith: any chance you could send/paste a log from c-vol when it should be using it? | 19:51 |
| jgriffith | patrickeast: but I could not get it to create the second volume in any case | 19:51 |
| patrickeast | jgriffith: hmm thats not good | 19:51 |
| jgriffith | patrickeast: well.. yesterday I could've... today not so much :( | 19:51 |
| jgriffith | patrickeast: my lab is officially *dead* | 19:51 |
| patrickeast | :( | 19:51 |
| jgriffith | I'm trying to get everything back up, but stuck waiting for IT to figure out what they did to the network | 19:51 |
| jgriffith | it started with just a single port.. then all ports | 19:52 |
| jgriffith | then pooof! | 19:52 |
| patrickeast | oh geez | 19:52 |
| jgriffith | and the machine running your code is a VM in that lab | 19:52 |
| *** ctina has quit IRC | 19:52 | |
| *** Rockyg has joined #openstack-cinder | 19:52 | |
| patrickeast | jgriffith: haha, well, assuming that the code doesn’t need to use clone_image, were there any other concerns? | 19:53 |
| *** ctina_ has quit IRC | 19:53 | |
| patrickeast | jgriffith: and ignoring the whole not working thing | 19:53 |
| *** mutoulbj has quit IRC | 19:54 | |
| *** lpetrut has quit IRC | 19:56 | |
| jgriffith | patrickeast: well, trying to get it to work :( | 19:56 |
| jgriffith | patrickeast: which I was unsuccesful in doing sadly | 19:56 |
| patrickeast | jgriffith: if you want to take another stab at it i would recommend pulling the latest patchset on the review… it fixed a ton of little things | 19:57 |
| patrickeast | jgriffith: but i am interested in your thoughts ‘super simple’ approach though | 19:57 |
| patrickeast | s/thoughts/thoughts on/ | 19:59 |
| jgriffith | patrickeast: so what I thought was leave it in the clone_image for now | 20:03 |
| jgriffith | patrickeast: but have it return and populate the database with the appropriate info | 20:03 |
| jgriffith | patrickeast: that way all the crazy lookups and translations go away | 20:03 |
| jgriffith | patrickeast: and it all goes in manager, nothing in flow or API | 20:04 |
| *** DericHorn-HP has quit IRC | 20:04 | |
| patrickeast | jgriffith: maybe i’m missing something, i thought clone_image was a volume driver method, how would it go to the manager | 20:04 |
| jgriffith | patrickeast: so the manager calls it, and we could give it a response | 20:05 |
| jgriffith | patrickeast: then subsequent calls just check the DB | 20:05 |
| patrickeast | jgriffith: should we put it in the BaseVD impl of it? | 20:05 |
| jgriffith | patrickeast: so everything is there already. | 20:06 |
| jgriffith | patrickeast: so here's the problem I had with the generic and I think we still have | 20:06 |
| patrickeast | jgriffith: clone_image is called from the flow https://github.com/openstack/cinder/blob/master/cinder/volume/flows/manager/create_volume.py#L594 and calls the driver directly | 20:06 |
| jgriffith | patrickeast: some devices will need to use snaps, some clones.. some maybe something else | 20:06 |
| patrickeast | jgriffith: right, thats why it just does a create volume from image, and create from source | 20:06 |
| patrickeast | jgriffith: and lets the drivers do whatever they need | 20:06 |
| patrickeast | jgriffith: all drivers can do those actions, right? | 20:07 |
| * patrickeast likes to make grand assumptions | 20:07 | |
| jgriffith | patrickeast: hehe | 20:07 |
| jgriffith | patrickeast: well that's a 'good' assumption probably | 20:07 |
| *** DericHorn-HP has joined #openstack-cinder | 20:08 | |
| patrickeast | jgriffith: so i like the idea of cutting out as much as we can, imo more simple = more better | 20:08 |
| jgriffith | patrickeast: and you're right.. sorry nothing goes through manager currently | 20:09 |
| patrickeast | jgriffith: and i think we can do some more database magic in the managers flow to avoid calling back down through the api | 20:09 |
| jgriffith | patrickeast: yeah, that's what I'm wondering as well | 20:09 |
| patrickeast | jgriffith: i guess the trade off is duplicating any of the create_volume api flow that already does the db/quota stuff in the manager create_volume flow | 20:09 |
| jgriffith | patrickeast: so really if I get the thing to work maybe I'll think what you have is the best thing since sliced bread :) | 20:09 |
| patrickeast | jgriffith: haha | 20:10 |
| patrickeast | jgriffith: i do need to figure out the whole private image thing though… | 20:10 |
| patrickeast | jgriffith: i’ve been banging on it a little bit on a local vm and that seems to be the only big issues i found | 20:11 |
| *** belmoreira has joined #openstack-cinder | 20:11 | |
| patrickeast | jgriffith: right now its trying to create the volume from image using the internal tenant context, which doesn’t work if the internal tenant can’t access the image from glance | 20:11 |
| *** daneyon has joined #openstack-cinder | 20:11 | |
| patrickeast | jgriffith: i didn’t notice it because tempest was happy and i was only using public images on my devstack : / | 20:12 |
| *** belmoreira has quit IRC | 20:15 | |
| *** rhagarty has joined #openstack-cinder | 20:17 | |
| *** rhagarty_ has joined #openstack-cinder | 20:17 | |
| openstackgerrit | Sean McGinnis proposed openstack/cinder: Remove useless logging from unit tests https://review.openstack.org/197723 | 20:18 |
| *** e0ne has joined #openstack-cinder | 20:18 | |
| *** garthb has quit IRC | 20:21 | |
| *** cburgess_ has quit IRC | 20:23 | |
| *** cburgess has joined #openstack-cinder | 20:23 | |
| *** marcusvrn1 has quit IRC | 20:28 | |
| *** Yogi1 has joined #openstack-cinder | 20:28 | |
| tbarron | mriedem: thanks for all the good stuff on the bugs I raised! I've been idle this afternoon as my family is under the impression that I'm on vacation this week and had some plans ... | 20:34 |
| mriedem | tbarron: np | 20:35 |
| mriedem | just lock yourself in the bathroom | 20:35 |
| tbarron | mriedem: :-) | 20:35 |
| *** earlephilhower has quit IRC | 20:35 | |
| *** earlephilhower has joined #openstack-cinder | 20:36 | |
| *** openstackgerrit has quit IRC | 20:37 | |
| tbarron | mriedem: I think the 3 retries may work. There isn't really anything different up to the point of the 'device busy' excetption between our iSCSI logs and, say, LVM. | 20:37 |
| tbarron | I downloaded the cryptsetup code and didn't find anything enlightening. | 20:37 |
| *** openstackgerrit has joined #openstack-cinder | 20:37 | |
| tbarron | /dev/sdh is not a valid LUKS | 20:38 |
| tbarron | device; formatting device for first use | 20:38 |
| tbarron | The successfull iscsi CI logs that I looked at also have the 'not a valid LUKS device; | 20:39 |
| tbarron | formatting device for first use' msg. | 20:39 |
| tbarron | I think that is expected. | 20:39 |
| mriedem | yeah, it's info level | 20:39 |
| *** VManiac16 has joined #openstack-cinder | 20:39 | |
| tbarron | cinder doesn't really do much here, nova (with providers) does all the formatting, encrypting, etc. | 20:39 |
| tbarron | Well, someone is knocking on the bathroom door :-) Let's get the retry fix merged as it is harmless and run CI again on your main fix. | 20:41 |
| tbarron | mriedem: thanks again! | 20:41 |
| *** nkrinner has quit IRC | 20:44 | |
| *** DericHorn-HP has quit IRC | 20:45 | |
| *** annashen has joined #openstack-cinder | 20:46 | |
| *** daneyon has quit IRC | 20:47 | |
| *** mriedem has quit IRC | 20:49 | |
| *** kjnelson has left #openstack-cinder | 20:53 | |
| *** laudo has joined #openstack-cinder | 20:53 | |
| *** Lee1092 has quit IRC | 20:55 | |
| VManiac16 | hemna: Are you around? | 20:56 |
| hemna | here and there | 20:56 |
| VManiac16 | Just curious if the HP XP7 CI was down. | 20:56 |
| hemna | no idea, I don't watch the XP CI | 20:57 |
| hemna | asselin, | 20:57 |
| hemna | asselin, where is nestor ? | 20:57 |
| tsekiyama | hemna: Hmm, your nova patch for os-brick switching seems hitting requirements issue... It looks like Jenkins gate-nova-requirements hate test-requirements-py3.txt | 20:58 |
| VManiac16 | hemna: No need to bother asselin | 20:59 |
| *** marcusvrn has joined #openstack-cinder | 21:00 | |
| *** jasondotstar has joined #openstack-cinder | 21:01 | |
| hemna | tsekiyama, did it hit that again? | 21:02 |
| hemna | *sigh* | 21:02 |
| hemna | last time I looked at the log and it didn't really say what the problem was | 21:02 |
| hemna | VManiac16, well, I don't watch the CI systems, asselin does for us. | 21:02 |
| VManiac16 | hemna: Ok ... thanks. | 21:03 |
| tsekiyama | hemna: yeah,, it is hidden until you hit the "Toggle CI" button on the gerrit review page.. | 21:04 |
| openstackgerrit | Diem Tran proposed openstack/cinder: Fix PEP 476 and debug format messages of Oracle ZFSSA Cinder drivers https://review.openstack.org/197736 | 21:05 |
| tsekiyama | hemna: My glance patch is hitting the same problem. https://review.openstack.org/#/c/186201/ I'm trying to look into it | 21:06 |
| *** e0ne has quit IRC | 21:08 | |
| asselin | VManiac16, hemna he's on vacatino | 21:10 |
| VManiac16 | asselin: Right ... lounging in So. Cal. | 21:11 |
| laudo | what is the default size of an IOP when creating read_iops_sec qos for a cinder backend? | 21:14 |
| *** IlyaG has joined #openstack-cinder | 21:21 | |
| *** marcusvrn has quit IRC | 21:23 | |
| *** edmondsw has quit IRC | 21:28 | |
| jgriffith | laudo: IOP is an Io Per Second | 21:29 |
| *** jasondotstar has quit IRC | 21:30 | |
| *** Yogi1 has quit IRC | 21:37 | |
| *** IlyaG has quit IRC | 21:44 | |
| *** Yogi1 has joined #openstack-cinder | 21:45 | |
| *** Yogi1 has quit IRC | 21:52 | |
| *** dannywilson has quit IRC | 21:55 | |
| *** dannywilson has joined #openstack-cinder | 21:56 | |
| *** annegentle has joined #openstack-cinder | 22:02 | |
| *** MentalRay has quit IRC | 22:09 | |
| *** chlong has joined #openstack-cinder | 22:10 | |
| *** thingee has quit IRC | 22:11 | |
| *** thingee has joined #openstack-cinder | 22:11 | |
| *** thingee has quit IRC | 22:11 | |
| *** thingee has joined #openstack-cinder | 22:11 | |
| openstackgerrit | Peter Penchev proposed openstack/cinder: Implement the update_migrated_volume for the drivers https://review.openstack.org/180873 | 22:16 |
| *** dims_ has joined #openstack-cinder | 22:17 | |
| *** annashen has quit IRC | 22:20 | |
| *** dims has quit IRC | 22:21 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/cinder: Updated from global requirements https://review.openstack.org/197545 | 22:21 |
| *** annashen has joined #openstack-cinder | 22:21 | |
| laudo | jgriffith: So there is no size in kb for iops? its per sec only? | 22:23 |
| *** annashen has quit IRC | 22:25 | |
| *** chlong has quit IRC | 22:30 | |
| *** chlong has joined #openstack-cinder | 22:30 | |
| *** lcurtis has quit IRC | 22:40 | |
| *** VManiac16 has quit IRC | 22:48 | |
| *** hodos has quit IRC | 22:57 | |
| *** dannywilson has quit IRC | 23:02 | |
| *** dannywilson has joined #openstack-cinder | 23:03 | |
| *** jasondotstar has joined #openstack-cinder | 23:09 | |
| *** annegentle has quit IRC | 23:10 | |
| *** DericHorn-HP has joined #openstack-cinder | 23:13 | |
| *** zhenguo has joined #openstack-cinder | 23:14 | |
| *** vokt has joined #openstack-cinder | 23:15 | |
| *** mutoulbj has joined #openstack-cinder | 23:22 | |
| *** ctina_ has joined #openstack-cinder | 23:22 | |
| *** IanGovett has quit IRC | 23:30 | |
| *** annegentle has joined #openstack-cinder | 23:34 | |
| *** lixiaoy1 has joined #openstack-cinder | 23:37 | |
| *** ctina_ has quit IRC | 23:41 | |
| *** mutoulbj has quit IRC | 23:46 | |
| *** cebruns has quit IRC | 23:49 | |
| *** DericHorn-HP has quit IRC | 23:50 | |
| *** cebruns has joined #openstack-cinder | 23:51 | |
| *** annashen has joined #openstack-cinder | 23:51 | |
| *** DericHorn-HP has joined #openstack-cinder | 23:52 | |
| *** jasondotstar has quit IRC | 23:52 | |
| *** DericHorn-HP has quit IRC | 23:53 | |
| *** annashen has quit IRC | 23:56 | |
| *** rmesta1 has quit IRC | 23:58 | |
| *** Yogi1 has joined #openstack-cinder | 23:59 | |
| *** laughterwym has joined #openstack-cinder | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!