| *** emagana has joined #openstack-cinder | 00:02 | |
| *** rongze has joined #openstack-cinder | 00:04 | |
| *** dims__ has joined #openstack-cinder | 00:04 | |
| *** dims_ has joined #openstack-cinder | 00:05 | |
| *** rongze has quit IRC | 00:05 | |
| *** rongze has joined #openstack-cinder | 00:06 | |
| *** ho has joined #openstack-cinder | 00:07 | |
| *** dims__ has quit IRC | 00:09 | |
| *** rongze has quit IRC | 00:11 | |
| *** rongze has joined #openstack-cinder | 00:11 | |
| *** rongze has quit IRC | 00:17 | |
| *** Mandell has joined #openstack-cinder | 00:20 | |
| *** zhenguo has joined #openstack-cinder | 00:38 | |
| *** primechuck has quit IRC | 00:39 | |
| *** thingee has joined #openstack-cinder | 00:44 | |
| *** patrickeast has quit IRC | 00:50 | |
| *** _cjones_ has quit IRC | 00:53 | |
| *** leeantho has quit IRC | 00:54 | |
| *** vilobhmm11 has quit IRC | 01:01 | |
| *** emagana has quit IRC | 01:02 | |
| *** emagana has joined #openstack-cinder | 01:03 | |
| *** emagana has quit IRC | 01:03 | |
| *** ozialien has joined #openstack-cinder | 01:03 | |
| *** ozialien has left #openstack-cinder | 01:12 | |
| *** krtaylor has joined #openstack-cinder | 01:13 | |
| *** rongze has joined #openstack-cinder | 01:26 | |
| *** adurbin_ has quit IRC | 01:29 | |
| *** rongze has quit IRC | 01:30 | |
| *** fanyaohong has quit IRC | 01:39 | |
| *** primechuck has joined #openstack-cinder | 01:41 | |
| *** harlowja is now known as harlowja_away | 01:47 | |
| *** theanalyst has quit IRC | 01:49 | |
| *** theanalyst has joined #openstack-cinder | 01:51 | |
| *** dims_ has quit IRC | 01:54 | |
| *** bill_az has quit IRC | 01:56 | |
| *** Apoorva_ has quit IRC | 02:03 | |
| *** jwang_ has quit IRC | 02:26 | |
| *** Apoorva has joined #openstack-cinder | 02:26 | |
| openstackgerrit | wanghao proposed openstack/cinder: query volume detail support volume_glance_metadata https://review.openstack.org/147738 | 02:28 |
|---|---|---|
| *** Apoorva has quit IRC | 02:30 | |
| *** Apoorva has joined #openstack-cinder | 02:31 | |
| *** Apoorva has quit IRC | 02:35 | |
| *** jwang_ has joined #openstack-cinder | 02:37 | |
| *** Mandell has quit IRC | 02:42 | |
| *** Mandell has joined #openstack-cinder | 02:42 | |
| *** liusheng has joined #openstack-cinder | 02:46 | |
| *** Longgeek has joined #openstack-cinder | 02:59 | |
| *** zhithuang has joined #openstack-cinder | 02:59 | |
| *** zhithuang is now known as winston-d_ | 03:00 | |
| winston-d_ | jdurgin: ping | 03:00 |
| *** winston-1_ has joined #openstack-cinder | 03:03 | |
| *** winston-d_ has quit IRC | 03:03 | |
| *** Longgeek has quit IRC | 03:04 | |
| *** Longgeek has joined #openstack-cinder | 03:06 | |
| *** liusheng has quit IRC | 03:07 | |
| *** liusheng has joined #openstack-cinder | 03:07 | |
| *** Mandell has quit IRC | 03:11 | |
| *** vilobhmm1 has joined #openstack-cinder | 03:20 | |
| *** Lee1092 has joined #openstack-cinder | 03:23 | |
| openstackgerrit | wanghao proposed openstack/cinder-specs: Add blueprint for support-force-delete-backup https://review.openstack.org/98325 | 03:23 |
| *** mdenny has quit IRC | 03:24 | |
| *** heyun has joined #openstack-cinder | 03:24 | |
| openstackgerrit | wanghao proposed openstack/cinder: query volume detail support volume_glance_metadata https://review.openstack.org/147738 | 03:30 |
| openstackgerrit | wanghao proposed openstack/cinder-specs: Add blueprint for support-force-delete-backup https://review.openstack.org/98325 | 03:44 |
| *** ishant has joined #openstack-cinder | 03:45 | |
| openstackgerrit | wanghao proposed openstack/cinder: Supprot for force-delete backups https://review.openstack.org/166127 | 03:52 |
| *** _cjones_ has joined #openstack-cinder | 03:54 | |
| *** _cjones_ has quit IRC | 03:59 | |
| *** jungleboyj has joined #openstack-cinder | 04:04 | |
| *** thingee has quit IRC | 04:15 | |
| *** Mandell has joined #openstack-cinder | 04:22 | |
| *** heyun has quit IRC | 04:37 | |
| *** heyun has joined #openstack-cinder | 04:37 | |
| *** IanGovett1 has joined #openstack-cinder | 04:39 | |
| *** IanGovett has quit IRC | 04:40 | |
| *** haomaiwa_ has joined #openstack-cinder | 04:43 | |
| *** xyang1 has quit IRC | 05:07 | |
| *** openstackgerrit has quit IRC | 05:21 | |
| *** openstackgerrit has joined #openstack-cinder | 05:21 | |
| *** sks has joined #openstack-cinder | 05:27 | |
| *** harlowja_at_home has joined #openstack-cinder | 05:31 | |
| *** jseiler_ has quit IRC | 05:32 | |
| *** jseiler_ has joined #openstack-cinder | 05:32 | |
| *** nikesh has quit IRC | 05:33 | |
| *** nkrinner has joined #openstack-cinder | 05:36 | |
| *** rushiagr_away is now known as rushiagr | 05:38 | |
| *** deepakcs has joined #openstack-cinder | 05:57 | |
| *** winston-1_ has quit IRC | 06:02 | |
| *** vilobhmm1 has quit IRC | 06:06 | |
| *** Maike has joined #openstack-cinder | 06:07 | |
| *** Maike_ has joined #openstack-cinder | 06:08 | |
| *** Maike has quit IRC | 06:12 | |
| *** emagana has joined #openstack-cinder | 06:16 | |
| *** harlowja_at_home has quit IRC | 06:18 | |
| *** ankit_ag has joined #openstack-cinder | 06:18 | |
| *** zerda has joined #openstack-cinder | 06:21 | |
| *** IanGovett has joined #openstack-cinder | 06:26 | |
| *** IanGovett1 has quit IRC | 06:27 | |
| *** TobiasE has joined #openstack-cinder | 06:29 | |
| *** bnemec has quit IRC | 06:32 | |
| *** lpetrut has joined #openstack-cinder | 06:34 | |
| *** IanGovett1 has joined #openstack-cinder | 06:34 | |
| *** IanGovett has quit IRC | 06:36 | |
| *** Longgeek_ has joined #openstack-cinder | 06:37 | |
| *** Longgeek has quit IRC | 06:39 | |
| *** IanGovett1 has quit IRC | 06:43 | |
| *** IanGovett has joined #openstack-cinder | 06:44 | |
| *** Mandell has quit IRC | 06:46 | |
| *** dulek has joined #openstack-cinder | 06:48 | |
| *** IanGovett has quit IRC | 06:48 | |
| *** IanGovett has joined #openstack-cinder | 06:48 | |
| *** IanGovett has quit IRC | 06:53 | |
| *** IanGovett has joined #openstack-cinder | 06:55 | |
| *** ho_ has joined #openstack-cinder | 06:57 | |
| *** winston-d_ has joined #openstack-cinder | 06:58 | |
| *** ho has quit IRC | 06:59 | |
| openstackgerrit | wanghao proposed openstack/cinder: Supprot for force-delete backups https://review.openstack.org/166127 | 07:03 |
| *** e0ne has joined #openstack-cinder | 07:08 | |
| *** markus_z has joined #openstack-cinder | 07:14 | |
| *** svasheka has quit IRC | 07:21 | |
| *** e0ne has quit IRC | 07:23 | |
| *** ronis has joined #openstack-cinder | 07:26 | |
| *** e0ne has joined #openstack-cinder | 07:28 | |
| *** alexpilotti has joined #openstack-cinder | 07:30 | |
| *** haomaiw__ has joined #openstack-cinder | 07:32 | |
| *** haomaiwa_ has quit IRC | 07:32 | |
| *** chlong has quit IRC | 07:33 | |
| *** dims__ has joined #openstack-cinder | 07:33 | |
| *** ronis has quit IRC | 07:35 | |
| *** aarefiev_ has quit IRC | 07:38 | |
| *** dims__ has quit IRC | 07:38 | |
| *** ndipanov has quit IRC | 07:44 | |
| *** ankit_ag has quit IRC | 07:46 | |
| *** jistr has joined #openstack-cinder | 07:50 | |
| *** abhiram_moturi has quit IRC | 07:50 | |
| *** abhiram_moturi has joined #openstack-cinder | 07:50 | |
| *** e0ne has quit IRC | 07:57 | |
| *** e0ne has joined #openstack-cinder | 08:01 | |
| *** rushiagr is now known as rushiagr_away | 08:04 | |
| *** sks has quit IRC | 08:08 | |
| *** e0ne has quit IRC | 08:09 | |
| *** emagana has quit IRC | 08:12 | |
| *** e0ne has joined #openstack-cinder | 08:14 | |
| *** e0ne has quit IRC | 08:18 | |
| *** turul has joined #openstack-cinder | 08:18 | |
| *** e0ne has joined #openstack-cinder | 08:22 | |
| *** fanyaohong has joined #openstack-cinder | 08:26 | |
| *** Miouge has joined #openstack-cinder | 08:28 | |
| *** turul is now known as afazekas | 08:30 | |
| *** Miouge_ has joined #openstack-cinder | 08:30 | |
| *** Miouge has quit IRC | 08:34 | |
| *** Miouge_ is now known as Miouge | 08:34 | |
| *** c0m0 has joined #openstack-cinder | 08:35 | |
| *** ndipanov has joined #openstack-cinder | 08:40 | |
| *** jordanP has joined #openstack-cinder | 08:42 | |
| *** e0ne has quit IRC | 08:42 | |
| *** IanGovett1 has joined #openstack-cinder | 08:43 | |
| *** IanGovett has quit IRC | 08:44 | |
| openstackgerrit | Tina Tang proposed openstack/cinder: Attach/detach batch processing in VNX driver https://review.openstack.org/154350 | 08:45 |
| *** e0ne has joined #openstack-cinder | 08:46 | |
| *** aarefiev has joined #openstack-cinder | 08:47 | |
| *** pcaruana has joined #openstack-cinder | 08:50 | |
| *** e0ne has quit IRC | 08:51 | |
| *** e0ne has joined #openstack-cinder | 08:53 | |
| *** e0ne has quit IRC | 09:00 | |
| *** e0ne has joined #openstack-cinder | 09:04 | |
| *** ronis has joined #openstack-cinder | 09:04 | |
| *** zerda has quit IRC | 09:07 | |
| *** e0ne has quit IRC | 09:09 | |
| *** rushiagr_away is now known as rushiagr | 09:10 | |
| *** rongze has joined #openstack-cinder | 09:11 | |
| *** winston-d_ has quit IRC | 09:12 | |
| *** winston-d_ has joined #openstack-cinder | 09:14 | |
| *** ho_ has quit IRC | 09:14 | |
| *** ho has joined #openstack-cinder | 09:14 | |
| *** alecv has joined #openstack-cinder | 09:15 | |
| *** rongze has quit IRC | 09:15 | |
| *** rongze has joined #openstack-cinder | 09:15 | |
| *** lpetrut has quit IRC | 09:16 | |
| *** anshul has joined #openstack-cinder | 09:22 | |
| *** rongze_ has joined #openstack-cinder | 09:26 | |
| *** rongze has quit IRC | 09:26 | |
| *** winston-d_ has quit IRC | 09:26 | |
| openstackgerrit | wanghao proposed openstack/cinder-specs: Support query volume filter by glance metadata https://review.openstack.org/174778 | 09:27 |
| *** rongze_ has quit IRC | 09:28 | |
| *** rongze has joined #openstack-cinder | 09:28 | |
| *** rongze has quit IRC | 09:29 | |
| *** ho has quit IRC | 09:35 | |
| openstackgerrit | TaoBai proposed openstack/cinder: Storwize driver should only report active wwpn port https://review.openstack.org/174783 | 09:41 |
| openstackgerrit | yogeshprasad proposed openstack/cinder: Add chap support to CloudByte cinder driver https://review.openstack.org/173733 | 09:41 |
| openstackgerrit | Dave Chen proposed openstack/cinder: set/unset volume image metadata https://review.openstack.org/147726 | 09:42 |
| *** aix has joined #openstack-cinder | 09:45 | |
| *** heyun has quit IRC | 09:46 | |
| *** alecv has quit IRC | 09:48 | |
| *** jamielennox is now known as jamielennox|away | 09:49 | |
| *** e0ne has joined #openstack-cinder | 09:50 | |
| *** xyang has quit IRC | 09:57 | |
| *** winston-d_ has joined #openstack-cinder | 09:59 | |
| *** ozamiatin has quit IRC | 10:08 | |
| *** winston-1_ has joined #openstack-cinder | 10:10 | |
| *** winston-d_ has quit IRC | 10:10 | |
| *** ho has joined #openstack-cinder | 10:11 | |
| openstackgerrit | wanghao proposed openstack/cinder: Supprot for force-delete backups https://review.openstack.org/166127 | 10:12 |
| *** heyun has joined #openstack-cinder | 10:13 | |
| *** lpetrut has joined #openstack-cinder | 10:15 | |
| *** kmartin has quit IRC | 10:20 | |
| *** winston-1_ has quit IRC | 10:22 | |
| *** winston-d_ has joined #openstack-cinder | 10:22 | |
| *** heyun has quit IRC | 10:23 | |
| openstackgerrit | Tina Tang proposed openstack/cinder: Create consistgroup from cgsnapshot support in VNX driver https://review.openstack.org/163706 | 10:33 |
| *** e0ne is now known as e0ne_ | 10:38 | |
| openstackgerrit | wanghao proposed openstack/cinder: Supprot for force-delete backups https://review.openstack.org/166127 | 10:39 |
| *** fanyaohong has quit IRC | 10:39 | |
| *** e0ne_ is now known as e0ne | 10:42 | |
| *** zhenguo has quit IRC | 10:50 | |
| *** dulek_ has joined #openstack-cinder | 10:59 | |
| *** dulek has quit IRC | 11:02 | |
| *** dims__ has joined #openstack-cinder | 11:02 | |
| *** sks has joined #openstack-cinder | 11:05 | |
| *** aix has quit IRC | 11:05 | |
| *** annegentle has joined #openstack-cinder | 11:07 | |
| *** winston-d_ has quit IRC | 11:15 | |
| *** ishant has quit IRC | 11:22 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder-specs: Сhecking the existence of volume https://review.openstack.org/174823 | 11:25 |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: SMBFS: Fix retrieving total allocated size https://review.openstack.org/158713 | 11:36 |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: Windows SMBFS: Fix image resize errors during volume creation https://review.openstack.org/174291 | 11:36 |
| *** deepakcs has quit IRC | 11:40 | |
| *** julim has joined #openstack-cinder | 11:46 | |
| *** dulek_ has quit IRC | 11:46 | |
| *** ho has quit IRC | 11:48 | |
| *** dulek has joined #openstack-cinder | 11:49 | |
| *** anshul has quit IRC | 11:49 | |
| *** anshul has joined #openstack-cinder | 11:50 | |
| *** anshul has quit IRC | 11:50 | |
| *** anshul has joined #openstack-cinder | 11:51 | |
| *** cbits has joined #openstack-cinder | 11:52 | |
| *** cbits has left #openstack-cinder | 11:55 | |
| openstackgerrit | Yuriy Nesenenko proposed openstack/cinder-specs: Checking the existence of volume https://review.openstack.org/174823 | 11:56 |
| *** e0ne is now known as e0ne_ | 11:58 | |
| *** aix has joined #openstack-cinder | 12:03 | |
| *** e0ne_ has quit IRC | 12:08 | |
| *** e0ne has joined #openstack-cinder | 12:10 | |
| *** annegentle has quit IRC | 12:12 | |
| *** dulek has quit IRC | 12:19 | |
| *** abehl has joined #openstack-cinder | 12:20 | |
| *** anshul has quit IRC | 12:21 | |
| *** dulek has joined #openstack-cinder | 12:22 | |
| *** sks has quit IRC | 12:25 | |
| *** timcl has joined #openstack-cinder | 12:31 | |
| *** dalgaaf has quit IRC | 12:37 | |
| *** jaypipes has joined #openstack-cinder | 12:38 | |
| *** akerr has joined #openstack-cinder | 12:38 | |
| *** primechuck has quit IRC | 12:38 | |
| *** Miouge has quit IRC | 12:48 | |
| *** anshul has joined #openstack-cinder | 12:50 | |
| *** abehl has quit IRC | 12:52 | |
| *** cbader has joined #openstack-cinder | 12:53 | |
| *** xyang has joined #openstack-cinder | 12:55 | |
| *** xyang has quit IRC | 12:57 | |
| *** cbader has quit IRC | 13:00 | |
| *** xyang1 has joined #openstack-cinder | 13:04 | |
| *** Miouge has joined #openstack-cinder | 13:05 | |
| *** c0m0 has quit IRC | 13:07 | |
| *** marcusvrn has quit IRC | 13:19 | |
| *** mwichmann has left #openstack-cinder | 13:22 | |
| *** dustins has joined #openstack-cinder | 13:23 | |
| *** rushiagr is now known as rushiagr_away | 13:23 | |
| *** bill_az has joined #openstack-cinder | 13:24 | |
| *** jungleboyj has quit IRC | 13:25 | |
| *** mriedem has joined #openstack-cinder | 13:26 | |
| *** timcl has quit IRC | 13:26 | |
| *** markus_z has quit IRC | 13:29 | |
| *** dulek has quit IRC | 13:33 | |
| *** rushil has joined #openstack-cinder | 13:35 | |
| *** dansmith is now known as superdan | 13:35 | |
| *** Miouge has quit IRC | 13:36 | |
| *** timcl has joined #openstack-cinder | 13:41 | |
| *** dulek has joined #openstack-cinder | 13:42 | |
| *** anshul has quit IRC | 13:45 | |
| *** Mandell has joined #openstack-cinder | 13:46 | |
| *** anshul has joined #openstack-cinder | 13:46 | |
| *** anshul has quit IRC | 13:46 | |
| *** anshul has joined #openstack-cinder | 13:47 | |
| *** eharney has joined #openstack-cinder | 13:50 | |
| *** winston-d_ has joined #openstack-cinder | 13:55 | |
| *** nkrinner has quit IRC | 13:58 | |
| *** jaypipes is now known as leakypipes | 13:59 | |
| *** winston-d_ has quit IRC | 14:00 | |
| *** winston-d_ has joined #openstack-cinder | 14:00 | |
| openstackgerrit | Sean McGinnis proposed openstack/cinder: Logging not using oslo.i18n guidelines (scheduler) https://review.openstack.org/173368 | 14:01 |
| *** bnemec has joined #openstack-cinder | 14:03 | |
| *** mriedem has quit IRC | 14:03 | |
| *** thangp has joined #openstack-cinder | 14:04 | |
| *** rushil has quit IRC | 14:05 | |
| *** abhiram_moturi has quit IRC | 14:06 | |
| *** timcl has quit IRC | 14:06 | |
| *** mriedem has joined #openstack-cinder | 14:06 | |
| *** rushil has joined #openstack-cinder | 14:07 | |
| *** rushiagr_away is now known as rushiagr | 14:08 | |
| ameade | I have a fairly complicated fix for live vm migration with attached volumes for the NetApp E-Series driver that I want to get merged asap. Free beer to anyone who reviews it | 14:10 |
| ameade | https://review.openstack.org/#/c/170990/ | 14:10 |
| *** Lee1092 has quit IRC | 14:10 | |
| *** anshul has quit IRC | 14:11 | |
| *** e0ne is now known as e0ne_ | 14:11 | |
| *** bswartz has quit IRC | 14:13 | |
| *** e0ne_ is now known as e0ne | 14:13 | |
| *** deepakcs has joined #openstack-cinder | 14:17 | |
| DuncanT | ameade: I'm looking now. I suspect a better fix in the long term is to fix nova not to do multi-attach during migrate.... there's no fundamental need for it. | 14:18 |
| DuncanT | ameade: That isn't going to happen any time soon though | 14:18 |
| DuncanT | ameade: Does this fix mean you can only have 256 attached volumes on your e-series if you want live migration to work though? | 14:19 |
| ameade | DuncanT: that would be great, I had to do some really interested logic in the meantime. | 14:19 |
| ameade | DuncanT: yes | 14:19 |
| ameade | that limit should go up in later models so i have a plan to query for that number | 14:19 |
| *** Lee1092 has joined #openstack-cinder | 14:20 | |
| ameade | we tested the heck out of this fyi | 14:20 |
| ameade | DuncanT: if that option is turned on, we hard limit to 256 total volumes so we can guarantee that any created volume could be attached | 14:22 |
| DuncanT | ameade: Thanks for the clarification. PAtch looks good if a little messy. It's nicely restricted to the netapp drivers though, which is good | 14:23 |
| ameade | DuncanT: thanks for having a look, i'll get you a beer in vancouver. | 14:23 |
| deepakcs | eharney, ping | 14:25 |
| *** alexpilotti has quit IRC | 14:27 | |
| *** ronis has quit IRC | 14:28 | |
| eharney | deepakcs: hi | 14:28 |
| deepakcs | eharney, in the vol snap impr bp, do we really need to maintain old nova compatibility ? thats causing too much un-necessary code in cidner side | 14:31 |
| eharney | deepakcs: i'm pretty sure we do, unless there's a good reason to break it | 14:32 |
| eharney | at least for some overlap between releases | 14:33 |
| *** dims__ has quit IRC | 14:33 | |
| deepakcs | eharney, whats the reason for nova to be old and cinder to be new ? I mean wouldn't a distro carry them hand in hand ? | 14:33 |
| eharney | deepakcs: rolling upgrades for one | 14:34 |
| *** TobiasE has left #openstack-cinder | 14:34 | |
| deepakcs | eharney, i am not much aware of that, hence these Qs... do you mean cinder would be upgraded w/o nova ? | 14:35 |
| *** annegentle has joined #openstack-cinder | 14:35 | |
| eharney | i think in general we don't want upgrading one of them before the other to break functionality | 14:35 |
| deepakcs | eharney, so we need to take care of new nova with old cinder too then ? | 14:37 |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: Windows SMBFS: Fix image resize errors during volume creation https://review.openstack.org/174291 | 14:38 |
| DuncanT | rolling upgrade, bug fix deployments in a live datacentre | 14:38 |
| eharney | deepakcs: yes, we can generally do that i believe | 14:39 |
| DuncanT | Old cinder new nova and old nova new cinder both need to work | 14:39 |
| deepakcs | eharney, the bp spoke specifically abt old nova - new cinder only, but per DuncanT looks like we need to support both scenario, that means support for 4 combinations | 14:40 |
| deepakcs | thats going to add a lot of "if old... else new.. " kind of stuff in both nova/cinder | 14:41 |
| eharney | iirc you may be looking at more of that than is really necessary due to wanting to rename fields | 14:41 |
| *** mtanino has joined #openstack-cinder | 14:41 | |
| winston-d_ | deepakcs: yeah, fyi, we'll upgrade cinder first, then nova, 'cos cinder upgrade is easier and nova has more dependencies plus for nova, it's not just controller nodes, huge amount of hypervisors too. | 14:42 |
| DuncanT | We have been known to upgrade in either order, depending on why we're upgrading | 14:43 |
| DuncanT | Sometimes it's easier to add a second, new API and keep both working, rather than do something odd in one | 14:43 |
| deepakcs | eharney, rename fields ? you mean progress --> compute_complete kind of thing ? | 14:46 |
| *** timcl has joined #openstack-cinder | 14:46 | |
| eharney | i was thinking you wanted to rename some of them, if not, then not a concern | 14:46 |
| deepakcs | eharney, if u rename, then one of the above 4 combination won't work | 14:48 |
| deepakcs | depending on which side u rename | 14:48 |
| eharney | yes, which is why i'm thinking you shouldn't do that :) | 14:48 |
| deepakcs | eharney, and i am not doing it :) not sure why u got that impression | 14:48 |
| eharney | i thought i saw something in one of the patchsets that did, but i'll have to go look in more depth | 14:49 |
| deepakcs | eharney, np, i was only thinking on old nova - old/new cinder, but now need to look at both side old/new.. so will send more patches with that in mind | 14:50 |
| deepakcs | s/thinking on/thinking only | 14:50 |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: Windows SMBFS: Fix image resize errors during volume creation https://review.openstack.org/174291 | 14:50 |
| *** annegentle has quit IRC | 14:51 | |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: SMBFS: Fix retrieving total allocated size https://review.openstack.org/158713 | 14:51 |
| winston-d_ | ameade: about e-series live migration patch, what would happen if admin turn multattach config option on for a backend that already have > 256 vols? | 14:51 |
| *** annegentle has joined #openstack-cinder | 14:51 | |
| ameade | winston-d_: they wouldnt be able to create any more but could still do other operations, we just couldnt guarantee that they can attach them all until they get less that 256 | 14:53 |
| winston-d_ | ameade: would some of the live migration fail? if so, is the failure pattern deterministic? | 14:53 |
| winston-d_ | ameade: ok, what if the backend has > 256 vols and all of them attached, what would happen turning that config option on? | 14:54 |
| ameade | winston-d_: on a brownfield environment it isn't deterministic | 14:54 |
| ameade | winston-d_: it would just mean that some live migrations would work and others wouldnt | 14:54 |
| ameade | which is lame but no way around it | 14:55 |
| winston-d_ | ameade: first come first serve or stll non-deterministic? | 14:55 |
| ameade | winston-d_: unfortunately, non-deterministic...since we can't change LUN ids on the fly in eseries we would have collisions | 14:56 |
| winston-d_ | ameade: ok, I'd say please document these | 14:57 |
| ameade | winston-d_: yes definitely | 14:57 |
| winston-d_ | a DocImpact for your patch as well? | 14:57 |
| winston-d_ | what about other operations like snapshot? | 14:58 |
| winston-d_ | is # of snapshot also be limited in this case? | 14:58 |
| ameade | probably not needed since it is only for the driver, I already have a patch to update our docs (which are linked to from openstack-manuals) | 14:58 |
| winston-d_ | ameade: ok, that's good enough | 14:59 |
| ameade | winston-d_: no other operations should remain unaffected | 14:59 |
| ameade | no,* | 14:59 |
| ameade | lol | 14:59 |
| ameade | winston-d_: thanks for having a look and great questions | 14:59 |
| *** markvoelker has joined #openstack-cinder | 15:00 | |
| winston-d_ | one thing I think the driver may do, is to have some logic change in reporting backend stats to scheduler. | 15:00 |
| *** markvoelker has quit IRC | 15:00 | |
| *** markvoelker has joined #openstack-cinder | 15:00 | |
| winston-d_ | i.e. if multiattach config option is on, and # vols >= 256 already, report 0 or negative free capacity to scheduler, so that scheduler won't place new vols to the backend | 15:01 |
| *** vilobhmm1 has joined #openstack-cinder | 15:01 | |
| ameade | winston-d_: yes good idea, I guess currently it would have the same effect but would was a scheduler retry | 15:01 |
| winston-d_ | the current behavior would be, schedule put a vol to your backend, and fail, then reschedule | 15:01 |
| winston-d_ | ameade: right, but I think driver can be more proactive in such case. | 15:02 |
| ameade | sure i agree | 15:02 |
| ameade | let me think on that for a sec | 15:02 |
| DuncanT | winston-d_: No need to mess with the capacity, the filter function is designed to handle exactly this situation | 15:03 |
| winston-d_ | ameade: just a idea, not that current behavior breaks anything | 15:03 |
| winston-d_ | DuncanT: you mean backend supplied filter function? | 15:03 |
| DuncanT | winston-d_: Just return a filter function that is 'backend.volumes < 256)' | 15:03 |
| DuncanT | winston-d_: Yeah | 15:03 |
| *** dims__ has joined #openstack-cinder | 15:04 | |
| akerr | can't use capacity 0 because it prevent volume extend | 15:05 |
| ameade | it may cause issue if someone wants to extend a volume | 15:05 |
| ameade | yeah ^^ | 15:05 |
| ameade | would it make sense to add reporting for 'volume capacity'? in other words, how many more volumes can be placed on this backend? | 15:05 |
| winston-d_ | akerr, ameade scheduler was bypassed in vol extend case | 15:05 |
| akerr | when was that done. I remember coding up the extend function for NFS driver so that it had to check the free capacity and oversub ratios | 15:06 |
| winston-d_ | akerr, ameade, nobody knows how much capacity a backend has when doing extend, which is wrong | 15:06 |
| ameade | ah, so the plan is to change that? | 15:07 |
| akerr | nvm i answered my own question. If the driver is doing the checking then it is bypassing the scheduler :) | 15:07 |
| winston-d_ | ameade: yeah, in Paris I talked about having all opertions go through scheduler, even if the 'scheduling logic' isn't needed. | 15:08 |
| winston-d_ | I gonna revisit that in Vancover | 15:08 |
| ameade | DuncanT: ah i see what you are saying | 15:08 |
| winston-d_ | ameade: so DuncanT made a good point about using backend supplied filter/evaluate function for such case | 15:09 |
| ameade | DuncanT: does that filter it out for all operations? may have the same problem? | 15:09 |
| *** Maike_ has quit IRC | 15:09 | |
| winston-d_ | ameade: only create new vols, retype, migrate, managing existing vols go through scheduler | 15:10 |
| winston-d_ | ameade: other operations bypass scheduler, unfortunately. | 15:11 |
| akerr | winston-d_: but your idea would change that, correct? | 15:11 |
| winston-d_ | akerr: yeah | 15:12 |
| ameade | wouldn't want retypes and migrate to not work if the backend is full either | 15:12 |
| akerr | winston-d_: do we feel it has a strong chance of making it in? I'd hate to modify the logic here to work with current implementation only to have to change it again when the scheduler gets involved | 15:13 |
| winston-d_ | akerr: otherwise, things like reserved_percentage become totally useless when extend vol, clone vol bypass scheduler | 15:13 |
| ameade | if we wanted to represent the truth, we would need volume capacity. anything else is really slightly hacky | 15:13 |
| *** vilobhmm1 has quit IRC | 15:13 | |
| ameade | it may be good to do that anyways as i'm sure some backends have limits on the number of volumes itself | 15:14 |
| winston-d_ | akerr: sorry, make what in? nothing bypasses scheduler anymore or ameade's live migration fix with scheduler tweaks? | 15:14 |
| ameade | heck on lower end models, even without this patch the limit is 512 | 15:14 |
| akerr | winston-d_: your fix to push everything through the scheduler and prevent bypasses | 15:15 |
| winston-d_ | akerr: once we get consense, i'd do my best to make it in in L1 | 15:16 |
| *** jistr is now known as jistr|mtg | 15:16 | |
| winston-d_ | akerr: functionality-wise, that change is transparent to drivers. | 15:16 |
| ameade | k i have a plan | 15:16 |
| ameade | how about this patch as-is, i'll make a bp for reporting 'volume_number_capacity', and we implement that? | 15:17 |
| winston-d_ | akerr: in other words, I don't expect driver to be changed even we let everything go through scheduler | 15:17 |
| winston-d_ | ameade: yeah, that'll work, breaking them down is actaully a good idea. | 15:18 |
| *** dannywilson has joined #openstack-cinder | 15:18 | |
| ameade | yeah i think the real problem is outside of this patch, just a low limit makes it obvious | 15:18 |
| winston-d_ | after all, my suggestion is an enhancment, not hard requirement. | 15:18 |
| ameade | i'll comment this plan on the patch with some explanation as well | 15:19 |
| winston-d_ | ameade: sounds good | 15:19 |
| ameade | winston-d_: thanks for bringing that up, feels like productive conversation | 15:19 |
| *** harlowja_at_home has joined #openstack-cinder | 15:19 | |
| tbarron | winston-d_: DuncanT: I have a different scheduler question. Is 'CONF.scheduler_max_attempts = 1' supposed to work? | 15:21 |
| winston-d_ | tbarron: yeah, I think so | 15:21 |
| winston-d_ | tbarron: i think that simply means no retries | 15:22 |
| tbarron | hmm, http://paste.openstack.org/show/204474/ | 15:22 |
| tbarron | I get four retries :-) and the attempt counter doesn't increment. | 15:22 |
| *** dims__ has quit IRC | 15:22 | |
| tbarron | This is with recent master. | 15:22 |
| winston-d_ | interesting, got a bug #? | 15:23 |
| tbarron | I guess I'll file one. Just wanted to make sure that I wasn't missing something. | 15:23 |
| winston-d_ | sure, I need to dig into the code, haven't touch that for quite a while. | 15:25 |
| tbarron | It's the first time I tried to set this. Did so because we have automated tests that run negative vol create cases. | 15:25 |
| tbarron | When we see an exception, which is expected, we do a delete on the volume for cleanup. | 15:25 |
| tbarron | But the delete is running in the middle of the 3-attempt sequence. | 15:26 |
| tbarron | There is no lock around the whole sequence of three create attempts. | 15:26 |
| tbarron | And the volume state gets set from creating back to error between each attempt. | 15:27 |
| tbarron | So there is no exclusion of the delete by the overall create sequence. | 15:27 |
| tbarron | That kinda sounds like another bug to me. Opinions? | 15:27 |
| *** vilobhmm1 has joined #openstack-cinder | 15:28 | |
| tbarron | I was in any case trying to work around that issue with 'scheduler_max_attempts = 1' and found that it doesn't seem to be working either. | 15:29 |
| *** xyang has joined #openstack-cinder | 15:29 | |
| *** vilobhmm1 has quit IRC | 15:30 | |
| *** Mandell has quit IRC | 15:31 | |
| *** jdurgin1 has joined #openstack-cinder | 15:31 | |
| *** timcl has quit IRC | 15:32 | |
| winston-d_ | tbarron: i think there is a bug, in the exact case, when max_attempts set to 1 | 15:35 |
| tbarron | winston-d_: yeah, I'll file it in Launchpad unless you think that has already been done. | 15:36 |
| winston-d_ | tbarron: please file a bug, and I will see if I can get you a quick fix to test | 15:36 |
| tbarron | winston-d_: https://bugs.launchpad.net/cinder/+bug/1445561 | 15:42 |
| openstack | Launchpad bug 1445561 in Cinder "cinder scheduler fails to handle CONF.scheduler_max_attempts = 1 " [Undecided,New] | 15:42 |
| winston-d_ | tbarron: thx | 15:43 |
| *** kmartin has joined #openstack-cinder | 15:43 | |
| tbarron | winston-d_: going to lunch, bbiab | 15:43 |
| tbarron | winston-d_: thank you! | 15:43 |
| *** jistr|mtg is now known as jistr | 15:44 | |
| *** Longgeek_ has quit IRC | 15:44 | |
| *** jdurgin1 has quit IRC | 15:47 | |
| *** deepakcs has quit IRC | 15:49 | |
| *** ganso_ has joined #openstack-cinder | 15:49 | |
| *** esker has joined #openstack-cinder | 15:50 | |
| *** tsekiyama has joined #openstack-cinder | 15:51 | |
| *** mtecer has joined #openstack-cinder | 15:53 | |
| openstackgerrit | Anton Arefiev proposed openstack/cinder: Add missing backups entry to default quota class https://review.openstack.org/174941 | 15:57 |
| *** mtecer has quit IRC | 15:58 | |
| *** dulek has quit IRC | 15:58 | |
| *** thangp has quit IRC | 15:59 | |
| *** r-daneel has joined #openstack-cinder | 16:00 | |
| *** kbyrne has quit IRC | 16:00 | |
| *** hemna has joined #openstack-cinder | 16:01 | |
| *** annegentle has quit IRC | 16:05 | |
| *** adurbin_ has joined #openstack-cinder | 16:06 | |
| *** vilobhmm1 has joined #openstack-cinder | 16:07 | |
| *** harlowja_at_home has quit IRC | 16:09 | |
| *** pcaruana has quit IRC | 16:10 | |
| *** _cjones_ has joined #openstack-cinder | 16:11 | |
| *** e0ne has quit IRC | 16:12 | |
| *** vilobhmm11 has joined #openstack-cinder | 16:13 | |
| *** vilobhmm11 has quit IRC | 16:13 | |
| *** vilobhmm12 has joined #openstack-cinder | 16:13 | |
| *** vilobhmm12 has quit IRC | 16:13 | |
| *** vilobhmm11 has joined #openstack-cinder | 16:14 | |
| *** vilobhmm1 has quit IRC | 16:15 | |
| *** marcusvrn has joined #openstack-cinder | 16:17 | |
| *** mriedem1 has joined #openstack-cinder | 16:19 | |
| *** mriedem has quit IRC | 16:21 | |
| *** kmartin_ has joined #openstack-cinder | 16:21 | |
| *** kmartin has quit IRC | 16:21 | |
| *** marcusvrn1 has joined #openstack-cinder | 16:22 | |
| *** marcusvrn2 has joined #openstack-cinder | 16:22 | |
| *** marcusvrn has quit IRC | 16:23 | |
| *** marcusvrn3 has joined #openstack-cinder | 16:23 | |
| *** dims__ has joined #openstack-cinder | 16:24 | |
| *** marcusvrn1 has quit IRC | 16:26 | |
| *** marcusvrn2 has quit IRC | 16:26 | |
| *** patrickeast has joined #openstack-cinder | 16:27 | |
| *** thingee has joined #openstack-cinder | 16:27 | |
| *** timcl has joined #openstack-cinder | 16:29 | |
| *** leeantho has joined #openstack-cinder | 16:30 | |
| *** crose has joined #openstack-cinder | 16:37 | |
| *** mdbooth has quit IRC | 16:37 | |
| *** thingee has quit IRC | 16:37 | |
| *** lcurtis has joined #openstack-cinder | 16:38 | |
| tbarron | So is it expected that a delete can run against a volume that is being created? | 16:39 |
| *** thingee has joined #openstack-cinder | 16:39 | |
| tbarron | ^^^ this question may not be quite as naive as it sounds :-) | 16:39 |
| jgriffith | tbarron: no | 16:39 |
| winston-d_ | tbarron: no, if volume is in 'creating' status | 16:39 |
| tbarron | winston-d_: +1 | 16:39 |
| winston-d_ | tbarron: but it's error already, then you can delete it. | 16:40 |
| jgriffith | winston-d_: hey ya... I didn't quite follow your comment on that detach change? | 16:40 |
| jgriffith | winston-d_: I mean... yeah, it's in begin_detaching | 16:40 |
| tbarron | winston-d_: jgriffith: so when scheduler tries more than once, it does to error between the schedule attempts. | 16:40 |
| tbarron | If a delete comes in then, it gains its lock and tries to delete. | 16:41 |
| jgriffith | tbarron: sorry, not catching what the question is? | 16:41 |
| jgriffith | tbarron: well, that's valid if it's status is error (as winston-d_ stated) | 16:41 |
| tbarron | jgriffith: but the create can run at the same time then, right? | 16:42 |
| tbarron | creating -> error -> delete | 16:42 |
| jgriffith | tbarron: usually if it's error it *should* be done | 16:42 |
| tbarron | concurrent with creating -> error > creating | 16:42 |
| jgriffith | tbarron: if not that's a bug | 16:42 |
| jgriffith | tbarron: we shouldn't go from creating->error->creating in the same call | 16:43 |
| jgriffith | tbarron: that's bad | 16:43 |
| jgriffith | if it's doing that | 16:43 |
| tbarron | jgriffith: we by default retry the create | 16:43 |
| jgriffith | tbarron: yeah, that's fine | 16:43 |
| winston-d_ | jgriffith: so begin_detaching should already set volume to 'detaching' status, which means, 'detach' call should only look for 'detaching' | 16:43 |
| jgriffith | tbarron: but we shouldn't go to error until it's known that's not going to work | 16:43 |
| jgriffith | tbarron: see what I mean? | 16:43 |
| openstackgerrit | Huang Zhiteng proposed openstack/cinder: VolMgr: reschedule only when filter_properties has retry https://review.openstack.org/174956 | 16:43 |
| tbarron | jgriffith: but we do go to error | 16:44 |
| ameade | winston-d_, DuncanT: commented on https://review.openstack.org/#/c/170990/ | 16:44 |
| *** mdbooth has joined #openstack-cinder | 16:44 | |
| jgriffith | tbarron: and as I said, then that's a bug IMO | 16:44 |
| winston-d_ | tbarron: ^^ can you try this ifx? | 16:44 |
| tbarron | jgriffith: I will feed you this then, taskflow sets us to error | 16:44 |
| winston-d_ | jgriffith: agree, vol status shouldn't go error until done all retries | 16:44 |
| jgriffith | tbarron: then it's a bug in taskflow | 16:45 |
| winston-d_ | tbarron: then blame taskflow | 16:45 |
| winston-d_ | jgriffith: :) | 16:45 |
| jgriffith | winston-d_: :) | 16:45 |
| tbarron | talking to josh yesterday on this, he believes we should be putting a concurrency lock around the whole set of create attempts | 16:45 |
| winston-d_ | or those who migrate code to TF | 16:45 |
| tbarron | rather than relying on volume state to do the exclusion | 16:45 |
| ameade | more anti HA stuff | 16:46 |
| jgriffith | tbarron: that's a whole different philosophical argument | 16:46 |
| winston-d_ | the whole datacenter should have one gaint lock | 16:46 |
| jgriffith | tbarron: bottom line, setting state in the flow before it's *done* is wrong | 16:46 |
| jgriffith | winston-d_: LOL | 16:46 |
| winston-d_ | s/one/only one/ | 16:46 |
| jgriffith | we should just skip this whole cluster/cloud thing altogether | 16:47 |
| winston-d_ | every operation would require to accquire that lock to procceed | 16:47 |
| jgriffith | everyting on just one thread | 16:47 |
| ameade | GIL | 16:47 |
| ameade | but a global cloud lock | 16:47 |
| winston-d_ | yeah, cloud level GIL | 16:47 |
| tbarron | winston-d_: the s/G/D/GIL | 16:47 |
| tbarron | winston-d_: there was no lock barring us from saying that at the same time | 16:47 |
| jgriffith | tbarron: where's the flow manager setting the state to error? | 16:48 |
| jgriffith | tbarron: LOL | 16:48 |
| tbarron | jgriffith: gimme a minute | 16:48 |
| *** jistr has quit IRC | 16:48 | |
| * ameade goes back to lurking | 16:48 | |
| winston-d_ | hey, i think that's world peace solution, we should get nominated by Nobel price | 16:48 |
| winston-d_ | save lives for 67.5% of software engineers all over the world | 16:50 |
| *** dims__ is now known as dimsum__ | 16:50 | |
| jgriffith | winston-d_: haha! | 16:50 |
| * thingee is not looking forward to decisions at the next summit | 16:50 | |
| jgriffith | winston-d_: so about that change... the problem is we seem to have some "bugs" upstream on detach | 16:50 |
| jgriffith | winston-d_: where "begin_detach" isn't being called | 16:51 |
| jgriffith | winston-d_: so we slip directly to that "detach" call | 16:51 |
| *** leeantho has quit IRC | 16:51 | |
| jgriffith | winston-d_: then we get into hemna 's new multi-attach code that raises | 16:51 |
| jgriffith | winston-d_: things go "boom" | 16:51 |
| winston-d_ | jgriffith: which is fine, but 'detach' should always assume a well-behaved detaching operation should always call 'beging_detaching' first, which would then set volume status to 'detaching'. | 16:51 |
| thingee | jgriffith: :( | 16:52 |
| winston-d_ | jgriffith: but i have to admit, i don't know what would happen when multi-attach comes into play. | 16:52 |
| *** Mandell has joined #openstack-cinder | 16:52 | |
| thingee | jgriffith: all I'm going to say is I was more than happy to merge that right at the beginning of Liberty, but an overwhelming amount of people didn't believe me on that and wanted to deal with bugs last minute in the development cycle. | 16:53 |
| * thingee hides | 16:54 | |
| tbarron | jgriffith: cinder.volume.flows.common.error_out_volume | 16:54 |
| tbarron | in my volume log, looks like this: 2015-04-17 10:48:28.341 DEBUG cinder.volume.flows.common [req-f2ab6271-98a4-4d38-85e1-c561fa7fe711 None None] Updating volume: c058894a-363a-4eb6-aa76-3ab20781fe69 with {'status': 'error'} due to: ??? from (pid=24510) error_out_volume /opt/stack/cinder/cinder/volume/flows/common.py:89 | 16:55 |
| tbarron | update = { | 16:56 |
| tbarron | 'status': 'error', | 16:56 |
| tbarron | } | 16:56 |
| tbarron | db.volume_update(context, volume_id, update) | 16:56 |
| tbarron | Then we schedule another create attempt. | 16:56 |
| winston-d_ | tbarron: let's file a bug | 16:57 |
| *** annegentle has joined #openstack-cinder | 16:58 | |
| jgriffith | thingee: ? | 16:58 |
| tbarron | winston-d_: kk, will do. Just wanted to make sure I wasn't missing something obvkous. | 16:58 |
| thingee | jgriffith: multi attach bugs | 16:58 |
| tbarron | *obvious | 16:58 |
| jgriffith | thingee: no... my question is more like WTF? | 16:58 |
| thingee | jgriffith: not following | 16:58 |
| winston-d_ | tbarron: btw, if you have time, can you try https://review.openstack.org/174956 | 16:58 |
| jgriffith | "overwhelming amount of people didn't believe me on that" ??? | 16:59 |
| thingee | yeah, I said I wanted to merge it first thing in liberty | 16:59 |
| jgriffith | thingee: besides, it's actually not a bug in the multi-attach code | 16:59 |
| jgriffith | thingee: that code is doing exactly what it should expect | 16:59 |
| tbarron | winston-d_: I will try it in a minute. | 16:59 |
| jgriffith | thingee: the bug is that fact that we're going into that routine without checking beforehand | 16:59 |
| thingee | we just have some side effects on other projects, like the issue I raised yesterday with nova | 16:59 |
| winston-d_ | tbarron: thx | 16:59 |
| thingee | there hasn't been enough time to catch these issues | 16:59 |
| jgriffith | thingee: ok | 17:00 |
| thingee | which is why I wanted to delay to liberty | 17:00 |
| thingee | jgriffith: oh and also I figured out why my CI is just now seeing this issue :( | 17:00 |
| jgriffith | sorry, I'm not familiar with the "issues" you have found inparticular with Nova etc | 17:00 |
| thingee | jgriffith: it's what we were discussing yesterday | 17:00 |
| thingee | jgriffith: with the raise of invalidvolume | 17:00 |
| hemna | mornin | 17:00 |
| winston-d_ | hemna: mornin, early Saturday here. :) | 17:01 |
| hemna | :) | 17:01 |
| jgriffith | thingee: " jgriffith: oh and also I figured out why my CI is just now seeing this issue :(" | 17:02 |
| jgriffith | ^^ ? | 17:02 |
| hemna | jgriffith, so was there a bug filed on this one? | 17:02 |
| jgriffith | hemna: I filed one on the API not checking yeah | 17:03 |
| *** markvoelker has quit IRC | 17:03 | |
| hemna | the simple fix is to just ignore the bogus detach call and return no? | 17:03 |
| thingee | jgriffith: so my preseed image didn't have a reclone=true | 17:03 |
| jgriffith | https://review.openstack.org/#/c/174634/ | 17:03 |
| jgriffith | thingee: ahhh... | 17:03 |
| jgriffith | preseed images... bad | 17:03 |
| *** Apoorva has joined #openstack-cinder | 17:04 | |
| thingee | jgriffith: well, initially I thought had might nightly preseed image creator running + git pull in /opt/stack/* | 17:04 |
| thingee | jgriffith: greatly speeds things up and works...just if you remember to do those thing :( | 17:04 |
| thingee | jgriffith: last night I was spending time doing git bisect trying to figure when things stopped working for me | 17:05 |
| *** kmartin_ has quit IRC | 17:05 | |
| jgriffith | hemna: yeah | 17:05 |
| jgriffith | hemna: probably so | 17:05 |
| jgriffith | hemna: but it bothers me to do that | 17:06 |
| jgriffith | hemna: so I'm totally cool with doing that of course | 17:06 |
| jgriffith | hemna: but it bugs me that we issue invalid RPC calls down the chain like that | 17:07 |
| jgriffith | hemna: we should be "smarter" | 17:07 |
| jgriffith | :) | 17:07 |
| jgriffith | hemna: I abandoned what I had, if you want to submit something that just skips/returns that's cool | 17:07 |
| jgriffith | hemna: or if you want me to submit it that's cool too | 17:07 |
| hemna | it's a catch 22 kinda | 17:07 |
| jgriffith | hemna: yeah | 17:08 |
| hemna | it should throw invalid volume IMHO | 17:08 |
| hemna | and anyone that calls cinder needs to be careful of calling APIs, instead of ignoring exceptions that might come back | 17:08 |
| hemna | bleh | 17:08 |
| jgriffith | hemna: true story :) | 17:09 |
| *** leeantho has joined #openstack-cinder | 17:09 | |
| thingee | hemna: we still need to catch invalid volume from nova's perspective, as discussed yesterday | 17:09 |
| *** kmartin_ has joined #openstack-cinder | 17:09 | |
| thingee | hemna: when doing a detach and there are no attachments that exist | 17:11 |
| openstackgerrit | Walter A. Boring IV (hemna) proposed openstack/os-brick: Update os-brick requirements https://review.openstack.org/173402 | 17:12 |
| jgriffith | tbarron: winston-d_ so that's annoying :( | 17:12 |
| *** Mandell has quit IRC | 17:12 | |
| *** russellb has quit IRC | 17:14 | |
| tbarron | winston-d_: your fix works for me, i.e. I tested with it and now only see one create attempt when my driver throws an exception back to the manager. | 17:16 |
| *** saltsa has left #openstack-cinder | 17:16 | |
| winston-d_ | tbarron: great | 17:16 |
| *** harlowja_away is now known as harlowja | 17:17 | |
| *** russellb has joined #openstack-cinder | 17:17 | |
| winston-d_ | tbarron: thx for verifying. now I can get off and get some sleep | 17:19 |
| winston-d_ | you guys have a good day and nice weekend | 17:19 |
| *** winston-d_ is now known as winston-d_zZZ | 17:19 | |
| tbarron | winston-d_zZZ: sleep well! | 17:19 |
| *** Mandell has joined #openstack-cinder | 17:20 | |
| *** Mandell has quit IRC | 17:21 | |
| *** Mandell has joined #openstack-cinder | 17:21 | |
| *** russellb has quit IRC | 17:25 | |
| openstackgerrit | Petrut Lucian proposed openstack/cinder: Windows: Improve vhdutils error messages https://review.openstack.org/174973 | 17:28 |
| *** russellb has joined #openstack-cinder | 17:29 | |
| *** timcl has quit IRC | 17:30 | |
| harlowja | tbarron how's your investigation going | 17:34 |
| * harlowja let me know if u need any details about TF (or other) | 17:34 | |
| *** e0ne has joined #openstack-cinder | 17:37 | |
| *** timcl has joined #openstack-cinder | 17:37 | |
| *** mriedem1 is now known as mriedem | 17:38 | |
| tbarron | harlowja: well, we found (and winston-d_zZZ fixed) a bug wherein CONF.scheduler_max_attempts = 1 was being treated as infinite instead of as 1 :-) | 17:41 |
| harlowja | whoops | 17:42 |
| harlowja | :-/ | 17:42 |
| tbarron | harlowja: the remaining issue is what we were talking about yesterday. Turns out there isn't a consensus about how to exclude deletes from starting in the course of a sequence of retrying creates. | 17:42 |
| harlowja | hmmm, ya, if a delete comes in while a rescheduled create is on the RPC bus, then its hard to lock it there :-/ | 17:43 |
| tbarron | winston-d_zZZ: and jgriffith: believe that it is wrong to reset the volume state from creating to error between create attempts. | 17:43 |
| *** esker has quit IRC | 17:43 | |
| tbarron | and suggested that I file a bug on that. | 17:43 |
| *** jordanP has quit IRC | 17:43 | |
| harlowja | hmmm, what about new state 'creating-errored' | 17:44 |
| tbarron | yesterday, you on the other hand, IIRC, asserted that we should be setting a lock around the sequence as a way of doing the exclusion. | 17:44 |
| jgriffith | harlowja: why change state in the middle of an operation? | 17:44 |
| jgriffith | harlowja: so what I mean is... until we acknowledge defeat shouldn't it just stay in "creating" | 17:44 |
| harlowja | jgriffith fair enough, maybe not useful | 17:44 |
| jgriffith | harlowja: since it's still trying? | 17:44 |
| jgriffith | harlowja: well.. I think substates would be TREMENDOUSLY useful | 17:44 |
| harlowja | never acknowledge defeat solider!! | 17:45 |
| harlowja | 10-4 | 17:45 |
| jgriffith | :) | 17:45 |
| tbarron | I don't have as much experience on this as you guys but my bias is for the simplest solution possible. | 17:45 |
| *** emagana has joined #openstack-cinder | 17:45 | |
| *** e0ne is now known as e0ne_ | 17:45 | |
| harlowja | tbarron ya; i guess it becomes a question of which one is simplest, probably just not set the state to error (until defeat is acked) is simplest | 17:46 |
| tbarron | If there are independent reasons for substates, fine. But otherwise, keep it simple and don't add more machinery. | 17:46 |
| tbarron | harlowja: so will anything break if we just pull those lines that reset the state to error? | 17:46 |
| harlowja | sure, i'd be nice to have create-attempt-1 or create-attempt-2 kind of states, but i guess thats more of a nice to have... | 17:47 |
| harlowja | let me see | 17:47 |
| * harlowja thought that was a shared function, so probably, lol | 17:47 | |
| tbarron | cinder.volume.flows.common.error_out_volume | 17:48 |
| *** annegentle has quit IRC | 17:50 | |
| *** e0ne_ is now known as e0ne | 17:52 | |
| *** Maike has joined #openstack-cinder | 17:53 | |
| harlowja | tbarron so what i'd try is to use the information @ https://github.com/openstack/cinder/blob/master/cinder/volume/flows/manager/create_volume.py#L170 and have that conditionally stop error_out_volume from being triggered | 17:53 |
| *** annegentle has joined #openstack-cinder | 17:54 | |
| harlowja | and maybe log a warning instead of activating error_out_volume and just let it be (and then when rescheduling stops this will really enter error) | 17:55 |
| tbarron | harlowja: that looks like a good approach to me. | 17:56 |
| *** e0ne is now known as e0ne_ | 17:57 | |
| harlowja | sweet | 17:58 |
| harlowja | if anyone is interested in y! stuff + ceph ( http://yahooeng.tumblr.com/post/116391291701/yahoo-cloud-object-store-object-storage-at ) <--- current object store kind of stuff (volume storage post someday...) | 18:04 |
| tbarron | harlowja: I raised https://bugs.launchpad.net/cinder/+bug/1445601 for this. | 18:04 |
| openstack | Launchpad bug 1445601 in Cinder "cinder is putting volume state to error while retrying cinder creates" [Undecided,New] | 18:04 |
| *** esker has joined #openstack-cinder | 18:04 | |
| harlowja | tbarron cool | 18:04 |
| *** aix has quit IRC | 18:06 | |
| *** crose has quit IRC | 18:08 | |
| *** lpetrut has left #openstack-cinder | 18:12 | |
| openstackgerrit | Vilobh Meshram proposed openstack/cinder: Driver get_stats refresh arg is useless https://review.openstack.org/174993 | 18:16 |
| *** annegentle has quit IRC | 18:21 | |
| *** annegentle has joined #openstack-cinder | 18:25 | |
| *** leakypipes has quit IRC | 18:25 | |
| *** rushiagr is now known as rushiagr_away | 18:26 | |
| *** dustins_ has joined #openstack-cinder | 18:26 | |
| *** timcl has quit IRC | 18:27 | |
| *** annashen has joined #openstack-cinder | 18:27 | |
| *** Mandell has quit IRC | 18:28 | |
| *** dustins has quit IRC | 18:29 | |
| *** jungleboyj has joined #openstack-cinder | 18:29 | |
| *** mtecer has joined #openstack-cinder | 18:30 | |
| *** e0ne_ is now known as e0ne | 18:30 | |
| *** Mandell has joined #openstack-cinder | 18:31 | |
| mtreinish | thingee: so http://logs.openstack.org/16/165616/2/check/check-tempest-dsvm-full/936bc61/console.html.gz#_2015-03-19_02_41_35_282 was the failure before with tempest detaching before deletes | 18:32 |
| mtreinish | thingee: hemna left the -1 to indicate that the detach was no longer needed | 18:32 |
| thingee | mtreinish: that's exactly the issue I'm hitting..security group in use | 18:33 |
| mtreinish | thingee: well the sec group thing is a follow on failure, because the server delete failed | 18:33 |
| mtreinish | so the sec groups are still in use when tempest goes to delete them in cleanup | 18:33 |
| thingee | mtreinish: alright, you wouldn't happen to have the related nova change to do the detach? | 18:34 |
| mtreinish | nope, sry. That's probably over an even bigger date range, because the lvm bug which caused to add the skip was outstanding for some time | 18:35 |
| mtreinish | thingee: although since hemna left the comment I want to say it might have been related to the multi-attach stuff | 18:35 |
| *** jungleboyj has quit IRC | 18:36 | |
| mtreinish | thingee: yeah it's on the cinder side, the server delete fails when calling to cinder returns a 500 | 18:39 |
| mtreinish | thingee: and then in the cinder logs it has: http://logs.openstack.org/16/165616/2/check/check-tempest-dsvm-neutron-full/6862961/logs/screen-c-vol.txt.gz?level=TRACE | 18:39 |
| thingee | mtreinish: I'll add a catch in nova and see if that resolves the problem | 18:40 |
| thingee | still not exactly sure why this is not reproducible in gate | 18:40 |
| mtreinish | thingee: I'm wondering if it's the multi-attach patches which changed the detach behavior if the volume isn't attached anywhere | 18:41 |
| mtreinish | like what used to happen if we issued detach and it wasn't attached (pre multi-attach) | 18:41 |
| thingee | hemna: ^ | 18:42 |
| mtreinish | because that cinder traceback is clearly the len(attachments) in cinder is 0 so getting attachment[0] will fail | 18:42 |
| thingee | yes | 18:42 |
| mtreinish | although that might have been fixed since hemna left the original comment on the unskip patch | 18:42 |
| thingee | alright I'll test it in my environment and propose a patch if it helps | 18:42 |
| *** winston-d_zZZ has quit IRC | 18:43 | |
| *** lpetrut has joined #openstack-cinder | 18:44 | |
| *** annashen has quit IRC | 18:47 | |
| thingee | mtreinish: thanks for your help as always. I have relatives visiting so might have to be later that I post results. unless hemna wants to post a patch to catch things on the nova side when invalidvolume is raised. | 18:51 |
| *** jungleboyj has joined #openstack-cinder | 18:51 | |
| thingee | that's too bad multi attach caused issues in the tempest test though | 18:51 |
| thingee | this has been quite a headache for me. | 18:52 |
| thingee | re https://github.com/openstack/tempest/commit/f557d37a9d46aa7dd137368c7147b5696f5cbe94 | 18:52 |
| mtreinish | thingee: sure, np. I'm not sure if I'll be around later today or not | 18:52 |
| *** akerr has quit IRC | 18:53 | |
| mtreinish | but I'll probably check my bouncer backlog at some point | 18:53 |
| thingee | mtreinish: no worries. we'll pick back up next week. | 18:53 |
| thingee | mtreinish: fwiw, I didn't want this merging late in kilo. there was just an overwhelming amount of push on multi-attach in late kilo, rather than merging first thing in liberty. | 18:54 |
| thingee | just felt like there wasn't enough time to gate on it | 18:54 |
| mtreinish | thingee: sure, I can understand that | 18:54 |
| mtreinish | heh, let's just blame jgriffith because this test was only ever skipped because of an lvm setup bug :) | 18:55 |
| thingee | poor jgriffith ... I'd rather blame david wang | 18:55 |
| * thingee hopes to meet david wang some day | 18:55 | |
| thingee | mystery person to propose summit sessions, we argue, and he never shows up. HA | 18:56 |
| thingee | alright bbl | 18:57 |
| mtreinish | heh, it's great when that happens | 18:57 |
| *** e0ne has quit IRC | 18:58 | |
| *** jungleboyj has quit IRC | 19:00 | |
| *** marcusvrn3 has quit IRC | 19:00 | |
| *** Mandell has quit IRC | 19:10 | |
| *** jungleboyj has joined #openstack-cinder | 19:13 | |
| *** Maike has quit IRC | 19:13 | |
| *** hemna has quit IRC | 19:15 | |
| *** jungleboyj has quit IRC | 19:19 | |
| *** annegentle has quit IRC | 19:19 | |
| *** bnemec is now known as beekneemech | 19:22 | |
| *** dustins_ has quit IRC | 19:27 | |
| *** hemna has joined #openstack-cinder | 19:40 | |
| openstackgerrit | hadi esiely proposed openstack/cinder: Store volume encryption metadata on each volume https://review.openstack.org/152284 | 19:45 |
| openstackgerrit | hadi esiely proposed openstack/cinder: Add test case for volume_encryption_metadata_get https://review.openstack.org/168177 | 19:45 |
| openstackgerrit | hadi esiely proposed openstack/cinder: Remove unnecessary checks for encrypted types https://review.openstack.org/168179 | 19:45 |
| *** Mandell has joined #openstack-cinder | 19:47 | |
| *** annashen has joined #openstack-cinder | 19:48 | |
| *** setmason has joined #openstack-cinder | 19:50 | |
| *** timcl has joined #openstack-cinder | 19:52 | |
| *** annashen has quit IRC | 19:53 | |
| *** patrickeast has quit IRC | 19:53 | |
| *** ronis has joined #openstack-cinder | 19:55 | |
| *** annashen has joined #openstack-cinder | 20:00 | |
| *** ndipanov has quit IRC | 20:13 | |
| *** Mandell has quit IRC | 20:16 | |
| *** openstackgerrit has quit IRC | 20:22 | |
| *** openstackgerrit has joined #openstack-cinder | 20:23 | |
| *** timcl has quit IRC | 20:24 | |
| *** ronis has quit IRC | 20:26 | |
| *** annashen has quit IRC | 20:28 | |
| *** Mandell has joined #openstack-cinder | 20:29 | |
| *** annashen has joined #openstack-cinder | 20:32 | |
| *** annashen_ has joined #openstack-cinder | 20:44 | |
| *** annashen has quit IRC | 20:44 | |
| *** emagana has quit IRC | 20:45 | |
| *** emagana has joined #openstack-cinder | 20:47 | |
| jgriffith | mtreinish: haah... FTR, there were actually a BOAT load of issues with that test including networking. Cinder/LVM just got tagged as the last straw :) | 20:47 |
| *** Lee1092 has quit IRC | 20:50 | |
| openstackgerrit | John Griffith proposed openstack/cinder: Standardize logging in volume.api.py https://review.openstack.org/175054 | 20:53 |
| *** Mandell has quit IRC | 20:55 | |
| *** geguileo has quit IRC | 20:56 | |
| openstackgerrit | Walter A. Boring IV (hemna) proposed openstack/os-brick: Brick: Fix race in removing iSCSI device https://review.openstack.org/168506 | 20:58 |
| *** vilobhmm11 has quit IRC | 20:58 | |
| *** logan2 has quit IRC | 20:59 | |
| *** logan2 has joined #openstack-cinder | 21:00 | |
| *** vilobhmm1 has joined #openstack-cinder | 21:00 | |
| *** vilobhmm1 has quit IRC | 21:00 | |
| *** vilobhmm1 has joined #openstack-cinder | 21:01 | |
| *** esker has quit IRC | 21:02 | |
| -openstackstatus- NOTICE: Gerrit will be unavailable between 22:00 and 23:59 UTC for project renames and a database update. | 21:04 | |
| *** Apoorva has quit IRC | 21:06 | |
| *** mriedem has quit IRC | 21:07 | |
| *** openstackgerrit has quit IRC | 21:23 | |
| *** openstackgerrit has joined #openstack-cinder | 21:23 | |
| *** Mandell has joined #openstack-cinder | 21:26 | |
| *** kfox1111 has joined #openstack-cinder | 21:30 | |
| kfox1111 | got a weird case where I have a volume I can't detach. nova volume detach spits out a no volume found, though its there, shows up all correctly in cinder list and is on /dev/vdb on the vm. | 21:31 |
| *** erlon has quit IRC | 21:31 | |
| kfox1111 | Is there a way to force it into in-use state? | 21:31 |
| kfox1111 | or should I force it available, then reattach it so I can detach it again? | 21:32 |
| *** emagana has quit IRC | 21:33 | |
| *** annegentle has joined #openstack-cinder | 21:34 | |
| *** vilobhmm1 has quit IRC | 21:35 | |
| *** annashen_ has quit IRC | 21:36 | |
| kfox1111 | ERROR (NotFound): volume_id not found: 491a9df5-834b-41bd-8d5c-6fba288c8c53 (HTTP 404) (Request-ID: req-bdab05fd-7522-4cbc-9438-16e7c2a4a0ce) | 21:36 |
| *** vilobhmm1 has joined #openstack-cinder | 21:37 | |
| *** annashen has joined #openstack-cinder | 21:37 | |
| kfox1111 | | 491a9df5-834b-41bd-8d5c-6fba288c8c53 | in-use | cybervis-volume-new | 5 | None | false | d305f6e2-ffd5-46b8-bd03-4601a4cc151e | | 21:37 |
| kfox1111 | I don't get it. :/ | 21:37 |
| *** emagana has joined #openstack-cinder | 21:41 | |
| *** emagana has quit IRC | 21:41 | |
| *** emagana has joined #openstack-cinder | 21:42 | |
| *** setmason has left #openstack-cinder | 21:44 | |
| *** jamielennox|away is now known as jamielennox | 21:48 | |
| *** markvoelker has joined #openstack-cinder | 21:49 | |
| *** markvoelker_ has joined #openstack-cinder | 21:50 | |
| *** vilobhmm1 has quit IRC | 21:51 | |
| *** markvoelker has quit IRC | 21:54 | |
| *** mtecer has quit IRC | 21:55 | |
| *** markvoelker_ has quit IRC | 21:58 | |
| -openstackstatus- NOTICE: Gerrit is unavailable until 23:59 UTC for project renames and a database update. | 22:04 | |
| -openstackstatus- NOTICE: Gerrit is unavailable until 23:59 UTC for project renames and a database update. | 22:06 | |
| *** ChanServ changes topic to "Gerrit is unavailable until 23:59 UTC for project renames and a database update." | 22:06 | |
| *** annegentle has quit IRC | 22:06 | |
| *** vilobhmm1 has joined #openstack-cinder | 22:08 | |
| *** annashen has quit IRC | 22:15 | |
| *** annashen has joined #openstack-cinder | 22:16 | |
| kfox1111 | any ideas how to force it to disconnect properly? | 22:22 |
| *** vilobhmm1 has quit IRC | 22:27 | |
| *** vilobhmm1 has joined #openstack-cinder | 22:28 | |
| *** bswartz has joined #openstack-cinder | 22:30 | |
| *** annegentle has joined #openstack-cinder | 22:38 | |
| *** Mandell has quit IRC | 22:41 | |
| *** winston-d_zZZ has joined #openstack-cinder | 22:41 | |
| *** annashen has quit IRC | 22:41 | |
| *** Mandell_ has joined #openstack-cinder | 22:43 | |
| *** winston-d_zZZ has quit IRC | 22:46 | |
| *** rushil has quit IRC | 22:46 | |
| *** lcurtis has quit IRC | 22:47 | |
| *** IanGovett1 has quit IRC | 22:48 | |
| *** annegentle has quit IRC | 22:49 | |
| *** annashen has joined #openstack-cinder | 22:54 | |
| *** ChanServ changes topic to "The OpenStack Block Storage Project Cinder | The New Kids On the Block | https://wiki.openstack.org/wiki/Cinder" | 23:04 | |
| -openstackstatus- NOTICE: Gerrit is available again. | 23:04 | |
| *** patrickeast has joined #openstack-cinder | 23:10 | |
| *** lpetrut has quit IRC | 23:14 | |
| *** annashen has quit IRC | 23:15 | |
| *** ganso_ has quit IRC | 23:17 | |
| *** jamielennox is now known as jamielennox|away | 23:30 | |
| *** patrickeast has quit IRC | 23:50 | |
| *** hemna has quit IRC | 23:52 | |
| *** annegentle has joined #openstack-cinder | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!