Wednesday, 2015-02-18

*** asmith_brcd has quit IRC00:02
*** ho has joined #openstack-cinder00:07
anishjgriffith: annegentle found a bug in cinder that breaks autohelp-wrapper00:07
anishfix http://ix.io/got00:07
*** ebalduf has joined #openstack-cinder00:07
anishdoes the fix seem right, or is something broken on my side ?00:08
jgriffithanish: looking....00:08
jgriffithanish: bahhh!00:08
jgriffithanish: yes00:08
jgriffithanish: I'm surprised pep8 doesn't barf on that?00:08
* anish raises hands00:08
anishjgriffith: do you want me to file a bug or is direct patch okay ?00:09
jgriffithanish: that one I'm personally fine w/ just a patch00:09
openstackgerritAnish Bhatt proposed openstack/cinder: Fix extraneous comma that breaks docbook generation  https://review.openstack.org/15685100:12
*** Apoorva_ has quit IRC00:16
*** Apoorva has joined #openstack-cinder00:17
jgriffithanish: annegentle Thanks!!00:19
*** lcurtis has quit IRC00:21
*** patrickeast has quit IRC00:26
*** r-daneel has quit IRC00:26
*** jaypipes has quit IRC00:27
anishannegentle: the docbook generated a lot more changes in flagmappings than the ones I wanted to fix. Is the preferred approach to do all of them at once, or only do the ones relevant to my bug ?00:30
*** patrickeast has joined #openstack-cinder00:31
*** jwcroppe has joined #openstack-cinder00:35
*** markvoelker has quit IRC00:37
*** markvoelker has joined #openstack-cinder00:37
*** smoriya has joined #openstack-cinder00:39
*** markvoelker has quit IRC00:42
jgriffithanish: I think you should file a general bug (fix flagmappings....) and hit them in one bug like that00:44
*** david-lyle has quit IRC00:44
anishjgriffith: gotcha00:47
*** sgotliv has quit IRC00:47
*** IanGovett has quit IRC00:54
*** jasondotstar has quit IRC00:55
*** garthb has quit IRC00:58
*** garthb_ has quit IRC00:58
openstackgerritWalter A. Boring IV (hemna) proposed openstack/os-brick: Remove some unused exceptions from Cinder  https://review.openstack.org/15686301:01
*** jordandh has quit IRC01:01
openstackgerritWalter A. Boring IV (hemna) proposed openstack/cinder: WIP: Remove Brick from cinder codebase  https://review.openstack.org/15555201:02
*** Mandell has quit IRC01:06
*** jordandh has joined #openstack-cinder01:07
*** hemna is now known as hemnafk01:08
*** ebalduf has quit IRC01:14
*** annegentle has quit IRC01:18
*** zzzeek has quit IRC01:22
*** Mandell has joined #openstack-cinder01:23
anishhemnafk: what's the new location of https://review.openstack.org/155552 ?01:24
*** MentalRay has joined #openstack-cinder01:28
*** zzzeek has joined #openstack-cinder01:31
MentalRayIm curious to know if people using ceph were able to resize and live migrate without issues on icehouse01:34
*** scottda_ has joined #openstack-cinder01:36
*** vilobhmm has quit IRC01:37
*** ybathia has quit IRC01:37
*** scottda_ has quit IRC01:37
*** sigmavirus24 is now known as sigmavirus24_awa01:38
*** scottda_ has joined #openstack-cinder01:39
*** jordandh has quit IRC01:39
*** rmesta has quit IRC01:40
*** jordandh has joined #openstack-cinder01:41
*** leeantho has quit IRC01:47
*** IanGovett has joined #openstack-cinder01:48
*** annashen has quit IRC01:51
*** rwsu is now known as rwsu-afk01:54
*** BharatK has joined #openstack-cinder01:56
*** scottda_ has quit IRC01:59
*** markvoelker has joined #openstack-cinder01:59
*** scottda_ has joined #openstack-cinder02:01
*** Mandell has quit IRC02:01
*** Mandell has joined #openstack-cinder02:08
*** briancline has quit IRC02:09
*** scottda_ has quit IRC02:09
*** briancline has joined #openstack-cinder02:09
*** changbl has joined #openstack-cinder02:10
*** david-lyle has joined #openstack-cinder02:11
openstackgerritJeegn Chen proposed openstack/cinder: Fix return value inconsistency in VNX Driver  https://review.openstack.org/15647002:14
*** jordandh has quit IRC02:14
*** mriedem has joined #openstack-cinder02:14
*** ebalduf has joined #openstack-cinder02:15
*** mriedem has joined #openstack-cinder02:16
*** xyang1 has quit IRC02:17
*** _cjones_ has quit IRC02:18
*** jordandh has joined #openstack-cinder02:20
*** ebalduf has quit IRC02:20
*** takedakn has joined #openstack-cinder02:24
*** takedakn has quit IRC02:26
*** takedakn has joined #openstack-cinder02:26
*** tsekiyam_ has joined #openstack-cinder02:33
*** tsekiyama has quit IRC02:37
*** tsekiyam_ has quit IRC02:38
*** mtanino has quit IRC02:43
*** xyang1 has joined #openstack-cinder02:43
openstackgerritJeegn Chen proposed openstack/cinder: Pool-aware scheduler support in EMC VNX Cinder driver  https://review.openstack.org/14740802:44
*** esker has joined #openstack-cinder02:48
*** takedakn has quit IRC02:48
*** I has joined #openstack-cinder02:50
*** I is now known as Guest2224602:50
openstackgerritJeegn Chen proposed openstack/cinder: Pool-aware scheduler support in EMC VNX Cinder driver  https://review.openstack.org/14740802:50
*** Guest22246 is now known as hemna_02:51
*** lcurtis has joined #openstack-cinder02:52
hemna_anyone have the k3 priorities ether pad url handy?02:52
*** casusbel_ has joined #openstack-cinder02:53
*** mwichmann has quit IRC02:54
*** kaisers1 has joined #openstack-cinder02:54
*** kaisers has quit IRC02:55
*** Apoorva_ has joined #openstack-cinder02:55
bswartzhttps://etherpad.openstack.org/p/cinder-k3-priorities02:55
bswartzhemna_: ^02:55
hemna_bswartz, thanks.   I didn't have it here at home02:56
*** casusbelli has quit IRC02:57
*** Apoorva has quit IRC02:58
*** Apoorva_ has quit IRC03:00
*** hemna__ has joined #openstack-cinder03:00
*** hemna_ has quit IRC03:00
*** jordandh has quit IRC03:01
*** akerr_ is now known as akerr_away03:06
*** patrickeast has quit IRC03:06
anishbswartz: chiscsi support shows up twice in that, intentional ?03:10
*** lcurtis has quit IRC03:10
bswartzit's marked merged, who cares?03:11
anishalso true03:12
*** boris-42 has quit IRC03:12
*** jordandh has joined #openstack-cinder03:12
*** esker has quit IRC03:14
*** ebalduf has joined #openstack-cinder03:17
*** hemna__ has quit IRC03:20
*** ebalduf has quit IRC03:21
*** Mandell has quit IRC03:21
*** harlowja_ is now known as harlowja_away03:22
*** IanGovett has quit IRC03:23
*** lcurtis has joined #openstack-cinder03:23
*** esker has joined #openstack-cinder03:24
*** kaufer has joined #openstack-cinder03:27
*** rushiagr_away is now known as rushiagr03:28
*** david-lyle has quit IRC03:35
*** david-lyle has joined #openstack-cinder03:35
*** Guest55371 has joined #openstack-cinder03:35
*** Guest55371 is now known as annegentle03:36
*** Mandell has joined #openstack-cinder03:38
*** david-lyle has quit IRC03:40
*** lcurtis has quit IRC03:41
*** mriedem has left #openstack-cinder03:42
*** coolsvap_ is now known as coolsvap03:43
*** tsekiyama has joined #openstack-cinder03:44
*** ljou has quit IRC03:44
*** tsekiyama has quit IRC03:48
*** harlowja_at_home has joined #openstack-cinder03:50
*** annashen has joined #openstack-cinder03:51
*** annashen has quit IRC03:52
*** annashen has joined #openstack-cinder03:52
*** MentalRay has quit IRC03:57
*** esker has quit IRC04:03
*** BharatK has quit IRC04:10
*** annashen has quit IRC04:15
*** ebalduf has joined #openstack-cinder04:17
*** takedakn has joined #openstack-cinder04:19
*** harlowja_at_home has quit IRC04:21
*** david-lyle has joined #openstack-cinder04:22
*** ho has quit IRC04:22
*** ebalduf has quit IRC04:22
*** david-lyle is now known as david-lyle_afk04:23
*** ho has joined #openstack-cinder04:25
*** BharatK has joined #openstack-cinder04:27
*** takedakn has quit IRC04:33
*** avishay_ has joined #openstack-cinder04:42
*** annashen has joined #openstack-cinder04:42
*** esker has joined #openstack-cinder04:44
openstackgerritAmit Saha proposed openstack/cinder: Fixed typo  https://review.openstack.org/15521304:45
*** esker has quit IRC04:45
*** resker has joined #openstack-cinder04:47
openstackgerritSteven Kaufer proposed openstack/cinder: Replication status periodic task optimization  https://review.openstack.org/15467304:50
*** sgotliv has joined #openstack-cinder04:52
*** annashen has quit IRC04:57
*** shilpa_ has joined #openstack-cinder04:59
*** resker has quit IRC05:00
*** ho__ has joined #openstack-cinder05:01
*** ho has quit IRC05:02
*** _cjones_ has joined #openstack-cinder05:04
*** kaufer has quit IRC05:06
*** dalgaaf has quit IRC05:07
*** jungleboyj has joined #openstack-cinder05:14
*** _cjones_ has quit IRC05:38
*** _cjones_ has joined #openstack-cinder05:39
*** zzzeek has quit IRC05:43
*** ronenkat has joined #openstack-cinder05:50
*** esker has joined #openstack-cinder05:51
*** deepakcs has joined #openstack-cinder06:00
*** markvoelker has quit IRC06:03
*** markvoelker has joined #openstack-cinder06:04
*** anshul has joined #openstack-cinder06:06
*** markvoelker has quit IRC06:08
openstackgerritOpenStack Proposal Bot proposed openstack/cinder: Imported Translations from Transifex  https://review.openstack.org/15240106:11
*** xyang has quit IRC06:13
*** jamielennox is now known as jamielennox|away06:17
*** nkrinner has joined #openstack-cinder06:17
*** casusbel_ has quit IRC06:30
*** casusbelli has joined #openstack-cinder06:31
*** lpetrut has joined #openstack-cinder06:34
*** BharatK has quit IRC06:38
*** Raj_ has joined #openstack-cinder06:43
*** BharatK has joined #openstack-cinder06:52
*** markvoelker has joined #openstack-cinder06:53
openstackgerrityatin proposed openstack/python-cinderclient: Added options that can be used while uploading volume to glance  https://review.openstack.org/15246006:55
*** markvoelker has quit IRC06:58
*** Ilja has joined #openstack-cinder06:59
*** Ilja has quit IRC07:00
*** Ilja has joined #openstack-cinder07:00
*** ronenkat has quit IRC07:01
*** Ilja has quit IRC07:01
*** Ilja has joined #openstack-cinder07:02
*** nshaikh has joined #openstack-cinder07:06
*** takedakn has joined #openstack-cinder07:10
*** leopoldj has joined #openstack-cinder07:19
*** vilobhmm has joined #openstack-cinder07:20
*** sgotliv has quit IRC07:21
*** vilobhmm has quit IRC07:21
*** _cjones_ has quit IRC07:23
*** _cjones_ has joined #openstack-cinder07:24
*** Mandell has quit IRC07:25
*** Mandell has joined #openstack-cinder07:25
openstackgerritVipin Balachandran proposed openstack/cinder: VMware: Skip vSAN for preallocated image download  https://review.openstack.org/15690407:26
*** _cjones_ has quit IRC07:28
*** Yogi1 has quit IRC07:40
*** esker has quit IRC07:41
*** lpetrut has quit IRC07:45
*** jwcroppe has quit IRC07:46
*** BharatK has quit IRC07:47
*** takedakn has quit IRC07:48
*** Raj_ has quit IRC07:49
*** Raj_ has joined #openstack-cinder07:52
*** afazekas has joined #openstack-cinder07:54
*** markvoelker has joined #openstack-cinder07:54
*** takedakn has joined #openstack-cinder07:58
*** markvoelker has quit IRC07:59
*** boris-42 has joined #openstack-cinder08:00
*** alecv has joined #openstack-cinder08:01
*** BharatK has joined #openstack-cinder08:01
*** hodos has joined #openstack-cinder08:02
*** hodos has quit IRC08:03
*** takedakn has quit IRC08:07
*** jistr has joined #openstack-cinder08:08
*** chlong has quit IRC08:10
*** ronis has joined #openstack-cinder08:11
*** TobiasE has joined #openstack-cinder08:15
*** takedakn has joined #openstack-cinder08:15
*** emaisne has joined #openstack-cinder08:15
*** rushiagr is now known as rushiagr_away08:21
*** lpetrut has joined #openstack-cinder08:23
*** avishay_ has quit IRC08:24
*** avishay__ has joined #openstack-cinder08:24
*** jwcroppe has joined #openstack-cinder08:25
*** ronenkat has joined #openstack-cinder08:25
*** rushiagr_away is now known as rushiagr08:28
*** sgotliv has joined #openstack-cinder08:30
*** ho__ has quit IRC08:37
*** akerr has joined #openstack-cinder08:38
*** eharney has quit IRC08:40
*** akerr_away has quit IRC08:40
*** eharney has joined #openstack-cinder08:41
*** Ilja has quit IRC08:51
*** Ilja has joined #openstack-cinder08:51
*** jordanP has joined #openstack-cinder08:54
*** markvoelker has joined #openstack-cinder08:55
*** TobiasE has left #openstack-cinder08:58
*** TobiasE1 has joined #openstack-cinder08:59
*** markvoelker has quit IRC09:00
*** asselin has quit IRC09:10
openstackgerritJeegn Chen proposed openstack/cinder: Fix return value inconsistency in VNX Driver  https://review.openstack.org/15647009:10
*** severson_ has quit IRC09:10
*** asselin has joined #openstack-cinder09:11
*** severson_ has joined #openstack-cinder09:11
*** asselin has quit IRC09:14
*** severson_ has quit IRC09:14
*** asselin has joined #openstack-cinder09:14
*** sseverson has joined #openstack-cinder09:15
*** TobiasE1 has quit IRC09:18
*** TobiasE1 has joined #openstack-cinder09:18
*** dulek has joined #openstack-cinder09:18
*** takedakn has quit IRC09:30
*** sgotliv has quit IRC09:35
*** marcusvrn has joined #openstack-cinder09:46
*** mkerrin has joined #openstack-cinder09:46
*** pschaef has joined #openstack-cinder09:47
openstackgerritAnton Arefiev proposed openstack/cinder: Import only modules: H302  https://review.openstack.org/14578009:53
*** markvoelker has joined #openstack-cinder09:56
*** markvoelker has quit IRC10:02
*** TobiasE1 has quit IRC10:03
*** cnesa has joined #openstack-cinder10:07
*** eedten has joined #openstack-cinder10:07
*** ndipanov has quit IRC10:10
*** ndipanov has joined #openstack-cinder10:20
*** takedakn has joined #openstack-cinder10:24
*** MentalRay has joined #openstack-cinder10:28
*** Raj_ has quit IRC10:28
*** MentalRay has quit IRC10:29
*** yuriy_n17 has joined #openstack-cinder10:29
*** marcusvrn1 has joined #openstack-cinder10:31
*** takedakn has quit IRC10:32
*** takedakn has joined #openstack-cinder10:34
*** marcusvrn has quit IRC10:35
*** Ilja1 has joined #openstack-cinder10:40
*** Ilja has quit IRC10:40
*** nellysmitt has joined #openstack-cinder10:49
*** avishay has joined #openstack-cinder10:50
openstackgerritAnton Arefiev proposed openstack/cinder: Add entry create and cast tasks to manage workflow  https://review.openstack.org/13907110:50
openstackgerritAnton Arefiev proposed openstack/cinder: Fix lvm manage existing volume  https://review.openstack.org/15693910:50
*** avishay__ has quit IRC10:50
*** ronenkat has quit IRC10:53
*** deepakcs has quit IRC10:55
*** takedakn has quit IRC10:56
openstackgerritBharat Kumar Kobagana proposed openstack/cinder: Passing privileged user to create nova assisted snapshots  https://review.openstack.org/15694010:56
*** smoriya has quit IRC10:56
*** ronenkat has joined #openstack-cinder10:57
*** kbyrne has quit IRC11:09
*** kbyrne has joined #openstack-cinder11:13
*** e0ne has joined #openstack-cinder11:14
*** kmartin has quit IRC11:18
*** erlon has quit IRC11:21
*** cnesa has quit IRC11:27
*** BharatK has quit IRC11:33
*** eedten has quit IRC11:33
*** aix has joined #openstack-cinder11:37
*** humble_ has joined #openstack-cinder11:38
*** e0ne is now known as e0ne_11:40
*** chlong has joined #openstack-cinder11:45
*** Ilja has joined #openstack-cinder11:48
*** Ilja has joined #openstack-cinder11:48
*** cnesa has joined #openstack-cinder11:49
*** Ilja1 has quit IRC11:50
*** e0ne_ has quit IRC11:50
*** takedakn has joined #openstack-cinder11:53
*** rl has joined #openstack-cinder11:56
*** rl has quit IRC11:57
*** rl has joined #openstack-cinder11:58
*** aix_ has joined #openstack-cinder11:58
*** rl is now known as rluediger11:58
*** aix_ has quit IRC11:59
*** aix has quit IRC11:59
*** aix has joined #openstack-cinder11:59
*** aix has quit IRC12:00
*** aix has joined #openstack-cinder12:00
rluedigerI have a problem with openstack juno related to cinder actions like creation or deletion of new cinder volumes are getting stuck12:00
*** sgotliv has joined #openstack-cinder12:00
*** eedten has joined #openstack-cinder12:01
rluedigerI've already increased the loglevel for cinder services on both controller node and block storage node but i can't find any error message in the logfiles12:02
rluedigerhas anyone here an hint for me where i should look to figure out the problem?12:02
rluedigerups forgot to say hello to all12:06
DuncanTrluediger: What do you mean by getting stuck? They just sit in creating?12:07
rluedigerwhen i create a new volume it stays in mode generating for hours or days12:08
DuncanT'mode generating'? That isn't a cinder term (unless you're running in a none-english locale?)12:09
*** jistr has quit IRC12:09
rluedigerDuncanT: on the other hand if i delete such hanging volumes i have to restart the tgt & cinder services on the block-storage node to get the volumes deleted12:09
rluedigerDuncanT: sorry of course i mean creating12:10
rluedigerDuncanT: <snip> | 6e389e98-009d-4953-8e72-5d6a33b6c2fa | creating | demo-volume1 |  1   |     None    |  false   |             |</snip>12:10
DuncanTOk, if you look in the API log, you should be able to find a line relating to that create. In that line, there will be a request id field. Grep for that request id in the scheduler log12:11
*** jistr has joined #openstack-cinder12:12
rluedigerDuncanT: o.k. I'll chekc for it12:12
DuncanTThe flow for create is api->scheduler->cinder-volume, and request id lets you follow the flow reasonably easily12:12
openstackgerritYuriy Nesenenko proposed openstack/cinder: Fix comments style according to the Hacking Rules  https://review.openstack.org/14582112:13
*** nellysmitt has quit IRC12:15
*** e0ne has joined #openstack-cinder12:18
*** MentalRay has joined #openstack-cinder12:20
*** emaisne has quit IRC12:24
*** takedakn has quit IRC12:24
*** emaisne has joined #openstack-cinder12:24
*** BharatK has joined #openstack-cinder12:25
*** tbarron has joined #openstack-cinder12:26
rluedigerDuncanT: sory i must switch to a new volume to find the right entries in the api log12:30
DuncanTrluediger: That is probably the easiest way to do it, yeah12:30
rluedigerDuncanT: now i found some entries for the new volume but the scheduler log remains empty until now12:30
*** ankit_ag has joined #openstack-cinder12:31
rluedigerservice cinder-scheduler status12:31
rluedigercinder-scheduler start/running, process 682312:31
DuncanTrluediger: Completely empty? That suggests a config problem somewhere. To start with, can you grep for that request id in the API log and pastebin the output (paste.openstack.org)12:32
MentalRayAny one running Ceph with Icehouse or Juno. I have want to know if they can make Migration and Resize work12:32
e0neMentalRay: what are you mean as 'migration' in ceph?12:33
*** Ilja has quit IRC12:34
MentalRayWhat I mean is that if I want to migrate an instance from a CN to another one it currently cause an issue (Same for resize) (It is a known bug but I cannot believe that there is no known fix for this)12:35
MentalRayhttps://bugs.launchpad.net/cinder/+bug/134881112:35
openstackLaunchpad bug 1348811 in Cinder "RBD + Nova migration fails because imports an existing disk" [High,Confirmed]12:35
*** IanGovett has joined #openstack-cinder12:35
*** ronenkat has quit IRC12:35
e0neMentalRay: afair, it requires some additional config values in nova12:36
rluedigerDuncanT: request id is something like this ? "req-3d203c82-2a6c-48d8-bf64-e096dc073078 a73bbf8fa8c046be9b32ad95115bb41f fb53cbe4fdda4a73ae42b3bd82df640d"12:36
DuncanTreq-3d203c82-2a6c-48d8-bf64-e096dc073078 is the bit you need12:37
e0neMentalRay: i don't have environment now to clarify it12:37
*** Ilja has joined #openstack-cinder12:38
rluedigerDuncanT: Hmmm im absolutly new to the IRC thing and to the paste.openstack.org thing as well. Do i have to put some reference to this IRC conversation to it?12:39
*** IanGovett has quit IRC12:41
*** ronenkat has joined #openstack-cinder12:41
*** Ilja has quit IRC12:42
*** IanGovett has joined #openstack-cinder12:42
smcginnisrluediger: You can just create a paste, then post the URL for it in IRC for others to look at.12:43
*** emaisne has quit IRC12:43
DuncanTNo, just copy and paste your log message in and hit the 'paste' button, it will give you a unique url for your text, that you can put in here12:44
*** Yogi1 has joined #openstack-cinder12:44
*** Ilja has joined #openstack-cinder12:44
MentalRaye0ne,  so you think that we only need to add some value to the config file and it will remove this current issue?12:44
*** sgotliv has quit IRC12:44
e0neMentalRay: may be12:45
rluedigerDuncanT: ahh ok pasteid is Paste #176803 (http://paste.openstack.org/show/176803/)12:45
MentalRayhum12:46
rluedigermeanwhile I've tested to restart the cinder-scheduler but nothing changed there is no entry with the same req-id as in the api log12:46
DuncanTrluediger: Ok, that looks like a perfectly normal create. That suggests you've got a config error somewhere. Are all your services running on one node or several?12:46
*** shilpa_ has quit IRC12:47
rluedigerDuncanT: i have 3 nodes (VMWARE Fusion Machines) and heer you can find which services are running on which node http://paste.openstack.org/show/176805/12:48
rluedigerDuncanT: But we are facing the same problem in our test environment with physical nodes as well12:49
DuncanTrluediger: Can you post up grep -i rabbit /etc/cinder.conf from each of the three machines, please?12:52
DuncanTSorry, the output of "grep -i rabbit /etc/cinder.conf"12:52
DuncanTrluediger: It looks like your services just aren't talking to each other12:53
DuncanTThe fact that they are VMWare virtual hosts shouldn't matter to cinder12:53
*** emaisne has joined #openstack-cinder12:53
*** nellysmitt has joined #openstack-cinder12:53
*** aix has quit IRC12:59
*** sgotliv has joined #openstack-cinder13:00
rluedigerDuncanT: if you mean /etc/cinder/cinder.conf you can find the result here http://paste.openstack.org/show/176809/13:00
rluedigerDuncanT: Oh the first paste was from the Controller node13:01
DuncanTI did indeed, sorry. I just want to confirm they are all talking to the same rabbit host13:01
rluedigerDuncanT: and on the block storage node it is exact the same result13:02
DuncanTOk, that crosses one easy answer off the list :-(13:02
*** ronenkat has quit IRC13:04
*** jordanP has quit IRC13:04
*** e0ne is now known as e0ne_13:05
*** jaypipes has joined #openstack-cinder13:05
*** eedten1 has joined #openstack-cinder13:08
*** Yogi1 has quit IRC13:08
*** e0ne_ is now known as e0ne13:09
*** rushiagr is now known as rushiagr_away13:09
*** markvoelker has joined #openstack-cinder13:11
*** eedten has quit IRC13:11
DuncanTrluediger: On controller1, can you run 'sudo rabbitmqctl list_queues name messages consumers' and paste the output, please? Let's see if the messages are stuck in a queue somewhere13:12
*** aix has joined #openstack-cinder13:12
*** nlevinki has joined #openstack-cinder13:12
rluedigerDuncanT: http://paste.openstack.org/show/176811/13:13
*** bswartz has quit IRC13:13
DuncanTHmmm, so either /something/ consumed the message, or it never got posted13:15
DuncanTrluediger: I am not sure if there is a better way of debugging this than I'm using - if we're lucky somebody will pop up and know the issue, until then, all I can do is walk through the mechanics one step at a time until we find something unexpected13:16
MentalRaye0ne, if you ever have the time to take a look or point me in some direction, I would really appreciate it.13:16
DuncanTrluediger: Can you stop cinder-scheduler, run another create, then dump the rabbit queue state again, please?13:17
*** ndipanov has quit IRC13:18
*** zigo has quit IRC13:18
*** zigo has joined #openstack-cinder13:20
rluedigerDuncanT: http://paste.openstack.org/show/176829/13:22
rluedigerDuncanT: I appreciate your help13:23
*** e0ne_ has joined #openstack-cinder13:24
*** pschaef has quit IRC13:25
DuncanTrluediger: So if the scheduler is stopped, there should be a message left on the queue, but there isn't, and consumers for the scheduler queue should have become zero but didn't13:25
*** e0ne has quit IRC13:27
rluedigerDuncanT: I issued the following as root on the controller service cinder-scheduler status and got the result cinder-scheduler stop/waiting13:27
DuncanTHmmm, it is possible that it didn't cleanly close the rabbit connection, so rabbit is still waiting for the tcp connection to time out... restarting rabbit will fix that13:28
rluedigerDuncanT: cinder service-list says http://paste.openstack.org/show/176830/13:29
openstackgerritJeegn Chen proposed openstack/cinder: Attach/detach batch processing in VNX driver  https://review.openstack.org/15435013:29
*** timcl has joined #openstack-cinder13:30
*** timcl has left #openstack-cinder13:30
*** ronenkat has joined #openstack-cinder13:30
*** e0ne has joined #openstack-cinder13:30
DuncanTService list goes by db status. Can you restart rabbit, and post the queue info dump again, please?13:31
*** ndipanov has joined #openstack-cinder13:31
rluedigerDuncanT: after restartin the rabbitmq it looks like http://paste.openstack.org/show/176835/13:32
*** e0ne_ has quit IRC13:33
DuncanTThere's still a consumer on the scheduler queue :-(13:33
DuncanTCan you run "ps auxx | grep cinder-sc" on all three nodes and just check there isn't a rogue process, please?13:33
rluedigerDuncanT: i saw but i have no idea where it comes from13:33
*** ronenkat has quit IRC13:34
rluedigerDuncanT: there is nothing on controller or block1 node http://paste.openstack.org/show/176841/ the third one which will be used as compute it totally shut down13:36
DuncanTrluediger: I'm quite confused :-(13:36
*** aix has quit IRC13:37
rluedigerDuncanT: ;) welcome to my world13:40
DuncanTrluediger: Oh, confused is my default state :-) It is not normally caused by basic deployment issues though13:41
DuncanTrluediger: Can you restart cinder-api and cinder-scheduler, and post the log of the restart, please? I might be able to spot something then13:42
rluedigerDuncanT: there is one more point i've observed. If i submit a cinder force-delete on the controller i have to restart the services on the block1 node to find a msg in the log that the volume what is not existing was deleted.13:42
rluedigerDuncanT: Yes I'll do so13:43
*** humble_ has quit IRC13:43
*** bswartz has joined #openstack-cinder13:44
DuncanTGot to step away for a few minutes, back in ~ ten minutes13:44
*** cnesa has quit IRC13:44
*** timcl has joined #openstack-cinder13:47
rluedigerDuncanT: api log http://paste.openstack.org/show/176852/ scheduler log http://paste.openstack.org/show/176853/13:47
rluedigerDuncanT: I've put some marker in the files right before the restart13:48
*** sandywalsh_ has joined #openstack-cinder13:48
*** krtaylor has quit IRC13:50
*** sandywalsh has quit IRC13:50
openstackgerritRodrigo Barbieri proposed openstack/cinder: Fix exception error on HNAS drivers  https://review.openstack.org/15353713:52
*** krtaylor has joined #openstack-cinder13:56
*** bswartz has quit IRC13:58
*** ankit_ag has quit IRC13:59
openstackgerritJeegn Chen proposed openstack/cinder: Attach/detach batch processing in VNX driver  https://review.openstack.org/15435013:59
*** annegent_ has joined #openstack-cinder14:01
*** bswartz has joined #openstack-cinder14:01
*** rluediger has quit IRC14:01
*** rl has joined #openstack-cinder14:02
*** rl is now known as rluediger14:02
openstackgerritxing-yang proposed openstack/cinder: Kilo Consistency Group API update  https://review.openstack.org/14456114:02
*** aix has joined #openstack-cinder14:03
*** timcl1 has joined #openstack-cinder14:03
*** kaufer has joined #openstack-cinder14:04
*** timcl has quit IRC14:05
*** rushiagr_away is now known as rushiagr14:07
*** kaisers1 has quit IRC14:13
*** eedten1 has quit IRC14:13
openstackgerritAnton Arefiev proposed openstack/cinder: Add entry create and cast tasks to manage workflow  https://review.openstack.org/13907114:14
openstackgerritAnton Arefiev proposed openstack/cinder: Fix lvm manage existing volume  https://review.openstack.org/15693914:14
*** eedten1 has joined #openstack-cinder14:14
*** eedten1 is now known as TobiasE14:14
rluedigerDuncanT: As already said I really appreciate your help but for today I've to leave. Maybe we can proceed tomorow. Many thanks from germany14:14
*** rluediger has quit IRC14:14
DuncanTrluediger: Sure, I'll be about tomorrow. Sorry, I got pulled off into other things14:15
*** avishay_ has joined #openstack-cinder14:16
*** avishay has quit IRC14:16
*** dustins has joined #openstack-cinder14:17
*** annegent_ has quit IRC14:18
*** annegent_ has joined #openstack-cinder14:19
*** shilpa_ has joined #openstack-cinder14:20
*** mwichmann has joined #openstack-cinder14:20
*** primechuck has joined #openstack-cinder14:20
*** nlevinki_ has joined #openstack-cinder14:21
*** nlevinki_ has quit IRC14:25
*** nlevinki has quit IRC14:25
*** dustins has quit IRC14:26
*** alonmarx has joined #openstack-cinder14:26
*** dustins has joined #openstack-cinder14:26
*** shilpa_ has quit IRC14:27
*** alonmarx_ has quit IRC14:29
*** Yogi1 has joined #openstack-cinder14:30
*** mriedem has joined #openstack-cinder14:31
*** changbl has quit IRC14:32
*** rushil has joined #openstack-cinder14:32
*** jungleboyj has quit IRC14:34
*** cnesa has joined #openstack-cinder14:43
*** ronenkat has joined #openstack-cinder14:45
*** kaufer has quit IRC14:46
*** jwcroppe has quit IRC14:47
*** anshul has quit IRC14:48
*** jcru has joined #openstack-cinder14:52
*** kvidvans has joined #openstack-cinder14:52
*** kvidvans has quit IRC14:52
*** kvidvans has joined #openstack-cinder14:53
*** kvidvans has quit IRC14:53
*** kvidvans has joined #openstack-cinder14:53
*** kvidvans has quit IRC14:53
*** krtaylor has quit IRC14:54
*** patrickeast has joined #openstack-cinder14:55
*** krtaylor has joined #openstack-cinder14:55
*** Ilja has quit IRC14:57
*** Ilja has joined #openstack-cinder14:57
*** kaufer has joined #openstack-cinder15:00
*** annegentle has quit IRC15:00
*** mtanino has joined #openstack-cinder15:03
*** ebalduf has joined #openstack-cinder15:04
*** asselin_ has joined #openstack-cinder15:04
*** jordandh has quit IRC15:04
*** jordandh has joined #openstack-cinder15:04
smcginnisDuncanT: Still around?15:07
DuncanTsmcginnis: Yup15:08
*** ebalduf has quit IRC15:08
smcginnisDuncanT: Wondering if you looked back in IRC. Some discussion yesterday afternoon (here) about using pip2pi, pypicache, and apt-cache for CI.15:08
smcginnisDuncanT: Wondering if you've ever looked in to any of that.15:09
DuncanTsmcginnis: Yeah, I've a couple of links openned from the discussion, not yet read15:09
smcginnisDuncanT: So not something you've tried out?15:09
DuncanTsmcginnis: We have internal (on cloud) apt mirrors I use, and I use a git object cache15:09
smcginnisDuncanT: Nice!15:09
smcginnisDuncanT: I'm probably going to try to set something up. I'll share whatever I find. Maybe not too valuable for you since you have local mirrors.15:10
smcginnisDuncanT: But could be.15:10
DuncanTI don't (yet) do any pip/pipi cache stuff, because when I tried it proved to be an epic pain in the ass15:10
smcginnisDuncanT: That's a concern of mine.15:10
DuncanTI don't run my own, so knowing how to do it would of course be useful15:10
*** leopoldj_ has joined #openstack-cinder15:10
*** thangp has joined #openstack-cinder15:10
smcginnisDuncanT: Things work as is. Would be nice to speed things up, but if it makes it fragile not worth it.15:11
DuncanTI feel you ought to be able to just set up squid since everything can be fetched via http(s), but experience suggests that very little actually hits that cache :-(15:11
smcginnisDuncanT: That was my first thought, but not sure if it would cache the way we would need it to.15:12
smcginnisDuncanT: Not a squid expert (or even an advanced novice) by any means though.15:12
DuncanTI set it up and looked at the cache hit/miss stats after a bunch of identical runs, they were terrible. After that I gave up on that approach. Would love to know why it doesn't work15:13
*** leopoldj has quit IRC15:13
DuncanTMy total knowledge came from two how-to's ;-)15:13
smcginnisThen you're twice as knowledgable than me.15:13
DuncanTGit object caching seems to make checkouts way faster though, and seems very solid/safe... I bake the cache into the image. Give me a sec and I'll grab the notes I used to set that up15:14
*** sseverson has quit IRC15:14
*** sseverson has joined #openstack-cinder15:14
*** jwcroppe has joined #openstack-cinder15:14
smcginnisCool15:15
*** jwcroppe has quit IRC15:16
*** sigmavirus24_awa is now known as sigmavirus2415:16
*** jwcroppe has joined #openstack-cinder15:17
*** e0ne is now known as e0ne_15:18
*** kaisers has joined #openstack-cinder15:19
*** fischerw has joined #openstack-cinder15:19
*** e0ne_ is now known as e0ne15:20
jgriffithe0ne: hey hey... good morning15:20
jgriffithe0ne: bad news... the backported LVM upgrade did NOT help15:20
e0nejgriffith: hi!15:20
DuncanTsmcginnis: http://paste.openstack.org/show/176901/15:20
DuncanTsmcginnis: That's straight out of my notes, and was written a while ago, but it looks like it should still work15:20
e0nejgriffith: :(. did you get the same error after upgrade lvm2?15:21
jgriffithe0ne: yep15:21
e0ne:(15:21
DuncanTjgriffith: You might find that paste link interesting too... git clone cinder in ~2 seconds15:21
smcginnisDuncanT: I see, so you just prep the image with a cache at the time? So when it's used just has to pull the latest?15:21
jgriffithDuncanT: I ran into problems with that15:22
jgriffithDuncanT: but might be that I set something up wrong15:22
jgriffithDuncanT: it seemed to run into hiccups figuring out when it actually needed to pull15:22
e0nejgriffith: what is your steps to reproduce this issue?15:22
DuncanTsmcginnis: When it pulls latest, it generally has far less to pull so it works faster15:22
jgriffithe0ne: just setup devstack, and run the tempest volume tests15:23
DuncanTjgriffith: I've not noticed any problems, but I'm not running anything in anger15:23
smcginnisDuncanT: Yeah, I would think it should help.15:23
jgriffithDuncanT: Yeah, quite possible I screwed somethign up15:23
smcginnisDuncanT: That delta should be fairly small if it's a recent image.15:23
jgriffithDuncanT: OR... I use HUB and had some weirdness with it ALWAYS cloning from local if/when avaialable15:24
jgriffithmaybe that was related?15:24
DuncanTsmcginnis: Doing thin checkouts or whatever git calls them should give a similar performance boost TBH, but I like being able to look at history15:24
DuncanTHUB?15:24
jgriffithDuncanT: I'll have to try it out, would be immensely helpful15:24
smcginnisDuncanT, jgriffith: I think I may know what the issue with this is.15:24
smcginnisIIRC, git cache is keyed on the server.15:24
jgriffithDuncanT: oh yes.. https://github.com/github/hub15:24
smcginnisSo if you have your local mirror you always pull from the same host.15:25
DuncanTsmcginnis: My cache directory has a giant mess of remotes, so that seems reasoanble15:25
smcginnisBut if you are using public it is a pool and you may end up grabbing from a different host, so none of it will hit cache.15:25
openstackgerritwang peter proposed openstack/cinder: CG Modification Support in EMC VNX Cinder Driver  https://review.openstack.org/15478415:25
*** spradeepv has joined #openstack-cinder15:25
smcginnisjgriffith: That could explain why it didn't work for you.15:25
jgriffithsmcginnis: sure fits15:26
openstackgerritThang Pham proposed openstack/cinder: Cinder objects base  https://review.openstack.org/13187315:26
e0nejgriffith: just to clarify that we are on the same page: do you get lockev lvm and stacktrace in syslog after trying to remove snapshots?15:27
jgriffithe0ne: so I just do "tox -e all -- volume"15:27
jgriffithe0ne: al the tests will pass15:27
jgriffithe0ne: but check out the slow list.. and then dig in to c-vol log :(15:27
e0nejgriffith: or just you're working on a very slow lvm commands15:27
jgriffithe0ne: ohhh.. no15:27
jgriffithe0ne: just super slow glitches15:27
jgriffithe0ne: something in our sequence15:28
DuncanTjgriffith: hub looks damn handy15:28
jgriffithe0ne: going to try a few different things today15:28
e0nejgriffith: ok. maby it's related15:28
jgriffithe0ne: also need to go back and look and see if this was in Juno as well15:28
jgriffithe0ne: it very well could be15:28
jgriffithDuncanT: yeah, I like it... but I don't really exploit all of the cool features it has15:29
emaisneHi, does anyone know which backend is used for the automized tempest tests?? e.g. https://review.openstack.org/#/c/153033/15:29
DuncanTjgriffith: I don't use github nearly as much as I could/should, and this would make it easier. The fork alone will encourage me to publish more15:29
jgriffithDuncanT: tab completion and abbreviated cloning are all I really use15:29
DuncanTjgriffith: I've dozens of minor patches to github projects sat around in my source directory15:30
e0neemaisne: it should be lvm15:30
DuncanTjgriffith: Because I'm too lazy to figure out pull requests15:30
jgriffithDuncanT: even just things like "git browse" are handy15:31
*** devlaps has joined #openstack-cinder15:32
*** spradeepv has left #openstack-cinder15:32
*** spradeepv has joined #openstack-cinder15:33
*** esker has joined #openstack-cinder15:33
*** rmesta has joined #openstack-cinder15:36
jgriffithemaisne: that's goign to use the reference LVM driver15:45
jgriffithemaisne: the OpenStack Jenkins CI tests use Cinder LVM15:46
*** jungleboyj has joined #openstack-cinder15:47
*** shilpa_ has joined #openstack-cinder15:47
jgriffithe0ne: if you want to tag team this a bit that would be awesome!15:47
jgriffithe0ne: I need to setup Juno and add some timing statements15:48
jgriffithe0ne: see if the problem has been aroung longer than I think maybe15:48
e0nee0ne: i try to reproduce in on master first15:48
jgriffithe0ne: cool....15:49
*** hemnafk is now known as hemna15:49
jgriffithe0ne: some things to look for; there are some Warning statements I added you'll see two of those most likely15:49
jgriffithBUT15:49
jgriffithe0ne: more interstingly, putils execute now gives completion time in seconds on all of it's calls15:49
hemnamornin15:49
jgriffithe0ne: start looking at those and prepare to be horrified15:50
jungleboyjhemna: Morning sir.15:50
jgriffithDuncanT: actually... you might have some insight here15:51
jgriffithDuncanT: you've been pretty helpful with strace in the past...15:52
e0nejgriffith: what do you mean "putils execute now gives completion time in seconds"?15:52
jgriffithDuncanT: http://goo.gl/kRng2N15:52
*** shilpa_ has quit IRC15:52
*** jecarey has joined #openstack-cinder15:52
jgriffithDuncanT: ^^ anything jump out there?15:52
DuncanTjgriffith: looking...15:52
DuncanTjgriffith: You know there's a flag for strace to include timing info, right?15:52
jgriffithDuncanT: DOH.. that would've been frikin nice to have in there!15:53
*** eedten1 has joined #openstack-cinder15:53
*** erlon has joined #openstack-cinder15:53
jgriffithe0ne: just a sec... I'll grab an example for ya15:54
DuncanT'strace -r' IIRC15:54
*** TobiasE has quit IRC15:54
jgriffithe0ne: http://goo.gl/xK6sY215:54
*** eedten2 has joined #openstack-cinder15:54
jgriffithe0ne: one theory I had was updating stats every 10 seconds15:55
jgriffithe0ne: but I bumped that out to like 120 and still had issues15:55
jgriffithe0ne: although I didn't look to see if the hangs were still in conjunction with a get stats call... hmmm15:56
*** eedten2 has quit IRC15:56
*** TobiasE has joined #openstack-cinder15:56
*** rhe00_ is now known as rhe0015:58
*** eedten1 has quit IRC15:58
*** Adriano_ has joined #openstack-cinder15:59
*** tbarron has quit IRC15:59
*** xyang has joined #openstack-cinder15:59
*** kmartin has joined #openstack-cinder16:00
*** tbarron has joined #openstack-cinder16:00
jgriffithAre we meeting today?16:00
jgriffithhaven't seen thingee around since last week16:00
e0nejgriffith: at lease, we've got agenda16:00
jgriffithasselin_: You up?16:00
e0nejgriffith: thingee is already at meeting16:01
jgriffithahh16:01
jgriffithhe just doesn't join openstack-cinder any more I guess16:01
asselin_i'm here ;)16:01
asselin_was in meeting-416:01
e0ne:(16:02
*** ronenkat_ has joined #openstack-cinder16:03
*** avishay_ is now known as avishay16:03
*** setmason has joined #openstack-cinder16:03
*** deepakcs has joined #openstack-cinder16:04
*** anshul has joined #openstack-cinder16:04
e0nejgriffith: what is the right place to ask about devstack? i've got something wrong with deps: http://paste.openstack.org/show/176938/16:04
*** spradeepv has left #openstack-cinder16:05
*** dustins has quit IRC16:06
*** tbarron1 has joined #openstack-cinder16:08
*** tbarron has quit IRC16:08
*** Ilja has quit IRC16:12
*** harlowja_at_home has joined #openstack-cinder16:14
hemnajaypipes, https://review.openstack.org/#/c/85856/  the cinderclient patch.16:16
*** BharatK has quit IRC16:17
*** tsekiyama has joined #openstack-cinder16:18
jaypipeshemna: ty sir16:18
*** rhagarty has joined #openstack-cinder16:20
*** rluediger has joined #openstack-cinder16:22
*** hongbin has joined #openstack-cinder16:24
*** asselin__ has joined #openstack-cinder16:25
hongbinHi cinder devs. I have a question about debugging cinder. Any response is appreciated.16:25
hongbinI used a heat template to manage the volume along with other openstack resources16:26
hongbinI got this error on c-vol16:26
hongbinhttp://pastie.org/995966216:26
hongbinI want to know what is a general approach to debug such problem?16:27
*** david-lyle_afk is now known as david-lyle16:28
*** asselin_ has quit IRC16:29
smcginnishongbin: Hi, weekly cinder meeting is going on right now. Check back in about 30 minutes and someone might be able to help.16:31
*** ebalduf has joined #openstack-cinder16:31
e0nehongbin: afaik, we have not fix for it. i've such issues time ti time with high disk i/o load16:32
hongbinsmcginnis: Oh, get that. Thx.16:32
*** ronis has quit IRC16:35
hongbine0ne: Is that a bug report for this issue? If no, I am happy to create one.16:39
*** changbl has joined #openstack-cinder16:39
e0nehongbin: it will be good. i didn't reproduce it with master so didn't filed it to launchpad16:40
*** harlowja_at_home has quit IRC16:43
*** pradipta has joined #openstack-cinder16:44
*** timcl1 has quit IRC16:45
hongbine0ne: Thanks. A bug report was created https://bugs.launchpad.net/cinder/+bug/1422831 .16:45
openstackLaunchpad bug 1422831 in Magnum "Error on deleting volume on bay update" [Undecided,New] - Assigned to hongbin (hongbin034)16:45
*** timcl has joined #openstack-cinder16:45
*** rwsu-afk is now known as rwsu16:46
e0nehongbin: thanks16:47
*** hongbin has quit IRC16:48
*** esker has quit IRC16:48
*** tbarron1 has quit IRC16:51
*** afazekas has quit IRC16:51
*** tbarron has joined #openstack-cinder16:51
*** asselin__ has quit IRC16:53
*** esker has joined #openstack-cinder16:55
*** timcl1 has joined #openstack-cinder16:55
*** anshul has quit IRC16:55
*** timcl has quit IRC16:57
*** avishay has quit IRC16:57
*** rushil has quit IRC17:00
*** thingee has joined #openstack-cinder17:00
bswartznothing like a nice deadline to motivate people17:00
openstackgerritAnton Arefiev proposed openstack/cinder: Import only modules: H302  https://review.openstack.org/14578017:01
e0nebswartz: except yesterday's deadline17:01
*** deepakcs has quit IRC17:01
thingeejgriffith, hemna, gary-smith: lets try to talk today about capabilities?17:01
jgriffiththingee: sure17:02
thingeebswartz: I brought it up I think a couple of meetings about for code needed to be posted and on the ML17:02
* bswartz is getting lunch now17:02
jgriffithwinston-d: do you have time as well?17:02
thingeebswartz: to get the initial agreement on the dates from ya'll17:02
bswartzthingee: no complaints here -- I just like to remind people17:02
winston-djgriffith: yup17:02
thingeebswartz: doing my best :)17:02
* bswartz bbl17:02
*** bswartz has quit IRC17:03
*** coolsvap is now known as coolsvap_17:03
kauferhemna: Do you have any issues with my plan for how to address the DB query for the replication period task?  Please see my reply to your comment here:  https://review.openstack.org/#/c/154673/4/cinder/volume/manager.py17:03
thingeeoh yeah and people please help with reviewing https://etherpad.openstack.org/p/cinder-k3-priorities17:03
thingeeeven if you're not core17:03
dulekjgriffith: Had you maybe a moment to look at my patch to allocation_capacity problem?17:03
thingeexyang1, e0ne, eharney, DuncanT, hemna thanks for the help so far17:04
jgriffithdulek: no, sorry I have not17:04
jgriffiththingee: are we talking about capabilities?17:04
jgriffithI'd like to take advantage of winston-d before he falls asleep :)17:05
winston-dthingee: hasn't been active lately on review, having vacation right now, Chinese New Year.17:05
thingeejgriffith: Oh I just said today, as in sometime today. If we want to do it now while winston-d is awake, sounds good to me!17:05
jungleboyjthingee: I will dig into that this afternoon.17:05
thingeejungleboyj: thank you!17:05
patrickeasthey guys, shamelesss plug for a sponser for my FFE http://lists.openstack.org/pipermail/openstack-dev/2015-February/056983.html17:05
jgriffithgary-smith: hemna work for you ^^ ?17:05
thingeeI need one more core to sponsor patrickeast's FFE http://lists.openstack.org/pipermail/openstack-dev/2015-February/056983.html17:05
jgriffithpatrickeast: consider me a sponsor17:05
thingeeI'm already behind it myself17:05
patrickeasti’ll offer many beers in vancouver for help with it ^17:05
winston-ddulek: I was reviewing it, hopefully can finish it today or tomorrow.17:05
thingeepatrickeast: I'm set!17:06
*** annegent_ has quit IRC17:06
openstackgerritErlon R. Cruz proposed openstack/cinder: Adds pool aware scheduling for HNAS drivers  https://review.openstack.org/14213317:06
openstackgerritErlon R. Cruz proposed openstack/cinder: Adds SSH communication to HNAS drivers  https://review.openstack.org/14290417:06
thingeepatrickeast: even though I initially delayed it.17:06
dulekwinston-d: Great, just wanted to know if everything's on track :)17:06
* thingee has figured out that he has to just delay people's work to FFE to get free beers later17:06
*** nshaikh has quit IRC17:06
patrickeasthaha17:06
thingeewinston-d, jgriffith, gary-smith, hemna: capability talk...shall we start?17:07
jgriffithsure17:07
winston-ddulek: i need more time when review code touches taskflow. ;)17:08
* thingee reminds himself of the scope and unscoped17:08
winston-dthingee: sure17:08
dulekwinston-d: It's not *that* hard, but let's not start this discussion again. :D17:08
*** patrickeast has quit IRC17:09
thingeehttps://review.openstack.org/#/c/150511/17:09
jgriffiththingee: seems gary-smith and hemna aren't around17:10
winston-ddulek: it's just I am not *that* familiar with taskflow, could be just me.17:10
thingeeok so we're saying we want to allow scoped keys, but we shouldn't disallowed everything to be scoped?17:10
hemnajgriffith, we are here17:11
jgriffithoh... sorry, you never responded17:11
hemnajgriffith, we were just hashing this stuff out here in our cube17:11
jgriffithgot ya17:11
hemnawhat we think should happen is a new set of cinder-specs to dial this stuff in17:11
hemnathe existing cinder-spec probably needs to get nuked for K17:11
hemnahttps://review.openstack.org/#/c/12764617:12
hemnado you guys want to do a google hangout ?17:12
hemnakaufer, sorry, been busy.17:13
*** lcurtis has joined #openstack-cinder17:13
jgriffithworks for me17:13
hemnakaufer, so I see your filtering patch in place17:13
hemnakaufer, I think we need to get that to land first17:13
hemnakaufer, and then update the replication patch to use it.17:13
thingeehemna: don't mean to be a pain, but if I can call in somehow...traveling so I'm tethered to my phone17:13
hemnathingee, can't you do hangout on your phone ?17:14
kauferhemna: The BP already has a +2 from thingee, just waiting for it to get merged17:14
hemnakaufer, take your cinder patch out of WIP then17:14
hemnakaufer, unless it's not ready.17:14
*** jistr has quit IRC17:15
kauferhemna: Nope, it's only a WIP since the BP is not merged17:15
*** sgotliv has quit IRC17:15
*** dannywilson has joined #openstack-cinder17:15
hemnajgriffith, can you take a look at kaufer's spec on the DB api filtering?17:15
hemnajgriffith, so we can get this stuff in to avoid the fetching every volume issue in the replication patch ?17:16
kauferjgriffith: https://review.openstack.org/#/c/155074/17:16
winston-dare we talking about capabilities now?17:16
hemnaI can't +2 in cinder-specs17:16
hemnawinston-d, I think so, if folks want to do a hangout17:16
hemnaI can set one up17:16
thingeehemna: yeah i can try. just send out the link17:17
kauferhemna: I didn't want the current patch set to depend on the BP/DB filtering code, I'll remove the WIP from the DB filtering code now17:17
hemnathingee, winston-d jgriffith gary-smith DuncanT https://plus.google.com/hangouts/_/gttgwwq4o4txqzxeuhw2qjjta4a17:17
*** lpetrut has quit IRC17:17
*** markstur has quit IRC17:18
winston-dgive me a minute, i need to get over the 'wall' to use google services.17:18
erlonthingee: winston-d: Mike, Huang, can you guys revisit this change (https://review.openstack.org/#/c/141868/)? As I commented in the thread, the unit tests where implemented in another PS, so, no reason to keep holding it.17:18
rhe00what kind of monster machines do people run their CI on?17:20
DuncanTrhe00: Cloud instances... 8 gig I think17:20
*** chlong has quit IRC17:22
*** e0ne is now known as e0ne_17:22
jgriffithrhe00: OpenStack instance here as well.... 8G ram and 4 VCPU17:22
jgriffith10 Gig disk17:22
rhe00I have a HP G8 16 VCPUs with 128 GB RAM and lvm backend storage.  It pretty much dies when nodepool is trying to set up a node with 8 GB 4 VCPUs and 40 GB disk. Horizon stops responding, etc.17:23
jgriffithrhe00: oh... you meant "what's you're cloud running on"17:23
rhe00yes17:23
jgriffithrhe00: I'm running a similar system, although it's a supermicro17:23
jgriffithrhe00: but I don't do nodepool etc17:23
*** e0ne_ is now known as e0ne17:24
rhe00I am bit concerned that I will run out of HW because I was planning to run more than one node on this machine to test both FC and ISCSI. Doesn't seem possible when the first node brings the machine to a halt.17:25
jgriffithrhe00: yeah, not sure what's up.  FWIW I have an all in one and I run 15+ instances at a time on it17:25
*** vilobhmm has joined #openstack-cinder17:26
*** Apoorva has joined #openstack-cinder17:28
dulekthingee: Spec for Cinder objects has objectification of backups mentioned.17:30
dulekthingee: I was able to implement it lately and want to send the patch today17:30
dulekthingee: Is it possible to get it on priorities etherpad once it's up in gerrit?17:31
*** e0ne has quit IRC17:31
*** vilobhmm_ has joined #openstack-cinder17:32
*** dulek has quit IRC17:32
*** dulek has joined #openstack-cinder17:33
*** Yogi1 has quit IRC17:35
openstackgerritAnish Bhatt proposed openstack/cinder: Improve error handling in refactored Tgt driver  https://review.openstack.org/15471317:35
*** dannywilson has quit IRC17:35
*** leeantho has joined #openstack-cinder17:35
*** _cjones_ has joined #openstack-cinder17:36
*** TobiasE has quit IRC17:38
*** cnesa1 has joined #openstack-cinder17:38
*** cnesa has quit IRC17:38
*** ronis has joined #openstack-cinder17:38
*** EmilienM is now known as EmilienM|afk17:40
*** tbarron has quit IRC17:41
thingeedulek: excellent. got a link to the review?17:41
*** yuriy_n17 has quit IRC17:42
dulekNot yet, struggling with git and gerrit a little. I'll send you one once I'm done.17:42
dulekthingee: There's still some work to do in the patch, but I think it's a fair start.17:42
*** harlowja_away is now known as harlowja_17:42
*** rushil has joined #openstack-cinder17:43
*** anshul has joined #openstack-cinder17:43
kauferthingee: I'm looking at the k3 priorities etherpad, I see that my BP changes for the API sorting is marked as pending approval on the cross project spec; I assume that you are referring to https://review.openstack.org/#/c/145544/ which is only for CLI and is not related to the reviews in question17:44
openstackgerritThang Pham proposed openstack/cinder: Snapshot and volume objects  https://review.openstack.org/13356617:44
kauferthingee: However, it is pending the approval of a change to the cinder spec to adopt the API WG server sorting guidelines: https://review.openstack.org/#/c/153061/17:45
thingeekaufer: in a meeting, I'll get back to you17:45
kauferthingee: Ok, thanks17:45
*** emaisne has quit IRC17:46
*** _cjones_ has quit IRC17:48
*** _cjones_ has joined #openstack-cinder17:48
openstackgerritEric Harney proposed openstack/cinder: LVM: Fix thin provisioning and mirrors stats reporting  https://review.openstack.org/15708317:54
openstackgerritMichal Dulko proposed openstack/cinder: WIP: Backup object  https://review.openstack.org/15708517:55
*** annegentle has joined #openstack-cinder17:55
*** annegentle has quit IRC17:56
*** annegentle has joined #openstack-cinder17:56
*** Yogi1 has joined #openstack-cinder17:57
*** alecv has quit IRC18:01
*** lpetrut has joined #openstack-cinder18:01
*** dustins has joined #openstack-cinder18:04
dulekthingee: Here's my patch: https://review.openstack.org/15708518:05
*** ybathia has joined #openstack-cinder18:06
dulekthingee: I'll try to address the items I've listed in the patch tomorrow18:06
dulekthingee: Have a nice day! :)18:06
openstackgerritEric Harney proposed openstack/cinder: Tests: Fix cxt target tests opening files  https://review.openstack.org/15708918:07
*** annashen has joined #openstack-cinder18:07
*** dulek has quit IRC18:09
*** bswartz has joined #openstack-cinder18:10
*** rushiagr is now known as rushiagr_away18:10
openstackgerritSteven Kaufer proposed openstack/cinder: Generic filter support for volume queries  https://review.openstack.org/15534318:11
openstackgerritSteven Kaufer proposed openstack/cinder: Replication status periodic task optimization  https://review.openstack.org/15467318:11
kauferjungleboyj, hemna: ^18:12
*** pradipta has quit IRC18:14
*** patrickeast has joined #openstack-cinder18:15
*** patrickeast has quit IRC18:15
*** krtaylor has quit IRC18:16
*** patrickeast has joined #openstack-cinder18:17
*** rajinir_r has joined #openstack-cinder18:20
thingeekaufer: thanks for the clarification. I once again was confused by all the specs.18:22
thingeeI'll move the review up for ready for review on the etherpad18:23
jungleboyjkaufer: Looking.18:23
*** garthb has joined #openstack-cinder18:23
*** garthb_ has joined #openstack-cinder18:23
thingeekaufer: I gave it a +2. We'll start the code reviews now, but I'll give a little time for people to comment on the specs if needed.18:25
thingeespec*18:25
*** jwang has quit IRC18:25
*** dannywilson has joined #openstack-cinder18:27
patrickeastthingee: jgriffith: hey guys, thanks for sponsoring my FFE, is there anything else I need to/should do now with it? (other than keep working on the feature)18:27
*** jwang has joined #openstack-cinder18:28
thingeepatrickeast: just work on it. I'll ask jgriffith for a +2 on the spec as well. I'll target it for k-3 after that.18:29
patrickeastthingee: ok cool18:29
thingeewould also like hemna to give a +1 since I know he has interest in this18:29
flip214hemna: any update on the transport driver multiplicity? Brick or john griffiths' target driver abstraction, what is the way to go?18:30
thingeejungleboyj: did I see right that the systemz is sponsored in nova?18:31
jungleboyjYes.  I think they have a decent chance of making it.18:32
*** alecv has joined #openstack-cinder18:32
jungleboyjkaufer: +2.  Looks good to me.18:33
jungleboyjneed to get hemna s input.18:33
kauferthingee: Sorry was away, thanks!18:33
kauferjungleboyj: Thanks, my only concern was with the constants in in the filter for 'replication_status'18:33
kauferjungleboyj: The DB filtering doesn't support something like 'replication_status'!='disabled' so I had to provide all of the valid values, which I pulled out of your spec, hopefully I covered them all18:34
thingeewinston-d: can you have a look at this review you participated in? It's blocking a couple of bps from being merged https://review.openstack.org/#/c/141868/1018:35
jungleboyjkaufer: Ok, if you got them from my Spec then you should be good.18:35
kauferjungleboyj: It would be nice to have those defined constants somewhered18:36
kaufers/somewhered/somewhere18:36
thingeehemna: are we planning brick extraction this week?18:37
thingeepatrickeast: for your two bps, CG and over subscription, are you going to be ready before march 1st?18:37
*** alecv has quit IRC18:37
*** tbarron has joined #openstack-cinder18:37
thingeepatrickeast: jenkins passes, not marked as wip18:37
*** leopoldj_ has quit IRC18:38
thingeeleeantho: is this going to not be marked as wip soon? https://review.openstack.org/#/c/155127/18:41
jungleboyjkaufer: where would we add that?18:41
thingeegugl2: code for this going to be up, passing jenkins, not marked as wip before march 1st? https://blueprints.launchpad.net/cinder/+spec/hplefthand-driver-supplies-filtering-weighing-functions18:41
thingeepatrickeast: also multipath enhancement with pure18:42
thingeerhagarty: flash cache support. are we still looking good to have the code posted before march 1st, passing jenkins, not marked as wip?18:43
thingeexyang1, hemna: so I think this code could be done better. https://review.openstack.org/#/c/144561/11/cinder/consistencygroup/api.py18:45
thingeexyang1, hemna: we go from string -> list -> string18:45
thingee:)18:45
jgriffithflip214: what do you mean?18:45
patrickeastthingee: sry was afk for a min, yea i have code for them right now just waiting for the dependent changes to go in first so they will actually work18:45
jgriffithflip214: the target stuff isn't in brick any longer18:45
jgriffithflip214: whether it should be or not is a whole different question :)18:46
xyangthingee: suggestions for improvement are welcome18:46
thingeexyang1, hemna: I haven't looked to see if the potential race conditions I raised in the previous patch sets were addressed, but I'm a bit hung up on that.18:46
flip214jgriffith: I'd like to know (for the DRBD-for-Nova thing) whether your target abstraction, brick, or whatever else is the right way to go.18:47
xyangthingee: on which?18:47
flip214apart from 2 lines on IRC (from you) I haven't got _any_ feedback...18:47
jgriffithflip214: so on the nova side you need to deal with the initiator piece18:47
jgriffithflip214: which means it needs to be in brick18:47
thingeexyang: we talked on google hangout about them.18:47
jgriffithflip214: eventually18:47
thingeexyang: you said you would have to do some research18:47
jgriffithflip214: it's part of what hemna wants to solve18:48
xyangthingee: sorry, I thought we talked about the 2nd patch.  I don't know which one you are referring to now18:48
flip214jgriffith: well, all that got moved to K anyway... so it's not that important anymore.18:48
jgriffithflip214: currently you need initiator work in /cinder/brick/initiators as well as in Nova18:48
kauferjungleboyj: So some are here, but it seems that 'copying' is missing: https://github.com/openstack/cinder/blob/master/cinder/replication/api.py#L35-L3618:48
jgriffithflip214: we're in K :)18:48
flip214just thinking that the 3 ways that are in https://github.com/LINBIT/cinder/tree/drbd-for-nova are bad for the reviewer, and my health.18:48
jgriffithflip214: you mean L?18:48
*** jkremer has joined #openstack-cinder18:48
flip214sorry, got moved to _after_ K, ie. L.18:48
hemnain another meeting18:48
hemnasorry guys18:48
jgriffithflip214: looking18:49
hemnathingee, yes I'm planning on brick extraction in K18:49
flip214thank you...18:49
thingeexyang: we did. I raised in the patch set though to you over google hangout about some race conditions. You weren't sure and said you would have to research into the problems I raised.18:49
hemnathingee, os-brick landed.18:49
jgriffithflip214: wait... what am I looking at exactly?18:49
kauferjungleboyj: Maybe a complete list could be in that module, then the period task filtering code could just reference that and remove 'disabled'18:49
hemnathingee, I'm working on updating the docs for os-brick so we can do a release of it.18:49
hemnathen update requirements.txt18:49
flip214jgriffith: the remaining question is whether the DRBD transport functions should stay in the drbd cinder driver, or be relocated to brick.18:49
jgriffithflip214: says DRBD for nova, it's Cinder18:49
hemnathen get my WIP cinder patch out of WIP18:50
jungleboyjkaufer: That doesn't sound like a bad idea in case that changes in the future.18:50
flip214jgriffith: yes. My question is about cinder.18:50
jgriffithflip214: can you specify what you mean by "transport functions"18:50
flip214the nova part is ~60 lines, no problem there.18:50
jgriffithflip214: which ones specifically18:50
jgriffithand I can answer that maybe :)18:50
flip214initiate_connection, ensure_export, create_export, ...18:50
kauferjungleboyj: Thoughts on if I should also tackle that in my patch?18:50
flip214jgriffith: https://github.com/LINBIT/cinder/blob/drbd-for-nova/cinder/volume/drivers/drbdmanagedrv.py#L494 and below...18:51
jgriffithflip214: that all goes under cinder/volume/targets/18:51
flip214the first functions use your Target Driver (iSCSI), the ones below are used for the DRBD protocol.18:51
jgriffithflip214: if your driver needs/wants to override methods that are in iscsi target driver that's cool18:51
jungleboyjHmmm, kind-of a separate change, but closely related.  I am fine with lumping that in.18:52
jungleboyjhemna: ^^ ?18:52
hemnajungleboyj, which ?18:52
jungleboyjkaufer s question.18:52
jgriffithflip214: seems like what you have is good to me18:52
jgriffithflip214: not sure of the question18:53
*** krtaylor has joined #openstack-cinder18:53
*** gary-smith has quit IRC18:53
jgriffithflip214: just change the _DRBD ones to actually override the ones in the base target driver18:53
flip214jgriffith: my question was the same as in my email from january... is that okay, or should I move the DRBD transport functions into the transport/ directory? upon which you answered, you agree with me.18:53
jgriffithflip214: I can post an example of how I did this in my SolidFire driver18:54
jgriffithflip214: I did not make a seperate driver in volume/targets18:54
flip214jgriffith: if the DRBD transport is chosen, the functions are already monkeypatched.18:54
jgriffithflip214: I created the class in my driver file18:54
thingeehemna: understood you have plans for K. :) The question though is will we see something ready this week. I mentioned at the midcycle meetup I wanted this early in K-3.18:54
hemnathingee, https://review.openstack.org/#/c/155552/18:55
hemnathingee, https://review.openstack.org/#/c/156863/18:55
flip214jgriffith: okay. thanks, I believe to know the direction Cinder wants to take now.18:55
thingeethat's wip though. I want to start reviewing something for merge.18:55
hemnahow is that for starters?18:55
jgriffithflip214: https://gist.github.com/j-griffith/c22cfc9e7c7695b9354918:56
thingeebecause we're blocking multipath stuff, etc18:56
hemnaI should be able to finish up the docs changes for os-brick today....if I can get out of meetings18:56
thingeehemna: just send kmartin to all of them for you18:56
jgriffithflip214: of course you may or may not need more than "pass" for the impl's :)18:56
hemnathingee, at some point today I wanted to chat with you and jaypipes about multiattach as well.18:56
flip214jgriffith: my implementation now takes the "iscsi_helper" config option to distinguish...18:56
*** e0ne has joined #openstack-cinder18:57
*** e0ne is now known as e0ne_18:57
*** e0ne_ is now known as e0ne18:57
xyangthingee: I went through comments you gave on modify CG https://review.openstack.org/#/c/144561/, I think I've addressed all your comments there.18:59
xyangthingee: the race condition is probably in the 2nd patch, not in this one19:00
thingeexyang: ah you might be right19:00
thingeexyang, DuncanT: so with regards to cinder objects....19:00
*** timcl has joined #openstack-cinder19:00
thingeeDuncanT: I know you had some success in testing the migrations?19:00
xyangthingee: you have lots of comments on create cg from cg snapshot that I have not addressed: https://review.openstack.org/#/c/145952/19:01
thingeexyang: yeah my mistake. I was reading 2nd patch as 2nd patch set on that first review.19:02
*** EmilienM|afk is now known as EmilienM19:02
thingeexyang: not 2nd patch, completely separate19:02
thingeexyang: I think that's where I was noticing potential race issues19:02
thingeexyang: is that still planned for k?19:02
*** kaufer has quit IRC19:02
xyangthingee: ok, no problem.  I'm still planning for it, just takes more time19:02
leeanthothingee, yes, I'll be removing the WIP tag from that patch soon.  I was waiting for the next version of hplefthandclient to be available on PyPi before removing it19:02
thingeesounds good19:02
*** casusbelli has quit IRC19:02
thingeeleeantho: thanks19:03
*** rushil has quit IRC19:03
thingeexyang: are you going to be able to spend time testing cinder objects patch by doing some db migrations?19:03
*** timcl1 has quit IRC19:04
thingeexyang: DuncanT already mentioned he has done some testing. I'm not sure what the extent of his testing is though.19:04
xyangthingee: I could do that, but I'm still fighting a new destack install problem19:04
thingeexyang: ugh, ok19:04
*** bswartz has quit IRC19:04
thingeexyang: I've also been testing the patch myself19:04
xyangthingee: ok, if I get my setup fixed, I'll test it19:05
thingeexyang: thanks19:05
xyangthingee: np19:05
thingeexyang: hopefully we'll get an update on xtremeio patch soon so we can merge that19:05
xyangthingee: i have a question regarding that19:05
DuncanTThingee: I added a field to the db manually, and nothing broke. I changed the type of a field, and was able to write the adaptor in the snapshot object and the code didn't care19:05
thingeeDuncanT: that's wonderful news19:06
xyangthingee: is there a way to override a default value in cinder.conf?19:06
thingeexyang: yes19:06
xyangthingee: other than changes cinder.conf19:06
DuncanTthingee: It is hardly extensive testing, but it is enough to make me think it is at least a great start. We won't know for sure until we try to use it in anger19:06
thingeexyang: https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L201419:07
DuncanTthingee: Multi-attach will be a great workout for it, since it will be making some quite major schema changes19:07
xyangthingee: thanks19:07
thingeeDuncanT: agreed19:07
*** annegentle has quit IRC19:08
*** gary-smith has joined #openstack-cinder19:08
thingeeok, I'm going to sign off for a bit. Haven't had breakfast and it's almost lunch time here. I'll bbl everyone19:08
thingeethanks for the productive times19:08
*** kaufer has joined #openstack-cinder19:10
*** dannywilson has quit IRC19:12
*** nellysmitt has quit IRC19:13
*** nellysmitt has joined #openstack-cinder19:14
*** asmith_brcd has joined #openstack-cinder19:15
*** nellysmitt has quit IRC19:18
*** anshul has quit IRC19:18
openstackgerritTom Swanson proposed openstack/cinder: Dell sc driver iscsi multipath enhancement  https://review.openstack.org/15674219:23
*** thingee has quit IRC19:28
*** Guest83939 has joined #openstack-cinder19:29
*** annegentle has joined #openstack-cinder19:29
*** timcl has quit IRC19:30
*** dustins has quit IRC19:30
*** annegentle has quit IRC19:31
*** Guest83939 is now known as annegentle19:31
*** lcurtis has quit IRC19:32
*** dustins has joined #openstack-cinder19:33
*** timcl has joined #openstack-cinder19:33
*** Mandell has quit IRC19:39
*** bswartz has joined #openstack-cinder19:39
*** patrickeast has quit IRC19:41
*** lcurtis has joined #openstack-cinder19:51
*** patrickeast has joined #openstack-cinder19:54
*** rluediger has quit IRC19:55
*** Mandell has joined #openstack-cinder19:55
*** dustins has quit IRC19:58
openstackgerrithadi esiely proposed openstack/cinder: Store volume encryption metadata on each volume  https://review.openstack.org/15228419:59
*** Ilja has joined #openstack-cinder20:00
*** dustins has joined #openstack-cinder20:00
*** vilobhmm_ has quit IRC20:01
*** vilobhmm has quit IRC20:01
*** vilobhmm has joined #openstack-cinder20:02
*** vilobhmm1 has joined #openstack-cinder20:02
*** Ilja1 has joined #openstack-cinder20:03
*** Ilja has quit IRC20:04
*** Ilja1 has quit IRC20:07
*** aix has quit IRC20:08
*** lpetrut has quit IRC20:10
*** Ilja has joined #openstack-cinder20:11
*** vilobhmm1 has quit IRC20:12
*** fandi has joined #openstack-cinder20:17
*** fandi has quit IRC20:18
*** fandi has joined #openstack-cinder20:20
*** patrickeast has quit IRC20:20
*** patrickeast has joined #openstack-cinder20:24
*** annashen has quit IRC20:25
openstackgerritIvan Kolodyazhny proposed openstack/cinder: Allow archiving deleted rows to shadow tables, for performance  https://review.openstack.org/13118220:25
*** Yogi1 has quit IRC20:25
*** devlaps has quit IRC20:27
*** dannywilson has joined #openstack-cinder20:27
openstackgerritThang Pham proposed openstack/cinder: Snapshot and volume objects  https://review.openstack.org/13356620:37
*** fandi has quit IRC20:39
openstackgerritOpenStack Proposal Bot proposed openstack/python-cinderclient: Updated from global requirements  https://review.openstack.org/15557920:41
*** dustins has quit IRC20:44
*** devlaps has joined #openstack-cinder20:44
*** jungleboyj has quit IRC20:45
openstackgerritSteven Kaufer proposed openstack/cinder: Generic filter support for volume queries  https://review.openstack.org/15534320:48
openstackgerritSteven Kaufer proposed openstack/cinder: Replication status periodic task optimization  https://review.openstack.org/15467320:48
*** _cjones_ has quit IRC20:48
*** nkrinner has quit IRC20:50
*** amoturi has joined #openstack-cinder20:57
*** tbarron has quit IRC20:58
openstackgerritIvan Kolodyazhny proposed openstack/cinder: Allow archiving deleted rows to shadow tables, for performance  https://review.openstack.org/13118220:59
*** Apoorva_ has joined #openstack-cinder20:59
*** haomaiw__ has joined #openstack-cinder21:00
*** esker has quit IRC21:00
*** Apoorva has quit IRC21:02
*** IanGovett1 has joined #openstack-cinder21:02
*** mriedem1 has joined #openstack-cinder21:02
*** haomaiwang has quit IRC21:03
*** mriedem has quit IRC21:03
*** aix has joined #openstack-cinder21:04
*** IanGovett has quit IRC21:04
*** Ilja has quit IRC21:06
*** dustins has joined #openstack-cinder21:09
*** nellysmitt has joined #openstack-cinder21:11
*** _cjones_ has joined #openstack-cinder21:14
kauferI have a question about how the driver classes are architected, specifically what the "get_volume_stats" function is used for, anyone have time to help answer some questions?21:14
DuncanTkaufer: the return value of that function is passed up to the scheduler every 30 seconds, and is used to make volume placement (aka scheduling) decissions21:16
*** xyang has quit IRC21:18
kauferOk, thanks DuncanT, so every driver must have an implementation of that function? Either directly or in a parent class?21:18
*** xyang has joined #openstack-cinder21:18
openstackgerritxing-yang proposed openstack/cinder: Kilo Consistency Group API update  https://review.openstack.org/14456121:18
openstackgerritAnish Bhatt proposed openstack/cinder: Improve error handling in refactored Tgt driver  https://review.openstack.org/15471321:22
*** annashen has joined #openstack-cinder21:22
*** xyang has quit IRC21:23
kauferDuncanT: I was trying to find the base class that defines that function in order to understand what the expected signature should be, I found that not all drivers implement the 'refresh' as a kwarg and am curious if that is a bug?21:23
*** xyang has joined #openstack-cinder21:23
*** timcl has left #openstack-cinder21:24
*** amoturi has quit IRC21:25
*** jordandh has quit IRC21:27
*** vilobhmm1 has joined #openstack-cinder21:29
*** bswartz has quit IRC21:29
*** amoturi has joined #openstack-cinder21:30
DuncanTkaufer: Not sure, sorry. Check the code in cinder/volume/manager.py to see if it ever actually gets called with refresh actually set I guess21:31
*** Yogi1 has joined #openstack-cinder21:31
DuncanTkaufer: I suggest implementing refresh for now, it's an interesting question though21:32
*** jordandh has joined #openstack-cinder21:32
*** jungleboyj has joined #openstack-cinder21:33
kauferDuncanT: I pushed some code that contained "self.driver.get_volume_stats()" assuming that refresh would default to False, but that failed in the CI since 7 drivers implemented the function as "def get_volume_stats(self, refresh)", while all of drivers implemented as "def get_volume_stats(self, refresh=False)"21:34
kauferDuncanT: I planned on creating a bug to address the 7 drivers that didn't implment refresh as a kwarg, but wanted some feedback first21:35
jgriffithkaufer: DuncanT FYI the only place that's called externally that I remember is volume/manager21:38
jgriffithwhich sets refresh to "True"21:38
jgriffithand must to work correclty21:38
jgriffithcorrectly even :)21:38
jgriffithkaufer: but yeah, the fact that others didn't implement the kwarg at all is bad bad bad21:39
kauferjgriffith, DuncanT: bug and fix to add the kwarg to the other drivers is on the way21:39
jgriffithkaufer: sweeet!21:39
*** jamielennox|away is now known as jamielennox21:51
*** patrickeast has quit IRC21:54
*** annashen has quit IRC21:54
*** patrickeast has joined #openstack-cinder21:55
*** e0ne has quit IRC21:56
openstackgerritSteven Kaufer proposed openstack/cinder: Implement refresh as kwargs in get_volume_stats  https://review.openstack.org/15716521:57
kauferjgriffith: ^21:57
*** kaufer has quit IRC21:59
*** lcurtis has quit IRC22:00
jgriffithkaufer, looks like everybody had it correct except HP22:02
jgriffithalthough I think some might be missing the call altogether :(22:02
DuncanTIf it is always called with refresh=True, in L should we clean it up to get rid of the args all together? I never saw the point of it TBH22:03
jgriffithDuncanT: so some were using it internally as well and toggling that (I thought)22:04
jgriffithDuncanT: seems like it might be valid, but I agree, if it's not called anywhere seems pointless to have it22:04
jgriffithDuncanT: let somebody readd if they need it in the future :)22:05
DuncanTjgriffith: I'll file a bug, and patch all the drivers early in L.22:05
DuncanTjgriffith: I suspect it is a hold-over from copying hypervisor stats calls from nova22:06
smcginnisDuncanT: Didn't kaufer just do that?22:06
jgriffithDuncanT: works for me, assuming like I said nobody leverages it internally22:06
jgriffithDuncanT: oh.. interesting22:06
*** lcurtis has joined #openstack-cinder22:06
DuncanTsmcginnis: He filed a bug saying not everybody implements it as a kwarg22:06
jgriffithsmcginnis: DuncanT 's talking about just removing the kwarg altogether22:06
smcginnisjgriffith: Only a handful. I won't name names.22:06
DuncanTsmcginnis: I'm talking about getting rid of the arg completely22:06
*** kaufer has joined #openstack-cinder22:07
kauferjgriffith: thanks for the review, sorry i lost my connection22:08
*** karimb has joined #openstack-cinder22:08
*** changbl has quit IRC22:09
*** annashen has joined #openstack-cinder22:11
jgriffithkaufer: no worries22:11
DuncanTkaufer: Not sure how much of the discussion you saw, but I suggested in future we just get rid of the arg since it is always set to True in practice. I'll prepare a patch tomorrow22:11
DuncanTkaufer: I've +2'd your patch since I suspect mine will cause people to complain22:12
*** amoturi has quit IRC22:12
jgriffithDuncanT: haha!22:12
jgriffithDuncanT: only if they're using it :)22:12
kauferDuncanT: Please see my change here:  https://review.openstack.org/#/c/154673/7/cinder/volume/manager.py22:12
smcginnisjgriffith, DuncanT makes sense to me. If it can be simplified, great.22:13
kauferDuncanT: That's how I found this, I was originally just invoking get_volume_stats()22:13
*** esker has joined #openstack-cinder22:13
*** esker has quit IRC22:13
kauferDuncanT: Maybe I should instead have that set to True?22:13
DuncanTkaufer: We can just cache that value in the manager during the normal call, rather than calling into the driver again22:13
DuncanTkaufer: Or, since it is just once during init, we can just do True, yeah22:15
kauferDuncanT: Oh, so I shouldn't call into the driver again to get the 'replication_status' value?22:15
DuncanTkaufer: At init time, there's probably no harm in calling in with True22:16
*** Yogi11 has joined #openstack-cinder22:16
jgriffithDuncanT: +122:16
jgriffithforgot that's all cached in the mangaer22:16
jgriffithmanager even :)22:16
kauferDuncanT: I set it to refresh=False since the driver was just initialized and I didn't think a refresh would be necessary22:16
DuncanTkaufer: At that point, it has never been called, so most drivers will be doing a refresh anyway until they do it in their own init22:17
jgriffithkaufer: there's a "host_stats" or something like that in there that you could just use, no?22:17
*** Yogi1 has quit IRC22:17
DuncanTkaufer: You should probably make https://review.openstack.org/#/c/154673/ depend on https://review.openstack.org/#/c/157165/ too, it'll make the CI systems happier if nothing else22:17
DuncanTAnyway, I've just spotted it is nearly half midnight, time for me to be off22:18
*** nellysmitt has quit IRC22:18
DuncanTGood luck22:18
jgriffithDuncanT: I got your strace with times22:18
jgriffithDuncanT: well... it's in progress at least22:18
kauferDuncanT: Thanks for the help!22:18
jgriffithDuncanT: it's a frikin open call22:19
*** vilobhmm2 has joined #openstack-cinder22:20
*** annegentle has quit IRC22:20
*** vilobhmm1 has quit IRC22:22
*** ronis has quit IRC22:22
kauferjgriffith: Humm, I don't see any "host_stats" in the volume manager22:23
jgriffithkaufer: hehe.. yeah, that's because it's called "vol_stats"22:24
*** ronenkat__ has joined #openstack-cinder22:24
jgriffithoh wait...22:24
jgriffithhold on22:24
*** ronenkat has quit IRC22:24
jgriffithself.stats22:25
jgriffithgeesh22:25
*** ronenkat_ has quit IRC22:25
*** ronenkat has joined #openstack-cinder22:25
*** setmason has left #openstack-cinder22:26
*** setmason has joined #openstack-cinder22:26
*** jordandh has quit IRC22:27
*** Apoorva_ has quit IRC22:27
kauferjgriffith: humm, seems that that only has 'pools' and 'allocated_capacity_gb'22:28
*** Apoorva has joined #openstack-cinder22:28
*** jecarey has quit IRC22:28
jgriffithkaufer: oh... well that's not very useful22:28
jgriffithkaufer: never mind... :(22:28
kauferno :(22:28
kauferjgriffith: Ok, seems like we need to call into the driver to get it, so then it's just it should be refresh=True or False22:29
kauferI traced it out and the volume stats are loaded at the time the new call is made22:29
*** jcru has quit IRC22:30
*** jordandh has joined #openstack-cinder22:30
kauferjgriffith: But we may not want to rely on that, see my comment here:  https://review.openstack.org/#/c/154673/4/cinder/volume/manager.py22:30
kauferjgriffith: So now that I've talked in circles, I think that it is probably safest to just have refresh=True22:31
*** lcurtis has quit IRC22:39
*** mriedem1 has quit IRC22:42
*** Mandell has quit IRC22:46
*** tbarron has joined #openstack-cinder22:46
*** asselin_ has joined #openstack-cinder22:50
*** IanGovett1 has quit IRC22:52
*** IanGovett has joined #openstack-cinder22:53
*** jordandh has quit IRC22:54
*** asselin_ has quit IRC22:55
*** openstackgerrit has quit IRC23:00
*** dustins has quit IRC23:00
*** openstackgerrit has joined #openstack-cinder23:00
openstackgerritPatrick East proposed openstack/cinder-specs: Add DB table for driver private data  https://review.openstack.org/15333323:03
*** jordandh has joined #openstack-cinder23:06
openstackgerritThang Pham proposed openstack/cinder: Snapshot and volume objects  https://review.openstack.org/13356623:07
*** thangp has quit IRC23:08
*** primechuck has quit IRC23:09
*** Apoorva_ has joined #openstack-cinder23:09
openstackgerritTomoki Sekiyama proposed openstack/os-brick: Failover to alternative iSCSI portals on login failure  https://review.openstack.org/15718423:10
*** agentle_ has joined #openstack-cinder23:10
openstackgerritTomoki Sekiyama proposed openstack/cinder: Failover to alternative iSCSI portals on login failure  https://review.openstack.org/14087723:10
*** bswartz has joined #openstack-cinder23:11
*** thingee has joined #openstack-cinder23:12
*** Apoorva has quit IRC23:12
bswartzthingee: ping23:15
*** jordandh has quit IRC23:18
*** chlong has joined #openstack-cinder23:19
*** kaufer has quit IRC23:19
*** jordandh has joined #openstack-cinder23:21
thingeebswartz: hey23:22
thingeebswartz: how's third party test with FC going for you guys?23:23
*** karimb has quit IRC23:25
*** markstur has joined #openstack-cinder23:25
*** karimb has joined #openstack-cinder23:25
*** lcurtis has joined #openstack-cinder23:28
*** annashen has quit IRC23:32
*** annashen has joined #openstack-cinder23:34
*** MentalRay has quit IRC23:34
*** jordandh has quit IRC23:35
*** krtaylor has quit IRC23:40
*** karimb has quit IRC23:41
*** Sam-I-Am has joined #openstack-cinder23:42
Sam-I-Amhello23:43
sigmavirus24hey thingee we met at PyTN and I thought I'd introduce you to Sam-I-Am who's a coworker having issues with Cinder + Ceph23:43
rhe00thingee: how often do you want status updates on ongoing CI work?23:43
Sam-I-Amwell, i provide a lot of support in #openstack and a particular bug keeps coming up23:44
sigmavirus24Sam-I-Am: has a bug that multiple people have run into so I thought I'd point him towards you :)23:44
thingeerhe00: anytime you got something interesting to tell me. I look at third party ci meetings with asselin and anteaya to gauge participation as well.23:44
Sam-I-Amtrying to figure out if theres any way to get it some attention for kilo23:44
thingeerhe00: how are things going?23:45
thingeesigmavirus24, Sam-I-Am: hello!23:45
rhe00thingee: I have been struggling for two weeks now. hit every bug, gotcha, configuration issue on the planet23:45
sigmavirus24o/23:46
thingeerhe00: I hear that. This is what I was talking about at the midcycle meetup.23:46
Sam-I-Amhttps://bugs.launchpad.net/cinder/+bug/134881123:46
openstackLaunchpad bug 1348811 in Cinder "RBD + Nova migration fails because imports an existing disk" [High,Confirmed]23:46
rhe00so I am pretty frustrated but trying to move forward23:46
thingeerhe00: well there are options as you know. I know you've been talking with patrickeast for help too23:46
thingeerhe00: and patrickeast apparently got things working pretty quickly23:46
thingeejbernard: ^23:47
patrickeastrhe00: if you are stuck on any of the config stuff feel free to bug me23:47
*** ebalduf has quit IRC23:47
thingeeSam-I-Am: jdurgin and jbernard are who you want to talk to. I don't work on the ceph driver anymore, but they would be able to help with that part23:47
*** ho has joined #openstack-cinder23:47
rhe00patrickeast: right now I am stuck on the publickey issue when nodepool is trying to login to the node to configure it23:47
Sam-I-Amthingee: cool, thx. whats the best time/way to reach them?23:48
anteayarhe00: posting stacktraces on paste.o.o really helps us to understand where you are23:48
thingeeSam-I-Am: I'm surprised josh durgin is not around right now, but he's on around 10 PST23:49
thingeenot sure about jbernard23:49
patrickeastrhe00: ah yea, the keys are kind of a pain, basically you just need to make sure all of them match up from the configurations23:49
Sam-I-Amthingee: thx23:49
patrickeastrhe00: and by match up i mean23:49
thingeeI have been noticing bugs are building up on the rbd driver though https://bugs.launchpad.net/cinder/+bugs?field.tag=rbd23:49
patrickeastrhe00: use the same key for all the nodepool stuff23:49
sigmavirus24thanks thingee23:50
Sam-I-Amthingee: i think because its becoming more popular?23:50
thingeeI will be meeting with sage (creator of ceph) himself this week...I was going to poke him to put a resource to help with the build up of bugs I've been noticing23:50
thingeeSam-I-Am: most definitely23:50
thingeeSam-I-Am: I just don't want them to forget about us :)23:50
rhe00patrickeast: I do use the same key. are we supposed to set up an ubuntu user and a jenkins user for nodepool or just a jenkins user? not quite clear on which user nodepool is trying to use when it hits the publickey auth failure23:51
rhe00patrickeast: I thought it would use the user specified in the nodepool.yaml file, but there is a comment in there somewhere about an ubuntu user as well23:51
patrickeastrhe00: yea, so this keypair needs to be in your openstack provider https://github.com/patrick-east/os-ext-testing-data/blob/master/etc/nodepool/nodepool.yaml.erb#L34 and the images need to be able to use it23:53
patrickeasteither through the openstack key injection magic, or pre-configured23:53
thingeepatrickeast: thanks for helping :)23:53
patrickeastif you are using the ubuntu cloud images it should set it up for you as long as the keypair is setup23:53
patrickeastand the name matches*23:53
patrickeastthingee: np23:53
rhe00patrickeast: one more question. how much space do you allocate for the disk on the slave node?  I read somewhere that I should have 40GB of disk space, but that seems like an awful lot.  Had to bind mount myself out of trouble as nova ran out of space in the instances folder on local disk.23:54
patrickeastrhe00: sec, checking what i made for my flavor23:54
asselinrhe00, ours are 'large' flavor23:55
patrickeastdevstack-node88192MB40GB0GB4096MB23:55
patrickeastso 4gb swap and 40 gb root23:55
rhe00asselin: that's 80 GB of hard drive, right?23:55
asselini need to check23:55
rhe00patrickeast: ok23:55
asselinnova devstack large :)23:55
*** jwcroppe has quit IRC23:56
*** asmith_brcd has quit IRC23:56
patrickeasti had to limit the disk space a bunch since our initiators don’t have very much local storage23:56
*** jwcroppe has joined #openstack-cinder23:56
asselinyes 80GB23:56
*** jordandh has joined #openstack-cinder23:56
patrickeastin the longer term i want to add support to nodepool to use volumes on the flash array23:56
patrickeastsince its sitting there connected to the initiator anyway23:56
asselinbut it doesn't use that much....we run 6 on 300GB23:57
asselinpatrickeast, it's not a nodepool issue...it's openstack provider.23:57
patrickeasthmm? i think its just a matter of adding in the flow to create volumes from images and booting from them instaed of direct boot from image23:58
asselinpatrickeast, i.e. you can configure nova & glance to use your storage array for images & instances23:58
patrickeastor maybe there is a shorter step i don’t know about23:58
asselinpatrickeast, ok i see23:58
patrickeastit would be cool too to have snapshots instead of images, going snapshot to new volume is almost a no-op for us23:59
rhe00patrickeast, asselin: thanks for the help so far! I'll keep working on it.23:59
patrickeastrhe00: np23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!