Friday, 2014-08-08

eglynnjogo: "just about to push" ==> to nova-specs?00:01
*** flwang_ has joined #openstack-ceilometer00:01
*** fnaval has quit IRC00:01
*** fnaval has joined #openstack-ceilometer00:01
openstackgerritPradeep Kilambi proposed a change to openstack/ceilometer: Support for per disk volume measurements
*** fnaval has quit IRC00:06
pradeglynn: ^^ incorporated the changes discussed. Please review00:08
eglynnprad: thanks for the quick turnaround, will probably be tmrw by the time I get to it (~1am here now)00:08
pradah np00:10
*** Qiming has quit IRC00:10
pradeglynn: have a good night :).. talk to you tomorrow00:10
eglynnprad: cool, you too!00:11
*** Dafna has joined #openstack-ceilometer00:11
*** flwang_ has quit IRC00:14
*** drjones has quit IRC00:20
*** _cjones_ has joined #openstack-ceilometer00:21
*** itisit has quit IRC00:29
*** sbfox has joined #openstack-ceilometer00:29
*** eglynn has quit IRC00:30
openstackgerritA change was merged to openstack/pycadf: Debug env for tox
*** amalagon_ has joined #openstack-ceilometer00:44
*** dmsimard is now known as dmsimard_away00:50
*** raymondr has joined #openstack-ceilometer01:01
*** Qiming has joined #openstack-ceilometer01:04
*** raymondr has quit IRC01:05
*** _cjones_ has quit IRC01:09
*** _cjones_ has joined #openstack-ceilometer01:09
*** _cjones_ has quit IRC01:15
*** ddieterly has joined #openstack-ceilometer01:24
*** flwang_ has joined #openstack-ceilometer01:39
*** atan8 has joined #openstack-ceilometer01:43
*** nosnos has joined #openstack-ceilometer01:45
*** flwang_ has quit IRC01:48
*** thomasem has joined #openstack-ceilometer01:59
*** thomasem has quit IRC02:04
*** raymondr has joined #openstack-ceilometer02:08
*** flwang_ has joined #openstack-ceilometer02:11
*** dmsimard_away is now known as dmsimard02:12
*** atan8 has quit IRC02:12
*** fnaval has joined #openstack-ceilometer02:23
*** flwang_ has quit IRC02:28
*** mitz has joined #openstack-ceilometer02:30
*** sbfox has quit IRC02:44
*** fnaval has quit IRC02:50
*** fnaval has joined #openstack-ceilometer02:50
*** dmsimard is now known as dmsimard_away02:52
*** fnaval has quit IRC02:55
*** raymondr has quit IRC02:59
*** nosnos has quit IRC03:04
*** sbfox has joined #openstack-ceilometer03:17
*** fnaval has joined #openstack-ceilometer03:28
openstackgerritMike Spreitzer proposed a change to openstack/ceilometer: Documented Stevedore usage and source details
sammeglynn: thanks for the ceilo.config file  , but in this what is use of rabbit?. Is it necessary to use that? do we need to install rabbit?03:43
*** dmsimard_away is now known as dmsimard03:59
*** dmsimard is now known as dmsimard_away04:04
*** nosnos has joined #openstack-ceilometer04:08
*** _nadya_ has joined #openstack-ceilometer04:10
*** fnaval has quit IRC04:16
*** fnaval has joined #openstack-ceilometer04:16
sammHey all, can you please tell me is there any need to use rabbit in order to use ceilometer?04:19
*** flwang_ has joined #openstack-ceilometer04:29
*** flwang_ has quit IRC04:33
*** ildikov has quit IRC04:36
*** jogo has left #openstack-ceilometer04:43
*** _cjones_ has joined #openstack-ceilometer04:45
*** atan8 has joined #openstack-ceilometer04:45
*** amalagon_ has quit IRC04:48
*** _cjones_ has quit IRC04:57
*** _cjones_ has joined #openstack-ceilometer04:57
*** drjones has joined #openstack-ceilometer04:58
*** drjones has quit IRC04:59
*** drjones has joined #openstack-ceilometer05:00
*** _cjones__ has joined #openstack-ceilometer05:01
*** atan8_ has joined #openstack-ceilometer05:01
*** atan8 has quit IRC05:01
*** _cjones_ has quit IRC05:02
*** atan8_ has quit IRC05:03
*** atan8- has joined #openstack-ceilometer05:03
*** atan8- has quit IRC05:04
*** Longgeek has joined #openstack-ceilometer05:04
*** drjones has quit IRC05:04
*** _cjones__ has quit IRC05:07
*** _cjones_ has joined #openstack-ceilometer05:08
*** _cjones_ has quit IRC05:20
*** _cjones_ has joined #openstack-ceilometer05:21
*** flwang_ has joined #openstack-ceilometer05:23
*** fnaval has quit IRC05:24
*** fnaval has joined #openstack-ceilometer05:25
*** _cjones_ has quit IRC05:26
*** fnaval has quit IRC05:30
*** flwang_ has quit IRC05:30
sammHi All, I hav eproblem in ceilometer when running command:  ceilometer-agent-notification    It gives error: AMQP server on localhost:5672 is unreachable05:32
*** Longgeek has quit IRC05:54
*** Longgeek has joined #openstack-ceilometer05:54
*** Longgeek_ has joined #openstack-ceilometer05:55
*** _nadya_ has quit IRC05:57
*** Longgeek has quit IRC05:59
*** k4n0 has joined #openstack-ceilometer06:02
*** ildikov has joined #openstack-ceilometer06:13
*** _nadya_ has joined #openstack-ceilometer06:27
*** Titilambert has quit IRC06:32
*** ildikov has quit IRC06:33
*** flwang_ has joined #openstack-ceilometer06:38
*** idegtiarov_ has joined #openstack-ceilometer06:44
openstackgerritHisashi Osanai proposed a change to openstack/ceilometer: enable to get swift's service type from configuration file
*** idegtiarov has quit IRC06:46
*** idegtiarov_ has quit IRC06:46
*** idegtiarov has joined #openstack-ceilometer06:46
*** ildikov has joined #openstack-ceilometer06:50
*** ildikov has quit IRC06:50
*** ildikov has joined #openstack-ceilometer06:51
*** safchain has joined #openstack-ceilometer06:52
*** _nadya_ has quit IRC07:03
*** _nadya_ has joined #openstack-ceilometer07:17
*** arezmerita has joined #openstack-ceilometer07:27
*** Dafna has quit IRC07:35
*** Dafna has joined #openstack-ceilometer07:37
*** flwang_ has quit IRC07:47
*** nosnos has quit IRC07:52
*** eglynn has joined #openstack-ceilometer07:53
*** nosnos has joined #openstack-ceilometer07:54
*** IvanBerezovskiy has joined #openstack-ceilometer08:03
*** flwang_ has joined #openstack-ceilometer08:13
*** sbfox has quit IRC08:15
*** flwang_ has quit IRC08:18
*** Longgeek_ has quit IRC09:11
*** Longgeek has joined #openstack-ceilometer09:11
*** promulo has quit IRC09:12
*** promulo has joined #openstack-ceilometer09:13
*** Longgeek has quit IRC09:16
*** Longgeek has joined #openstack-ceilometer09:23
*** claudiub has joined #openstack-ceilometer09:34
*** _nadya_ has quit IRC09:39
*** Longgeek has quit IRC09:43
*** Longgeek has joined #openstack-ceilometer09:44
*** Longgeek has quit IRC09:47
*** cdent has joined #openstack-ceilometer09:50
*** Qiming has quit IRC09:54
*** pnascimento has joined #openstack-ceilometer10:11
*** nosnos has quit IRC10:16
*** nosnos has joined #openstack-ceilometer10:17
*** flwang_ has joined #openstack-ceilometer10:18
*** ccrouch1 has joined #openstack-ceilometer10:21
*** alexpilotti has joined #openstack-ceilometer10:22
*** flwang_ has quit IRC10:22
*** promulo has quit IRC10:23
*** rwsu has quit IRC10:23
*** ccrouch has quit IRC10:23
*** promulo has joined #openstack-ceilometer10:25
*** nosnos has quit IRC10:26
*** rwsu has joined #openstack-ceilometer10:27
*** pnascimento has quit IRC10:45
*** _nadya_ has joined #openstack-ceilometer10:49
*** Qiming has joined #openstack-ceilometer10:52
*** renatoarmani has joined #openstack-ceilometer11:49
*** Longgeek has joined #openstack-ceilometer11:56
*** julim has joined #openstack-ceilometer12:01
*** vrovachev has joined #openstack-ceilometer12:03
*** _nadya_ has quit IRC12:06
*** flwang_ has joined #openstack-ceilometer12:19
*** flwang_ has quit IRC12:24
*** eglynn is now known as eglynn-lunch12:29
*** prad has quit IRC12:33
*** _nadya_ has joined #openstack-ceilometer12:37
*** thomasem has joined #openstack-ceilometer12:44
*** thomasem has quit IRC12:44
*** eglynn-lunch has quit IRC12:44
*** thomasem has joined #openstack-ceilometer12:45
sammeglynn: Hi ceilometer problem is resolved to some extent, I think.12:46
*** dhellmann_ is now known as dhellmann12:47
*** _nadya_ has quit IRC12:48
*** _nadya_ has joined #openstack-ceilometer12:49
openstackgerritIlya Tyaptin proposed a change to openstack/ceilometer: Improve performance of libvirt inspector requests
*** promulo has quit IRC12:51
*** gordc has joined #openstack-ceilometer12:56
*** ddieterl_ has joined #openstack-ceilometer13:04
*** ddieterly has quit IRC13:04
*** _nadya_ has quit IRC13:04
*** fnaval has joined #openstack-ceilometer13:04
*** ddieterl_ has quit IRC13:17
*** ddieterly has joined #openstack-ceilometer13:18
*** promulo has joined #openstack-ceilometer13:31
*** eglynn-lunch has joined #openstack-ceilometer13:32
*** renatoar_ has joined #openstack-ceilometer13:32
*** renatoarmani has quit IRC13:34
*** _nadya_ has joined #openstack-ceilometer13:35
gordcityaptin: do you know what the the profile_collector_performance command would look like if i ran it against a default devstack?13:37
*** dmsimard_away is now known as dmsimard13:40
ityaptingordc: Hi!13:40
*** ddieterly has quit IRC13:44
ityaptingordc: command will look the same as command it example. But you have to prepare config file and db urls for normal test running.13:44
ityaptingordc: sorry, "command it example" -> "command in example" :)13:45
eglynn-officesamm: good news! :(13:46
eglynn-officesamm: good news! :)13:46
* eglynn-office doesn't know his smiles from his frowns ;)13:47
gordcityaptin: hmm. i'll give that a try.13:47
gordccan you also expand on step3. regarding venv?13:48
*** russellb is now known as rustlebee13:50
ityaptin gordc: it's venv with installed my fork of ceilometer.  I'll add it to doc.13:51
*** jasondotstar has joined #openstack-ceilometer13:52
ityaptingordc: Oh, I only reply phrase in doc. What kind of info about venv you want to see?13:53
*** jasondotstar has quit IRC13:54
*** jasondotstar has joined #openstack-ceilometer13:54
gordcityaptin: i'm not sure what it means... could you like example commands i should run? (it's friday, so my brain is off)13:55
ityaptingordc: give me a minute :)13:56
gordcawesome. no rush.13:57
sammegylnn:Can you please tell me is it necessary to use rabbitMQ with celiometer13:57
*** eglynn-lunch is now known as eglynn13:59
eglynnsamm: it isn't, you could use qpid instead14:00
eglynnsamm: (but of course you do need *some* AMQP provider ... almost nothing in OpenStack will work without AMQP)14:01
*** prad has joined #openstack-ceilometer14:02
*** fnaval has quit IRC14:02
*** Longgeek has quit IRC14:04
openstackgerritgordon chung proposed a change to openstack/ceilometer: normalise resource data
ityaptingordc: Please, create file /tmp/ceilometer.conf with this: Also be accurate with credentials of your keystone and rabbit. Please, change it in file if they are different.14:06
ityaptingordc: 2 step: Create file /tmp/db_urls with your backend urls. for example14:08
ityaptingordc: Install ceilometer for with "python install"  and run command from
pradHello _nadya_ , could you please give your opinion on this discussion
prad_nadya_,  eglynn and gordc agree on the latest approach, just want to make sure you're cool with it as well14:10
*** flwang_ has joined #openstack-ceilometer14:20
*** vrovachev has quit IRC14:24
*** flwang_ has quit IRC14:25
*** fnaval has joined #openstack-ceilometer14:26
*** mspreitz has joined #openstack-ceilometer14:26
*** Titilambert has joined #openstack-ceilometer14:30
*** arezmerita has quit IRC14:35
*** k4n0 has quit IRC14:39
cdenteglynn: new version of (javelin) is up. I think this gets the closest I'm going to be able to get14:40
eglynncdent: thanks! ... looking14:42
*** _nadya_ has quit IRC14:45
*** _nadya_ has joined #openstack-ceilometer14:45
*** raymondr_ has joined #openstack-ceilometer14:46
_nadya_prad: sorry, I was afk. I will take a look in 2-3 hours, is it ok?14:46
prad_nadya_, sure, that would be great14:46
openstackgerritPradeep Kilambi proposed a change to openstack/ceilometer: Support for per disk volume measurements
pradeglynn: addressed rest of your concerns ^^14:48
*** mspreitz has quit IRC14:52
*** raymondr_ has quit IRC14:53
gordcityaptin: have you seen this?
*** sbfox has joined #openstack-ceilometer14:56
*** ildikov has quit IRC14:56
*** sbfox has quit IRC14:59
*** sbfox has joined #openstack-ceilometer14:59
*** _cjones_ has joined #openstack-ceilometer15:01
*** raymondr_ has joined #openstack-ceilometer15:02
*** IvanBerezovskiy has left #openstack-ceilometer15:02
*** renatoar_ has quit IRC15:02
*** renatoarmani has joined #openstack-ceilometer15:02
*** sbfox has quit IRC15:03
eglynncdent: just reviewed that, a few suggestions inline15:05
eglynnprad: thanks, will look shortly15:06
*** Qiming has quit IRC15:10
*** _cjones_ has quit IRC15:10
*** _cjones_ has joined #openstack-ceilometer15:10
*** _cjones_ has quit IRC15:15
openstackgerritEugeniya Kudryashova proposed a change to openstack/python-ceilometerclient: Add endpoint opt into auth_plugin
*** jaypipes has joined #openstack-ceilometer15:31
*** ddieterly has joined #openstack-ceilometer15:32
*** jaypipes is now known as leakypipes15:32
boris-42eglynn hi15:33
eglynnboris-42: yo15:34
boris-42eglynn ^ recently there were bug merged in ceilometer python client15:34
boris-42eglynn that killed rally gaes=)15:34
boris-42eglynn here is the fix
pradeglynn: sorry little confused with your new comments ... dint i already do that?15:34
eglynnboris-42: is that related to ?15:35
boris-42eglynn lemme see15:35
*** raymondr_ has quit IRC15:36
*** andreykurilin has joined #openstack-ceilometer15:36
eglynnprad: you left in the unnecessary "if expected_device:" (line 62)15:36
boris-42eglynn it's differnt15:37
pradeglynn: aha15:38
eglynnboris-42: can a bug be filed for this issue and then linked in the commit message?15:38
boris-42eglynn sure thing15:38
*** _cjones_ has joined #openstack-ceilometer15:39
openstackgerritPradeep Kilambi proposed a change to openstack/ceilometer: Support for per disk volume measurements
boris-42eglynn oh need this one
*** claudiub has quit IRC16:00
openstackgerritEugeniya Kudryashova proposed a change to openstack/python-ceilometerclient: Add endpoint opt into auth_plugin
*** safchain has quit IRC16:04
*** _nadya_ has quit IRC16:10
*** fabiog has joined #openstack-ceilometer16:11
*** flwang_ has joined #openstack-ceilometer16:21
*** itisit has joined #openstack-ceilometer16:23
*** raymondr_ has joined #openstack-ceilometer16:23
eglynn-officelsmola: are you around?16:24
eglynn-officelsmola: (related to that dupe'd sample issue you reported on IRC earlier in the week)16:25
*** flwang_ has quit IRC16:25
*** sbfox has joined #openstack-ceilometer16:48
cdentgordc: you around?16:55
*** sbfox has quit IRC16:58
*** renatoarmani has quit IRC17:02
*** fnaval has quit IRC17:06
*** fnaval has joined #openstack-ceilometer17:07
*** sbfox has joined #openstack-ceilometer17:10
*** samm has quit IRC17:11
*** fnaval has quit IRC17:11
*** fnaval has joined #openstack-ceilometer17:29
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Avoid duplication of discovery for multi-sink sources
*** sbfox1 has joined #openstack-ceilometer17:33
*** eglynn-office is now known as eglynn-officeafk17:34
gordccdent: stepped out for lunch.17:34
cdentno worries gordc, just been poking at get_meters on your branch17:35
gordccool! find anything interesting?17:35
cdentI'm trying to speed up my tests for my experiments: What's the best way to say "only run db related tests against mysql"?17:35
*** sbfox has quit IRC17:35
gordccdent: well you can pass in the list of tests... something like 'tox -epy27'... not sure if it's possible to select only sql tests though17:37
gordcyou might need to do some modifications.17:37
cdentnext question:17:38
cdentwhen you run `ceilometer meter-list` what is that we are actually expecting back: a list of unique meter names, or a list of unique meter/resource-id/user-id/project-id  tuples?17:39
gordccdent: that's what i was trying to figure out yesterday... it's somewhat convoluted...17:41
*** eglynn has quit IRC17:41
cdentIf it is the latter, I think the query can be simplified by quite a lot17:42
gordcas i understand it. it should return back a list of unique meter/resource17:42
gordcbut not user/project17:42
gordcthe best way to figure it out would be to see how mongodb backend gets meter data17:42
cdentyeah, that's what I'm planning to do17:43
cdentI'll likely post up my changes as not-be-merged patchset dependent on yours, for sake of comparison. Is that good, or would you prefer something different?17:43
gordci probably stared at the query for too long but all the simpler solutions i could think of didn't satisfy the tests we have written.17:43
cdentHa, yeah, that's part of why I ask, I'm not sure the tests are necessarily right...17:44
gordcthat's fine. there's better way but i'm not sure how to do it... :)17:45
*** _nadya_ has joined #openstack-ceilometer17:45
gordccdent: does your patch involve rewriting tests?17:45
cdentnot yet, no17:45
cdentI'm just struggling to understand a failure I'm seeing17:45
gordcok... yeah, i wouldn't mind hashing out a more logic backend design... but i think we may need to push changes involving test changes to v317:46
gordc"I'm just struggling to understand a failure I'm seeing"  that was me the past two days.17:46
*** raymondr_ has quit IRC17:46
*** fnaval has quit IRC17:55
*** alexpilotti has quit IRC17:55
*** fnaval has joined #openstack-ceilometer17:56
nealphdhellmann: are there any reference materials to go to for dealing with how to deploy pecan in a multi-threaded way?17:58
*** sbfox1 has quit IRC17:58
*** sbfox has joined #openstack-ceilometer17:59
ryanpetrellotl;dr use uwsgi, gunicorn, or apache+mod_wsgi18:00
*** fnaval has quit IRC18:00
nealphryanpetrello: concern is that we're seeing a performance hit when adding Apache, so now we're looking for thread-handling options that would allow us to avoid the apache+mod_wsgi approach...18:02
ryanpetrelloyea, I generally *do not* recommend Apache to folks deploying WSGI apps (if they can help it)18:03
nealphperformance hit = increase in api response time.18:03
ryanpetrelloI think you’ll be much happier w/ uwsgi or dunicorn18:03
nealphcool..but if I'm trying to avoid adding package dependancies is there any tuning that can be done?18:05
* nealph is being a bit difficult :)18:05
cdentyou can add more workers and more threads to the wsgidaemon (if you're running in that mode) nealph18:08
cdentare you getting latency under load, or just plain latency18:09
nealphcdent: latency under load...good question.18:10
*** ssakhamuri has joined #openstack-ceilometer18:10
cdentbecause if it is just plain slow per request, I've got no useful advice :)18:10
*** eglynn has joined #openstack-ceilometer18:11
nealphcdent: curious about adding more threads...via wsgi config or some other methodology (feel free to punt me to docs at this point)18:12
cdentI suspect some of this stuff is out of date but: and18:13
*** jasondotstar has quit IRC18:15
nealphand with that, we're back with apache. :D Does Nova api use mod_wsgi?18:15
* nealph starts to dig through code...18:16
cdentnealph: I thought you want to stick with apache?18:16
cdentwhat thing that is running pecan are you making go?18:18
*** _nadya_ has quit IRC18:21
nealphcdent: well, no, Apache introduces latency on a per-request basis. Now, you might be thinking "you'll have to trade horizontal scalability for a bit of per-request latency"...maybe so. This is in support of a scaled-out ceilometer-api.18:21
*** flwang_ has joined #openstack-ceilometer18:22
cdentmount ceilometer/api/app.wsgi under whatever wsgi server you like? uwsgi is my choice these days.18:23
cdentbiab, must dine18:24
*** flwang_ has quit IRC18:26
eglynngordc: hola! ... quick question about bigger-data-sql
eglynngordc: does it feel like there's enough consensus now to get the spec landed?18:46
eglynn(assuming a fresh spec patchset, updated to line up the etherpad and the WIP patch)18:46
*** rbowen has quit IRC18:49
*** rbowen has joined #openstack-ceilometer18:49
*** _cjones_ has quit IRC18:54
*** drjones has joined #openstack-ceilometer18:55
gordceglynn: sorry about that. was in a meeting... i think i'm ok with the current design. i'll refresh the spec to reflect the latest patchset.18:58
eglynngordc: great, thank you sir!19:02
cdenteglynn, did you see the conversation including this question earlier: when you run `ceilometer meter-list` what is that we are actually expecting back: a list of unique meter names, or a list of unique meter/resource-id/user-id/project-id  tuples?19:02
eglynncdent: sorry missed that, on the bike home19:03
cdentthe more I look at the code the more I think that earlier optimizations done to reduce size of the samples being joined against resource info has lead to weird tests or expectations19:03
eglynncdent: IIUC it would be, unique (meter, resource-id) pairs19:03
*** ccrouch1 has quit IRC19:03
cdentThat's my guess too, but that's not what this test verifies:19:03
* eglynn looks19:05
cdentit wants a result of 119:05
cdentbut the availble results that match 'user-id' are:19:05
cdent[(u'instance', u'resource-id', u'user-id'), (u'instance', u'resource-id-alternate', u'user-id')]19:05
cdentthis subquery:
cdentlimits the results19:06
eglynnhmmm, I'm missing something here obviously19:07
gordccdent: iiuc, it's because mongo doesn't store all unique resources (unlike sql)... it only stores the latest instance of resource.19:07
eglynnso the pairs in this case being (instance, resource-id) and (instance, resource-id-alternate), or?19:07
cdent(at least I think so, I get the results above when I turn the SQL into something much more simple)19:07
eglynncdent: what values did you expect to see?19:08
cdentthe answer I expect, given your expression of "unique (meter, resource-id) paris" is the list I've pasted above19:09
cdentTo get the results that mongo presents you have to do some perilously complex SQL19:09
cdentand the only reason mongo presents the results it does it because it doesn't keep as much data around19:09
*** fnaval has joined #openstack-ceilometer19:10
gordccdent: i actually ran down this path yesterday... it's because of above reason... it only keeps latest record of resource. so in mongodb, the resource-id-alternate resource exists... but it doesn't have that user-id (because a later sample changes it)19:11
* cdent nods19:11
gordci guess you're asking if that is right or wrong?19:11
eglynnyeah just looking at how the test sets up the data19:11
eglynnthat's the artificial thing I think19:11
cdentIt seems.... unfair... to expect the sql side to bend over backwards to produce what is in fact incorrect results19:11
eglynnas in, artificial == "not reflecting reality"19:12
cdentespecially when the sql side could be much simpler19:12
eglynn^^^ same resource-id, different user-id ==> I don't think that can happen in reality in openstack19:12
eglynni.e. there's no mechanism I know of to reassign ownership of an existing resource19:13
*** _nadya_ has joined #openstack-ceilometer19:13
gordceglynn: i'm not sure it can either.. but we do have a lot of tests that are written to test this.19:13
cdentis that a bug in the tests we ought to fix before trying to fix the database?19:13
cdentbecause if we are trying to fix the database to bad tests, we're going to go through unnecessary hoops19:13
eglynncdent: I think mongo is doing the realistic thing, just storing the latest resource representation19:14
cdentthe sql I've got going reverses the join, so we are only pulling stuff out of the Sample table as needed, rather than needing to use the Sample as a starting point19:14
cdenteglynn: that's the long term goal for the sql side too, but for the sake of incremental changes...19:15
gordccdent: i'm not sure it's a code bug becasue mongo/hbase backends explicitly support it (through column updates)... it might be a design bug for 'realworld' case.19:15
cdentwith my sql there explain shows three simple steps instead of 5 primary or derived tables19:15
eglynncdent: so your sql change is to stop producing the "incorrect results" that match what mongo produces?19:16
cdentmy sql change returns "unique (meter, resource-id) pairs"19:17
gordccdent: i know what you're talking about... i could've built a much simpler query... but it just wouldn't work with tests so i figured i'd leave it for v319:17
gordci think the current query matches current performance... but i can't verify with hard numbers because i can't run ilya's tests.19:18
cdentgordc: I figured that was the case and I was probably beating on already beaten walls...19:18
eglynncdent: so without your change the test produces: [(u'instance', u'resource-id', u'user-id'), (u'instance', u'resource-id-alternate', u'user-id')] amiright?19:18
cdentgordc: But just looking at the generated queries it seems rather too obvious the ways in which things are broken it is hard to not want to fix!19:18
gordccdent: hopefully you find a different path... i kept trying to simplify query... then i got tired19:18
eglynncdent: whereas with your change the test produces what exactly?19:19
cdenteglynn: the opposite, without my change, the test produces just one result, with my change it produces the list19:19
gordceglynn: it would just produce first item19:19
gordcthe second item exists with different user id in mongo/(probably hbase)19:20
cdentgiven what is stored in the database, and definition of "meter-list", my results are "correct"19:20
gordc this sample overwrites19:20
cdent(by some strange definition of "correct")19:20
eglynncdent: I don't follow, are (instance, resource-id) & (instance, resource-id-alternate) not unique pairs?19:21
gordccdent: i think the issue is that mongo/hbase only store latest... but to do that in sql would be bad for performance because of all the updates19:21
cdentthey are unique paris, they are a list of two items, the _original_ code demands 119:21
cdentthe original code, for sql, achieves the result of "1" by trimming the available data in  a subquery19:22
cdentin order to get results that mirror mongo19:22
eglynnok, got it ... finally ;)19:23
cdentmy contention is that that subquery is actually resulting in incorrect results, if you take the whole database into account19:23
gordceglynn: it took me 2 days to get to where you are.19:23
cdent_and_ is also really expensive19:23
eglynnTBH this is a highly artificial situation, if it can't happen in the wild19:23
*** raymondr_ has joined #openstack-ceilometer19:23
cdentI've done manual tests (making some instances, listing some meters) and the results are sensible19:24
* eglynn wonders if we shouldn't just strip out all those multi-{user|project|source}-per-resource tests?19:24
eglynn... seeing as they're costing us in terms of confusion and complexity in the tests19:24
gordceglynn: i wondered same thing... thought it be safer as part of v3.19:25
eglynn... yet seem to add no value, since those scenarios can't occur19:25
eglynngordc: yeah, fair point19:25
cdentmeh! :)19:26
eglynn... a typical Friday evening suggestion :)19:26
*** ildikov has joined #openstack-ceilometer19:28
*** eglynn has quit IRC19:32
cdentfor my own sense of friday evening edification I'm going to keep traveling down my current path, just to see what I can learn19:32
gordccdent: can you make note how many tests need to be changed?19:35
cdentyes, will do19:35
cdentjust made one change just now, and am running again19:35
*** ildikov has quit IRC19:41
openstackgerritgordon chung proposed a change to openstack/ceilometer-specs: 'big-data' sql part 2
boris-42gordc hi19:46
*** renatoarmani has joined #openstack-ceilometer19:54
*** julim has quit IRC20:06
*** openstackgerrit has quit IRC20:20
*** flwang_ has joined #openstack-ceilometer20:23
*** renatoarmani has quit IRC20:25
*** promulo has quit IRC20:26
*** flwang_ has quit IRC20:27
cdentThat's as far as I'm going to get today. Hard to say, yet, if it will be promising.20:34
*** cdent has quit IRC20:34
*** _nadya_ has quit IRC20:36
*** robsparker has quit IRC20:42
*** dmsimard is now known as dmsimard_away20:46
*** robsparker has joined #openstack-ceilometer21:00
*** itisit has left #openstack-ceilometer21:06
*** gordc has quit IRC21:07
*** thomasem has quit IRC21:24
*** atan8 has joined #openstack-ceilometer21:49
*** fnaval has quit IRC21:55
*** fnaval has joined #openstack-ceilometer21:56
*** ddieterly has quit IRC21:58
*** fnaval has quit IRC22:00
*** raymondr_ has quit IRC22:05
*** fnaval has joined #openstack-ceilometer22:13
*** raymondr has joined #openstack-ceilometer22:17
*** flwang_ has joined #openstack-ceilometer22:24
*** prad has quit IRC22:24
*** drjones has quit IRC22:26
*** _cjones_ has joined #openstack-ceilometer22:26
*** flwang_ has quit IRC22:29
*** openstackgerrit has joined #openstack-ceilometer22:38
*** jdonalds has joined #openstack-ceilometer22:44
*** raymondr has quit IRC22:46
*** jdonalds has quit IRC22:50
*** _cjones_ has quit IRC22:51
*** _cjones_ has joined #openstack-ceilometer22:51
*** flwang_ has joined #openstack-ceilometer23:00
*** _cjones_ has quit IRC23:17
*** _cjones_ has joined #openstack-ceilometer23:18
*** flwang_ has quit IRC23:27
*** _nadya_ has joined #openstack-ceilometer23:37
*** _nadya_ has quit IRC23:41
*** promulo has joined #openstack-ceilometer23:55

Generated by 2.14.0 by Marius Gedminas - find it at!