| boris-42 | ildikov ping | 00:02 |
|---|---|---|
| *** _cjones_ has quit IRC | 00:27 | |
| *** fnaval has joined #openstack-ceilometer | 00:39 | |
| *** liusheng has quit IRC | 01:01 | |
| *** shakamunyi has joined #openstack-ceilometer | 01:05 | |
| *** shakayumi has joined #openstack-ceilometer | 01:16 | |
| *** shakamunyi has quit IRC | 01:18 | |
| *** liusheng has joined #openstack-ceilometer | 01:19 | |
| *** thomasem has quit IRC | 01:24 | |
| *** thomasem has joined #openstack-ceilometer | 01:59 | |
| *** julim has joined #openstack-ceilometer | 02:08 | |
| *** yjiang5 is now known as yjiang5_away | 02:15 | |
| *** julim has quit IRC | 02:25 | |
| *** fabio has quit IRC | 02:26 | |
| *** shakayumi has quit IRC | 02:58 | |
| *** Qiming has joined #openstack-ceilometer | 03:13 | |
| *** fnaval has quit IRC | 04:56 | |
| *** ildikov has quit IRC | 05:03 | |
| *** ildikov has joined #openstack-ceilometer | 05:17 | |
| *** Qiming has quit IRC | 05:46 | |
| openstackgerrit | A change was merged to openstack/ceilometer: Removed StorageEngine class and it's hierarchy https://review.openstack.org/87812 | 05:56 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/ceilometer: Imported Translations from Transifex https://review.openstack.org/88506 | 06:03 |
| *** Qiming has joined #openstack-ceilometer | 06:12 | |
| *** alexpilotti has quit IRC | 06:28 | |
| *** sileht has quit IRC | 06:30 | |
| *** _nadya_ has joined #openstack-ceilometer | 06:30 | |
| *** sileht has joined #openstack-ceilometer | 06:34 | |
| *** _nadya_ has quit IRC | 07:02 | |
| *** ildikov has quit IRC | 07:05 | |
| *** _nadya_ has joined #openstack-ceilometer | 07:05 | |
| *** ildikov has joined #openstack-ceilometer | 07:07 | |
| *** aswadrangnekar has joined #openstack-ceilometer | 07:17 | |
| *** piyushmasrani has quit IRC | 07:22 | |
| *** flwang has joined #openstack-ceilometer | 07:26 | |
| *** vigneshvar_ has joined #openstack-ceilometer | 07:30 | |
| vigneshvar_ | hi | 07:31 |
| vigneshvar_ | i have query on alarms | 07:32 |
| vigneshvar_ | i have set up an alarm for cpu_util < 95% just to test the alarm but when i do alarm list , it shows me "insufficient data' | 07:33 |
| vigneshvar_ | what could be the possible reason and how can i figure out the problem | 07:33 |
| *** ihrachyshka has joined #openstack-ceilometer | 07:34 | |
| vigneshvar_ | an help would be great | 07:35 |
| liusheng | your ceilometer and ceilometerclient is latest? | 07:41 |
| *** nacim has joined #openstack-ceilometer | 07:45 | |
| vigneshvar_ | yes it is latest | 08:10 |
| vigneshvar_ | used the master branch as on may 3 | 08:10 |
| vigneshvar_ | 2014 | 08:10 |
| *** eglynn has joined #openstack-ceilometer | 08:19 | |
| aswadrangnekar | hi | 08:27 |
| aswadrangnekar | https://github.com/openstack/python-ceilometerclient/blob/master/ceilometerclient/common/http.py#L48 | 08:28 |
| aswadrangnekar | https://github.com/openstack/python-ceilometerclient/blob/master/ceilometerclient/common/http.py#L137 | 08:28 |
| *** renlt has joined #openstack-ceilometer | 08:28 | |
| aswadrangnekar | On line 48 auth_token is fetched from the dictionary which according to me is a string, and on line 137 it is a treated as a callable - is this a bug? | 08:29 |
| *** ekarlso has quit IRC | 08:29 | |
| *** ekarlso has joined #openstack-ceilometer | 08:29 | |
| *** flwang has quit IRC | 08:32 | |
| *** ildikov has quit IRC | 08:35 | |
| *** ildikov_ has joined #openstack-ceilometer | 08:35 | |
| scroiset_ | aswadrangnekar: hi, see https://github.com/openstack/python-ceilometerclient/blob/master/ceilometerclient/client.py#L65 and L87 .. seems the token provided is always a callable. | 08:39 |
| *** flwang has joined #openstack-ceilometer | 08:40 | |
| aswadrangnekar | scroiset_: thanks! it solves my query | 09:02 |
| *** flwang has quit IRC | 09:24 | |
| *** ihrachyshka has quit IRC | 09:29 | |
| *** renlt has quit IRC | 09:40 | |
| *** Qiming has quit IRC | 09:41 | |
| *** aviau has quit IRC | 09:50 | |
| *** eoutin has joined #openstack-ceilometer | 09:57 | |
| *** AMike has joined #openstack-ceilometer | 10:02 | |
| *** AMike has joined #openstack-ceilometer | 10:02 | |
| aswadrangnekar | scroiset_: ping | 10:04 |
| scroiset_ | aswadrangnekar: pong | 10:05 |
| aswadrangnekar | scroiset_: I did not understand one point though - why is it required that token is callable? | 10:06 |
| aswadrangnekar | I checked in nova python client it does not have it callable | 10:07 |
| aswadrangnekar | any clues? | 10:08 |
| scroiset_ | aswadrangnekar: *thinking | 10:08 |
| aswadrangnekar | scroiset_: ok | 10:08 |
| scroiset_ | so initially .. seems to be a lazy pattern to authenticate against keystone when the token is really needed. | 10:16 |
| scroiset_ | when a raw token is passed, the same mechanism is use with a lambda .. to be compatible with the first case | 10:17 |
| aswadrangnekar | scroiset_: ok thanks! | 10:19 |
| *** ihrachyshka has joined #openstack-ceilometer | 10:29 | |
| *** aswadrangnekar has left #openstack-ceilometer | 10:31 | |
| *** ihrachyshka has quit IRC | 10:35 | |
| *** DuncanT has joined #openstack-ceilometer | 10:59 | |
| DuncanT | Hi. Quick question: I'm thinking of adding api request timing to cinder for every request. Is this the sort of info ceilometer can easily consume? (I've not run ceilometer, though I'm written a few things that consume the same event stream) | 11:00 |
| vigneshvar_ | hi, i have problem with ceilometer not trig 14gering an alarm . I am using the master branch as on may | 11:04 |
| vigneshvar_ | hi, i have problem with ceilometer not triggering an alarm . I am using the master branch as on may 14 | 11:04 |
| eglynn | DuncanT: ... the swift middleware for ceilometer currently intercepts incoming API calls in a way that could be re-moulded for your purpose | 11:05 |
| eglynn | DuncanT: ... https://github.com/openstack/ceilometer/blob/master/ceilometer/objectstore/swift_middleware.py | 11:05 |
| eglynn | DuncanT: ... however you might want to consider batching up the timings, as opposed to emitting a sample for every single API request-response cycle | 11:06 |
| DuncanT | eglynn: Thanks, I'll take a look at that code | 11:06 |
| DuncanT | I can certainly look at batching up the timings | 11:07 |
| eglynn | DuncanT: cool | 11:07 |
| DuncanT | I just wanted to check that a ceilometer event was a sensible place to send the results | 11:07 |
| eglynn | DuncanT: yeah, it could certainly be, especially if the aggregation exposed by the ceilo statistics API is sufficient for your purpose | 11:08 |
| eglynn | vigneshvar_: "not triggering" == "alarm state unexpectedly does not transition to alarm" | 11:08 |
| eglynn | vigneshvar_: or "not triggering" == "alarm state stuck in insufficient data" | 11:08 |
| eglynn | vigneshvar_: "not triggering" == "alarm state transitions, but no action is seen" | 11:09 |
| DuncanT | eglynn: I've not looked at the ceilometer stats API yet, I've got a devstack install of ceilometer so I'll go learn something about it now :-) | 11:09 |
| eglynn | DuncanT: ... the ceilo API allows simple aggregation (min, max, average, sum, count, stddev, cardinality) over variable-length periods | 11:10 |
| eglynn | DuncanT: ... but doesn't yet support calculation of things like percentiles | 11:10 |
| *** _nadya_ has quit IRC | 11:10 | |
| DuncanT | eglynn: That sounds like it will do what I want for now... and the 'yet support' bit gives me a good sign that adding anything missing would, in principle, be welcome | 11:11 |
| eglynn | DuncanT: cool enough, yeah we added some addition aggregation functions in icehouse, more to come possibley in juno | 11:12 |
| *** flwang has joined #openstack-ceilometer | 11:12 | |
| eglynn | DuncanT: ... reason I mentioned percentiles is that these are often used to look at API responsiveness | 11:13 |
| eglynn | DuncanT: (i.e. more concerned when p90 or p99 spikes, as opposed to the average) | 11:13 |
| DuncanT | eglynn: Yeah, I'm not yet sure how to alert on the data, step one is to collect it and analyse it during incidents | 11:14 |
| eglynn | DuncanT: cool | 11:14 |
| DuncanT | eglynn: I'm quite convinced that analysing stats is the future of monitoring, and I thing there are some smart algorithms for spotting issues, I've never had the the time or the data sets to really play. I have a fundamental allergy to statistical methods from my uni days (a badly taught set of modules) that I *really* need to shake | 11:15 |
| vigneshvar_ | eglynn: alarm state stuck in insufficient data | 11:16 |
| eglynn | DuncanT: yeah, I guess the question is how sophisticated we should go with the analytics supported *directly* by ceilo | 11:17 |
| eglynn | DuncanT: ... remember you can always grab the raw datapoints and analyse off-line | 11:18 |
| eglynn | vigneshvar_: what statistic is the alarm based on? | 11:19 |
| DuncanT | eglynn: I suspect the way forward is to develop offline analysis, prove the general usefulness then talk about how to get them directly integrated with ceilo | 11:19 |
| eglynn | vigneshvar_: check the period of the alarm against the configured interval for the agent collecting that metric | 11:19 |
| eglynn | DuncanT: ... yeap could be | 11:20 |
| eglynn | DuncanT: ... we're also discussing at summit much more aggressive pre-aggregation/roll-up of incoming metrics | 11:20 |
| eglynn | DuncanT: ... which would point towards more exotic aggregation being done off-line | 11:20 |
| *** Qiming has joined #openstack-ceilometer | 11:21 | |
| DuncanT | eglynn: Please make pre-aggregation configurable, or certain analyses become impossible | 11:21 |
| eglynn | vigneshvar_: ... also check the query condition in the alarm rule, whether this would yield any data if applied to a corresponding statistics query | 11:21 |
| vigneshvar_ | alarm is set for cpu_util | 11:22 |
| vigneshvar_ | eglynn: alarm is set for cpu_util | 11:22 |
| eglynn | DuncanT: ... yep, noted! | 11:22 |
| eglynn | DuncanT: ... I think it would have to be heavily configurable, beyond the cheapest/simplest aggregates (i.e. avg, min, max etc.) | 11:23 |
| eglynn | vigneshvar_: period? | 11:23 |
| vigneshvar_ | eglynn: for testing i have set the condition to cpu_util < 95.0. i will just check the period and send you | 11:23 |
| eglynn | vigneshvar_: $ ceilometer alarm-show -a $ALARM_ID | 11:23 |
| vigneshvar_ | eglynn: period: during 1 x 600s | 11:24 |
| vigneshvar_ | eglynn: period | 600 | 11:25 |
| eglynn | vigneshvar_: what interval is defined for the cpu source in the /etc/ceilometer/pipeline.yaml on the compute node? | 11:25 |
| eglynn | vigneshvar_: what query is set for the alarm? | 11:26 |
| vigneshvar_ | eglynn: just a min will check | 11:26 |
| vigneshvar_ | eglynn: cpu_source - interval:600 | 11:27 |
| eglynn | vigneshvar_: right that's ok | 11:28 |
| vigneshvar_ | egynn: metadata.user_metadata.stack == 44837952-3f1f-4497-8e59-fb314a3ea337 AND project_id == 24ba4f14daa74fc18e4a6a1d1c8689e9 | 11:28 |
| vigneshvar_ | egynn: thats the query | 11:28 |
| *** _nadya_ has joined #openstack-ceilometer | 11:29 | |
| vigneshvar_ | egynn : stack and project ids are correct | 11:29 |
| eglynn | vigneshvar_: one sec | 11:30 |
| eglynn | vigneshvar_: sorry back | 11:32 |
| eglynn | vigneshvar_: so how are the traget instances created? | 11:32 |
| eglynn | vigneshvar_: are you manually spinning up those instances? | 11:33 |
| vigneshvar_ | i used the autoscaling.yaml file from hot templates | 11:33 |
| vigneshvar_ | egylnn: i used the autoscaling.yaml file from hot templates | 11:33 |
| eglynn | vigneshvar_: what user metadata *exactly* is set on the instances? | 11:33 |
| eglynn | vigneshvar_: is it {"metering.stack": "44837952-3f1f-4497-8e59-fb314a3ea337"} | 11:34 |
| eglynn | vigneshvar_: or just: {"stack": "44837952-3f1f-4497-8e59-fb314a3ea337"} | 11:34 |
| eglynn | vigneshvar_: $ nova show INSTANCE_NAME | 11:34 |
| vigneshvar_ | eglynn: Just a min just checking it | 11:35 |
| vigneshvar_ | eglynn: it is metering.stack | 11:35 |
| eglynn | vigneshvar_: k ... reason I asked is that ceilometer will only persist user metadata in the resource metadata if the user metadata is prefixed with 'metering.' | 11:36 |
| eglynn | vigneshvar_: k, so next thing to look at is the project ID constraint | 11:37 |
| vigneshvar_ | eglynn: what must be checked | 11:37 |
| eglynn | vigneshvar_: $ keystone tenant-list | grep 24ba4f14daa74fc18e4a6a1d1c8689e9 | 11:38 |
| eglynn | vigneshvar_: ... is that the *same* tenant as owns the instance? | 11:39 |
| *** nacim has quit IRC | 11:39 | |
| vigneshvar_ | eglynn: give me a minute | 11:40 |
| eglynn | vigneshvar_: take all the time that you need | 11:41 |
| eglynn | vigneshvar_: the fact that the "project_id == 24ba4f14daa74fc18e4a6a1d1c8689e9" constraint is included in the alarm rule implies that this is not an alarm created with admin privilege | 11:41 |
| eglynn | vigneshvar_: ... so the stats for a resource owned by *another* tenant would not be visible when the alarm is evaluated | 11:42 |
| vigneshvar_ | eglynn: yes the alarm is created by a user called demo | 11:43 |
| vigneshvar_ | eglynn: demo - 24ba4f14daa74fc18e4a6a1d1c8689e9 | 11:43 |
| eglynn | vigneshvar_: ... and the instance, owned by? | 11:43 |
| vigneshvar_ | eglynn: and the instance is owned by demo | 11:43 |
| eglynn | vigneshvar_: ... $ nova show INSTANCE_NAME | grep tenant | 11:43 |
| vigneshvar_ | eglynn: tenant_id | 24ba4f14daa74fc18e4a6a1d1c8689e9 | 11:44 |
| eglynn | vigneshvar_: k, that looks correct also | 11:45 |
| eglynn | vigneshvar_: hmmm | 11:47 |
| eglynn | vigneshvar_: k try running this directly: | 11:47 |
| eglynn | vigneshvar_: $ ceilometer sample-list -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9" | 11:47 |
| eglynn | vigneshvar_: (as the demo user/tenant) | 11:47 |
| eglynn | vigneshvar_: check with ... $ echo $OS_USERNAME $OS_TENANT_NAME | 11:48 |
| vigneshvar_ | eglynn: user and tenant are demo | 11:49 |
| vigneshvar_ | eglynn: Output of sample list is quite big | 11:49 |
| vigneshvar_ | eglynn: It shows lot of value from 1 to 85 | 11:50 |
| eglynn | vigneshvar_: sorry I meant to say to use statistics ... | 11:51 |
| eglynn | vigneshvar_: $ ceilometer statistics -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9" | 11:51 |
| eglynn | vigneshvar_: (as opposed to sample-list) | 11:51 |
| vigneshvar_ | eglynn: yap got it. | 11:51 |
| vigneshvar_ | eglynn: you want me to paste the whole line here or any specific values | 11:52 |
| eglynn | vigneshvar_: no just checking that the query returns sane values | 11:53 |
| ekarlso | eglynn: is ceilo gonan be a timeseries serivce ? :p | 11:53 |
| eglynn | vigneshvar_: ... i.e. that the same query as used by the alarm evaluator can be run directly | 11:53 |
| eglynn | ekarlso: are you going to summit? | 11:53 |
| ekarlso | eglynn: nope sadly | 11:54 |
| eglynn | ekarlso: we'll be discussing some related ideas here http://junodesignsummit.sched.org/event/22cb077a1722e710955b78aaef700ba7#.U2jKhoqPXQI | 11:54 |
| eglynn | vigneshvar_: ... have a look at your alarm evaluator log | 11:55 |
| *** nacim has joined #openstack-ceilometer | 11:55 | |
| vigneshvar_ | eglynn: ok i will check that | 11:55 |
| eglynn | vigneshvar_: are you running devstack? | 11:55 |
| eglynn | vigneshvar_: ... or an actual install? | 11:55 |
| vigneshvar_ | eglynn: devstack | 11:55 |
| eglynn | vigneshvar_: k, screen -r, then look at the ceilometer-alarm-evaluator screen | 11:56 |
| vigneshvar_ | eglynn: just checking it | 11:56 |
| eglynn | vigneshvar_: you should see period statistics queries in the logs (one for each alarm) | 11:57 |
| *** _nadya_ has quit IRC | 11:58 | |
| vigneshvar_ | eglynn: i dont find ceilometer-alarm-evaluator......... only ceilometer-acompute,ceilometer-acentral,ceilometer-collector, ceilometer-api | 11:59 |
| vigneshvar_ | eglynn: only those are present | 11:59 |
| eglynn | vigneshvar_: that's your problem right there methinks | 11:59 |
| eglynn | vigneshvar_: grep enable ~/devstack/localrc | 11:59 |
| eglynn | vigneshvar_: http://docs.openstack.org/developer/ceilometer/install/development.html | 12:00 |
| eglynn | vigneshvar_: note: "enable_service ceilometer-alarm-evaluator,ceilometer-alarm-notifier" | 12:00 |
| vigneshvar_ | eglynn: ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral,ceilometer-collector,ceilometer-api | 12:01 |
| vigneshvar_ | eglynn: ENABLED_SERVICES+=,ceilometer-alarm-notify,ceilometer-alarm-eval | 12:01 |
| vigneshvar_ | both are already enabled and stack had no errors | 12:01 |
| eglynn | vigneshvar_: s/ceilometer-alarm-eval/ceilometer-alarm-evaluator/ | 12:02 |
| eglynn | vigneshvar_: s/ceilometer-alarm-notify/ceilometer-alarm-notifier/ | 12:02 |
| vigneshvar_ | eglynn: i will change it and rerun the stack and try it once again | 12:02 |
| eglynn | vigneshvar_: ... k | 12:03 |
| vigneshvar_ | eglynn: thanks a lot. I will get back with you once i am done with it | 12:07 |
| eglynn | vigneshvar_: np! | 12:07 |
| vigneshvar_ | eglynn: Is there a way to update only these two services without actually disturbing the devstack setup | 12:08 |
| eglynn | vigneshvar_: well easiest would be simply to run these services manually | 12:09 |
| vigneshvar_ | eglynn: ok i will do that. thanks | 12:10 |
| eglynn | vigneshvar_: ... e.g. ceilometer-alarm-evaluator --config-file /etc/ceilometer/ceilometer.conf ... etc. | 12:10 |
| *** fnaval has joined #openstack-ceilometer | 12:16 | |
| *** thomasem has quit IRC | 12:24 | |
| *** thomasem has joined #openstack-ceilometer | 12:25 | |
| *** ildikov_ has quit IRC | 12:27 | |
| *** ildikov has joined #openstack-ceilometer | 12:27 | |
| *** promulo has quit IRC | 12:30 | |
| *** eoutin has quit IRC | 12:52 | |
| *** aviau has joined #openstack-ceilometer | 12:55 | |
| vigneshvar_ | eglynn: I have enabled the services | 12:58 |
| vigneshvar_ | eglynn: and i can see the logs | 12:58 |
| eglynn | vigneshvar_: and what's the alarm state now? | 12:58 |
| vigneshvar_ | eglynn: But it is just the same | 12:58 |
| eglynn | vigneshvar_: are you seeing the statistics queries in the alarm evaluator logs? | 12:59 |
| vigneshvar_ | eglynn: Yes i can query stats | 13:00 |
| *** kun_huang has joined #openstack-ceilometer | 13:02 | |
| eglynn | vigneshvar_: ... you can see the query? ... what's the response? ... is it non-empty? | 13:03 |
| eglynn | vigneshvar_: ... in a functioning case, the relevant log output would look something like http://paste.openstack.org/show/79158/ | 13:04 |
| *** nacim has quit IRC | 13:05 | |
| *** nacim has joined #openstack-ceilometer | 13:06 | |
| *** eoutin has joined #openstack-ceilometer | 13:06 | |
| *** zul has joined #openstack-ceilometer | 13:07 | |
| *** gordc has joined #openstack-ceilometer | 13:08 | |
| vigneshvar_ | eglynn: http://paste.openstack.org/show/79159/ | 13:08 |
| vigneshvar_ | eglynn: no output | 13:09 |
| vigneshvar_ | eglynn: shows [] | 13:09 |
| eglynn | vigneshvar_: ... didn't you say earlier that the alarm period was 600s? | 13:10 |
| vigneshvar_ | eglynn: i think it is another alarm | 13:11 |
| eglynn | vigneshvar_: the bound timestamps are 2014-05-03T19:39:22 and 2014-05-03T19:41:22 | 13:11 |
| vigneshvar_ | eglynn: there are two alarms. one for scale up and scale down | 13:11 |
| vigneshvar_ | eglynn: that machine has current time "Sat May 3 19:44:24 GMT 2014" | 13:12 |
| eglynn | vigneshvar_: the bounding timestamps are only 2 minutes apart, with period=60s | 13:12 |
| eglynn | vigneshvar_: ... that indicates an eval window of 120s, not 600s | 13:12 |
| eglynn | vigneshvar_: ... whereas the interval that the metric is gathered over is 600s, right? | 13:13 |
| vigneshvar_ | eglynn: yes you are right | 13:15 |
| eglynn | vigneshvar_: ... so you must either: | 13:15 |
| eglynn | vigneshvar_: ... (a) change the interval for the cpu source to 60s and restart the compute agent | 13:15 |
| eglynn | vigneshvar_: ... or: | 13:15 |
| eglynn | vigneshvar_: ... (b) chage the alarm period to be 600s | 13:15 |
| eglynn | *change | 13:16 |
| openstackgerrit | Koert van der Veer proposed a change to openstack/ceilometer: Implemented metering for cinder's snapshots https://review.openstack.org/92365 | 13:18 |
| vigneshvar_ | eglynn: there were two alarms. one alarm is 60s and other 600s | 13:18 |
| vigneshvar_ | eglynn: i think one i pasted is 60s one | 13:19 |
| eglynn | vigneshvar_: are both still in insufficient_data? | 13:19 |
| eglynn | vigneshvar_: $ ceilometer alarm-list | 13:19 |
| vigneshvar_ | eglynn: yes | 13:19 |
| vigneshvar_ | eglinn: both insufficient data | 13:20 |
| eglynn | vigneshvar_: can you post the log excerpt for the 600s alarm? | 13:20 |
| vigneshvar_ | eglynn: sure just a min | 13:20 |
| *** erecio has quit IRC | 13:21 | |
| *** erecio has joined #openstack-ceilometer | 13:22 | |
| vigneshvar_ | eglynn: http://paste.openstack.org/show/79161/ | 13:23 |
| *** changbl has quit IRC | 13:24 | |
| *** julim has joined #openstack-ceilometer | 13:25 | |
| eglynn | vigneshvar_: ... what user are you running the ceilometer-alarm-evaluator under? | 13:25 |
| vigneshvar_ | eglynn: the user which i used to run stack.sh (username: openstack) | 13:26 |
| vigneshvar_ | i run it in the screen which is already created by devstack | 13:26 |
| eglynn | vigneshvar_: ... I mean the OS_USERNAME etc. | 13:27 |
| vigneshvar_ | eglynn: demo | 13:27 |
| eglynn | vigneshvar_: surely that should be admin? | 13:28 |
| eglynn | vigneshvar_: nah ignore that | 13:29 |
| eglynn | vigneshvar_: ... determined by the service_credentials in the ceilometer.conf | 13:29 |
| vigneshvar_ | eglynn: give me a min | 13:30 |
| flwang | gordc: ping | 13:31 |
| gordc | flwang: whatup? | 13:31 |
| flwang | gordc: do you know is there any company are using Ceilometer in production environment? thanks | 13:32 |
| *** _nadya_ has joined #openstack-ceilometer | 13:32 | |
| *** nacim has quit IRC | 13:33 | |
| gordc | eglynn: you guys use it? ^ | 13:33 |
| eglynn | vigneshvar_: ... I'm presuming the service_credentials in the ceilometer.conf are correct | 13:33 |
| eglynn | vigneshvar_: ... otherwise it would not be possible for say the compute agent to query nova-api to discover the local instances | 13:33 |
| gordc | flwang: i assume enovance and dreamhost use it to some extent. | 13:33 |
| flwang | gordc: cool, thanks | 13:34 |
| eglynn | gordc: ... well Red Hat isn't operating a public cloud, but we do use it for some smaller scale internal labs | 13:34 |
| eglynn | gordc, flwang: ... yeah AFAIK DreamHost | 13:34 |
| flwang | gordc: as for eNovance and Dreamhost, I'm curious is there a billing system | 13:34 |
| gordc | flwang: i think Dreamhose uses it for billing... dhellmann would be able to answer that (or direct you to someone who could) | 13:35 |
| eglynn | flwang: ... ask dhellmann, IIRC there is called "The Dude" or something similar | 13:35 |
| dhellmann | eglynn, flwang, gordc : we've actually moved to a quota-based flat billing structure | 13:36 |
| flwang | eglynn: gordc: got it. thanks a lot | 13:36 |
| flwang | dhellmann: ah, you're here :) | 13:36 |
| dhellmann | the dude code is still around, but uses the v1 API | 13:36 |
| flwang | dhellmann: Dude is a code name for the billing system, right? | 13:36 |
| *** rdmcnair has joined #openstack-ceilometer | 13:36 | |
| dhellmann | Dude is a "bacronym" for "Dreamhost Usage Data Exporter" | 13:37 |
| gordc | dhellmann: was just about to ask... | 13:37 |
| dhellmann | it is the tool we used to move data out of ceilometer, into our existing billing system | 13:37 |
| dhellmann | we built a lot of this stuff after going on a Big Lebowski kick, so we have another tool called Rug that ties our network systems together :-) | 13:38 |
| flwang | dhellmann: ah, so it's most like an adapter between CM and your existed billing system, right? | 13:38 |
| dhellmann | flwang: yes, that's right | 13:38 |
| dhellmann | it ran queries in ceilomter, and recorded the results in the billing system | 13:38 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/ceilometer: Automatic discovery of TripleO Overcloud hardware https://review.openstack.org/92370 | 13:38 |
| flwang | dhellmann: got it. may I get your thoughts about why there is no billing components for OpenStack? | 13:39 |
| *** Qlawy has left #openstack-ceilometer | 13:40 | |
| flwang | dhellmann: is it too customized to get a common one? | 13:40 |
| vigneshvar_ | eglynn: os_username = ceilometer inside ceilometer.conf | 13:41 |
| eglynn | vigneshvar_: check with ... $ grep os_ /etc/ceilometer/ceilometer.conf | awk '{printf("%s=%s\n", toupper($1), $3);}' > scrc ; . ./scrc | 13:41 |
| eglynn | vigneshvar_: ... then repeat: $ ceilometer statistics -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9" | 13:41 |
| dhellmann | flwang: yes, exactly. we decided very early on that most deployers would already have something for billing, so building a tool they could integrate with their existing systems would make more sense. Also, the rules for what people wanted to charge for might be very complex. | 13:42 |
| *** fnaval has quit IRC | 13:43 | |
| flwang | dhellmann: i see. thanks for the patient clarification | 13:43 |
| dhellmann | flwang: any time! | 13:44 |
| vigneshvar_ | eglynn: you want me to do it on the terminal where . ceilometer-alarm-evaluator | 13:44 |
| eglynn | vigneshvar_: nope, just run any shell, then first run: $ . ~/devstack/openrc demo demo | 13:45 |
| eglynn | vigneshvar_: .., then $ grep os_ /etc/ceilometer/ceilometer.conf | awk '{printf("%s=%s\n", toupper($1), $3);}' > scrc ; . ./scrc | 13:45 |
| eglynn | vigneshvar_: ... that's set up your shell to use the same credentials | 13:46 |
| eglynn | vigneshvar_: ... that'll | 13:46 |
| eglynn | vigneshvar_: ... if not creds, the other obvious candidate is timestamp drift | 13:46 |
| eglynn | vigneshvar_: ... are both the compute node *and* the alarm evaluator host seeing the same current time | 13:47 |
| eglynn | vigneshvar_: ... same == "same ballpark" | 13:47 |
| eglynn | vigneshvar_: ... the times mentioned earlier looked way off | 13:47 |
| vigneshvar_ | eglynn: the timestamps are correct. That machine time was not set properly | 13:48 |
| vigneshvar_ | eglynn: It is a single node setup | 13:48 |
| eglynn | vigneshvar_: have you run the above? | 13:48 |
| *** _nadya_ has quit IRC | 13:49 | |
| vigneshvar_ | eglynn: after setting up the new credential what should i run on that to test | 13:50 |
| *** nacim has joined #openstack-ceilometer | 13:51 | |
| *** _nadya_ has joined #openstack-ceilometer | 13:52 | |
| eglynn | vigneshvar_: $ ceilometer statistics -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9;timestamp>2014-05-03T19:15:08;timestamp<=2014-05-03T19:35:08' | 13:52 |
| vigneshvar_ | eglynn: no data | 13:52 |
| vigneshvar_ | eglynn: just an empty table | 13:52 |
| eglynn | vigneshvar_: again without the timestamp constraints | 13:52 |
| eglynn | vigneshvar_: $ ceilometer statistics -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9' | 13:53 |
| vigneshvar_ | eglynn: then i get the data | 13:53 |
| ekarlso | eglynn: whats' so bad with the metadata ? | 13:53 |
| eglynn | vigneshvar_: ... and the conclusion is? | 13:53 |
| vigneshvar_ | eglynn: time problem | 13:54 |
| eglynn | vigneshvar_: ... yeap, smells that way, doesn't it? | 13:54 |
| eglynn | vigneshvar_: ... how about correcting the current time? | 13:54 |
| vigneshvar_ | eglynn: yap you are right | 13:54 |
| eglynn | vigneshvar_: ... better still set up NTP | 13:54 |
| vigneshvar_ | eglynn: does nt it affect other services ? | 13:54 |
| eglynn | vigneshvar_: ... it may well do, in some subtle ways | 13:55 |
| vigneshvar_ | eglynn: so can i set up current time | 13:55 |
| eglynn | ekarlso: ... what's bad about the "weight of highly repetitive free-form metadata"? | 13:56 |
| eglynn | ekarlso: ... well it's weighty for a start | 13:56 |
| eglynn | ekarlso: ... and highly repetitive ;) | 13:56 |
| ekarlso | eglynn: how else u store metadata then ? | 13:57 |
| eglynn | ekarlso: in a more lightweight form, that doesn't repeat rarely-changing resource metadata for every single datapoint | 13:58 |
| ekarlso | eglynn: so what, store md once then or for a resource? | 13:58 |
| *** julim has quit IRC | 13:58 | |
| eglynn | ekarlso: ... that's what we need to agree at summit, how to handle the fact that the metadata is rarely changing but *not* completely static | 14:00 |
| *** julim has joined #openstack-ceilometer | 14:02 | |
| ildikov | eglynn, ekarlso: if we can identify the parts of metadata that are static, than maybe we will be a half step closer | 14:03 |
| eglynn | ildikov: yeap ... or possibly treat changes in metadata as being akin to a "new identity" for the resource | 14:05 |
| ildikov | eglynn, ekarlso: but I'm living in a dream world, where metadata is validated against a model or somehow not 100% free form :) | 14:05 |
| vigneshvar_ | eglynn: how to check if any new data is collected ? | 14:06 |
| eglynn | vigneshvar_: use the sample-list command line above | 14:07 |
| ildikov | eglynn: hmm, that new identity thing sounds a bit complicated related to that the data is stored for a longer time period, where these identities are changing | 14:07 |
| eglynn | vigneshvar_: $ ceilometer sample-list -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9" | 14:07 |
| vigneshvar_ | eglynn: i dont find any data with current date as i have set now | 14:08 |
| *** prad has joined #openstack-ceilometer | 14:08 | |
| eglynn | vigneshvar_: has a 600s interval rolled around yet? | 14:08 |
| eglynn | vigneshvar_: if you're impatient ... | 14:09 |
| *** fnaval has joined #openstack-ceilometer | 14:09 | |
| eglynn | vigneshvar_: ... change the interval for the cpu source in the pipeline.yaml to say 60s | 14:09 |
| eglynn | vigneshvar_: ... restart the compute agent | 14:09 |
| eglynn | vigneshvar_: $ sleep 60 ; ceilometer sample-list -m cpu_util -q 'metadata.user_metadata.stack=44837952-3f1f-4497-8e59-fb314a3ea337;project=24ba4f14daa74fc18e4a6a1d1c8689e9" | 14:10 |
| eglynn | ildikov: ... yeah all open for discussion in ATL methinks, another approach mooted was to only store the latest metadata and reply on events to bracket relevant changes to the resource state | 14:12 |
| eglynn | ildikov: ... e.g. instead of having the instance state=suspended for some datapoints and state=active for others, the suspension/resume events effectively carry the bracketed time ranges | 14:13 |
| vigneshvar_ | eglynn: no new data at all | 14:14 |
| ildikov | eglynn: yeap, it's an option, but we will see what direction the summit talks will go to | 14:14 |
| eglynn | ildikov: yep | 14:15 |
| ildikov | eglynn: I'm still behind with reading the mails, it's all your fault ;) | 14:15 |
| eglynn | vigneshvar_: look at the compute agent logs | 14:15 |
| eglynn | ildikov: LOL :) | 14:15 |
| *** promulo has joined #openstack-ceilometer | 14:15 | |
| vigneshvar_ | eglynn: got new samples | 14:16 |
| vigneshvar_ | eglynn: alarm state true | 14:17 |
| vigneshvar_ | eglynn: state - alarm | 14:17 |
| eglynn | vigneshvar_: \o/ | 14:17 |
| vigneshvar_ | eglynn: thanks a lot got it working . You are awesome | 14:17 |
| eglynn | vigneshvar_: np! | 14:18 |
| *** eoutin has quit IRC | 14:20 | |
| *** alexpilotti has joined #openstack-ceilometer | 14:20 | |
| gordc | dhellmann: i added a comment to notifier middleware bp: https://blueprints.launchpad.net/oslo.messaging/+spec/grad-notifier-middleware ... some concerns about where to graduate code to. | 14:34 |
| openstackgerrit | Artur Svechnikov proposed a change to openstack/python-ceilometerclient: Add methods to resource classes https://review.openstack.org/91554 | 14:35 |
| *** vigneshvar_ has quit IRC | 14:51 | |
| *** shakayumi has joined #openstack-ceilometer | 14:52 | |
| *** DuncanT has left #openstack-ceilometer | 14:53 | |
| *** jergerber has joined #openstack-ceilometer | 15:04 | |
| openstackgerrit | A change was merged to openstack/ceilometer: Corrections of spelling, rephrasing for clarity https://review.openstack.org/92184 | 15:07 |
| *** lsmola has quit IRC | 15:18 | |
| *** fnaval has quit IRC | 15:21 | |
| *** fnaval has joined #openstack-ceilometer | 15:23 | |
| *** shakayumi has quit IRC | 15:31 | |
| *** AMike has quit IRC | 15:32 | |
| *** shakamunyi has joined #openstack-ceilometer | 15:32 | |
| *** zul has quit IRC | 15:34 | |
| *** _cjones_ has joined #openstack-ceilometer | 15:35 | |
| *** alexpilotti_ has joined #openstack-ceilometer | 15:41 | |
| *** alexpilotti has quit IRC | 15:44 | |
| *** alexpilotti_ is now known as alexpilotti | 15:44 | |
| *** eglynn has quit IRC | 15:46 | |
| *** fnaval has quit IRC | 15:47 | |
| *** eglynn has joined #openstack-ceilometer | 15:52 | |
| *** changbl has joined #openstack-ceilometer | 15:54 | |
| *** _nadya_ has quit IRC | 15:56 | |
| *** zul has joined #openstack-ceilometer | 15:58 | |
| *** fnaval has joined #openstack-ceilometer | 15:59 | |
| *** Ruetobas has quit IRC | 16:01 | |
| *** Ruetobas has joined #openstack-ceilometer | 16:03 | |
| *** mengxd has joined #openstack-ceilometer | 16:06 | |
| openstackgerrit | Koert van der Veer proposed a change to openstack/ceilometer: Implemented metering for cinder's snapshots https://review.openstack.org/92365 | 16:07 |
| *** zul has quit IRC | 16:07 | |
| *** Ruetobas has quit IRC | 16:07 | |
| *** Ruetobas has joined #openstack-ceilometer | 16:13 | |
| *** zul has joined #openstack-ceilometer | 16:13 | |
| openstackgerrit | Koert van der Veer proposed a change to openstack/ceilometer: Implemented metering for cinder's snapshots https://review.openstack.org/92365 | 16:19 |
| *** _nadya_ has joined #openstack-ceilometer | 16:26 | |
| *** Qiming has quit IRC | 16:27 | |
| dhellmann | gordc: you might be right, I'll have to look at the dependency graph again | 16:38 |
| gordc | dhellmann: cool cool. | 16:39 |
| dhellmann | gordc: if the notifier middleware moves to oslo.messaging, wouldn't the audit middleware (and oslo.middleware) just depend on oslo.messaging? | 16:40 |
| *** eglynn has quit IRC | 16:40 | |
| gordc | dhellmann: right... but the audit middleware exists in oslo.middleware... and notifier middleware depends on oslo.middleware (assuming the base middleware code gets moved there (https://github.com/openstack/oslo-incubator/blob/master/openstack/common/middleware/base.py)) | 16:42 |
| *** shakamunyi has quit IRC | 16:42 | |
| dhellmann | gordc: I think moving the auditing middleware to messaging may have been a mistake (though I know it was done to fix adoption issues with oslo.messaging). I'm not sure how to correct it, now, though. :-/ | 16:43 |
| dhellmann | we can't just put all of the middleware in the messaging lib | 16:43 |
| dhellmann | and we don't want oslo.messaging to depend on oslo.middleware, even to handle a backwards-compatibility issue like the auditing middleware | 16:44 |
| gordc | dhellmann: right. well we can have notifier middleware in oslo.middleware so it'll just be oslo.middleware dependent on oslo.messaging (except now you're pulling in oslo.messaging for something only a subset of middlewares use) | 16:45 |
| dhellmann | yeah, I think that's why I suggested putting the notifier code in oslo.messaging | 16:46 |
| dhellmann | gordc: this is hurting my head :-) | 16:46 |
| gordc | dhellmann: i'd assume the issue would be present regardless of audit middleware... any middleware that wants to extend notifier middleware will create a cyclic dependency. | 16:46 |
| dhellmann | gordc: is any of this simpler if we assume that no code from the incubator is going into any of these libraries? because that's my assumption. | 16:47 |
| gordc | :) glad to spread the headache. | 16:47 |
| dhellmann | gordc: how is there a cycle? does the notifier middleware use a base class that's going to be in oslo.middleware? | 16:47 |
| gordc | dhellmann: ^^ the base.py file above... | 16:48 |
| gordc | dhellmann: that said, it's pretty generic so we could just keep two copies around. | 16:48 |
| dhellmann | gordc: ok, I missed that | 16:48 |
| dhellmann | nah, we'll just move all of the middleware to oslo.middleware, I guess | 16:48 |
| gordc | dhellmann: ok... yeah, i'm not sure if there's a scenario where can avoid not pulling in oslo.messaging into oslo.middleware... unless we drop notifier middleware completely. | 16:50 |
| dhellmann | well, we can't drop it if people are using it | 16:51 |
| dhellmann | it might have to go into its own library or something, but that may stir up other issues | 16:51 |
| dhellmann | eglynn: what are the chances of moving http://junodesignsummit.sched.org/event/22cb077a1722e710955b78aaef700ba7 so it doesn't conflict with any oslo sessions? | 16:52 |
| gordc | right... i mean the goal is to drop audit middleware at some point but notifier middleware is its own thing so not sure if that can be dropped. | 16:52 |
| dhellmann | gordc: I don't suppose I could convince you to write up the analysis of those middleware modules and make some recommendations for how to organize them? | 16:55 |
| *** nacim has quit IRC | 16:56 | |
| gordc | sure. i can write up an ether... what type of info you looking for? what middleware does and its dependencies? | 16:56 |
| dhellmann | gordc: yeah, just a list of the modules and how they depend on each other and other libraries | 17:00 |
| *** kun_huang has quit IRC | 17:11 | |
| *** ildikov has quit IRC | 17:16 | |
| *** mengxd has quit IRC | 17:20 | |
| *** yjiang5_away is now known as yjiang5 | 17:23 | |
| *** zul has quit IRC | 17:25 | |
| *** dperaza has joined #openstack-ceilometer | 17:45 | |
| *** _nadya_ has quit IRC | 17:51 | |
| *** _nadya_ has joined #openstack-ceilometer | 17:53 | |
| *** nekron99 has joined #openstack-ceilometer | 17:55 | |
| *** nekron99 has quit IRC | 17:55 | |
| *** nekron99 has joined #openstack-ceilometer | 17:56 | |
| *** _nadya_ has quit IRC | 17:57 | |
| *** cjchand has joined #openstack-ceilometer | 18:13 | |
| *** alexpilotti has quit IRC | 18:17 | |
| *** alexpilotti has joined #openstack-ceilometer | 18:20 | |
| *** _nadya_ has joined #openstack-ceilometer | 18:27 | |
| *** ildikov has joined #openstack-ceilometer | 18:37 | |
| *** zul has joined #openstack-ceilometer | 18:44 | |
| *** zul has quit IRC | 19:02 | |
| *** zul has joined #openstack-ceilometer | 19:06 | |
| *** promulo has quit IRC | 19:15 | |
| *** _nadya_ has quit IRC | 19:25 | |
| *** _nadya_ has joined #openstack-ceilometer | 19:25 | |
| *** promulo has joined #openstack-ceilometer | 19:31 | |
| *** _nadya_ has quit IRC | 19:38 | |
| *** cjchand has quit IRC | 19:48 | |
| *** cjchand__ has joined #openstack-ceilometer | 19:48 | |
| openstackgerrit | gordon chung proposed a change to openstack/ceilometer: fix indices in event tables https://review.openstack.org/91501 | 19:53 |
| openstackgerrit | gordon chung proposed a change to openstack/ceilometer: fix event model https://review.openstack.org/92454 | 19:53 |
| *** eglynn has joined #openstack-ceilometer | 19:56 | |
| *** _nadya_ has joined #openstack-ceilometer | 20:08 | |
| *** julim has quit IRC | 20:31 | |
| *** mengxd has joined #openstack-ceilometer | 20:40 | |
| *** nekron99 has quit IRC | 20:45 | |
| *** _nadya_ has quit IRC | 20:46 | |
| *** jdob has quit IRC | 20:55 | |
| *** prad_ has joined #openstack-ceilometer | 21:03 | |
| *** prad has quit IRC | 21:05 | |
| *** prad_ is now known as prad | 21:05 | |
| *** xdmeng has joined #openstack-ceilometer | 21:09 | |
| *** mengxd has quit IRC | 21:12 | |
| *** xdmeng has quit IRC | 21:15 | |
| *** mengxd has joined #openstack-ceilometer | 21:16 | |
| *** mengxd has quit IRC | 21:17 | |
| *** rdmcnair has quit IRC | 21:20 | |
| *** mengxd has joined #openstack-ceilometer | 21:28 | |
| *** jaypipes has quit IRC | 21:30 | |
| *** xdmeng has joined #openstack-ceilometer | 21:31 | |
| *** mengxd has quit IRC | 21:34 | |
| *** xdmeng has quit IRC | 21:39 | |
| *** xdmeng has joined #openstack-ceilometer | 21:39 | |
| *** cjchand__ has quit IRC | 21:41 | |
| *** eglynn has quit IRC | 21:51 | |
| *** zul has quit IRC | 22:07 | |
| *** zul has joined #openstack-ceilometer | 22:09 | |
| *** zul has quit IRC | 22:23 | |
| *** prad has quit IRC | 22:39 | |
| *** changbl has quit IRC | 22:49 | |
| *** fnaval has quit IRC | 22:59 | |
| *** cjchand has joined #openstack-ceilometer | 23:02 | |
| *** liusheng has quit IRC | 23:06 | |
| *** jergerber has quit IRC | 23:14 | |
| *** fnaval has joined #openstack-ceilometer | 23:37 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!