15:01:51 #startmeeting ceilometer 15:01:53 Meeting started Thu Jan 8 15:01:51 2015 UTC and is due to finish in 60 minutes. The chair is eglynn-office. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:54 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:56 The meeting name has been set to 'ceilometer' 15:02:05 o/ 15:02:06 o/ 15:02:07 Happy New Year folks, I hope y'all had a nice break :) 15:02:09 o/ 15:02:11 o/ 15:02:16 Happy New Year 15:02:37 o/ 15:02:43 hey, happy new year guys! 15:02:56 good break indeed! 15:03:05 apols for the low profile this week, I've been heads-down on some RH installer work for my sins 15:03:18 #topic Kilo-2 status 15:03:23 o/ 15:03:27 #link https://launchpad.net/ceilometer/+milestone/kilo-2 15:03:32 happy new year :) 15:03:48 I've added blocked LP BPs pending the ceilo-specs reviews landing 15:04:04 eglynn-office: as for the low profile, we're all in warming up phase I think 15:04:24 yeap, it's like cranking up an old engine on a cold morning ;) 15:04:43 so first question is, did I miss out anything that should be targetted at kilo2? 15:05:01 nealph: e.g. the pipeline config work, were you thinking k2 or k3? 15:05:35 eglynn-office: I think https://blueprints.launchpad.net/ceilometer/+spec/self-disabled-pollster could be kilo-2 15:05:35 (kilo-2 is week of Feb 5th) 15:05:43 llu-laptop: coolness, thanks 15:05:43 eglynn-office: targeting k2 for bringing up the db and everything except api.... 15:06:05 o/ 15:06:05 nealph: cool, should we split the BP into two parts in that case? 15:06:06 that's pretty agressive, but I think we have a few more resources to throw at it. 15:06:28 nealph: (so that we can track against 2 separate milestones) 15:06:47 eglynn-office: if that's the preference, you bet. 15:07:39 eglynn-office: will do that by eod tomorrow. 15:08:08 nealph: excellent, if you could file a BP https://blueprints.launchpad.net/ceilometer/+addspec with a brief list of the k2 changes , I'll target it at kilo-2 15:08:11 nealph: i'd like to start work with api after discussion if you don't mind 15:08:32 idegtiarov: yeah, I owe you a chat. :) 15:08:35 nealph: ... i.e. no need for yet another ceilo-specs proposal 15:08:58 idegtiarov: still waking up from the break... 15:09:08 eglynn-office: sweet! 15:09:26 nealph: yep! so do i! :) 15:10:25 anything else on the kilo-2 radar? 15:10:56 I think gabbi (the http declarative test harness) is sufficiently happy now to actually use 15:11:06 So I've started assembling some real tests 15:11:12 Seems to work. 15:11:16 cdent: coolness, thanks! :) 15:11:24 The only trouble is I don't know the API very well. 15:11:27 cdent: great news!!! 15:11:45 cdent: so I presume patches are welcome? 15:11:56 cdent: i.e. divide and conquer to get API coverage? 15:12:00 yeah sure 15:12:19 I think the first step is probably to merge a very basic thing 15:12:24 and then people can build on that 15:12:37 I can make that happen early next week 15:12:58 cool, yeah, once your initial patches are landed and running happily in the gate, should be easier for other folks to help out 15:13:01 cdent: let's sync sometimes next week, unfortunately I saw a lot of API code lately :-) 15:13:44 okee doke 15:14:08 fabiog: BTW thanks again for that RBAC patch in kilo-1, we've backported internally to juno & icehouse 15:14:39 eglynn-office: glad to help 15:14:39 fabiog: ... i.e. to be carried as an extra patch in the distro, obviously too featureful to backport to stable upstream 15:15:30 k, let's move on to the pasta making :) 15:15:38 #topic gnocchi status 15:16:10 jd__: lots of patches landed this week, nice! 15:16:15 #link https://review.openstack.org/#/q/status:merged+project:stackforge/gnocchi,n,z 15:17:18 biggest outsanding in the review queue is currently I guess ceph driver and the cross-metric aggregation follow-ups 15:18:01 ceph is ready, it just need to be rebased 15:18:31 sileht: I owe you reviews on both of those patchsets, I'll get to them once I've tamed staypuft/astapor (prolly tmrw) 15:18:32 cross-metric, have unresolved issue about missing datapoint 15:19:29 eglynn-office, jd__ have some suggestions, but I'm waiting for your thought, to be sure 15:19:54 sileht: yeap, apols for the slowness of the response on that ... I'll chime in on gerrit 15:20:00 o/ oh, sorry, folks :) 15:20:12 forgot about meeting 15:20:24 DinaBelova: np! :) 15:20:37 hi! Dina! 15:20:40 eglynn-office, the discustion is here: https://review.openstack.org/#/c/140638/5/gnocchi/carbonara.py 15:21:13 I've also work on some HTML rendering 15:21:26 and I'm currently working on a statsd frontend 15:22:01 sileht: on a first thought, jd__'s suggestion to "compute the to_timestamp to be last possible datapoints in common for all the timeseries" seems reasonable 15:22:12 ... in any case, I'll reply in gerrit 15:22:32 jd__: yeah, I noticed "statsd" referenced in a topic branch name on a review :) 15:22:38 once I'm done I'll probably starts working on ceilometer-alarm support or something 15:23:02 nice, finally the statsd fanboys will have something to cheer about :) 15:23:27 s/fanboys/fanpersons/ :) 15:23:57 jd__, what are you going to start with in the alarms support? 15:24:06 * DinaBelova is just actually interested 15:24:42 I've been thinking about some opentsdb related raw-downsampled data magic 15:25:12 and suddenly understood I have no clear understanding of how heat autoscaling use case will be working with gnocchi 15:25:28 I mean I see several possibilities, etc. - and remember all our discussions 15:25:58 DinaBelova: the strongly typed server_group attribute on the Instance resource type is the key point 15:26:23 eglynn-office, yeah... but I see kind of issue with the thing how does heat define stack 15:26:41 DinaBelova: oh, yeah? 15:26:50 I mean heat stack defining alarm sets the period 15:27:14 yep ... a-ha, lining that up with the archive policy? 15:27:15 actually it'll be the granularity in the statistics request 15:27:19 eglynn-office, yes 15:27:29 yeah, good point 15:27:31 we can only do that storing some amount of raw data 15:27:41 and playing with aggregated data where possible 15:27:51 combining it with raw one where not 15:28:01 I did not remember the discussions about this thing 15:28:14 in a sense it's a version of the current issue of lining up the relevant pipeline.yaml interval with the alarm period 15:28:41 the alarm period has to be a whole number multiple of the pipeline source interval 15:29:07 eglynn-office, not actually - I'm mostly about the fact, that archive policies are set globally by admin in the cloud, although stack can be created by anyone with any period of the alarm 15:29:47 eglynn-office, moreover, heat is interested in points like now, now-period, now - 2*period, etc., gnocchi is working with normalized timestamps 15:30:11 it's not ptobably the global issue, I just don't see the exact solution for now 15:30:11 DinaBelova: ... pretty much like the pipeline.yaml is set administratively, whereas anyone can create alarms? 15:30:19 yes 15:30:28 oh, sorry, got your previous point 15:30:42 yeah, these things have kind of common roots 15:31:01 DinaBelova: I don't fully see the issue with normalized timestamps ... is this the clamping to the natural wall-clock boundaries? 15:31:32 DinaBelova: ... i.e. 1 minute or 5 minute or one hour boundaries, not anchored on utcnow 15:32:39 eglynn-office, please correct me if I'm wrong, but I somehow thought that gnocchi is working with normalized timestamps - like when we're aggregating 1 hour, we're doing this from 1 am to 2 am (let's say), not starting the clock from the first written mertic 15:32:54 DinaBelova: yes, exactly 15:33:20 ok, so let's say heat now uses the alarm that basically is making statistics request 15:33:41 DinaBelova: ... so yeah, you're right the "meaning" of the periodization changes 15:34:18 eglynn-office, heat now basically creates alarm that asks ceilo api about smth like ceilometer statistics -m cpu_util -q metadata.user_metadata.stack=stackval -p 600 -a avg 15:34:21 let's say 15:34:41 or smth like, it does not matter 15:34:59 but heat uses request (now - smth, now) period to ask 15:35:11 DinaBelova: yes, instead of the latest period being [now minus 5mins, now] say, it becomes [last 5min boundary, now] 15:35:12 not the from (1am, 2am) 15:35:32 so the latest period is generally just partially populated 15:35:59 eglynn-office, that does not seem to be the direct mapping between current logic and how it'll be looking like 15:36:10 becasue 5min boundary might be not stored in gnocchi 15:36:19 if it won't be this archive definition 15:36:33 and we can't store all the periods that might be asked by heat 15:36:56 as this might be any period of time 15:37:17 not only pre-aggregated granularities we store 15:38:18 eglynn-office, I would say, that in this case we need to store all raw data as well - and use pre-aggregates where possible 15:38:19 well we don't need to store all the periods, or? ... if the alarm period was small whole number multiple of an existing archive policy granularity for the metric, then we could the final aggregation step in the alarm evaluator? 15:38:25 combining them with raw data 15:38:41 eglynn-office, moment 15:38:50 * DinaBelova reading and trying to get the point 15:39:11 say the alarm period in 10 mins, and we only store 5 min granularity 15:39:57 the evaluator could itself aggregate over the contiguous pairs of 5 min datapoints 15:40:01 not ideal TBH 15:40:23 as it collapses down to the "mean-of-means" problem 15:40:43 but we're doing that anyway for the cross-metric aggregation, right sileht? 15:41:30 eglynn-office, what would we do, is we store 100 points of 5-min granularities and 100 points of 1-day granularities, and alarm period is 1 min? or 2 mins? 15:42:13 1-hour I meant, not 1-day, but anyway 15:42:16 DinaBelova: in that case, all bets are off ... similarly to currently when the alarm period is 60s but the pipeline interval is 600s 15:42:54 eglynn-office, but this does not seem to be 'normal'.... we might want to fix it somehow? 15:43:04 * sileht read backlog 15:43:20 and ok, let's say we store 5 mins granularity, and alarm asks for 7 min period 15:43:24 7, not 10 15:43:47 in this case alarm period is bigger than the minimum granularity 15:43:50 eglynn-office, yes "cross-metric aggregatio" does "mean-of-means" 15:44:48 DinaBelova: yeah, AFAICS this can only work if the period for the alarm is a whole-number multiple of the existing granularity for the metric 15:44:56 DinaBelova: ... otherwise we have to get into somehow aggregating over the raw data on-demand, as you said 15:45:09 sileht: yeah, that's what I thought ... thanks for the confirmation 15:45:18 eglynn-office, but we cannot guarantee alarm will be created with period = n * granularity 15:45:47 DinaBelova: exactly ... so it seems like a variant of the current issue with the pipeline interval? 15:46:04 eglynn-office, well, kind of 15:46:26 DinaBelova: yeah, not exactly the same, but similar 15:46:35 DinaBelova: TBH I'm a bit more concerned about the normalized timestamp issue that you mentioned 15:46:51 DinaBelova: since it opens to the latest period being very lightly populated 15:47:10 DinaBelova: and hence skewed by the quickest instances reporting 15:47:29 eglynn-office, simply I do not think we need to restrict only alarm_period = n * granularity.... because that's odd to make any ceilometer users to fit archives they do not know about, actually 15:47:35 eglynn-office, well, this as well 15:48:10 eglynn-office, I suppose we might want to implement complex logic with raw/pre-aggregated data processing 15:48:16 with combining them 15:48:27 DinaBelova: hmm, yeah worth thinking about 15:48:37 DinaBelova: BTW currently we work around that skewed latest period by allowing the sample count to be used to discard low quality datapoints 15:48:44 although that's the big headache with complex aggregation fns like mean/dev 15:48:46 i.e. where the sample count is anomolously low 15:49:03 eglynn-office, hm, may you please provide the example? 15:49:26 sorry, /me still on the vacation, low CPU-brain performance 15:49:27 :) 15:50:21 eglynn-office, do you mean we'll just through away some datapoints in case if gnocchi decided they are 'low quality' on some condition? 15:51:25 DinaBelova: see https://github.com/openstack/ceilometer/commit/a4c7411a 15:51:34 * DinaBelova looking 15:52:16 a-ha, got it 15:52:18 DinaBelova: basically the idea is for the alarm evaluator to throw away any datapoints that were formed from an unusually low number of samples 15:52:51 eglynn-office, how is the 'unusuality' of samples processed? 15:53:21 I don't want to interrupt, but as I remember we have one more topic for the today's meeting 15:53:25 DinaBelova: based on variance ... 2 standard deviations below the average sample count IIRC 15:53:30 ildikov, thanks 15:53:33 ildikov: yeah, we'd better move on 15:53:35 eglynn-office, yeah, found this line 15:53:48 DinaBelova: let's catch up this later or trmw? 15:53:54 eglynn-office, yeah, ok 15:53:56 #topic Mid-Cycle attendance and plans 15:54:07 so this is less than 2 weeks away 15:54:28 yep 15:54:33 can folks intending to travel please confirm their attendence on the etherpad by EoW? 15:55:02 I've had a query from a Cisco contributor about potentially sending a couple of attendees 15:55:05 DinaBelova: it's a pity that you cannot come :( 15:55:24 but we need to ensure we've the minimum quorum of core contributors 15:55:46 eglynn-office: and that would be 15:55:47 so if you've got approval from the bean-counters, please update pad 15:55:48 ? 15:56:19 fabiog: I really like to see 5 established ceilo contributors 15:56:37 eglynn-office: ok 15:56:38 #link https://etherpad.openstack.org/p/galway-jan-2014-ceilometer-sprint 15:56:39 I think currently we have 3 declared 15:56:45 ildikov, yeah... I actually regret about this 15:57:50 I will also need an email with full nam, check-in and check-out dates for people that want to take advantage of the HP rate hotel 15:58:08 cool, I mentioned that to the Cisco guy 15:58:25 we're running out of time, as per usual ... 15:58:37 #topic open discussion 15:58:51 just one quick thing from the cross-project meeting 15:59:11 eads up on sdague's logging guidelines 15:59:17 *heads up on sdague's logging guidelines 15:59:21 (first mooted around the ATL summit IIRC) 15:59:28 ... looking like it's finally going to be adopted 15:59:35 #link https://review.openstack.org/#/c/132552/3/specs/log-guidelines.rst 16:00:00 ^^^ please take a look at the latest if you can spare the time 16:00:10 eglynn-office: cool, thanks for the heads-up! 16:00:26 thanks for your time folks, I guess that's a wrap 16:00:33 #endmeeting ceilometer