15:01:27 #startmeeting ceilometer 15:01:27 Meeting started Thu Jan 22 15:01:27 2015 UTC and is due to finish in 60 minutes. The chair is eglynn-office. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:29 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:32 The meeting name has been set to 'ceilometer' 15:01:47 o/ 15:01:52 <_elena_> o/ 15:02:04 o/ 15:02:18 o/ 15:02:34 <_nadya_> o/ 15:02:50 hi 15:03:00 hi y'all 15:03:25 #topic kilo-2 15:03:30 #link https://launchpad.net/ceilometer/+milestone/kilo-2 15:03:37 o? 15:03:48 o/ 15:04:28 does the status of all those BPs look accurate? 15:04:55 I bumped the instance-restart, as I don't Einav is working actively on it 15:05:07 don't think 15:05:17 i'll probably need to push elasticsearch support to k-3 (i haven't tried looking at how to test in gate yet) 15:05:54 gordc: cool enough 15:06:11 gordc: what does a local install of elasticsearch look like? 15:06:18 (in terms of complexity) 15:06:48 eglynn-office: it isn't that bad. basically the code i posted back in november. 15:07:05 h-ha, k ... just wondering about potential push-back from infra about installing on CI nodes 15:07:10 *a-ha 15:07:11 the queries we have in our api aren't that complex so building the queries weren't hard. 15:07:40 o/ 15:07:41 i think the complaint is that actually running elasticsearch eats RAM like crazy. 15:07:59 it was fine for just doing testing locally on my thinkpad though. 15:08:07 is it tunable I wonder, to dial down cache sizes etc. 15:08:48 eglynn-office: possibly... i believe there is a memory based backend which is typically used for testing 15:08:54 o/ 15:09:20 * gordc haven't really looked at it recently... just been glancing at it during downtime 15:09:40 gordc: cool enough, all that'll prolly come out in wash anyway in the discussion with infra about installing for CI 15:10:10 eglynn-office: yeah... i'll need to talk with clarkb later once i've dug into it. 15:10:24 coolness 15:10:42 any got anything else to flag for kilo-2? 15:11:04 feb 5 is the magic date ... https://wiki.openstack.org/wiki/Kilo_Release_Schedule 15:11:36 eglynn-office, I have that https://review.openstack.org/#/c/149063/ 15:12:12 eglynn-office, it's fresh it's allow to write gnocchi alarm stuff on gnocchi side 15:12:30 o/ 15:12:32 sileht: a-ha cool, looks very reasonable on a first quick scan 15:13:36 sileht: ... I'll review in more detail after meeting, but looks like we can get it in quickly 15:14:46 eglynn-office, sure 15:16:43 fabiog: quick dumb question about the RBAC stuff landed in kilo-1 15:16:53 fabiog: ... in the absence of a matching rule in the policy.json, default is to allow amiright? 15:17:15 eglynn-office: no, still apply the rule of admin or not 15:17:50 eglynn-office: so if there is no rule and you are admin, you have total control, otherwise you are bound to the project 15:18:29 fabiog: for non-admin ... in the absence of match rule for API operation, then apply segregation rule as always, but allow the API operation to proceed? 15:18:42 *matching rule 15:19:08 eglynn-office: yes 15:19:32 coolness, just as I thought, needed a sanity check :) 15:19:39 eglynn-office: unless it has a @admin-required 15:19:55 yeap 15:20:02 k, let's move on from kilo-2? 15:20:45 #topic gnocchi 15:21:36 just a heads-up that jd__ and I were discussing options around where to move the gnocchi code 15:21:50 (we touched on this in last week's meeting also) 15:23:04 jd__: so the idea is to propose gnocchi for big-tenting under the openstack/ namespace, right? 15:23:51 yup eglynn-office 15:24:04 if the rest of the Ceilometer folks are interested obviously 15:24:28 but since it sounds like we are going to leverage Gnocchi heavily in the future I guess it'd be a good idea to bring them closer 15:24:33 I think more repos is better than less repos 15:24:34 jd__, well, we are :) 15:24:38 I'm ;) 15:25:15 the advantage of doing that as opposed moving directly into the ceilo repo would be to avoid continueing to add directly to the existing monolith 15:25:16 I'm too :) 15:25:55 eglynn-office: sounds good to me :) 15:25:56 also advantageous in terms of supporting the "standalone gnocchi" usecase 15:26:04 the same way we hand oslo.* libs? still under Ceilometer/telemetry umbrella? 15:26:06 i.e. as a general purpose metrics store 15:26:24 gordc: yes 15:26:36 jd__: cool cool. works for me 15:26:42 a lot of projects have more than one repository FWIW already 15:27:16 gordc: ... still with a separate (but expanded) gnocchi-core group 15:27:19 jd__, gordc: Agree, it looks cool for me too) 15:28:15 eglynn-office: gordc: yeah matching Oslo practice 15:28:31 one further advantage would be gnocchi independently attracting contributors 15:28:55 ... i.e. folks interested in metrics storage on its own merits 15:29:04 independent of the data collection pipeline 15:29:23 eglynn-office: so in the long term gnocchi will replace Ceilometer? I thought there would have been some sort of "merging" ... 15:29:35 fabiog, not 'replace' 15:29:41 ceilometer will use gnocchi 15:29:45 fabiog: gnocchi will be used by ceilo, not replace 15:30:24 that's basically kind of API to write/query datapoints 15:30:38 eglynn-office: but now you have part of the API in 1 service and part in a second one ... do you need to deploy two API servers? 15:31:28 fabiog: or treat gnocchi as a "backend service" 15:31:43 eglynn-office: ok, that makes sense 15:32:49 eglynn-office: so Ceilometer API v3 would be the place where other Openstack services interact and Gnocchi is internal to Ceilometer, did I get it right? 15:33:08 eglynn-office: Gnocchi API I mean 15:33:22 if someone wants to build a Ceilometer API v3 on top of Gnocchi probably 15:33:28 I'm not sure it makes more sense right now 15:33:54 we have 3 different APIs for 3 different purposes that are not tied together (events/samples/alarms) 15:33:56 jd__: but if it is backend service you are not going to expose its API to generic clients ... 15:34:09 fabiog: the ceilometer v3 API could be partailly layered over the gnocchi API (specifically an equivalent of the statistics API implemented as a pass-thru' to gnocci) 15:34:29 eglynn-office: yeah what's not clear is the value the Ceilo APIv3 would add over Gnocchi's API yet :) 15:35:40 jd__: yeap ... say if instead gnocchi API is called direct, the python-ceiloclient would need to be aware of two endpoints from the ks service catalog, amiright? 15:35:47 eglynn-office: maybe we should have a deeper discussion on how to integrate/leverage the two projects. It was one of the mid-cycle topics 15:36:08 ... i.e. gnocchi would expose it's own publicURL endpoint in the service catalog 15:36:15 eglynn-office: yes, though likely that'd be gnocchi client I guess 15:36:17 jd__: is there a place where we can see response gnocchi will return (to compare how different it is from current ceilometer api v2 responses?) 15:36:38 eglynn-office: I wouldn't be offended by seeing 3 endpoints in the long term 15:36:56 gordc: the documentation is pretty good in this regard 15:36:56 gordc: http://gnocchi.readthedocs.org/en/latest/rest.html 15:37:00 gordc: tox -e docs and open it 15:37:19 eglynn-office: yeah that one is not up to date anymore since we started building dynamic doc :( 15:37:19 cool cool. i was too lazy to look. :) 15:37:19 gordc, http://gnocchi.readthedocs.org/en/latest/rest.html 15:37:38 I wish we could build the doc on OS CI but I don't think we can until we are off stackforge 15:37:51 gordc: so yeah better doc is tox -e docs 15:37:52 jd__, btw am I right that right now when we query gnocchi we grab list of datapoints basically, although some of Ceilo requests may return one concrete value? 15:38:07 jd__, does gnocchi.readthedocs.org is updated manually ? 15:38:09 like avg for all period of time VM is working? 15:38:27 sileht: no it's automatic but you need a complete env (SQL) to build the doc now so RTFD fails 15:38:45 jd__, oh ok 15:38:45 DinaBelova: there is no direct analogue of that on-the-fly aggregation in gnocchi 15:39:01 DinaBelova: hum I think so yes, you'd have to do it yourself 15:39:23 eglynn-office, jd__ - a-ha, ok, I'll take a look on it 15:39:29 will add to todos :D 15:39:29 DinaBelova: because I think it'd only work with some aggregation methods (like min/max/mean) but not all 15:39:48 I think we have a much better understanding of statistics in Gnocchi :-> 15:39:53 thanks to eglynn-office 15:40:03 DinaBelova: so if the v3 API is based on gnocchi, then on-demand periodization is replaced by up-front declaration of the needed granularities via the archive policy 15:40:54 eglynn-office, yeah, thanks 15:41:33 also I hope to make this chain https://review.openstack.org/#/c/148271/ working today 15:41:45 will send changes on review :) 15:42:30 so the other thing I wanted to mention about gnocchi is that we've a bunch of integration tasks that we need to ensure are covered 15:42:54 e.g. alarm evaluation (that sileht has starting working on :)) 15:43:35 eglynn-office, I have a working PoC to help me to design the Alarm Rule description 15:43:44 figuring out the v3 API within the restrictions of the gnocchi pre-aggregation 15:44:08 ✔︎ alarm evaluation 15:44:31 eglynn-office: not sure what that's mean but OK :D 15:44:31 sileht: yep, I saw https://review.openstack.org/149182 , nice! (... I left a quick suggestion on gerrit) 15:45:16 also figuring out the CLI side (i.e. extend python-ceiloclient or write a separate python-gnocchiclient) 15:45:24 Also I want to tests gnocchi dispatcher via ceilometer test tools for different storages. if someone interesting, i create etherpad with test plan. https://etherpad.openstack.org/p/gnocchi_dispatcher_test_plan 15:45:28 also ensuring full resource type coverage 15:46:13 idegtiarov, were you going to send volumes resources change on review today? 15:46:28 due to the moment eglynn-office mentioned? 15:46:55 DinaBelova: already done 15:47:17 idegtiarov, may you please cover all this task? 15:47:26 I mean adding all these resources/meters to gnocchi 15:47:31 DinaBelova: yep, will do 15:47:41 I'll put together a list of these integration tasks for next week, will be looking for volunteers to get them all squared away for kilo-3 15:47:58 eglynn-office, cool, really nice thing 15:48:09 I'm eager to see that 15:48:12 <_nadya_> we will add alarm evaluation to gnocchi? It seems to be ceilometer responsibility... Ok then, looks like conversation about Gnocchi may continue forever :) 15:48:55 _nadya_: read the backlog :) 15:49:17 _nadya_: silehthas proposed an alarm eval PoC to gnocchi, but this could optionally replace "classic" ceilo alarm eval 15:49:35 DinaBelova: actually I would like to propose some refactoring according to resource adding in gnocchi, hope will propose WIP patch early next week 15:49:48 idegtiarov, stevedore thing? 15:50:10 DinaBelova: yes it is :) 15:50:14 cool-cool 15:50:42 eglynn-office, both can be used together ! 15:50:49 <_nadya_> eglynn-office: yep, I see. Just thought that fabiog's question about replacing may appear again :) 15:51:28 sileht: not an mutually exclusive proposition? 15:52:10 sileht: ... I thought you'd choose to load the classic threhsold evaluator, or the gnocchi-based one, but never both in the same deployment? 15:52:17 eglynn-office and _nadya_ I would like to see a longer term picture of how these two components will co-exist in the future, does someone have such a picture? 15:52:39 eglynn-office, I'm thinking about combining ceilometer and gnocchi alarm with the CombinationAlarmRule 15:52:57 and if that has not been defined yet, I think it would be a good think to work on 15:53:09 thing 15:53:14 fabiog: yes, it's a work-in-progress 15:53:20 eglynn-office, this allows to smoothly migrate to gnocchi 15:53:37 eglynn-office: is there a wiki or some docs about it? 15:53:53 eglynn-office: please :-) 15:54:21 <_nadya_> fabiog: I'm absolutely agree with you. I think that we need more time for it ... 15:54:22 fabiog: jd__ had a wiki but it's not up-to-date ... let's add that as a task 15:55:17 eglynn-office: I have a doodle for the Batching suggestion 15:55:26 eglynn-office: I added to the agenda topic 15:55:38 fabiog: yep, let's move on that in open discussion 15:55:44 #topic open discussion 15:55:45 eglynn-office: if we write something it'd be cool to have that in one of the ceilo or gnocchi doc, not a wiki, so it's actually "useful" 15:56:10 jd__: fair point 15:56:15 jd__: ... a spec perhaps? 15:56:32 jd__: (... so as to make it a discussion) 15:56:47 fabiog: do you want to mention your doodle now? 15:56:51 eglynn-office: at least 15:57:20 eglynn-office:yes http://doodle.com/cwv5nmv3hxadvreu 15:57:42 folks pls vote on that ^^^ 15:57:51 fabiog: what's the native TZ for the poll? 15:57:53 so please people subscribe for the slot, I will then set-up a Google Hangout and I will present the idea, if that has traction I will prepare a BP 15:57:53 PST? 15:58:01 it is PDT 15:58:05 PST 15:58:11 Pacific Standard Time 15:58:21 so the first slot is the same time as the Ceilo meeting 15:58:23 UTC-8:00 15:58:28 yes 15:58:33 cool\ 15:58:49 I will check the votes EOD friday 15:59:39 cool thanks fabiog 15:59:43 <_nadya_> fabiog: thanks a lot! It would be great to attend 15:59:52 anything else folks wanted to raise? 16:00:27 the shot-clock has beaten us again 16:00:29 ... thanks for your time folks 16:00:34 #endmeeting ceilometer