15:00:10 #startmeeting ceilometer 15:00:11 Meeting started Thu Oct 16 15:00:10 2014 UTC and is due to finish in 60 minutes. The chair is eglynn-office. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:13 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:15 The meeting name has been set to 'ceilometer' 15:00:18 hey y'all 15:00:26 o/ 15:00:27 hey! 15:00:27 o/ 15:00:29 o/ 15:00:30 o/ 15:00:35 o/ 15:00:56 o/ 15:01:15 #topic Juno close-out 15:01:22 o/ 15:01:27 consider it closed! :) 15:01:30 o/ 15:01:34 o/ 15:01:52 thanks all for your efforts over juno :) 15:01:56 yay! 15:02:08 yeehaw! 15:02:18 <_nadya_> o/ 15:02:31 agreed. good stuff all around. 15:03:01 release notes could do with a final sanity check 15:03:04 #link https://wiki.openstack.org/wiki/ReleaseNotes/Juno#OpenStack_Telemetry_.28Ceilometer.29 15:03:37 note that cdent found a nasty issue in the new ipmi-agent when testing the packaging for Fedora 15:03:41 https://bugs.launchpad.net/ceilometer/+bug/1381600 15:03:43 Launchpad bug 1381600 in ceilometer/juno "ceilometer-agent-ipmi fails on unparseable data" [High,In progress] 15:03:54 fix is landed, but too late to make the last RC 15:04:20 so we'll backport to stable/juno and the distros will have to carry as an extra patch 15:04:54 eglynn-office, what do you think about idegtiarov's Mongo pathces backporting as well? 15:05:02 eglynn-office: for the release notes, separate storage for alarm? 15:05:02 when they'll be landed for kilo? 15:05:19 llu-laptop: good point, I'll add that 15:05:46 eglynn-office: i think we should add we support notification publisher now instead of rpc. (i think that was juno... or was it icehouse?) 15:05:47 DinaBelova: yes they can land for kilo, on the edge of suitability for stable though 15:06:29 eglynn-office, okay, cool, let's think about if we need to backport them to juno when they'll land in Kilo 15:06:51 gordc: a-ha, it was juno IIRC ... I'll check the log to be 100% sure 15:07:14 gordc, juno I believe 15:07:15 eglynn-office: yeah i think sileht got that in really early in the cycle. 15:07:25 gordc, ++ 15:07:26 gordc: yep, I think you're right 15:07:29 it was Juno 15:07:34 o/ 15:07:43 DinaBelova: here's the stable policy ... https://wiki.openstack.org/wiki/StableBranch#Appropriate_Fixes 15:07:46 cool cool. yeah let's add that in. 15:07:57 eglynn-office, thanks, will o through it for sure 15:08:29 cool, anything else on Juno? 15:08:48 eglynn-office, U just don't like the idea keeping this replica set bug in the juno.... 15:08:54 I* 15:09:14 DinaBelova: yep, I understand 15:09:25 #topic TSDaaS/gnocchi status 15:09:29 jlibosva: 15:09:40 darn tab completion! 15:10:01 jd__: anything notable in gnocchi-land? 15:10:14 eglynn-office, I guess you wanted to say "jd__ - floor is yours!" 15:10:18 :) 15:10:22 hi guys 15:10:27 I'm still writing documentation 15:10:40 I'm off this week so it got a little delayed 15:11:03 sileht made good progress toward aggregation of entities 15:11:04 jd__, btw, thank you so much for this effort! ityaptin spent much time on gnocchi investigation while its perf testing becasue of docco lack 15:11:16 I need to review the dispatcher, but sounds like sileht did good stuff too 15:11:38 DinaBelova: I can send the doc for review now if you want and if that can already help ityaptin 15:11:46 jd__: yeah, there's a proposal on getting around the circularity issue based on project_id 15:11:46 jd__, eglynn-office - ityaptin will share the doc in a hour 15:11:50 jd__, thanks! 15:11:52 it's not finished but I can update the patchset when I advance 15:11:55 it'll be really cool 15:12:06 eglynn-office: I missed that where it is? 15:12:20 jd__: https://review.openstack.org/#/c/128922/ 15:12:24 DinaBelova: ok I'll push it today and will send the link on IRC if 15:12:28 s/if// 15:12:34 jd, thank you sir! 15:12:46 DinaBelova: cool, the suspense is killing me! :) 15:12:52 eglynn-office :D 15:12:56 eglynn-office: ok that's just me not yet up to date on my mails then :) 15:13:23 * jd__ is visiting Raleigh 15:13:24 folks, I'll share the doc today - you'll have the opportunity to look on current perf with swift 15:13:42 jd__: isn't there a law against reading mails while on vacation? :) 15:13:52 jd__, sileht, but I found one issue with dispatcher auth 15:14:08 ityaptin, Yes I disable it for now :p 15:14:22 jd__: a-ha, got it, the new mothership :) 15:14:56 an other issue I have is that we need to limit the number of parallel request that the dispatcher does to gnocchi 15:15:03 sileht, yeah, I noticed 15:15:53 sileht: is that issue with the acks being delayed untill all threads return really a bug (or a feature) in oslo.messaging? 15:16:02 eglynn-office: yeah I'm off for upstream work not for work ;) 15:16:36 eglynn-office, this is the normal behavior, but it's worth for this usecase 15:16:40 sileht, yes! I tested it with make_test_data script form ceilometer/tools, and it's working in one thread how i supposed it will be. 15:17:04 I had not enough time to use TSDB, yet. 15:17:09 I'll try to collect TSDB results asap 15:17:15 to have the opportunity to use these results for the summit talk 15:17:24 ityaptin: great, thanks! 15:17:30 ityaptin: can you talk about these results yet? I'm curious 15:18:03 eglynn-office, we want that the message is not ack until the gnocchi calls are done, but we don't want process a millions of samples in parallel because eventlet thing is good to continue to process messages 15:18:18 <_nadya_> ityaptin: you gonna use Dina's patch set for tsdb? which is on review now? 15:18:40 <_nadya_> ityaptin: or smth is already merged? 15:19:06 sileht: a-ha, got it ... I was wondering if each individual ack could be done eagerly after the individual POST to gnocchi returned 15:19:28 (instead of waiting for all the gnocchi calls to complete, IIUC) 15:19:34 with dispatcher I have several issues at the moment, but post entities and post measures show stable results for big amount 15:19:50 eglynn-office, yes I'm trying to build something like that 15:20:04 sileht: a-ha, cool! 15:20:28 Gnocchi doc WIP https://review.openstack.org/#/c/128958/ 15:20:30 jd__: One entity post nearly 0.5 sec, 1k measures - near 0.5 sec as well 15:21:03 ityaptin: sounds like everything scales out :) 15:21:06 jd__: nice! 15:21:31 _nadya_: Yes, I am going to pull Dina's patchset and try to deploy and configure OpenTSBD 15:21:33 jd__: who drew the picture? 15:21:43 jd__, eglynn-office - we're supposing it'll look much better for the TSDB 15:21:51 jd__, thanks for the link 15:21:55 gordc: my little bro 15:22:08 LOL :) 15:22:21 <_nadya_> ityaptin: ok, I will add some comments there 15:22:29 gordc: http://instagram.com/p/rwuDnDgOqj/ 15:22:43 _nadya_ : cool) 15:22:53 jd__: :) so you paid a beer. 15:23:00 gordc: indeed! :) 15:23:18 I see the family resemblence there :) 15:23:37 cool, shall we move to the summit topics discussion? 15:23:41 * jd__ nods 15:23:48 #topic Kilo summit design session topics 15:24:33 so the idea was to do a quick pitch for each idea 15:24:54 do you want to pich this idea about doing pitches? 15:25:24 Well, I may start first, as I need to collaborate with eglynn-office for our topics :) 15:25:52 jd__: well, do you think the topics are fully self-explanatory as it stands? 15:26:03 if so, no need for pitches per se 15:26:04 eglynn-office: maybe you want to relink the google doc? 15:26:07 or we can just go through the list of proposals in the order they are placed in the doc 15:26:17 #link http://bit.ly/kilo-ceilometer-summit-topics 15:27:19 eglynn-office: I think it's going to be long to pitch everything right now if that's what you have in mind 15:27:32 OK, let's make the "pitch" element optional 15:27:46 perhaps eglynn-office or someone can go through one by one and ask if more info needed? if not move on? 15:27:50 do these all have bps? 15:28:05 gordc, gnocchi ones no, I believe 15:28:06 gordc: not all of them 15:28:11 just shout "pass" if the topic summary on the spreadsheet is sufficient 15:28:17 nm.. i just saw the column... 15:28:28 stupid half screen. 15:28:28 gordc: yeah, I added that :-) 15:28:35 fabiog_: good idea 15:28:44 jd__: "Bringing Gnocchi and Ceilometer together" anything to add there? 15:28:53 I think that some of these sessions are not properly aligned with our Gnocchi effort 15:29:06 eglynn-office: I think it speaks for itself? :) 15:29:21 eglynn-office: I would like to know what is the scope 15:29:27 jd__: would it cover migration/co-existence also? 15:31:03 eglynn-office: whatever we think it should; co-existence is my favorite 15:31:15 llu-laptop: your proposal might be connected to this: https://review.openstack.org/#/c/105130/ 15:31:19 jd__: cool 15:31:25 eglynn-office: jd__: it should IMHO 15:31:43 eglynn-office: does Gnocchi need to go to incubation before becoming a co-existing tool for Ceilometer? 15:31:44 llu-laptop: don't know how much overlap but i think it's tackling same problem. 15:32:04 fabiog_, there is the idea to merge this code to ceilo after POC will be tested ok 15:32:16 so it won't be the separated repo/project 15:32:17 gordc: yes, somthing similar like that 15:32:18 it's a telemetry project we don't need incubation 15:32:25 DinaBelova: or it might :) 15:32:31 jd__, yeah 15:32:44 anyway, it's integrated telemetry program 15:32:50 k, we don't need to foreshadow the summit discussion 15:32:54 if we agree yes :) 15:32:59 like sahara folks are having stackforge repos as well - for the tools 15:32:59 * jd__ nods 15:33:04 llu-laptop: cool cool. seems like a logical problem to fix. 15:33:05 cool 15:33:08 yeah, sorry 15:33:10 fabiog_: "Role Base Access Control for Ceilometer API" anything to add? 15:33:22 that's policy implementation? 15:33:43 oh yes that's it (just read the doc :p) 15:33:45 jd__: yes. currently we have a very limited ability to define who can do what 15:33:45 +1 for this 15:34:08 though i'm not sure we need to _discuss_, I think we need to _act_ 15:34:13 jd__, fabiog_ - I remembber we had Bp for this 15:34:20 yes we have already 15:34:23 jd__: basically is to have policy enforcement against rules 15:34:27 we just need someone to do the work 15:34:51 yep, in general if the path ahead is clear, less need for a summit discussion 15:34:59 jd__: I can work on this. I have some experience with KS 15:35:03 agreed 15:35:16 fabiog_: coolness, sorry I took that topic out of order 15:35:20 really +1 for just implementing 15:35:25 sweet! resource! 15:35:26 yep 15:35:39 I guess KS expert might impl it much better than any of us 15:35:42 and with better result 15:35:43 fabiog_: "Individual Meter/Sample Retention Policy" anything to add? 15:36:01 I already commented on this one that it's kind of orthogonal with Gnocchi AFAIU 15:36:06 seems this falls out of gnocchi very naturally 15:36:17 jd__, yeah, it 15:36:22 eglynn-office: I think we lack of the ability here to remove extra messages 15:36:25 is definitely part of this initiative * 15:36:47 fabiog_, hmm, extra messages? 15:36:56 sorry, I possibly ca'nt get the point 15:36:59 samples I guess? 15:37:14 DinaBelova: yes notifications. Oslo does not control what is sent. So we need to filter 15:37:15 DinaBelova: samples for one meter being retained for longer than another meter 15:37:29 a-ha, ok, got it 15:37:39 DinaBelova: especially for Billing. You don't need all the notfications 15:38:05 well, but anyway it seems to need be reviewed as a gnocchi part 15:38:08 amiright? 15:38:16 k, sounds like something that would be useful, though essentially covered by one aspect of gnocchi 15:38:27 fabiog_: "Notification Filter" anything to add there? 15:38:32 fabiog_, DinaBelova: there is some suggestion of work on this in Oslo, too. 15:38:43 So for discussion, perhaps visit that too. 15:39:06 I think it might not be sufficient, but should be considered. 15:39:14 eglynn-office: sorry I got them confused 15:39:23 I was talking about notifcation filters right now 15:39:36 doesn't the existing pipeline allow us to filter out meters, and hence the notifications that gave rise to these samples? 15:39:57 eglynn-office: not for notifications AFAIK 15:39:58 I'd think so too 15:40:02 meta note: I think we should prioritize the in person meetings for those things which aren't far enough along to even know what to spec 15:40:29 * eglynn-office doesn't understand the need for additional filtering out for notifications 15:40:42 cdent: agreed in general 15:40:48 eglynn-office: I would like to have the same ability to express the meters I am interested that are coming from notifications 15:41:19 cdent: so the question is not so much "do we want to do this?" more "would this profit from F2F discussion?" 15:41:22 fabiog_: that should exist. (if not, it's a bug) 15:41:48 fabiog_: a-ha, got it 15:41:48 gordc: No, it does exist if you sent stuff to Events 15:42:31 eglynn-office: because we cannot control what a service emits as notification, but we can filter it at our end if it is not of any interest 15:42:46 eglynn-office: does it make sense? 15:43:08 fabiog_: do you mean that you want to disable some notification listener in ceilo? 15:43:12 gordc: afaik notifier takes all "handled" events and republishes.... 15:43:27 fabiog_: yeah, I'd need to check if the pipeline source filtering also applies to notification-driven samples on the publish 15:43:32 does not sound like we need F2F for that 15:43:53 yeah, defo sounds useful, but not necessarily a f2f topic 15:43:55 llu-laptop: no I want to remove the notifications from the queue and just drop them if not in the list of things I want to retain 15:44:19 fabiog_: yep, got it now ... sorry for being dense there! 15:44:24 nealph: "Configuration (pipeline) using persistent data store" anything to add? 15:44:32 fabiog_: I support this idea 15:44:52 so: Gnocchi, Monasca, in-tree func tests, Kafka, Notif contracts these are my pick for things that'd benefit from F2F discussion, the rest doesn't need that IMHO 15:44:53 ildikov: thanks :-) 15:45:01 eglynn-office: nothing specific to add. Etherpad up at https://etherpad.openstack.org/p/configuration_via_data_store 15:45:14 nealph: thanks, nice! 15:45:18 eglynn-office: ^ summarized for you :p 15:45:28 jd__: cool 15:45:53 thanks to ildikov for helping drive it. Fabio will be my proxy at the summit. :) 15:45:55 fabiog_: I'm a great fan of better configuration or well let's say a bit more fine-grained 15:46:29 cool 15:46:41 eglynn "Functional areas to target for collaboration with Monasca" anything to add? 15:46:50 nealph: cool, I think it would be a great thing to have 15:47:06 nope, self explanatory 15:47:06 * jd__ notes the schizophrenic behaviour 15:47:10 LOL :) 15:47:32 fabiog_, nealph: "Apply Limit/Next for open ended queries" anything to add? 15:47:48 is this related to pagination? 15:48:03 eglynn-office: yes. This could be also solved implementing pagination. I noted that Ceilo does not support that 15:48:12 llu-laptop: that or an alternative. 15:48:22 kind of a default limit if none specified? 15:48:23 eglynn-office: I prefere this simple limit and next approach 15:48:34 I think pagination is paritally supported, sad though 15:48:36 eglynn-office: yes in the config 15:48:46 cool, got it 15:48:50 fabiog_: agree we need a limit... don't know if we need a session for it. 15:48:59 llu-laptop: eglynn-office: this is really a performance concern, as Horizon implements a wide-open query. 15:49:13 * jd__ doesn't want to hear about pagination anymore 15:49:30 jd__: LOL, terrible past 15:49:31 we lost at least a developer over that 15:49:34 sounds like one for the contributor meetup on the Firday perhaps? 15:49:34 jd__: lol it's back! 15:49:34 :-) 15:49:46 jd__:I've heard rumors.... :) 15:49:50 eglynn-office: sure 15:49:50 I'll make a tshirt for it 15:50:07 eglynn "Switchover to in-tree functional tests" anything to add? 15:50:08 nope, self explanatory 15:50:17 and definitely we need it 15:50:19 imho 15:50:44 yep, it'll be a big deal for kilo if QA and the gate is to scale 15:50:45 def needs to be some chat on how 15:50:53 yeah, ++ 15:51:13 cdent: absolutely ... heat & neutron have blazed the trail somewhat 15:51:18 I need to know how to make all this thing tested :D Vadim will kill me without the decision :D 15:51:27 cdent: /me needs to crib from their play-book 15:51:42 eglynn "Mapping gnocchi semantics to InfluxDB" anything to add? 15:51:45 eglynn-office, I would suggest to merge our two sessions (InfluxDB / OpenTSDB) into the one. They seem to be most valuable backends for the future, so we need to find out the most performant wrapper to fit both of them + Swift. 15:52:04 DinaBelova: yep, sounds good 15:52:06 InfluxDB has new features released that’ll help continuous aggregation, OpenTSDB has no retention policies for now (although HBase has), they have different aggregation functions, etc. - we need to map gnocchi terms to the terms of these DBs 15:52:19 DinaBelova: do want to update the spreadsheet to consolidate? 15:52:27 well, yeah, why not 15:52:32 eglynn-office: is Paul Dix planning on attending (do we need him to?) 15:52:41 eglynn-office, although it'll be ~40 mins I guess 15:52:45 gordc: great question, I'll find out 15:52:49 I guess if we’ll discuss Influx+OpenTSDB together, it might take 30 mins, in case of some preparations done by me and Eoghan (for each DB)... dunno for sure 15:52:56 eglynn-office, I'll fix the doc 15:53:00 DinaBelova: "Mapping gnocchi semantics to OpenTSDB" anythinh to add? 15:53:03 eglynn-office: cool cool 15:53:13 eglynn-office ^^ the same as for yours 15:53:19 cool 15:53:20 pradk: "Metering Ceph storage cluster" anything to add? 15:53:22 we need to merge them 15:53:33 yep, agree 15:53:54 eglynn-office, nothing much to add.. just that would be good one for face time with some ceph experts to bounce some ideas 15:53:59 pradk: how different is from polling from Swift? 15:54:15 pradk: or are you going to implement notifications 15:54:18 fabiog_: ceph standalone as opposed backend for swift 15:54:30 fabiog_: ceph standalone as opposed *to backend for swift 15:54:32 fabiog_, exactly what eglynn-office said 15:54:42 pradk: "Alarm Enhancements" anything to add? 15:55:05 pradk: sure, but are you going to poll or send notifications 15:55:25 eglynn-office, nothing much to add there. Just need some clarity on if this is possible or needs code.. we can discuss this at project pod if needed 15:55:34 pradk: cool 15:55:34 eglynn-office, fixed the docco 15:55:40 DinaBelova: thx! 15:55:43 np 15:55:45 Yathiraj Udupi/ Debo Dutta "Kafka Publisher" anything to add? 15:55:53 fabiog_, probably both 15:56:01 seems like it could maybe be rolled into the monasca session? 15:56:10 pradk: thanks for the clarification 15:56:17 llu-laptop: "pollster self-disable mechanism" anything to add? 15:56:25 eglynn-office, i think there is someone who subj,fitted spec and code for kafka already? 15:56:55 not much, just want to find a way to not loading/running pollsters which is estimated to be faliure 15:57:27 pradk: spec https://review.openstack.org/#/c/126964/ 15:57:44 llu-laptop: "YAML definition for snmp metrics" anything to add? 15:57:58 lsmola would like that :) 15:58:04 eglynn-office, code https://review.openstack.org/#/c/126989/ 15:58:44 pradk: thanks! 15:58:49 just like gorc just pointed, it's similar to the https://review.openstack.org/#/c/105130/ 15:58:52 cdent: "Notifications {as-a-contract, standardization, maximization}" anything to add? 15:59:03 nope, I think the summary gets it 15:59:15 cdent: cool 15:59:16 cdent: you planning a cross proj session as well? 15:59:43 well since I'm not going to be there, not me personally, but sandy walsh and I have been talking about it on the linked email thread 16:00:00 anyone want to volunteer to lead/facilitate that session as cdent won't be there in person? 16:00:30 k, at least we did a first pass on all topics 16:01:02 can folks go away and think some and then start marking your interest on the spreadsheet 16:01:10 remember we only have 6 slots 16:01:12 thanks! 16:01:15 thanks! 16:01:22 time bell rings 16:01:24 laters 16:01:30 eglynn-office, quick update for the next topic - jd__, ildikov & I marked us for these common activities 16:01:32 we're over time but justa quickie ... call for volunteers on https://wiki.openstack.org/wiki/CrossProjectLiaisons 16:01:37 eglynn-office ^^ 16:01:43 >.> 16:01:49 DinaBelova: +1 16:01:50 DinaBelova, ildikov: thanks! 16:01:59 k, beaten by the clock 16:02:04 #endmeeting ceilometer