21:00:01 #startmeeting Ceilometer 21:00:01 #meetingtopic Ceilometer 21:00:01 #chair nijaba 21:00:01 #link http://wiki.openstack.org/Meetings/MeteringAgenda 21:00:01 ATTENTION: lease keep discussion focussed on topic until we reach open discussion topic 21:00:02 Meeting started Wed Nov 21 21:00:01 2012 UTC. The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:03 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:04 The meeting name has been set to 'ceilometer' 21:00:07 Current chairs: nijaba 21:00:20 o/ 21:00:21 o/ 21:00:25 o/ 21:00:27 o/ 21:00:34 o/ 21:00:54 #topic actions from previous meeting 21:01:01 #topic jd and nijaba to start preparing a video demo of ceilometer 21:01:06 no real progress there this week. We clearly do not have much time to work on this at the moment. I guess we'll have to postpone this a bit... 21:01:18 #agreed 21:01:22 :) 21:01:27 unless anyone else wants to work on it 21:01:43 no volunteer? 21:01:51 ok moving on... 21:01:53 #topic eglynn propose agent item on ceilo interaction for nova IRC meeting 21:02:14 so I proposed it to them, but they didn't bite 21:02:24 sorry wrong topic! 21:02:35 #rewind 21:02:48 we discussed the cilo/nova interaction at the nova weekly meeting 21:03:12 and option 5 (a versioned stable lib wrapping th enova virt bit we need) was the preference of the nova folks 21:03:31 provisionally at least, pending the details being fleshed out 21:03:46 who would own the lib? Ceilo? 21:04:01 nova primarily 21:04:07 k 21:04:08 with ceilo contributions to it 21:04:18 and who has the action to flesh out the details? 21:04:27 so I've been trying to smoke out the versioning and releas emgmt implications 21:04:34 #link http://lists.openstack.org/pipermail/openstack-dev/2012-November/003123.html 21:04:49 I still have the action to flesh out 21:05:12 ... and yjiang5 has some ideas also 21:05:26 should you guys action yourselves on that? 21:05:36 yes, I did some investigate on the API 21:05:37 hi, bit late ... 21:05:49 #action eglynn drive fleshing out of nova-virt API to completion 21:05:57 thanks eglynn 21:06:11 ooo, eglynn is driving 21:06:11 #topic nijaba to close survey and publish result prior to next meeting 21:06:22 I did this via an email to the mailing list: 21:06:22 #link http://lists.openstack.org/pipermail/openstack-dev/2012-November/003124.html 21:06:22 We'll have time to discuss this in a bit 21:06:33 #topic asalkeld investigate diamond for use to generate stats 21:06:43 so I had a look 21:07:05 but not really good fit for ceilometer 21:07:16 but good fit for diagnostic 21:07:24 or tracing 21:07:32 interesting 21:07:34 it's kinda like tach 21:07:35 any ideas we can steal for the multi-publishing changes? 21:07:59 well it doen't have a concept of different rates 21:08:09 which would be neat 21:08:16 it would :) 21:08:19 wait, it does 21:08:21 so not really 21:08:39 really 21:08:44 I had a system set up here polling ceph at one rate and everything else at another rate 21:08:50 yeah, you can control that in the config file 21:09:01 I'll have to dig up the settings again, I deleted that vm 21:09:22 o, I meant more caching, aggregation 21:09:25 it controlls the polling rate, which is then the same as the publishing rate for those meters 21:09:38 oh, yeah, I think it assumes upstream does that 21:10:03 I think what we want to find is the best way to minimize polling and intreations, rig 21:10:04 we might want to sample often but publish less often 21:10:22 yep, say for CPU util 21:10:26 that's a point for transformer 21:10:27 (for monitoring) 21:10:30 that can be achieved through transfomer 21:10:36 ya 21:10:46 * jd__ high five yjiang5 21:11:11 the transformer drops some samples on the floor, or? 21:11:22 (to dial down the publishing rate) 21:11:23 eglynn: or use them to build a % value 21:11:28 (or aggregates) 21:11:43 dropping shouldn't be a transformer feature itself 21:11:43 a-ha, OK 21:12:22 thought yjiang5 menat using a transformer to step down the sample rate for fewer publishes ... 21:12:24 possibly we can have transfomer pipeline? some drop, some aggeragete ? possibly 21:12:34 ok, so should we have a topic on transformer design next week? 21:12:34 s/menat/meant/ 21:12:44 yep, long thread in gerrit 21:13:00 not sure a meeting topic will fit 21:13:03 maybe yjiang5 can prep soemthing? 21:13:09 sure 21:13:12 yeah, we need something written up to talk about 21:13:12 (gerrit highly unsuited to long discussions ...) 21:13:12 thanks 21:13:20 I would discuss on the ML firstly 21:13:25 +1 21:13:27 yep 21:13:32 +1 21:13:38 the subject is so complex, async communication will work better as a first step 21:13:42 #action yjiang5 to drive a topic on transformer next week meeting 21:13:54 arghhh 21:14:01 trike t 21:14:06 astrike that then 21:14:10 #unaction ! 21:14:16 ;) 21:14:25 undo? 21:14:28 #rewind too 21:14:45 ? 21:14:52 #action yjiang5 to start a thread on transformer 21:15:05 sure 21:15:13 #topic eglynn propose IRC meetup with Synaps folks to discuss collaboration model 21:15:13 * nijaba guess they did not bite :) 21:15:23 so I proposed it to them, but they didn't bite 21:15:33 however they assure me they're continueing to discuss internally 21:15:35 bummer 21:15:42 and will have a conclusion by week's end 21:15:57 (apparently busy with a bunch of other stuff ...) 21:16:00 re-#action then? 21:16:00 so, let's wait for their decision 21:16:04 cool 21:16:04 yep 21:16:06 so they didn't say no, but haven't said yes? 21:16:13 exactly 21:16:40 #action eglynn to report on synaps' decision 21:16:42 will be wrapped up one way or the other this week 21:16:53 #topic dhellmann to confirm with ttx that tarball job is correct 21:17:06 it was not, but has been fixed so it is now 21:17:16 \o/ thx 21:17:17 cool! thanks dhellmann 21:17:27 #topic nijaba to transform bp-less features into bp 21:17:27 So I spent quite a bit of time on this, you can see the results at 21:17:27 #link https://blueprints.launchpad.net/ceilometer/grizzly 21:17:27 I also updated the roadmap page, but I think this is now truely redudant info and we should get rid of it, wdyt? 21:17:35 the infra team also added a gating job to ensure all of our contributors have signed the CLA 21:17:48 nice! 21:17:51 yep, get rid, duplication is badness for this sort of stuff 21:17:54 nijaba: I hate duplicate 21:18:00 * nijaba too 21:18:11 nijaba: maybe just erase the contents and point to launchpad? saves history... 21:18:15 let's kill the grizzly then! 21:18:24 #action nijaba to get rid of roadmap page content to point to lp 21:18:30 otherwise good work nijaba :) 21:18:36 thanks 21:18:46 yep, much neater! 21:18:53 I guess that's it for last week's actions 21:19:03 #topic Folsom stable update release? 21:19:11 So far, it seems that we have made some good progress that should not break the compatibility with Folsom and some of the bug fixes are somehow critical for production environments. What would you guys think if we were producing a Folsom stable update at the same time of the g2 milestone? 21:19:38 we might even be able to say that g2 is compatible with folsom 21:19:43 this may mean that we should wait until we commit breaking changes after that milestone, though 21:19:54 interesting, do we have a stable/folsom branch? 21:20:02 jd__ has a patch under way to update the CI system to test changes against folsom 21:20:07 I agree with dhellmann on that 21:20:10 eglynn: we don't but can have one 21:20:23 nijaba: yes, we do have a stable/folsom branch 21:20:28 stable/folsom should be a 2012.2.1 release with fix only, but I don't think this is what we want 21:20:28 couldn't we create one, backport selected fixes to that and keep master trucking ahead? 21:20:40 we made a branch right before ODS 21:20:42 dhellmann: ah... the 0.1 branch? 21:20:52 yeah, I thought we called it stable/folsom 21:20:56 * dhellmann goes to git 21:20:58 selecting fix is going to be big grunt work :( 21:21:09 *fixes 21:21:20 cp ceilometer-2012.g2 ceilometer-folsom 21:21:28 remotes/origin/stable/folsom 21:21:51 asalkeld: that would break the rule I think 21:21:56 the main issue with compatibility has been nova 21:22:09 so the stable-maint criteria for backport selection are ... user-visible fix, low impact, low risk 21:22:09 it's saner to keep Folsom compatibility until g2 21:22:14 so far we're compatible, so we could just merge master into the stable branch 21:22:30 dhellmann: I don't think it's policy compliant /o\ 21:22:36 exactly 21:22:40 so, anyone against this? It should be simple IIUC and don't break compat until g3 21:22:45 oh, we have to pick individual patches? 21:22:53 dhellmann: oh yes we have to. 21:23:00 so users expect stable releases to be minimal and contain carefully selected fixes 21:23:02 :-( 21:23:04 stable/folsom is meant to fix bugs like eglynn said 21:23:12 jd__: does this apply to inbation? 21:23:13 ah, ok, that makes sense 21:23:14 not trunk chasing 21:23:25 well, we could just say our g2 release is also compatible and not touch stable/folsom then 21:23:27 nijaba: the point of incubation is to act like others players, right? ;) 21:23:28 incubation even? 21:23:38 dhellmann: I think it's the best option here 21:23:42 jd__: hmmmpfff... 21:23:44 agreed 21:23:48 ok 21:24:08 I've already updated tox.ini to run unit tests against folsom and the -infra team is about to merge my change forcing gate checks againt Folsom 21:24:18 just say yes to this and it'll happen! ;) 21:24:20 #agreed g2 should be declared folsom compatible 21:24:26 one other thing I have found is that our version of anyjson is not compatible with folsom, so I had to make a change for our internal packaging 21:24:36 it conflicts with the version folsom nova wants, IIRC 21:24:40 dhellmann: feel free to send a patch then :) 21:24:54 yeah anyjson dependency caused issues before 21:24:55 unfortunately, changing it will break grizzly compatibility 21:25:05 (nova, quantum etc. have a much older dep IIRC) 21:25:09 dhellmann: we can create a second pip-requires? 21:25:18 so we may actually need a branch at g2 for compatibility with just that change 21:25:25 let's update nova and quantum? ;) 21:25:41 they're updated in grizzly already, I think 21:25:47 yep they are 21:25:55 #link http://wiki.openstack.org/StableBranch#Appropriate_Fixes 21:25:57 ah, ok 21:25:57 so grizzly is consistent, but newer than folsom 21:26:11 maybe we need an unstable/folsom branch ;-() 21:26:16 LOL 21:26:19 lol 21:26:42 dhellmann: what's the problem with supporting several anyjson on our side? 21:27:15 hmm, that might work 21:27:55 #action dhellmann test more lenient anyjson support for folsom and grizzly compatibility 21:28:08 dhellmann: if you need some help about that, ping me 21:28:16 not sure you'll need, but I care about Folsom too 21:28:24 #link http://wiki.openstack.org/GrizzlyReleaseSchedule as a reminder 21:28:35 g2 is jan 10 21:28:45 jd__: thanks, I'll let you know! 21:29:10 #topic Discuss results of Poll 21:29:24 so, did you have time to look at the results 21:29:27 ? 21:29:41 any action we need to take? 21:30:04 I look at the result, and I don't have any comment 21:30:08 the first few high priority items were interesting 21:30:10 I think you set this in the blueprints priorities? 21:30:21 I promise I didn't stuff extra ballots! 21:30:38 I set the priorities according to our internal vote, not the poll so far 21:31:10 nijaba: ok, not sure there's a big difference? 21:31:10 does anything in the results change our priorities? I think those internal changes would be great, but I agree with their current priority. 21:31:15 #link http://lists.openstack.org/pipermail/openstack-dev/2012-November/003124.html 21:31:52 * I want to have monitoring information in a central place 21:31:53 82,8% 24 21:31:55 ooo 21:32:27 +1 ;) 21:32:52 I think the horizon plugin suggestion showed that we forgot to input items that were already on the roadmap on which we did not vote 21:33:09 nijaba: good point 21:33:10 so I added https://blueprints.launchpad.net/ceilometer/+spec/horizon-plugin 21:33:11 and 21:33:20 jd__: how's that prototype you were working on coming along? 21:33:24 https://blueprints.launchpad.net/ceilometer/+spec/user-api 21:33:41 are you guys in on horizon integration atm ? 21:33:46 should we set some medium prio on those? 21:33:55 dhellmann: prototype? on horizon? not me 21:34:06 zykes-: what do you mean? 21:34:09 jd__: you had some graphs, I thought that's what they were for 21:34:28 dhellmann: debug graphs, not plugin stuff yet 21:34:32 dhellmann: ah, nop, it's just a debug interface sending you html rather json when talking to ceilometer-api 21:34:40 s/rather/rather than/ 21:34:52 jd__: ah 21:34:53 and the horizxon plugin needs a user api first, I think 21:35:01 nijaba: yeah, probably 21:35:15 security wise, I think it is a must 21:35:20 we could extend the API to user-API pretty easily now that we have Keystone 21:35:22 nijaba: i'll save it for open-disc 21:35:40 a user API, as opposed to the existing REST API? 21:35:43 ok, so, set taht for medium prio? 21:35:52 eglynn: so a user can't see resource usage for someone else's stuff 21:36:07 eglynn: yes, same api, but contrained to current tenant 21:36:13 a-ha OK, a non-admin-see-the-world API, got it 21:36:14 my idea for that was just to update the api implementation to always add the tenant_id parameter to a query unless the user has admin 21:36:23 so the user api is the same as the admin api 21:36:38 dhellmann: on a differrent port or baed on admin status? 21:36:50 based on the credentials we get from the keystone middleware 21:36:59 dhellmann: ok 21:37:01 I assume those are tucked into the request in a way that we can get to them 21:37:06 dhellmann: you mean only allow URI with in it? 21:37:23 dhellmann: feel free to fill in some details in https://blueprints.launchpad.net/ceilometer/+spec/user-api 21:37:26 jd__: no, allow any url, but internally always restrict by the current tenant_id even if the URL doesn't do that 21:37:39 #action dhellmann update user-api blueprint 21:37:46 dhellmann: thanks! 21:37:47 dhellmann: ah yes, good idea 21:38:00 exactly, as nova does for non-admin queries 21:38:12 We are rebuilding the API server, will user-api depends on that/ 21:38:20 Integrate with monitoring standards such as SNMP, CIM and SMI-S seems to big for this release, right? 21:38:34 I would say so 21:38:45 nijaba: yes, way too big 21:38:46 k 21:38:54 what about https://blueprints.launchpad.net/ceilometer/+spec/meter-post-api ? 21:38:58 I didn't understand 2 of 3 of your stuff, so yes. 21:39:22 nijaba: that should be a small change to the api 21:39:28 say this seems alot like a cw api 21:39:28 * nijaba likes the jd__ test, similar to litmus, 21:39:34 but better 21:40:19 how to auth? 21:40:29 asalkeld: probably 21:40:29 admin only? 21:40:31 post + user stuff 21:40:42 eglynn: at least admin only as a first step would be great 21:40:51 jd__: cool 21:40:53 admin or special role 21:40:55 I am working on monitoring api 21:40:57 yep 21:41:15 asalkeld: native version of CW API? 21:41:18 as stated on the mailing list, this is something PaaS platforms are requesting 21:41:18 at some point we could work towards one api if it made sense 21:41:23 eglynn, ya 21:41:27 cool 21:42:08 this a nice api too http://dev.librato.com/v1/metrics 21:42:19 so, should I mark this as approved/medium, low? 21:43:12 #vote on prio for post meter? high, medium, low 21:43:25 #startvote on prio for post meter? high, medium, low 21:43:26 Begin voting on: on prio for post meter? Valid vote options are high, medium, low. 21:43:28 Vote using '#vote OPTION'. Only your last vote counts. 21:43:31 #vote low 21:43:35 asalkeld: measure_time resolution seems a little coarse 21:43:39 #vote low 21:43:42 #vote low 21:43:45 #vote low 21:43:55 #vote medium 21:44:13 anyone else? 21:44:23 ok, done 21:44:26 #endvote 21:44:26 Voted on "on prio for post meter?" Results are 21:44:27 medium (1): jd__ 21:44:28 low (4): nijaba, dhellmann, eglynn, asalkeld 21:44:33 eglynn, sure - I just meant the rest api 21:44:33 asalkeld: but otherwise looks nice on a first read ... 21:44:46 #startvote on prio for horizon and user api? high, medium, low 21:44:47 Begin voting on: on prio for horizon and user api? Valid vote options are high, medium, low. 21:44:48 Vote using '#vote OPTION'. Only your last vote counts. 21:44:56 #vote medium 21:44:58 #vote low 21:45:01 #vote medium 21:45:10 #vote medium 21:45:16 low 21:45:24 syntax error 21:45:29 I will be off for 10-15min (kids to school) 21:45:34 #vote low 21:45:49 #vote low 21:46:03 (thanks, jd__) 21:46:06 countdown to 10... 21:46:19 #endvote 21:46:20 Voted on "on prio for horizon and user api?" Results are 21:46:21 medium (3): jd__, nijaba, yjiang5 21:46:21 then fireworks? 21:46:22 low (3): dhellmann, eglynn, asalkeld 21:46:39 argh.. we have parity 21:46:41 ok let's say the PTL wins :-> 21:46:54 * nijaba feels empowered! 21:47:05 medium it is then 21:47:26 anything else on the poll? 21:48:00 #topic Review blueprints and progress 21:48:27 so for g2 we currently have 21:48:29 #link https://launchpad.net/ceilometer/+milestone/grizzly-2 21:48:29 it looks like we only have one g2 blueprint not done, if I'm reading the list right 21:48:45 ah, 2 21:48:50 dhellmann: 2 actually 21:49:07 we still have more than a month 21:49:07 anything left for db access? 21:49:16 let me check about db access 21:49:19 should we be a bit more agressive? 21:49:43 yeah, db access is removed EXCEPT from nova_notifier, but that doesn't matter since it's run INSIDE nova 21:49:46 I remember only the nova notifier 21:49:52 I think we should close this one 21:50:09 enayone disagrees? 21:50:14 what is the status of removing db access from the nova compute agent? 21:50:22 when that lands, our notifier will break 21:50:45 yep, good about the no-db-compute work 21:51:01 that's got a pretty long time horizon tho' AFAIK 21:51:19 dhellmann: done 21:51:36 no, the notifier is run by nova itself 21:51:43 not by ceilometer-something 21:51:44 ok, so I am marking it as implemented then 21:51:55 #link https://etherpad.openstack.org/grizzly-nova-no-db-compute 21:52:00 anyone cares to commit on another blueprint for g2? 21:52:25 jd__: I think some time during grizzly there won't be a database handle or settings available to the notifier plugin, either. 21:52:33 I think we should be conservative about adding things 21:52:50 dhellmann: ok, maybe, but I'd say that's another problem :) 21:52:50 we have a lot of cleanup work to do, and figuring out the monitoring stuff is going to take a while, too 21:53:02 jd__: ok :-) 21:53:21 yep, we can always add stuff opportunistically late in the cycle if by any chance we end up with loads of spare bandwidth 21:53:26 eglynn, asalkeld: what about qpid testing? 21:53:42 that should not be too risky 21:53:45 #link https://blueprints.launchpad.net/nova/+spec/no-db-compute 21:53:58 nijaba: good question, I'll see if I can take that or hand it off 21:54:27 change the target when you have a feel for it 21:54:38 nijaba: cool, will do 21:54:42 thanks 21:55:18 so the only thing left is swift, and I thnk we should see progress soon 21:55:32 cool 21:55:44 fingers crossed 21:55:49 dhellmann: you must be impatient on this one ;) 21:55:54 haha 21:56:04 :-) 21:56:17 ok, anything else on this? 21:56:34 #topic Open discussion 21:57:04 I'm making good progress on the pecan & wsme work. I should have something to share next week, I think. 21:57:17 zykes-: you wanted to propose something? 21:57:22 dhellmann: cool 21:57:40 dhellmann: oh, cool. same as for eglynn for updating the bp 21:57:54 #action dhellmann update pecan port blueprint 21:58:04 nijaba: fyi for people that care, StackSherpa is releasing a Java implementation of billing towards openstack, I'll be working with them on porting to Python 21:58:30 meaning porting it over to Bufunfa that uses Ceilometer as source for one 21:58:31 zykes-: link? 21:58:37 back 21:58:45 nice 21:58:50 nijaba: Just talked to them a few mins ago 21:59:02 they'll release a video tonight for it and java code opens next week 21:59:05 zykes-: is it open source? 21:59:10 they sounded very keen on collaborating 21:59:18 nijaba: it will be after what they said 21:59:22 dhellmann, looked at the ceilometerclient yet? 21:59:23 k 21:59:33 asalkeld: not in depth :-( 21:59:43 we need a ceiolmeterclient in openstack/ 21:59:55 with cli 21:59:57 it looked like yours was similar to the other libraries, right? 22:00:00 nijaba: I can report next week :) 22:00:03 dhellmann: where is the tree of the client? 22:00:06