15:00:07 <nijaba> #startmeeting Ceilometer
15:00:07 <nijaba> #meetingtopic Ceilometer
15:00:07 <nijaba> #chair nijaba
15:00:07 <nijaba> #link http://wiki.openstack.org/Meetings/MeteringAgenda
15:00:08 <openstack> Meeting started Thu Nov 15 15:00:07 2012 UTC.  The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:10 <openstack> The meeting name has been set to 'ceilometer'
15:00:12 <openstack> Current chairs: nijaba
15:00:16 <nijaba> Hello everyone! Show of hands, who is around for the ceilometer meeting?
15:00:16 <nijaba> o/
15:00:19 <dhellmann-afk> o/
15:00:23 <hungry-eglynn> o/
15:00:26 <dhellman> nick dhellmann
15:00:36 * dhellmann needs more caffine
15:00:37 <yjiang5> o/
15:01:24 <nijaba> #topic actions from previous meeting
15:01:33 <nijaba> #topic dhellmann update versioning in ceilometer repo to match openstack standards
15:01:39 <jd__> hi
15:01:52 <dhellmann> the versioning work is done
15:02:05 <nijaba> nice!  thanks a lot.
15:02:08 <dhellmann> there's some sort of issue with the version number in the readthedocs.org build, which I have on my list to fix
15:02:51 <nijaba> ok.  no need to track this as an action?
15:03:02 <dhellmann> I don't think so
15:03:10 <timjr> howdy cowpokes :)
15:03:12 <nijaba> #topic jd__ and nijaba to start preparing a video demo of ceilometer
15:03:20 <nijaba> no real progress there this week.  re assigning for next week.
15:03:29 <anniec> o/
15:03:37 <nijaba> #topic jd__ and nijaba to start preparing a video demo of ceilometer
15:03:48 <nijaba> #action jd__ and nijaba to start preparing a video demo of ceilometer
15:03:53 <nijaba> that's better :)
15:04:02 <nijaba> #topic eglynn to writup a nova integration proposal to be discussed next week
15:04:28 <eglynn> the discussion with the nova folks is still ongoing, summarized here ...
15:04:31 <nijaba> so we saw eglynn's message on the ml
15:04:32 <eglynn> #link http://wiki.openstack.org/EfficientMetering/FutureNovaInteractionModel
15:04:51 <timjr> I'll probably be submitting a metrics patch in a week or two...
15:04:51 <nijaba> looks like nothing has been agreed on yet
15:05:02 <timjr> but I'm not handling the billing use case at present
15:05:14 <eglynn> TL;DR: lots of options, but definite push-back on the nova-compute RPC idea
15:05:21 <nijaba> how do we get this to converge?
15:05:30 <eglynn> (i.e. this is considered a private API)
15:05:46 <eglynn> nijaba: a couple of the live options need buy-in fromt he nova folks
15:06:04 <eglynn> if I don't get a response on the ML today, I'll bring it up at their weekly meeting this evening
15:06:06 <timjr> this is the library I'll be using: https://github.com/timjr/tomograph
15:06:07 <jd__> and so far the only option that nova is ready to adopt is #1
15:06:15 <nijaba> would us putting this as a topic in a nova irc meeting help?
15:06:24 <timjr> I'd love to talk with you guys about additional use cases (like billing) or improvements to it
15:06:26 <nijaba> we need to converge sooner than later on this
15:06:29 <eglynn> nijaba: I can propose this to vishy
15:06:52 <eglynn> agreed, lets push for a quick agreemnt on the way forward
15:06:55 <nijaba> sounds good.  any other suggestions?
15:06:56 <jd__> timjr: sure, could you bring this back at the end of the meeting during open discussion?
15:07:16 <timjr> jd__:  sure thing
15:07:22 <jd__> timjr: thanks :)
15:07:50 <jd__> nijaba: I think eglynn summarized everything well :)
15:08:15 <nijaba> eglynn: do you want to action yourself for the nova irc meeting?
15:08:43 <eglynn> #action eglynn propose agent item on ceilo interaction for nova IRC meeting
15:08:57 <nijaba> thanks eglynn
15:09:00 <nijaba> #topic nijaba to send an invite to fill the survey
15:09:08 <nijaba> I had to fight a bit with survey monkey, but the survey is now finaly out.
15:09:22 <nijaba> 15 responses so far.  we'll let it run until next meeting?
15:09:40 <dhellmann> that sounds like a good idea
15:09:42 <dhellmann> should be plenty of time
15:09:51 <jd__> +1
15:10:08 <nijaba> sandywalsh_: commented that the "features" were not clear enough for people outside the project
15:10:12 <eglynn> yep, that's reasonable
15:10:13 <nijaba> I think he is right
15:10:29 <nijaba> so, I'll make a note for next cylce
15:10:47 <sandywalsh_> thanks
15:11:02 <nijaba> #action nijaba to close survey and publish result prior to next meeting
15:11:15 <nijaba> #link http://www.surveymonkey.com/s/JNQ2PCJ
15:11:28 <nijaba> could also be useful :)
15:11:33 <nijaba> #topic asalkeld investigate diamond for use to generate stats
15:11:41 <nijaba> I am afraid asalkeld is asleep
15:11:53 <eglynn> yep, its his off-week
15:12:05 <nijaba> anyone has comments on this topic?  or should we reaction for next week?
15:12:20 <jd__> i'd say re-action
15:12:24 <dhellmann> +1
15:12:28 <nijaba> #action asalkeld investigate diamond for use to generate stats
15:12:42 <nijaba> That's all for last week's action, moving on...
15:12:51 <nijaba> #topic Discuss synaps integration
15:13:08 <nijaba> eglynn: I think you had some interations with synaps folks, right?
15:13:08 <eglynn> the Synaps folks are continuing their internal discussion, so no definite conclusion as yet
15:13:30 <eglynn> but they are actively considering the idea
15:13:38 <nijaba> it seems that they agree on using ceilometer, but not yet to fullyn join the project?
15:14:03 <eglynn> I would say it'll take another day or two to play out, but yeah edging towards a looser link-up
15:14:12 <nijaba> anyone from synaps online?
15:14:16 <sandywalsh_> eglynn: which specific part of using rpc is getting push back? The only thing I hear is using it for instrumentation.
15:14:25 <dhellmann> eglynn: how much duplication of effort do you think that will produce?
15:14:36 <eglynn> nijaba: bad time in their TZ
15:14:43 <nijaba> eglynn: right...
15:15:07 <eglynn> sandwalsh_ the idea of RPC being externally called (i.e. nova RPC message emitted by ceilo)
15:15:19 <dhellmann> eglynn, nijaba: maybe we could arrange a separate meeting at a better time for them?
15:15:31 <eglynn> dhellmann: yes, that's a good idea
15:15:32 <nijaba> dhellmann: we should at least try
15:15:41 <sandywalsh_> eglynn: oh, yeah, that would be unnecessary
15:15:47 <dhellmann> at least to discuss
15:15:55 <nijaba> eglynn: do you want to take the action, or do you want me to?
15:16:00 <eglynn> #action eglynn propose IRC meetup with Synaps folks to discuss collaboration model
15:16:09 <nijaba> cool
15:16:26 <nijaba> let's move on then
15:16:29 <nijaba> #topic Tarball generation
15:16:39 <nijaba> that's a topic from the last project meeting on tue
15:17:02 <eglynn> sandywalsh_ it one of the options discussed here for ceilo to retrieve nova diagnostics http://lists.openstack.org/pipermail/openstack-dev/2012-November/002791.html
15:17:07 <jd__> was that a problem?
15:17:12 <nijaba> eglynn gracfully replaced me as I was fighti with 3g and could not join (thanks!!)
15:18:02 <timjr> the nova rpc mechanism is not a robust interface... if things other than nova are going to call it, I think it needs a lot of work
15:18:03 <sandywalsh_> eglynn: I guess my fear is that ceilo is becoming a kitchen sink and losing focus. Yes, it could do diagnostics, but should it?
15:18:27 * nijaba sense a topic drift...
15:18:37 <dhellmann> yes, please, folks, let's stick with the agenda
15:18:39 <sandywalsh_> eglynn: two summits ago its focus was clear: billing, but now it's all things to all people
15:18:54 <dhellmann> nijaba: what exactly is the issue with the tarballs?
15:19:08 <eglynn> the actions from the project status meeting were ...  ceilometer crew to ask for a ceilometer-tarball job and pimp up their grizzly roadmap
15:19:12 <sandywalsh_> timjr: agreed, nova rpc should only be used by nova
15:19:27 <jd__> c'mon folks, be nice :)
15:19:34 <eglynn> i.e. ask the CI team to set up the ceilometer-tarball job
15:19:34 <nijaba> eglynn: the last part being the next topic
15:19:40 <dhellmann> we have a tarball job: https://jenkins.openstack.org/view/Ceilometer/job/ceilometer-sdist-tarball/
15:19:44 <dhellmann> is that the wrong one?
15:20:00 <jd__> yeah I think this has been resolved no?
15:20:01 <nijaba> not sure. we should ask the infra team
15:20:11 <eglynn> dhellmann: I'm not sure, I guess I should clarify that with ttx and/or infra team
15:20:11 <nijaba> that's from last tue
15:20:38 <nijaba> dhellmann: can you check with ttx/infra team?
15:20:42 <dhellmann> eglynn: ok. it seems to be producing them with the correct version numbers now, too (maybe that was it?)
15:21:04 <eglynn> dhellmann: yeah, could have been if that was only resolved after the Tue status meeting
15:21:07 <dhellmann> nijaba: sure, I can send an email
15:21:15 <jd__> log is there http://eavesdrop.openstack.org/meetings/project/2012/project.2012-11-13-21.01.log.html
15:21:17 <dhellmann> eglynn: yes, the version # change went in after that
15:21:20 <jd__> I think we got this covered already
15:21:20 <nijaba> I guess a simple "hey, I think I have done everything right, can you check" would be nice
15:21:23 <jd__> but we can check
15:21:36 <dhellmann> #action dhellmann to confirm with ttx that tarball job is correct
15:21:41 <nijaba> thanks!
15:21:43 <jd__> thanks dhellmann
15:22:00 <nijaba> #topic Transforming roadmap items into blueprints
15:22:08 <nijaba> that's another request from the project meeting
15:22:19 <ttx> dhellmann: if it produces "ceilometer-2013.1.tar.gz" tarballs it's not using the right versioning algorithm.
15:22:25 <nijaba> so I'll happily make sure we have a bp for each feature on the roadmap
15:22:30 <eglynn> yep, so ttx primarily drives that meeting from the BP lists
15:22:32 <jd__> makes sense, we just need volunteers I guess
15:22:39 <nijaba> and try to assign implementor when I know that
15:22:42 <jd__> nijaba: thanks!
15:23:08 <ttx> shoud be now producing "ceilometer-2013.1~g2~20121115.234234.tar.gz" or something
15:23:12 <eglynn> cool, then the BPs can be further fleshed out with details by the assigneee
15:23:16 <dhellmann> ttx: ok, I'll ping you after the meeting to see if I can get better details about how to fix that
15:23:27 <nijaba> #action nijaba to transform bp-less features into bp
15:23:36 <ttx> dhellmann: sure. Might not even be on your side
15:23:58 <nijaba> so I'll mark everyone that is high or medium as "approved"
15:24:07 <yjiang5> nijaba: where can we get the roadmap?
15:24:38 <nijaba> please make sure that when you'll submit an implementation to mark "address bp #XXX" so that status is tracked
15:24:38 <eglynn> #link http://wiki.openstack.org/EfficientMetering/RoadMap
15:24:46 <nijaba> thanks eglynn
15:24:52 <yjiang5> thanks.
15:25:20 <nijaba> ttx: how do you use bp from status tracking?
15:25:36 <nijaba> do you just look at the global bp status?
15:25:46 <nijaba> or do you do something fancier?
15:26:07 <ttx> not really fancier. Mostly tracking implementation status
15:26:17 <ttx> oh you mean do I use workitems ? no
15:26:27 <nijaba> yeah, I was wondering :)
15:26:43 <nijaba> ttx: thanks for clarifying
15:26:54 <ttx> people can't get the status right, so i didn't even try workitems
15:27:01 <nijaba> #agreed ceilometer team to use bp to track status of features
15:27:44 <jd__> like we had choice :)
15:27:53 <nijaba> ok, I think we ran out of agenda topic
15:27:55 <nijaba> jd__: lol
15:28:11 <anniec> :) open discussion time?
15:28:15 <nijaba> #topic Open discussion
15:28:15 <timjr> sorry for jumping in on you guys... I claim early morning brain
15:28:21 <jd__> timjr: ;)
15:28:23 <sandywalsh_> question: should I expect some feedback on my Unified Instrumentation and Metering proposal? http://wiki.openstack.org/UnifiedInstrumentationMetering
15:28:42 <timjr> sandywalsh_: I definitely thought the pictures were cool
15:28:53 <sandywalsh_> :/
15:28:57 <jd__> lol
15:29:06 <jd__> better than nothing I guess
15:29:06 <eglynn> sandywalsh_ I was using the "diagnostics" term loosely earlier
15:29:28 <eglynn> (just to mean the kind of info that ceilo currently grabs from polling libvirt)
15:29:35 <shengjie> hey guys, first time attending the meeting here, I want to get some feedback on my blueprint : https://blueprints.launchpad.net/ceilometer/+spec/hbase-storage-backend
15:29:43 <timjr> so, I think we can meet most of yahoo's metrics needs with a 20-30 line patch to nova and smaller patches to the other components
15:29:49 <timjr> I have a prototype working
15:30:13 <timjr> so I think our stance is changing to one of, you guys can have our patch if you want it, but it's not gonna be that hard to maintain it in house either
15:30:19 <nijaba> shengjie: I think it should depend on the multi-publisher blueprint
15:30:21 <eglynn> timjr: wanna submit those 20.-30 lines as a draft patch?
15:30:22 <anniec> asalkeld was asking us on progress on metering
15:30:30 <anniec> so repost tim's link earlier here: https://github.com/timjr/tomograph
15:30:34 <timjr> that tomograph library, https://git.corp.yahoo.com/timjr/tomograph, is a custom-made lib that makes the patch small
15:30:37 <timjr> eglynn: definitely
15:30:38 <anniec> sorry .. on monitoring i mean
15:30:49 <timjr> I wasn't sure... can people see the draft reviews?
15:30:52 <anniec> asalkeld was asking us on progress on monitoring
15:31:06 <timjr> a friend was saying I might be better off publishing a regular review but -1'ing it
15:31:20 <shengjie> nijaba: how come?
15:31:22 <eglynn> timjr: that would work too, and reach a wider audience
15:31:52 <anniec> if you guys can take look at the link of tomograph i just post, tim has a real nice screenshot on the trace graph and monitoring graph, it's kinda cool with small amount of patch to achieve our needs
15:32:00 <nijaba> shengjie: I assume you want to use Hbase in parallel to the current storage, am I wrong?
15:32:05 * eglynn looking
15:32:11 <sandywalsh_> so, not sure what the answer here is ... we're going to maintain multiple notification consumers?
15:32:28 <dhellmann> shengjie: a Hbase backend sounds like a nice addition
15:32:59 <nijaba> dhellmann: as a replaqcement storage engine, or as parallel storage?
15:33:24 <dhellmann> nijaba: as an option
15:33:43 <eglynn> instead of mongodb, not as well as, presumably ...
15:34:00 <shengjie> dhellmann: I guess given the nature of hbase, it might go parallel, i dunno yet.
15:34:03 <dhellmann> the idea with the storage plugin api was to let deployers choose a backend
15:34:03 <nijaba> dhellmann: as an option at the same leve as sqlalchemy or mongo?  ok
15:34:27 <nijaba> then no need to depend and shengjie should go ahead with his impelmentation then?
15:34:48 <shengjie> becoz store users/projects in hbase seems like an overkill, storing meter data is definitely a good use case
15:34:54 <shengjie> yes,
15:35:00 <sandywalsh_> hello? was my previous topic skipped?
15:35:03 <shengjie> i can go ahead and start the implementation
15:35:26 <jd__> nijaba: yeah hbase as a storage engine, nothing more :)
15:35:28 <dhellmann> shengjie: a driver that talked to hbase could also talk to another backend for storing the user and project data
15:35:32 <timjr> the approach taking by twitter folks, and presumably facebook too, is to have everything send stuff to scribe and then let scribe deal with forwarding it to things like hbase... it's worth considering
15:35:36 <nijaba> so. should I make it as direction "aproved"?
15:35:41 <jd__> shengjie: did yousee  my reply on the list?
15:35:42 <anniec> sandywalsh_ seems like no one is looking at timjr and mine link  either :P
15:35:46 <dhellmann> sandywalsh_: we had a lot of cross-talk, let us catch up
15:35:49 <shengjie> dhellmann: yes, that's more like it
15:36:02 <sandywalsh_> anniec: seems to be a lot of that
15:36:03 <eglynn> sandywalsh_ "we're going to maintain multiple notification consumers?" << not sure about the context on this
15:36:08 <jd__> nijaba: ah yes, I only changed "definition" to approved
15:36:12 <dhellmann> anniec & timjr : I'll look at the code, but can't read and follow the meeting at the same time
15:36:12 <sandywalsh_> eglynn: my proposal
15:36:28 <nijaba> jd__, shengjie: approved!
15:36:33 <jd__> great :)
15:36:52 <nijaba> so, I propose we focus on sandywalsh_'s proposal now
15:36:55 <eglynn> sandywalsh_ sorry yeah, I see the topic now
15:37:00 <sandywalsh_> anniec: I'd love to see the patch tim has put together though
15:37:17 <shengjie> thanks, guys, i think we are done with Hbase backend now
15:37:29 <timjr> yeah, I gotta clean it up a little first, but will post soon enough
15:37:37 <timjr> I think people will really like the zipkin style tracing
15:37:49 <timjr> I had no idea that keystone's db backend was so screwy, for example
15:37:51 <anniec> i like it.  you got 1
15:37:58 <sandywalsh_> timjr: I've been talking with lucy about how the two might work ... need to learn more about it
15:38:21 <timjr> lucy?
15:38:34 <sandywalsh_> timjr: from cloudkick
15:38:55 * sandywalsh_ searches for irc handle
15:39:23 <sandywalsh_> timjr: my first blush concern was using REST for logging messages ... Loggly was unable to handle that volume
15:39:46 <timjr> if you have debug logging on, it is indeed quite some volume
15:40:15 <sandywalsh_> timjr: Lucy Mendel ... she was dealing with someone from Yahoo, thought it was you
15:40:16 <timjr> I thought an elegant way to hook things like metrics, or ceilometer, up to nova was to just add a logging handler to the existing logging, and then maybe some more log messages
15:40:26 <timjr> that way people that don't use it won't have to care in the least
15:40:31 <yjiang5> sandywalsh_: if we have debugging logging, I'd suggest to seperate with other communication channel.
15:40:32 <timjr> but it is kind of a hack
15:41:05 <sandywalsh_> Consider my proposal in the ML thread this morning about notifiers having different handlers for different event types
15:41:11 <sandywalsh_> that would work and no major changes
15:42:12 <timjr> notifications, as they presently exist, are a little too high level for our use case
15:42:18 <timjr> we want to know about rpc latencies and such
15:42:29 <sandywalsh_> but, again, we have the problem of instrumentation vs usage/stage ... which is what I need to understand from the ceilo team what the mandate is
15:42:50 <timjr> but the main thing I'm here for, actually, is to see if people are still interesting in unifying metering with monitoring at all
15:42:51 <sandywalsh_> timjr: I'm currently working on rpc latencies in StackTach ... I have all the data in there now
15:43:07 <sandywalsh_> timjr: likewise
15:43:21 <timjr> from the nova point of view, I can imagine people want as few hooks as they can get away with
15:43:35 <sandywalsh_> timjr: my first approach was the Inflight Service, but now I know I can do it all with StackTach
15:43:46 <sandywalsh_> timjr: +1
15:44:15 <dhellmann> sandywalsh_: I thought at the summit we agreed to work our way up from the bottom, so we're starting with sharing code for instrumentation, metering, monitoring, and publishing
15:44:19 <yjiang5> sandywalsh_: where can I get your rpc latency data?
15:44:43 <sandywalsh_> yjiang5: I'll be putting a branch to stacktach up early next week
15:44:50 <yjiang5> sandywalsh_: thanks.
15:45:10 <sandywalsh_> dhellmann: yes, which is why I wrote that proposal ...
15:45:26 <dhellmann> sandywalsh_: yes, so what part of the mandate do you need clarified? :-)
15:45:36 <sandywalsh_> dhellmann: to highlight the differing needs of instrumentation and monitoring/meterings/usage/etc
15:46:13 <dhellmann> we never finished the work on use cases that we started the last day of the summit, maybe we should finish documenting those?
15:46:27 <sandywalsh_> the proposal highlights that both requirements need different implementations and we shouldn't try to do both with one solution
15:46:56 <sandywalsh_> this is an echo of the current thread on the ML
15:47:12 <timjr> I definitely think it would be cool to unify the metering and the monitoring/metrics stuff, but I don't think I know the metering use case well enough to do it myself
15:47:23 <timjr> I will leave it to you guys to tell me why my interface won't work :)
15:47:33 <sandywalsh_> :) that's fair
15:47:55 <sandywalsh_> timjr: if you want to chat offline after this meeting about my rpc latency idea, I'm free
15:48:00 <timjr> sure
15:48:12 <timjr> sandywalsh_: there's actually a dude working on zipkin tracing at rackspace
15:48:15 <timjr> IRC nick dreid
15:48:34 <sandywalsh_> timjr: must be with lucy's group
15:48:57 <anniec> will be nice to find out who lucy's talking to at Yahoo
15:49:04 <sandywalsh_> so, ceilo group ... look forward to some feedback on unifying the two requirements :)
15:49:17 <sandywalsh_> anniec: I'll check my email after
15:50:20 <nijaba> ok. looks like we have some actions and running out of exchanges?
15:50:38 <timjr> yep, out of exchanges :)  thanks for letting me crash the meeting
15:50:58 <anniec> will be nice for this team to review sandywalsh_ and timjr 's proposal as action
15:51:03 <eglynn> sandywalsh_ I'll try to respond to your mail on the list shortly
15:51:06 <anniec> and give us some feedback
15:51:12 <nijaba> timjr: open discussion is just meant for it!  feel free to add topics to the meeting agenda too next time
15:51:29 <timjr> nijaba: thanks
15:51:30 <sandywalsh_> anniec: Lucy is in dreid's group and dreid was working with timjr :) haha
15:51:43 <sandywalsh_> eglynn: thanks
15:51:44 <anniec> got it .. thanks!
15:51:48 <timjr> by working with I guess he means we were on the zipkin IRC channel at the same time :)
15:51:50 <dhellmann> yes, please, give us some heads up when there's something to be reviewed before a meeting by putting it on the agenda!
15:52:02 <sandywalsh_> timjr: quite likely
15:52:19 <timjr> dhellmann: I posted it in a rush last night :)
15:52:50 <dhellmann> timjr: np, then, just something to keep in mind :-)
15:52:52 <timjr> dhellmann: please feel free to critique the code or whatever, I'd love to hear your opinion
15:53:02 <timjr> I am still a bit of a python newb
15:53:09 <nijaba> anything else (starting the 30 sec coundown)?
15:53:28 <nijaba> countdown too...
15:53:51 <nijaba> ok. looks like we are done for today!
15:53:57 <nijaba> thanks everyone!
15:54:01 <nijaba> #endmeeting