15:00:07 #startmeeting Ceilometer 15:00:07 #meetingtopic Ceilometer 15:00:07 #chair nijaba 15:00:07 #link http://wiki.openstack.org/Meetings/MeteringAgenda 15:00:07 ATTENTION: please keep discussion focused on topic until we reach the open discussion topic 15:00:08 Meeting started Thu Jan 24 15:00:07 2013 UTC. The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:09 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:11 The meeting name has been set to 'ceilometer' 15:00:13 Current chairs: nijaba 15:00:19 Hello everyone! Show of hands, who is around for the ceilometer meeting? 15:00:19 o/ 15:00:22 o/ 15:00:25 o/ 15:00:30 o/ 15:00:30 o/ 15:00:33 o/ 15:00:35 o/ 15:00:38 o/ 15:00:46 o/ 15:01:00 good to see everyone and new faces ! 15:01:04 * jd__ sees new hands 15:01:08 #topic actions from previous meeting 15:01:13 o/ 15:01:18 15:01:25 #topic nijaba to specify draft policy on wiki for units 15:01:47 15:01:50 * dragondm waves hi 15:01:59 * nijaba has been a bad boy and has not had time to work on this 15:02:07 #action nijaba to specify draft policy on wiki for units 15:02:26 That's it for last week 15:02:45 #topic relevance of https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-agent-object-storage 15:02:52 jd__: floor is yours 15:02:58 there was an action "llu to get in touch with the healthmon team to see what their reaction is to our plan for integration, putting the ml in cc". Divakar was trying to setup a meeting to go over the plan - any luck on this? 15:03:32 I have updated the ceilometer wiki with the healthnmon plans http://wiki.openstack.org/Ceilometer/CeilometerAndHealthnmon 15:03:49 * jd__ let the floor to YehiaBeyh and divakar 15:04:54 i'm looking at the wiki 15:04:54 As per my proposal 15:04:58 Healthnmon as the source of metering data (for compute to start with) 15:05:30 Healthnmon currently has implemented drivers for KVM to collect the required meters data for both Compute and Instances running on the compute remotely. The same data can be leveraged by Ceilometer thru Healthnmon APIs as initial step 15:05:57 Ceilometer Centralized Agent mechanism can be leveraged to pull the required metering data from Healthnmon 15:06:34 Healthnmon is working on the required drivers for vCenter ESX and Hyper-V as well 15:07:03 so this plan is the opposite of the plan we've discussed previously? 15:07:10 Collecting the data from healthnmon would help in getting the data for all the hypervisors 15:07:18 that are supported thru openstack 15:07:50 we need drivers for all of the hypervisors for metering, so we need to implement those in ceilometer at some point anyway 15:07:53 dhellmann: this is what it sounds like 15:07:58 Are you are talking about some virch url://of-some-remote-hypversir? 15:08:12 (belated o/) 15:08:14 I don't think it buys anything to use healthnmon as another abstraction layer for that 15:08:29 jd__: agreed 15:08:30 jd__: +1 15:09:02 I've been looking at implementing for xenserver... 15:09:05 I admit it can be a good source of information, but we want to support the hypervisors Nova does directly 15:09:11 how is the plan different from what was discussed previously? 15:09:17 divakar: can you discuss a bit your objections with the plan described in 6.1 and 6.2 of the wiki page? 15:09:47 YehiaBeyh: in the previous plan the data would flow from ceilometer to healthnmon, in the new plan that is reversed 15:10:12 can we not do this through the existing hypervisor polling mechanism in Nova and not require the agent (or have to depend on healthmon) for it? 15:10:13 that really only works for deployments where the admins want both healthnmon monitoring and ceilometer metering 15:10:30 sandywalsh: the plan is to get a version of that code in ceilometer and take it out of nova 15:10:42 sandywalsh: that's probably at least an H change for nova 15:10:47 hmm 15:10:54 sandywalsh: there isn't such polling for now AFAIK 15:10:56 As part of healthnmon we are already collecting the inventory data + alerting + utilization data (meters) 15:11:03 jd__: yup, we use it 15:11:14 sandywalsh: you retrieve CPU time and disk IO from nova ? 15:11:28 we already poll in nova for bw data 15:11:36 jd__: whatever the hypervisor offers 15:11:49 jd__: essentially, whatever the agent is doing, we can do in the polling 15:11:54 the low level code for disk/cpu is there 15:12:06 So healthnmon has the required data for making the decisions while scheduling, monitoring, metering and also for analytics 15:12:09 we just need to emit it. 15:12:10 dragondm: I believe those bw stats only reported by the xenapi driver though 15:12:13 hey, folks, let's please focus on divakar's proposal for now 15:12:21 we can discuss APIs in a bit 15:12:34 ok 15:12:34 dhellmann: it's related is it not? 15:12:49 sandywalsh: one thing at a time 15:12:50 Let me summarize 15:13:01 healthnmon has the required data for making the decisions while scheduling, monitoring, metering and also for analytics 15:13:38 We are looking at providing a single point of data around monitoring, scheduling and support for use cases around analytics, autonomics and planning 15:14:08 divakar: that "single point" is overlapping with a couple of other projects at this point :-) 15:14:22 metering also can be driven from this data and hence the proposal 15:15:14 At this point I see that Heat is looking for scheduling data + alerting data which can be integrated as well 15:15:21 yes, it seems that we are hitting again the same issue of goals for each projects which have a big overlap 15:16:03 lot of good work is being put into multi publisher in ceilo 15:16:19 I would propose that we rationalize around this 15:16:24 dare one suggest the next design summit, seems like we're impinging too many different areas 15:16:39 n0ano: I was driving to this 15:16:55 specially since we are in the last milestone of the project 15:17:06 and need to finalize existing bp 15:17:15 divakar: fwiw, we've stopped stacktach development in favor of CM development for this very reason ... project overlap. 15:17:29 this is also the reason for me being here as well 15:17:31 project overlap 15:17:55 Monitoring is not a overlap at this point 15:18:04 I think we should start preparing the meetings at the summit to deep dive on this 15:18:05 divakar: with stacktach it is 15:18:22 nijaba: +1000 15:18:24 but please, let's move back to the advertised topics 15:18:29 +1 15:18:32 jd__: please go ahead 15:19:00 what is the major overlap here? 15:19:13 is it the data collection? 15:19:21 #topic relevance of https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-agent-object-storage 15:19:21 For me, yes 15:19:32 so this blueprint is about creatin another new agent 15:19:34 collection, storage, api, aggregation, propagation 15:19:49 technically this is isn't required since it can be done with the central agent 15:19:53 -is 15:20:17 I'm not sure it's a really good idea to split the central agent again and create more namespace 15:20:35 jd__: what's the motivation for doing this specific polling in a separate agent? 15:20:46 why were you thinking about spliting it in the first place? to be able to disable it? 15:21:04 dhellmann: in case you just run swift? 15:21:12 * sandywalsh votes to kill the agent altogether ... nova is the touch point to the hypervisor. 15:21:22 nijaba: yes, I think it was about being able to enable/disable things, but we can do this now 15:21:24 +1 15:21:37 +1 15:21:41 jd__: agreed, this is now par of multi publisher, IIUC 15:21:44 jd__: the existing config option to disable pollsters isn't very rich 15:21:56 dhellmann: but it's enough to do that IIUC 15:22:02 nijaba: now it is even more yes 15:22:19 jd__: yeah, I hit send too soon -- the existing option should work, but the new publisher config will make it even easier 15:22:27 sandywalsh: this is for swift. 15:22:41 ok, so sounds like we all agreed to say that this blueprint is obsolete 15:22:42 ? 15:22:48 +1 15:22:50 +1 15:22:56 jd__: +1 15:22:58 I would argue against multiple agents - they all serve a similar domain purpose and a much richer configuration may make deployment simpler (same package, different cfg) 15:23:06 this is what I though :-) I'll change its status, thanks guy 15:23:09 +1 15:23:10 +1 15:23:16 +1 15:23:19 guyS 15:23:20 yep, +1 to non-agent-proliferation 15:23:34 +1 15:23:35 we can move to the next topic I guess 15:23:44 +1 (no agents) 15:23:44 #topic blueprint review 15:23:57 #link - https://launchpad.net/ceilometer/+milestone/grizzly-3 15:24:16 we have quite a few not started ones 15:24:47 can you guys tell me if you think we should postpone them or keep them? 15:24:51 I'm working on Publisher counters frequency receival now, since the multiple publisher patch is under review 15:24:52 all my synaps wqork is on hold again this (snowed under with downstream work) 15:25:01 nijaba: I am going to try to start https://blueprints.launchpad.net/ceilometer/+spec/move-listener-framework-oslo next week 15:25:02 * jd__ updated https://blueprints.launchpad.net/ceilometer/+spec/pollster-global-keystone-auth 15:25:08 yjiang5_home: should I mark it as started? 15:25:20 yjiang5_home: great 15:25:32 nijaba: I will send patch out possibly early next week. 15:25:49 yjiang5_home: great ,thanks 15:26:01 nijaba: https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-agent-object-storage still appears in g3, maybe you should remove it? I don't know why it shows 15:26:40 I think https://blueprints.launchpad.net/ceilometer/+spec/api-aggregate-average is implemented, dhellmann you confirm? 15:26:56 jd__: fixed 15:27:02 jd__: yes, that's part of the "statistics" endpoint in the v2 api 15:27:17 marking complete 15:27:43 perfect 15:28:02 multi-publisher seems in good shape to me 15:28:07 eglynn: what about qpid? 15:28:17 (I think I'm the only one who did a review since a moment now :) 15:28:30 jd__: thanks for your review. 15:28:49 nijaba: lets keep that, I'll find time for it next week 15:28:56 yjiang5_home: I saw you updated it, I'll try to take a look again :) 15:29:10 jd__: thanks. 15:29:16 eglynn: great, thanks 15:29:51 I'm back online today, so I should have time to look over the multipublisher stuff this afternoon 15:30:19 ok, so now we have a TON of bp marked for grizzly with no milstone: https://blueprints.launchpad.net/ceilometer/grizzly 15:30:38 I think synaps is going to slide until H 15:30:45 * jd__ proposes to tie dhellmann to review.openstack.org 15:30:52 eglynn: what's your pov? 15:30:55 nijaba: agree that a strong possibility 15:31:12 eglynn: should I remove the target for now? 15:31:15 (I"m been snowed under with downstream work the last couple weeks) 15:31:18 we can always chnage back 15:31:24 fair enough 15:31:39 perfect, I'll do 15:31:51 #action nijaba to untarget synaps bp for now 15:32:15 any other that should be untargetd from grizzly? 15:32:29 or, which one should we target to g3? 15:32:41 and i'll untarget the others 15:32:43 * dhellmann looks for the bug about sqlalchemy metadata queries 15:32:57 I think test-db-backends should be done now. 15:33:02 we should add https://bugs.launchpad.net/ceilometer/+bug/1098603 15:33:03 Launchpad bug 1098603 in ceilometer "need to handle missing units values in existing mongodb data" [Medium,New] 15:33:12 and https://bugs.launchpad.net/ceilometer/+bug/1093625 15:33:15 Launchpad bug 1093625 in ceilometer "no metaquery implementation in sqlalchemy DB backends" [Medium,Confirmed] 15:33:51 dhellmann: +1. would you care to do that? 15:33:52 do we want to follow up on the healthnmon/integration proposal in email or do we need a special meeting? 15:33:53 that 2nd maybe big 15:34:14 ah I created a BP for 1093625 this morning :( 15:34:23 YehiaBeyh: meeting, eventually voice? 15:34:33 yes 15:34:44 nijaba: done 15:34:46 nijaba: I think https://blueprints.launchpad.net/ceilometer/+spec/counter-disabling is already in multiple publisher code. 15:34:46 can we be in on that? 15:34:59 #action nijaba to propose a doodle for helthmon meeting 15:35:08 nijaba: https://blueprints.launchpad.net/ceilometer/+spec/sqlalchemy-metadata-query FWIW 15:35:09 in addition, if we do need to deep dive into this we can propose something for the summit 15:35:27 yjiang5_home: agreed 15:35:34 yjiang5_home: yes it is, this is why it's assigned to me, so I can click on Implemented without writing code :-D 15:35:35 #action nijaba to propose a doodle for helthmon meeting - what does that mean? 15:35:36 nijaba: can we participate in the voice meeting? 15:35:54 YehiaBeyh: doodle to pick a date/time that suits most of ut 15:35:59 s/ut/us 15:36:03 jd__: :) 15:36:04 voice, as in G+ hangout or the linkes? 15:36:09 that sounds great 15:36:19 s/linkes/likes/ 15:36:27 sandywalsh: yes, weĺl share a bridge info once a date/time is picked 15:36:33 nijaba: thanks 15:36:59 I can setup the meeting.. please do let me know what all wants to be part of that conference call 15:37:01 np 15:37:50 can the list of participant be added to the notes so Divakar can set up the meeting. thx 15:38:26 i'd like in on that meeting as well. 15:38:36 divakar: I'll share with you the list of respondant to the doodle 15:38:45 please include me as well if possible 15:38:55 nijaba: that will be great 15:38:56 Thank you nijaba 15:39:08 just watch the ml for a doodle invite, respond to it, and you will be invited 15:39:09 nijaba: thank you 15:39:38 ok, back to the bps, anything else we should clean up? 15:40:06 nijaba: i think test-db-backends should be marked done. 15:40:21 llu-laptop, nijaba: +1 15:40:34 llu-laptop: yup 15:40:35 good job has been done on this 15:40:43 hats off 15:42:12 ok, let's move on 15:42:17 #topic open discussion 15:42:21 FOSDEM planning 15:42:34 eglynn: very good point! 15:42:47 IIRC from that mail thread, the plan was ... 15:42:54 jd__: eglynn and I should have a quick voice call to prep 15:43:01 nijaba 5mins intro, eglynn 10mins architecture, jd_ 10mins contributing to ceilo 15:43:06 yup 15:43:12 are we still good with that split? 15:43:16 sure 15:43:20 +1 for me 15:43:21 cool 15:43:33 should we aim to have drafts to for review for say Monday? 15:43:40 (in advance of a call?) 15:43:44 this is for a presentation? 15:43:45 or do the call first? 15:43:50 dhellmann: yep 15:43:53 nice 15:44:04 eglynn: prep first our slides :) 15:44:15 I saw nova and other openstack projects replaces the nosetest with testr for parellel unittest, shall we follow that fashion? 15:44:17 then meeting on tue 15:44:17 dhellmann: https://fosdem.org/2013/schedule/event/openstack_ceilometer/ 15:44:21 or wed 15:44:22 nijaba: cool 15:44:42 llu-laptop: probably, but I've no clue what we should do, I think -infra took care of that so far but I may be wrong 15:44:44 eglynn: nice++ 15:45:00 jhopper: jd__: the idea of against multiple agent is quite interesting, I noticed not big differenece in bin/ceilometer-agent-compute and "bin/ceilometer-agent-central, possibly we can work to merge them? https://review.openstack.org/#/c/20123/ can be an preparation for it. 15:45:09 llu-laptop: do you have any idea why they didn't just use py.test? 15:45:18 I absolutely agree 15:45:24 +1 on that 15:45:28 There are numerous de-duplications that we could take advantage of 15:45:34 yjiang5_home: yeah https://review.openstack.org/#/c/20123/ is about this, exactly, there's minimum difference after 15:45:37 not to mention provide a more contigious namespace for the agents 15:45:37 +1 on that 15:45:39 I think the testr supports parrallel testing 15:45:43 or rather their flavors 15:45:46 llu-laptop: so does py.test 15:46:37 dhellmann: then I don't know why. It just seems that nova/olso/glance/etc. all turns to testr 15:46:38 yjiang5_home: it should be possible to create one agent that takes a list of plugin namespaces to use to load pollsters 15:46:43 llu-laptop: ok 15:46:59 why. not. kill. the. agent? 15:47:14 sandywalsh: how many times do you want the same answer? 15:47:21 if it's ok, i'd like to help use testr in ceilometer 15:47:23 I haven't heard an answer 15:47:42 sandywalsh: we have 2 agents for different purposes. not everything is about nova. 15:47:56 dhellmann: yes, that's what we want. 15:47:57 sandywalsh: we will also eventually be moving the stats collection out of nova 15:47:59 i tend to agree with sandywalsh. why can't we rely on openstack notifications. 15:48:12 dhellmann: then change the underlying project (glance, etc) to use notifications properly 15:48:16 for all projects. not just nova. 15:48:38 danspraggins: have you worked with swift before? 15:48:49 nijaba: i have not. 15:49:15 danspraggins: we're moving in that direction, too, but in the mean time... 15:49:19 danspraggins: unlikely they will accept such a modification 15:49:35 nijaba: if there is a hard road block on a particular system, then sure, use an agent. But in 90% of the situations, notifications should and can be used. 15:49:36 dhellmann: fair enough. 15:49:39 danspraggins: but yes, this is the edn goal 15:49:51 nijaba: good deal. 15:49:53 Cool 15:50:00 but in the meean time. we need to be able to capture what we need NOW 15:50:11 nijaba: makes sense. 15:51:30 jhopper: I will create a bp for single agent. 15:51:49 I would be more than happy to cut my teeth on it unless the effort is already spoken for 15:52:05 dhellmann, jd, eglynn: did you have any proposals for new core devs? 15:52:23 I am already looking into what it would take to get nova to emit the needed stats. 15:52:41 dragondm: thanks! 15:52:52 dragondm: great 15:52:53 hi 15:52:55 anyone use uses oslo, should be willing to use notifications 15:53:03 s/use/who/ 15:53:11 sandywalsh: yup 15:53:19 anything else for open topic? 15:53:28 (ill see abt writing up some bp's for nova & ceilio to that end) 15:53:44 dragondm: There is some discussion about get data from nova in mailing list and some challenge. 15:53:45 how about adopting the HACKING guide? 15:53:47 jd__: we are on core dev proposals 15:54:01 yjiang5_home: rather the work required to implement the bp 15:54:06 dragondm: s/is/was/ 15:54:56 nijaba: sorry missed your question :) likely yjiang5_home 15:54:57 nijaba: re new core devs, I'd propose yjiang5_home if he's willing 15:54:59 yjiang5_home: yup, been following. 15:55:26 also I think llu-laptop has been doing great work 15:55:33 (if he's willing also ...) 15:55:34 in which case, with yjiang5_home approval, I'll start a formal mail thread for approval 15:55:41 +1 to both yjiang5_home and llu-laptop 15:55:44 jd__: eglynn: dragondm: really thanks 15:55:45 +1 eglynn for llu-laptop 15:55:47 ditto for llu-laptop 15:56:13 thanks for your #action nijaba :-) 15:56:16 #action nijaba to formaly start ml thread about core dev startus for yjiang5_home and llu-laptop 15:56:26 thanks, I'd love to 15:56:52 sounds like a done deal, but we have to follow the official process 15:57:17 indeed 15:57:41 ok, looks like we've ran out of off topics :) 15:57:44 how about adopting the HACKING guide? 15:57:54 sandywalsh: ?? 15:58:08 sandywalsh: you mean like coding style etc? 15:58:20 #link https://github.com/openstack/nova/blob/master/HACKING.rst 15:58:23 I think he means nova's code guidelines 15:58:23 https://github.com/openstack/nova/blob/master/HACKING.rst 15:58:46 yup, import order, etc 15:58:57 I think we've been trying to follow that, but it would be good to go through and make sure now that we're incubated 15:58:57 good point for a ml discussion? 15:58:58 makes code reviews easier when coming over from other projects 15:59:06 sandywalsh: do yiou want to start the thread? 15:59:14 sure 15:59:17 thanks 15:59:35 #action sandywalsh to start a thread about adopting https://github.com/openstack/nova/blob/master/HACKING.rst 15:59:45 ok, we are out of time 15:59:52 thanks a lot everyone! 16:00:02 #endmeeting