15:00:05 #startmeeting Ceilometer 15:00:06 #meetingtopic Ceilometer 15:00:06 Meeting started Thu Jan 10 15:00:05 2013 UTC. The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:06 #chair nijaba 15:00:06 #link http://wiki.openstack.org/Meetings/MeteringAgenda 15:00:06 ATTENTION: please keep discussion focused on topic until we reach the open discussion topic 15:00:07 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:09 The meeting name has been set to 'ceilometer' 15:00:11 Current chairs: nijaba 15:00:17 Hello everyone! Show of hands, who is around for the ceilometer meeting? 15:00:17 o/ 15:00:20 o/ 15:00:24 o/ 15:00:26 o/ 15:00:27 o/ 15:00:53 o/ 15:00:54 jd__: around? 15:01:01 nijaba: yes 15:01:09 totally missed the time! 15:01:14 #topic actions from previous meeting 15:01:14 #topic llu to report agreement result to wiki page 15:01:14 #link http://wiki.openstack.org/Ceilometer/CeilometerAndHealthnmon 15:01:29 llu-laptop: anything to report? 15:01:31 i've put the plan in the wiki 15:01:51 nice! thank you 15:02:11 #topic nijaba to get in touch with the healthmon team to see what their reaction is to our plan for integration 15:02:29 I have not done much on this, but maybe now in overlap with llu-laptopÅ› work? 15:02:58 o/ 15:03:01 i havn't got in touch with the Healthnmon guy 15:03:15 ok, so I'll re-action myself on this 15:03:31 #action nijaba to get in touch with the healthmon team to see what their reaction is to our plan for integration 15:03:41 #topic nijaba to find out if EmilienM wishes to commit on https://blueprints.launchpad.net/ceilometer/+spec/api-aggregate-average 15:03:46 Emilien is a bit shy about his python skills. We may need to find someone else... 15:03:57 Self and divakar from Health and Mon have joined the IRC 15:04:15 Barath_: welcome! 15:04:17 nijaba: the v2 api implementation asalkeld started includes this 15:04:34 it does? cool!! 15:04:53 yeah, he went ahead with the new statistics endpoint that calculates all of that stuff at the same time 15:04:57 very slick 15:05:03 should I make this bp as a dep of the v2 api work then? 15:05:47 yes, let's do that 15:05:53 #info already implemented in v2 api 15:06:07 #action nijaba to mak this bp as a dep for v2 api 15:06:20 #topic nijaba to send an email to openstack-dev listing the orphan bps 15:06:27 I just did this 15:06:27 #link http://lists.openstack.org/pipermail/openstack-dev/2013-January/004398.html 15:06:44 #topic nijaba to add bp skeleton to remind ourselves to implement higher level tests for db backends 15:06:55 this was done 15:06:55 #link https://blueprints.launchpad.net/ceilometer/+spec/test-db-backends 15:07:07 That's it for last week action 15:07:21 #topic G2 release status 15:07:29 We have 2 release critical bugs we need to fix urgently as ttx is trying to cut the release today 15:07:29 #link https://bugs.launchpad.net/ceilometer/+bug/1098204 15:07:29 #link https://bugs.launchpad.net/ceilometer/+bug/1098206 15:07:29 We need a plan and keep ttx in the loop 15:07:32 Launchpad bug 1098204 in ceilometer "API ACL authentication is broken" [Critical,Confirmed] 15:07:33 Launchpad bug 1098206 in ceilometer "policy.json is never found" [Critical,Confirmed] 15:07:52 s/release/milestone but yes 15:07:52 jd__: are you working on fixes? 15:07:57 I'm right now 15:08:04 jd__: any eta? 15:08:22 should be ok by the end of the day 15:08:27 you'll need to land them in master then proposed backport for milestone-proposed 15:08:41 ttx: that's what I though 15:08:44 http://wiki.openstack.org/GerritJenkinsGithub#Submit_Changes_in_master_to_milestone-proposed 15:08:52 the first one already have a fix to review 15:09:26 ttx: what's the ultimate deadline? 15:09:57 no deadline, you're in incubation. We can do it tomorrow morning EU time 15:10:11 ok great 15:10:17 So, it is normally not customary to produce release notes for milestone 15:10:32 the fixes will be there before, but I'd like to test them IRL to be sure if there's enough time :) 15:10:42 nijaba: i used to do blogposts to highlight features landed in a milestone 15:10:48 however, since we decided earlier that this would be a folsom comptible milestone, it might still be good to announce it a bit 15:10:52 less "official" 15:11:14 I can do it, if noone objects 15:11:16 a blog post sounds like a good idea 15:11:22 yep 15:11:23 +1 15:11:42 #action nijaba to prepare a blogpost announcing the release 15:11:50 s/releae/milestone 15:12:07 anyhting else regarding the release? 15:12:12 err milestone!!! 15:12:32 I'll get it right someday.... 15:12:44 I guess that's a no 15:12:53 nothing from me 15:12:54 #topic Discussion about meters unit's in /meters 15:12:59 2 critical bugs is enough for me 15:13:02 There is a review in the pipe for providing meter's unit in /meter 15:13:02 #link https://review.openstack.org/18413 15:13:02 it would be nice if we could settle on this... 15:13:19 jd__: I imagine. thnaks for thaking care of those! 15:13:33 dhellmann: eglynn: gpernot: the floor is yours 15:13:55 (others too, but those are the one that started the discussion) 15:14:19 I don't have a strong opinion. I sort of like having a unique unit name, but see the point of being consistent, too. 15:14:34 I don't have much to add, I've +2'ed, for me this is good enough :) 15:14:36 If eglynn hadn't objected, I think I would have approved the code. 15:14:38 my preference would be not to proliferate the unit types for the 'count'-style meters 15:14:58 but not strong enough objection to -1 or -2 the patch 15:15:27 eglynn: I see some value in providing some understanding of what is counted for gauges 15:16:02 nijaba: my thought was that this understanding was already encapsulated in the meter name 15:16:15 eglynn: what about using a coming "gauge:" prefix 15:16:53 nijaba: as in, guage:images or gauge:packets? 15:17:02 eglynn: ie "gauge:instances" 15:17:04 why that? 15:17:07 eglynn: yes 15:17:20 nijaba: I'm not sure that would really improve matters much 15:17:24 jd__: to simplofy machine processing of units, normalizing is useful 15:17:28 unit is totally unrelated to the type of measure 15:17:59 IIRC, all of the units in dispute were for the things we were counting, right? 15:18:08 yep 15:18:10 nijaba: there being no scaling factor for these gauges, the unit is always going to be a bit redundant 15:18:13 counting the fact the the weather is +2 Celcius degree (delta) or 24 C (gauge) doesn't change the unit :) 15:18:13 dhellmann: yep 15:18:36 jd__: point taken 15:19:16 so, what should we do? 15:19:21 the point of using 'instance' and 'container' rather than None is that you can know that you can't compare or add these two things because it has no sense (apples and bananas) 15:19:42 jd__: can we compare/sum over different meters anyway? 15:19:45 I've seen other systems use "each" as a unit for counted things, where the thing is described elsewhere 15:20:14 CW uses 'Count' and 'Count/second' 15:20:20 eglynn: nothing stops you to do that, and it's simpler if you know the unit (you can check what you're doing) 15:20:41 eglynn: that's a good word, too 15:21:24 jd__: so you could say sum over image.download and image.serve? 15:21:37 jd__: (both with unit=images in that patch IIRC) 15:21:41 nijaba: I'd suggest to extend this patch to g3, considering this is in fact part of the API, or we can change this in future without backward compatibility? 15:21:51 eglynn: sounds like a good example, if you want to charge for both for example or aggregate for stats :) 15:21:57 yjiang5_home: it has already been moved to g3 15:22:03 :) 15:22:14 jd__: OK, I didn't realize you could do that ... 15:22:23 eglynn: but if you know the unit, you can makes the API raises an error because unit are != if you're trying to add image.serve and swift container :) 15:23:00 jd__: i don't like using the unit for validation 15:23:26 gpernot: you prefer to return wrong values? 15:23:31 jd__: OK, that's one decent justification for this approach 15:23:38 network.outgoing, may have 2 counters with same unis, counting different things... 15:23:40 eglynn: thanks :) 15:23:58 gpernot: then you can select the unit you want, no? 15:24:06 smart validation would also need to take into account scaling conversion 15:24:16 I think we need to require that a given meter is always measured in the same units from all data sources 15:24:23 e.g. Kb + Mb OK, Kb + bananas not OK 15:24:36 eglynn: yes, but I think we agreed to keep SI unit (meaning second, and not ns) that means the pollster needs to convert to SI if needed 15:24:53 jd__: CPU time is already in ns IIRC 15:24:59 dhellmann: +1 15:25:06 eglynn: yes, actually I think it's better to say 'B' and not 'KB' or 'MB' 15:25:16 eglynn: I know, that should be considered as a bug IMHO 15:25:29 exactly, we should be recording the data in the smallest unit we have availble, and let the API caller convert to something else if they want 15:25:32 dhellmann: what would be the reason? 15:25:52 AFAIK in CW if you ask for a bytes metric in Gb say, it'll do the conversion before returning 15:25:52 jd__: simplicity of our implementation 15:25:52 jd__:ease of use for client apps ? 15:25:57 jd__: granularity 15:26:00 dhellmann: I think all unit have decimal, so smallest isn't a good criteria, SI is much better I think :) 15:26:18 jd__: I mean if we have access to ns, we should store and use that, rather than seconds 15:26:25 dhellmann: oh sure! 15:26:35 nobody will suggest to drop information :) 15:26:50 and we should only return values to the caller in the units we have them in, no conversions 15:26:51 no rounding basically, right? 15:27:20 eglynn:sure 15:27:21 the API implementation shouldn't even be worried about units. it has numbers and it does math on them, that's it. 15:27:37 no conversion => so if I ask for the sum of a meter in Kb and another in Gb, that's an error? 15:27:40 the units are provided as a convenience for the api user 15:27:56 eglynn: I don't think we want to provide a way for the API user to specify the units they want at all. 15:28:06 they'll take what we give them, and be happy! :-) 15:28:15 lol 15:28:18 lol 15:28:35 eglynn: all meters should be in bytes, so what you describe will raise an error ultimately yes 15:28:36 but if one meter stored as Gb and another as Kb, these could be summable, or? 15:28:54 eglynn: there is not currently an API for computing values across meters 15:28:55 * nijaba votes for using uk royal units 15:28:55 * nijaba runs 15:29:02 and I don't think we need one 15:29:08 * jd__ shots nijaba 15:29:30 dhellmann: I thought jd__ said that is currently supported? 15:29:38 (cross-meter aggregation) 15:29:41 IIUC 15:29:48 I don't know if it is in v2, it may be? 15:29:52 he's suggesting that we do it, but we do not now and I don't think it's necessary 15:30:12 no, the meter specification in the v2 api is one at a time 15:30:17 I don't know if it's necessary but I don't want to close doors too soon 15:30:35 a-ha, OK ... that seemed like the best argument for needing the specific unit for the counts 15:30:36 especially if it's quite cheap to keep them open for later 15:30:50 jd__: the problem is, as we've shown in this discussion, it's not cheap 15:30:54 considering how the scope of the project is elarging :) 15:31:02 you have to know a lot more about the meter, and whether the measurements can be combined 15:31:11 that's info the caller will have implicitly, but we would need to make explicit 15:31:23 that complicates our implementations, especially the storage backend that is doing all of the actual work 15:31:31 if we let the caller do it, they just have to make 2 calls 15:31:45 true that, adding network I/O requests makes sense, adding network and disk requests less so ... 15:32:10 right 15:32:22 the API is still mostly about metering, afaict 15:32:22 the 15:32:23 m 15:32:34 the new monitoring stuff is going to go through the new publishing pipelines, right? 15:32:46 yep 15:32:55 dhellmann: +1. multi-publisher will imply othe api access 15:33:03 for billing you want to list line-items anyway 15:33:04 until someone wants to build a monitoring stuff on top of ceilometer-api 15:33:14 :) 15:33:18 so even if you charge for network and disk i/o, you want those pulled out 15:33:23 jd__: which would not make sense 15:33:31 jd__: polling that API isn't going to be a good way to monitor :-) 15:34:05 by monitor, I mean things like graphing 15:34:26 bringing it back to units, asalkeld made a good point about describing the units somewhere 15:34:29 (on gerrit) 15:34:39 using the SNMP MIB as an example 15:34:49 do we need something of that ilk? 15:34:58 i.e. to make it clear that B = 'bytes' 15:35:05 etc. 15:35:11 MIB implies machine-parsable, right? 15:35:18 I think so 15:35:23 we have it documented, do we need a machine-parsable version? 15:35:48 how would it help to provide a mapping between "B" and "bytes"? 15:35:49 I don't see a use case 15:35:52 yeah, docco probably suffices 15:36:13 so if I need to start counting something new, the process would be 15:36:27 add the meter with a newly invented unit 15:36:36 and step 2, update the docco 15:36:55 we could, eventually, generate that list of meters and units from the plugins when we build the docs 15:37:17 we would need to expand the API for the plugins a bit, but not in complicated ways 15:37:26 but yes, for now, it's 2 steps 15:37:42 k, seems reasonable as long as we follow that convention 15:37:48 fine with me too :) 15:38:09 so to clarify, does that mean we agreed to use unique units for counting, instead of "count"? 15:38:32 yep, grugingly ;) 15:38:53 ok :-) 15:39:04 so lets get 18413 landed, could we sneak it in for g-2 at this late stage? 15:39:12 or OK just retargeting to g-3? 15:39:13 the other thing I wasn't sure about with that change was having the units included in every counter 15:39:28 eglynn: I think it has already been retargeted, no? 15:39:38 a-ha, yes it was ... 15:40:11 including the units in the counter means there isn't a need to explicitly declare a meter before something starts publishing data to it. that seems appealing, but is that what we want? 15:40:38 dhellmann: that also buys you the ability to change unit in time or to know that unit changed 15:41:10 jd__: right, that gets back to my earlier point: we don't want the units being changed 15:41:14 dhellmann: in fact, there is a user requireed in IRC to know all meter information before the counter published 15:41:19 jd__: shouldn't the unit really be immutable? 15:41:46 ok but the fact is that there's no check anywhere that the unit is always the same 15:42:01 so an external probe can send counters and suddently change units 15:42:15 nothing indicates it's a problem 15:42:15 but changing it would surely break the aggregation logic? 15:42:30 jd__: yes, so we have to document that restriction 15:42:30 eglynn: for sure 15:42:32 eglynn: if it specifies different unit, why should it be broken? 15:42:38 it shouldn't be an issue in practice 15:42:53 jd__: can't add say bytes and seconds 15:43:23 can we say, for the grizzly release, that we're not going to deal with changing units at all? we have enough other things to do 15:43:31 I agree that there's 3 different ways to fix this: 1. write in the doc that this will break your installation 2. check at record time that the unit is always the same 3. handle it correctly when doing computing and request in the API 15:43:43 if we want to start getting fancy with error detection or even unit conversion, we can discuss those things for H? 15:43:56 ok, I think we have settled on using names for units instead of count or each. right? Then we can thave another meeting about the unti policy, which the patch in review does not depend on? 15:44:03 dhellmann: that sounds like option 1, fine with me :) 15:44:10 yep agreed, punt to H, stick to #1 for now 15:44:10 nijaba: yes 15:44:14 jd__: cool 15:44:33 depends on doc is always not so good IMHO. 15:44:50 yjiang5_home: I'd prefer option 3 too, but we can start with option 1 anyway :) 15:44:58 jd__: why not 2? 15:45:01 it's better than nothing 15:45:21 yjiang5_home: likely because I like to do complicated things :-)) 15:45:29 hehe :) 15:45:42 we need to move on 15:45:48 * dhellmann nods 15:45:50 #agreed use name of for units that provide counts, instead of count or each 15:46:16 jd__: I have a question to the keystone bug. Is it caused by keystoneclient change in last minutes for g2 milestone, or because we didn't catch it in our test? 15:46:21 #action nijaba to start a thread on ml about unit policy 15:46:37 #topic Ceilometer and Healthmon 15:46:48 yjiang5_home: let me answer on #openstack-metering 15:46:55 jd__: sure. 15:47:03 I saw Barath_ online 15:47:25 llu-laptop: looks like he dropped off 15:47:34 divakar too 15:47:45 :( 15:48:22 so I guess we are back to my action to get in touch with them to see if they agree on llu-laptop plan? 15:48:26 Barath_ just pinged me to give me their email address and want to talk about that in email. 15:48:59 llu-laptop: sounds good. do you want to start the discussion, putting the ml in cc? 15:49:54 yes. I'll drop them the mail and cc the ml. 15:50:06 llu-laptop: thanks a lot! 15:50:37 #agreed llu-laptop to initiate conversation with healthmon 15:50:51 #topic Bugshashing day retrospective. what worked, what didn't, what we should consider doing differently next time ... etc. 15:51:06 as I was not there, I do not have much to say 15:51:12 jd__: ? 15:51:29 yeah, I think we did quite a good job to fix some bugs 15:51:30 one thing asalkeld mentioned before was that the choice of day was bad for APAC contributions 15:51:41 but I didn't see anyone from outside the project 15:51:42 i.e. Friday bleeds into their w/e 15:51:55 yeah, maybe next time we can pick a Tuesday or Wednesday 15:51:57 eglynn: ah good to know indeed, didn't think about that 15:51:58 so next time I think a Tuesday or Wednesday would be better 15:52:08 yep 15:52:23 jd__: no, I didn't either 15:52:37 agree with jd__ that the lack of randomers disappointing 15:53:01 jan 4th did not help much for randommer 15:53:07 some stats are still online at http://www.naquadah.org/~jd/bugdaystats/output/ceilometer.html FTR 15:53:28 maybe we need to seed the backlog with some more juicy/tempting/interesting-looking bugs next time 15:53:35 so we didn't gain new contributors, but we did fix bugs and learned about how to run one of these things 15:53:45 I count that as at least a partial success :-) 15:53:45 yep agreed 15:53:57 lets do it again before g3 15:53:59 sure :) 15:54:02 +1 15:54:09 maybe with a longer lead-in time 15:54:35 and not next to a holiday 15:54:37 or weekend 15:54:40 should we poikc the date now? 15:54:51 why not 15:55:01 sure, when is the g3 date again? 15:55:04 G3 is Feb 21st 15:55:22 * eglynn has jury duty the week of Feb 11th 15:55:35 so a week before that is feb 11th. 15:55:49 which would put eglynn out 15:55:58 yep 15:56:14 what about the week before that, then? 15:56:18 works for me 15:56:20 is that too soon? 15:56:23 but then feb 3rd is really close 15:56:33 3 weeks away 15:56:35 yeah, that's a point 15:56:51 the 7th would be 2 weeks before the g3 deadline 15:56:52 and eglynn as a LOT of bp to do 15:57:16 actually I can prolly join in after GMT EoD during the week of the 11th 15:57:25 maybe we should wait until after g3 then? do something before the release? 15:57:40 that would work too 15:57:51 during the pre-final stabilization period 15:58:00 we are supposed to be done with features by then, right? so fixing bugs makes sense. 15:58:07 exactly 15:58:08 right 15:58:34 March 7 then? 15:58:44 RC start on march 14th 15:59:15 I'll be on vacation from 6th to 17th March 15:59:17 I'll be at PyCon US from 12-18th 15:59:50 nijaba: otherwise, doodle? :) 15:59:53 would March 5th work? 16:00:06 (for jd__) 16:00:12 march 5th seeems good 16:00:26 fine with me 16:00:29 sure 16:00:50 cool 16:00:52 #agreed next bug squashing day on march 5th! 16:01:06 #topic Open Discussion 16:01:22 any quick topics as we are already past the hour? 16:01:34 nothing here 16:01:37 no 16:01:39 nowt from me 16:01:43 no 16:02:08 nop 16:02:09 Any specs for meter-post-api ? 16:02:22 great. So thanks again for a great meeting everyone. See you next week on wednesday at 21UTC 16:02:24 egallen: not yet 16:02:42 #endmeeting