15:00:48 #startmeeting ceilometer 15:00:49 Meeting started Thu May 30 15:00:48 2013 UTC. The chair is jd__. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:51 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:53 The meeting name has been set to 'ceilometer' 15:01:01 o/ 15:01:04 o/ 15:01:04 \o' 15:01:04 o/ 15:01:06 o/ 15:01:07 o/ 15:01:09 o/ 15:01:29 o/ 15:01:41 hey everyone 15:01:43 o/ 15:01:48 o/ 15:01:48 o/ 15:02:14 let's go! 15:02:21 #topic Last week action: jd__ and dhellmann to write formal statement about limiting support for pre-grizzly versions of ceilometer 15:02:32 * jd__ whispers 15:02:46 The ceilometer team has limited capacity to provide support for older versions of the project. Because the project graduated from incubation around the time of the grizzly release, that is the first version for which we will provide regular ongoing support following the standard deprecation cycle for OpenStack. The grizzly version of ceilometer is cannot be installed on the same server with earlier versions of OpenStac 15:02:47 because of conflicting package requirements, but is API compatible with the folsom release if installed separately. 15:03:52 nitpick: s/ceilometer is cannot be/ceilometer cannot be/ 15:04:00 otherwise looks good to me! 15:04:16 corrected 15:04:33 where should we publish that? 15:04:43 somewhere in our docs? 15:04:49 maybe in the installing section? 15:04:58 and also probably to the openstack mailing list 15:05:14 how about in the readme? 15:05:18 should we mention the stable/grizzly branch explicitly, or is that implicit in "ongoing support"? 15:05:42 actually, yeah, I think it probably is implicit ... 15:05:48 I think that's implicit? 15:05:52 yep, agreed 15:06:08 if there's a link to that policy, I can include a link when I add this to the docs 15:06:11 i.e, in the root code directory.. 15:06:41 nealph: we have a lot of people trying to install from tarballs, so I'm not sure they will see it in the readme :-/ 15:06:49 I was typing that 15:07:11 the doc that is online is probably the most read, so let's go with that 15:07:12 dhellmann: by "that policy", do you mean the overall stable branch policy? i.e. as expressed on https://wiki.openstack.org/wiki/StableBranch 15:07:18 who wants to take the #action to make a patch? 15:07:20 yeah 15:07:22 I'll do it 15:07:27 cool 15:07:28 was thinking 'in addition to' online... 15:07:44 readme is pretty empty, i'm not sure anyone uses it... it does point to http://launchpad.net/ceilometer 15:07:45 but that said, I agree with the tarball comment. 15:08:23 I don't think it hurts to put in readme, as an addition though 15:08:48 * nealph tends to look there first 15:09:28 nealph, +1 15:09:31 so anyone up to the task? 15:09:34 on sending to the mailing list ... the dev list is good, but should we also consider openstack-announce? 15:10:03 we can ask ttx for -announce I guess 15:10:04 (i.e. much less volume, reserved for important announcements etc, might hit a different audience ...) 15:10:04 I will 15:10:12 cool 15:10:19 jd__: I'm working on a patch now 15:10:24 fair enough. i guess since we don't need to update statement that often it isn't a concern having it in multiple places. 15:10:28 sorry, regarding readme. 15:10:31 dhellmann: great 15:11:03 #action dhellmann make a patch on Ceilometer documentation to include the statement about Grizzly support 15:11:32 #link https://review.openstack.org/31062 15:11:42 so fast 15:12:03 cool 15:12:06 #action dhellmann to send email to the openstack list about the support policy 15:12:26 oh, do you guys think we need to use the announce list? 15:12:39 dhellmann: maybe, ask ttx for his opinion and permission 15:12:44 that feels a little formal, but I can talk to ttx if you want 15:12:45 ok 15:13:35 moving on then :) 15:13:39 #topic Review Havana-2 milestone 15:13:45 #link https://launchpad.net/ceilometer/+milestone/havana-2 15:13:59 if you're not aware h1 is on its way today 15:14:28 so we should focus on delivering h2 now 15:15:13 this time every blueprint has a owner, which is great :) 15:15:31 think about updating the status of your blueprint as you work on them 15:16:12 I don't have much comment on this for this week anyway; any question? 15:17:06 Roland has prepared a spec for https://blueprints.launchpad.net/ceilometer/+spec/specify-event-api ... but had to do it as a new BP. Can I merge that with my previous spec? 15:17:32 btw> dhellmann, you should give that a peek. In your wheelhouse. 15:17:40 sandywalsh: it's on my list :-) 15:17:43 :) 15:18:14 sandywalsh: yes, do you need me to do anything? 15:19:18 jd__, not sure. I could just point my spec to his wiki page. But is there a way I can give him the ability to change the BP directly? Perhaps assign him as the assignee or drafter? 15:19:44 the bps are linked already, do they need to be "merged"? 15:20:14 I'm not objecting, just asking... 15:20:28 yeah, he had to create a new one because it wouldn't let him assign the spec himself 15:20:30 sandywalsh: maybe, I'm not LP expert :) 15:20:41 k, we'll mess with it and clean it up 15:20:44 but I can do assignment of bp of things like that if needed 15:21:21 that's all from me :) 15:22:30 #topic blueprint discussion: support for auditing event - mrutkows, gordc 15:22:38 #link https://blueprints.launchpad.net/ceilometer/+spec/support-standard-audit-formats 15:22:41 hi everyone 15:22:49 mrutkows, gordc floor is yours 15:23:04 still kinda new to Ceilomter, but over last month been working hard to learn code and come up to speed 15:23:19 basically what we are trying to do with this blueprint, is establish a configurale audit path that leverages ceilometer architecture and features, but can be used to create/log records that have normative data that can be attested to by an auditor 15:24:06 this sounds similar to the work sandywalsh is doing with recording notifications 15:24:11 I've read through the proposal, and it seems like a good idea to me 15:24:22 i have more of a security standards background... and it came to my attention that people wanted to code special logging for Nova 15:24:24 dhellmann: yes, but based on standards 15:24:27 * dhellmann hasn't had a chance to read the whole thing, so may be misunderstanding 15:24:37 and we thought leveraging Ceilometer was a better path 15:24:39 dhellmann: but the idea behind is really the same 15:25:03 this would be easy to implement on the nova side to make a notification driver that would output in this "standard" format. There would have to be a corresponding parser on the CM side. If it's optional on both sides, I think it's a nice addition. Keeps the enterprise customers happy. 15:25:03 we want the path to supprt std. formats for normative comparison 15:25:13 mrutkows: something I don't get, is how you want to store your CADF data? 15:25:18 so we proposed it be a configurable path 15:25:26 I love the idea of standardizing. We did have a few people object to the idea of requiring ceilometer for deploying nova, for example, but that was related to the scheduler. This could be less tightly coupled, so shouldn't be an issue. 15:25:34 mrutkows, my apologies for not formally responding before this meeting. 15:25:37 well, for now as metadata as this is the path thats easy for now 15:25:51 dhellmann, is there a bp sandywalsh work? 15:26:13 gordc, our approach has been to use the existing nova notification format 15:26:22 hi Sandy, yes I have seen many things in blueprints and other topics that we may have alignment in many ideas 15:26:27 gordc: several :-) 15:26:33 mrutkows: ok, I see this in your WIP code indeed, but this seems under efficient, you're using counters but you don't really count anything 15:26:46 mrutkows: what sandywalsh is working on will be much more useful I think 15:27:16 I don't think they're mutually exclusive. We're really just talking about the wire format, yes? 15:27:27 mrutkows: in the end I think you want a notification driver for Oslo that emits CADF and a collector that registers CADF data in a well-designed db 15:27:30 jd: ive been hoping to align with Sandy's proposals, notification path (filter) seems correct path (push) 15:27:41 jd__, +1 15:27:53 jd: i agree 15:28:01 mrutkows: which in the end is orthogonal to Ceilometer functionnalities, even if you use the same wire 15:28:21 it seemed that starting within Ceilometer and then promoting to OSLO after proven seemed best 15:28:22 whereas I see event recording as sandywalsh is working a leverage for us to build more counters 15:28:24 jd__, except we can use the existing Event/Trait model to store the event as per normal. The incoming collector translator would just be different 15:28:45 is the point to get CADF data into ceilometer, or to get it out? 15:28:59 dhellmann: to get it in, but there's no value for ceilometer 15:29:14 mrutkows: I actually think you should start a side project on this 15:29:21 Primarily get it in, getting out would be a conversation for another day, as long as we can store it for now then im happy 15:29:32 CADF does have a query syntax 15:29:59 but that is something of a pipe dream at this stage 15:29:59 you'd call it "OpenStack Audit Service" :) 15:30:16 would it make more sense to add a new query API that used the CADF format for queries and responses and accessed the ceilometer database? 15:30:16 OpenStack can brand it sure =) 15:30:44 dellmann: your my hero 15:30:46 I need to read more. I'm not sure why the wire format between nova and ceilometer needs to change. 15:31:05 Hmmm... Y'know there is a side project already out there designed to take OS notifications and translate them to different formats... 15:31:13 we haven't defined a robust query api for Events yet, might be something worth considering (so long as it's not bloated api-by-committee :) 15:31:15 dhellmann: would make myself and any info available to you 15:31:21 dhellmann: yeah, but I really thinks it's a whole different set of functionnality at this stage 15:31:26 audit isn't metering 15:31:30 jd__: sure 15:31:36 it's called Yagi. 15:31:52 dhellmann, I assume it's if the endpoint isn't CM 15:31:56 but it does seem like it goes towards our revised goal of providing the data we collect for lots of purposes 15:32:10 this could be our first API extension :-) 15:32:38 although I don't see an issue with making it a first class part of the API, if it's just returning data in a different format 15:32:41 anyway even if we keep this into ceilometer, this is going to need a new collector and/or db schema IMHO 15:32:45 jd: there is a relationship between metering and auditing 15:33:11 the cadf spec. acks. this fact if your interested in reading 15:33:13 jd__: well, no, that was my point -- if they're just mapping events/counters to these new record types, couldn't we do that with the existing data? 15:33:42 dhellmann: I've a tendency to think "audit" means "store raw stuff that we are sure nobody messed with" 15:34:00 such as the notification events sandywalsh is adding support for? 15:34:04 jd__: I tend to agree with your view 15:34:06 dhellmann: yes 15:34:07 dhellmann: perhaps, but the CADF data model assures all info needed for ISO, NIST, COBIT etc are in the record in a normative way 15:34:31 or one solution would be to twist sandywalsh's work to use CADF format instead 15:34:36 mrutkows: in the the record, or in the api response? 15:34:38 perhaps we can use this exercise to identify what the essential data is so any format could map 15:34:56 mrutkows: ok, well, using CADF format throughout does make it seem like a bigger project 15:35:06 nealph: in the record,the record eventually has to be immutable and even signed 15:35:30 nealph: audit records cannot be created on demand 15:35:37 jd__, how it's stored in the db shouldn't really matter. It's how it appears on the wire and how it's published that really matters. 15:35:39 notifications aren't signed now, but that would be a straightforward addition 15:36:04 dragondm, do you think the right approach would be to translate in Yagi or via a specific notification driver? 15:36:12 sandywalsh: for audit purpose, I'm not really sure 15:36:18 sandy: yes consider it a wire format, but if we can allow it to also be signed and stored in a format a customer needs, that is better 15:36:21 hmm..... 15:36:58 mrutkows, customers shouldn't be hitting the CM db directly anyway 15:37:00 it might depend on their requirements. A driver would be simpler. 15:37:09 dragondm: +1 15:37:16 sandy: auditors want to grab logs eventually and a format has to be one that can be relied upon (and better a standard format) 15:37:37 dragondm, that's what I'm thinking. No need to stand up another server 15:38:06 mrutkows, but couldn't they do that from the raw events? Why do they have to use the logs? 15:38:10 sandy: ideally customers should not 15:38:29 mrutkows, we wouldn't want to dump raw events in the logs if they contain sensitive info 15:38:32 sandy: we have lots to talk about i can see =) 15:38:37 :) 15:39:04 sandy: the companies that worked on CADF know that dumping raw events are not enough 15:39:21 sandy: background and discussion would take many beers 15:39:22 I'll reread and give proper feedback. I don't think there's anything in there that worries me. I think it call all be done as drivers/plugins and as optional components. 15:39:42 s/call/can/ 15:40:15 sandy: thanks ) 15:40:30 mrutkows: points for a nice BP though. :) 15:40:50 nealph: thanks much 15:41:11 sounds great :) 15:41:43 jd: thanks for allowing us to discuss today 15:42:00 mrutkows, mailing list or wiki markup for comments? What's best for you? 15:42:15 wiki markup seems to be more robust 15:42:16 * dhellmann likes the ceilometer architecture diagram in the wiki page 15:42:50 dhellmann: yeah me too 15:42:55 mrutkows, will do 15:43:20 mrutkows: would be a good idea to use your ceilometer diagram from the wiki in our doc actually 15:43:58 jd__, was just going to suggest the same 15:44:00 jd: that would be great 15:44:26 jd: I can provide the powerpoint originals if needed or tweak as needed 15:44:44 mrutkows: original files would be good, too, for updates later 15:45:11 jd: i am actually working on another level of the diagram, to show an overlay of what major files comprosie each box 15:45:15 comprise 15:45:27 you can add the original ppt as an attachment to the wiki page. That's what I usually do. 15:45:32 dhellmann: ok will post to wiki 15:45:35 I think there is a diagram update in the pipeline now... 15:46:08 * nealph searching for the changeid 15:46:29 https://review.openstack.org/#/c/27835/ 15:46:42 nealph: ^^^ it that the one you mean? 15:46:44 Gordon will be helping me to review WIP code and with my first check in 15:46:47 nealph: there is, but we've been having some trouble communicating the useful bits of the architecture to include 15:46:48 yep :) 15:47:28 nealph: Tong Li works in my group 15:48:04 nealph: I can work with him to see if he can make use of mine 15:48:11 great...maybe merge the text updates from that change and leverage this diagram? 15:49:01 nealph: will read the patch comments 15:49:08 and see if I can address with Tong 15:50:08 k. will add that suggestion to patch comments. 15:50:47 #topic Open discussion 15:50:55 if you want to address something else or continue :) 15:51:08 ttx suggested that we just send the support announcement to the main mailing list, so I will do that 15:51:44 reminder: the freeze for 2013.1.2 is today (release due on June 6th) 15:51:53 so if you've anything you think should be in the second release off stable/grizzly, please tag and/or backport 15:51:59 (I'll do a trawl thru' the git log in any case before EoD today) 15:52:39 eglynn: is that schedule published somewhere? 15:52:58 dhellmann: one sec, I'll dig it out ... 15:54:14 https://wiki.openstack.org/wiki/Havana_Release_Schedule ? 15:54:30 #link http://lists.openstack.org/pipermail/openstack-stable-maint/2013-April/000479.html 15:54:46 scroll down to "Finally, future stable/grizzly releases will be ..." 15:54:51 thanks 15:55:10 I think I joined the stable team after that list was established 15:55:20 a-ha, cool 15:55:37 I'm still getting set up with notifications and such :-) 15:56:21 jd__: is there a bp for non-admin user authentication? 15:57:27 BrooklynChen: there was, this is already implemented 15:59:54 #endmeeting