15:00:48 <jd__> #startmeeting ceilometer
15:00:49 <openstack> Meeting started Thu May 30 15:00:48 2013 UTC.  The chair is jd__. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:51 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:53 <openstack> The meeting name has been set to 'ceilometer'
15:01:01 <sandywalsh> o/
15:01:04 <dragondm> o/
15:01:04 <nealph> \o'
15:01:04 <eglynn> o/
15:01:06 <mjmulloc> o/
15:01:07 <llu-laptop> o/
15:01:09 <dhellmann> o/
15:01:29 <gordc> o/
15:01:41 <jd__> hey everyone
15:01:43 <mrutkows> o/
15:01:48 <nijaba> o/
15:01:48 <n0ano> o/
15:02:14 <jd__> let's go!
15:02:21 <jd__> #topic Last week action: jd__ and dhellmann to write formal statement about limiting support for pre-grizzly versions of ceilometer
15:02:32 * jd__ whispers
15:02:46 <dhellmann> The ceilometer team has limited capacity to provide support for older versions of the project. Because the project graduated from incubation around the time of the grizzly release, that is the first version for which we will provide regular ongoing support following the standard deprecation cycle for OpenStack. The grizzly version of ceilometer is cannot be installed on the same server with earlier versions of OpenStac
15:02:47 <dhellmann> because of conflicting package requirements, but is API compatible with the folsom release if installed separately.
15:03:52 <eglynn> nitpick: s/ceilometer is cannot be/ceilometer cannot be/
15:04:00 <eglynn> otherwise looks good to me!
15:04:16 <dhellmann> corrected
15:04:33 <jd__> where should we publish that?
15:04:43 <dhellmann> somewhere in our docs?
15:04:49 <dhellmann> maybe in the installing section?
15:04:58 <dhellmann> and also probably to the openstack mailing list
15:05:14 <nealph> how about in the readme?
15:05:18 <eglynn> should we mention the stable/grizzly branch explicitly, or is that implicit in "ongoing support"?
15:05:42 <eglynn> actually, yeah, I think it probably is implicit ...
15:05:48 <dhellmann> I think that's implicit?
15:05:52 <eglynn> yep, agreed
15:06:08 <dhellmann> if there's a link to that policy, I can include a link when I add this to the docs
15:06:11 <nealph> i.e, in the root code directory..
15:06:41 <dhellmann> nealph: we have a lot of people trying to install from tarballs, so I'm not sure they will see it in the readme :-/
15:06:49 <jd__> I was typing that
15:07:11 <jd__> the doc that is online is probably the most read, so let's go with that
15:07:12 <eglynn> dhellmann: by "that policy", do you mean the overall stable branch policy? i.e. as expressed on https://wiki.openstack.org/wiki/StableBranch
15:07:18 <jd__> who wants to take the #action to make a patch?
15:07:20 <dhellmann> yeah
15:07:22 <dhellmann> I'll do it
15:07:27 <eglynn> cool
15:07:28 <nealph> was thinking 'in addition to' online...
15:07:44 <gordc> readme is pretty empty, i'm not sure anyone uses it... it does point to http://launchpad.net/ceilometer
15:07:45 <nealph> but that said, I agree with the tarball comment.
15:08:23 <llu-laptop> I don't think it hurts to put in readme, as an addition though
15:08:48 * nealph tends to look there first
15:09:28 <sandywalsh> nealph, +1
15:09:31 <jd__> so anyone up to the task?
15:09:34 <eglynn> on sending to the mailing list ... the dev list is good, but should we also consider openstack-announce?
15:10:03 <jd__> we can ask ttx for -announce I guess
15:10:04 <eglynn> (i.e. much less volume, reserved for important announcements etc, might hit a different audience ...)
15:10:04 <nealph> I will
15:10:12 <eglynn> cool
15:10:19 <dhellmann> jd__: I'm working on a patch now
15:10:24 <gordc> fair enough. i guess since we don't need to update statement that often it isn't a concern having it in multiple places.
15:10:28 <nealph> sorry, regarding readme.
15:10:31 <jd__> dhellmann: great
15:11:03 <jd__> #action dhellmann make a patch on Ceilometer documentation to include the statement about Grizzly support
15:11:32 <dhellmann> #link https://review.openstack.org/31062
15:11:42 <llu-laptop> so fast
15:12:03 <jd__> cool
15:12:06 <dhellmann> #action dhellmann to send email to the openstack list about the support policy
15:12:26 <dhellmann> oh, do you guys think we need to use the announce list?
15:12:39 <jd__> dhellmann: maybe, ask ttx for his opinion and permission
15:12:44 <dhellmann> that feels a little formal, but I can talk to ttx if you want
15:12:45 <dhellmann> ok
15:13:35 <jd__> moving on then :)
15:13:39 <jd__> #topic Review Havana-2 milestone
15:13:45 <jd__> #link https://launchpad.net/ceilometer/+milestone/havana-2
15:13:59 <jd__> if you're not aware h1 is on its way today
15:14:28 <jd__> so we should focus on delivering h2 now
15:15:13 <jd__> this time every blueprint has a owner, which is great :)
15:15:31 <jd__> think about updating the status of your blueprint as you work on them
15:16:12 <jd__> I don't have much comment on this for this week anyway; any question?
15:17:06 <sandywalsh> Roland has prepared a spec for https://blueprints.launchpad.net/ceilometer/+spec/specify-event-api ... but had to do it as a new BP. Can I merge that with my previous spec?
15:17:32 <sandywalsh> btw> dhellmann, you should give that a peek. In your wheelhouse.
15:17:40 <dhellmann> sandywalsh: it's on my list :-)
15:17:43 <sandywalsh> :)
15:18:14 <jd__> sandywalsh: yes, do you need me to do anything?
15:19:18 <sandywalsh> jd__,  not sure. I could just point my spec to his wiki page. But is there a way I can give him the ability to change the BP directly? Perhaps assign him as the assignee or drafter?
15:19:44 <dhellmann> the bps are linked already, do they need to be "merged"?
15:20:14 <dhellmann> I'm not objecting, just asking...
15:20:28 <sandywalsh> yeah, he had to create a new one because it wouldn't let him assign the spec himself
15:20:30 <jd__> sandywalsh: maybe, I'm not LP expert :)
15:20:41 <sandywalsh> k, we'll mess with it and clean it up
15:20:44 <jd__> but I can do assignment of bp of things like that if needed
15:21:21 <sandywalsh> that's all from me :)
15:22:30 <jd__> #topic blueprint discussion: support for auditing event - mrutkows, gordc
15:22:38 <jd__> #link https://blueprints.launchpad.net/ceilometer/+spec/support-standard-audit-formats
15:22:41 <mrutkows> hi everyone
15:22:49 <jd__> mrutkows, gordc floor is yours
15:23:04 <mrutkows> still kinda new to Ceilomter, but over last month been working hard to learn code and come up to speed
15:23:19 <mrutkows> basically what we are trying to do with this blueprint, is establish a configurale audit path that leverages ceilometer architecture and features, but can be used to create/log records that have normative data that can be attested to by an auditor
15:24:06 <dhellmann> this sounds similar to the work sandywalsh is doing with recording notifications
15:24:11 <jd__> I've read through the proposal, and it seems like a good idea to me
15:24:22 <mrutkows> i have more of a security standards background... and it came to my attention that people wanted to code special logging for Nova
15:24:24 <jd__> dhellmann: yes, but based on standards
15:24:27 * dhellmann hasn't had a chance to read the whole thing, so may be misunderstanding
15:24:37 <mrutkows> and we thought leveraging Ceilometer was a better path
15:24:39 <jd__> dhellmann: but the idea behind is really the same
15:25:03 <sandywalsh> this would be easy to implement on the nova side to make a notification driver that would output in this "standard" format. There would have to be a corresponding parser on the CM side. If it's optional on both sides, I think it's a nice addition. Keeps the enterprise customers happy.
15:25:03 <mrutkows> we want the path to supprt std. formats for normative comparison
15:25:13 <jd__> mrutkows: something I don't get, is how you want to store your CADF data?
15:25:18 <mrutkows> so we proposed it be a configurable path
15:25:26 <dhellmann> I love the idea of standardizing. We did have a few people object to the idea of requiring ceilometer for deploying nova, for example, but that was related to the scheduler. This could be less tightly coupled, so shouldn't be an issue.
15:25:34 <sandywalsh> mrutkows, my apologies for not formally responding before this meeting.
15:25:37 <mrutkows> well, for now as metadata as this is the path thats easy for now
15:25:51 <gordc> dhellmann, is there a bp sandywalsh work?
15:26:13 <sandywalsh> gordc, our approach has been to use the existing nova notification format
15:26:22 <mrutkows> hi Sandy, yes I have seen many things in blueprints and other topics that we may have alignment in many ideas
15:26:27 <dhellmann> gordc: several :-)
15:26:33 <jd__> mrutkows: ok, I see this in your WIP code indeed, but this seems under efficient, you're using counters but you don't really count anything
15:26:46 <jd__> mrutkows: what sandywalsh is working on will be much more useful I think
15:27:16 <sandywalsh> I don't think they're mutually exclusive. We're really just talking about the wire format, yes?
15:27:27 <jd__> mrutkows: in the end I think you want a notification driver for Oslo that emits CADF and a collector that registers CADF data in a well-designed db
15:27:30 <mrutkows> jd: ive been hoping to align with Sandy's proposals,  notification path (filter) seems correct path (push)
15:27:41 <sandywalsh> jd__, +1
15:27:53 <mrutkows> jd: i agree
15:28:01 <jd__> mrutkows: which in the end is orthogonal to Ceilometer functionnalities, even if you use the same wire
15:28:21 <mrutkows> it seemed that starting within Ceilometer and then promoting to OSLO after proven seemed best
15:28:22 <jd__> whereas I see event recording as sandywalsh is working a leverage for us to build more counters
15:28:24 <sandywalsh> jd__, except we can use the existing Event/Trait model to store the event as per normal. The incoming collector translator would just be different
15:28:45 <dhellmann> is the point to get CADF data into ceilometer, or to get it out?
15:28:59 <jd__> dhellmann: to get it in, but there's no value for ceilometer
15:29:14 <jd__> mrutkows: I actually think you should start a side project on this
15:29:21 <mrutkows> Primarily get it in, getting out would be a conversation for another day, as long as we can store it for now then im happy
15:29:32 <mrutkows> CADF does have a query syntax
15:29:59 <mrutkows> but that is something of a pipe dream at this stage
15:29:59 <jd__> you'd call it "OpenStack Audit Service" :)
15:30:16 <dhellmann> would it make more sense to add a new query API that used the CADF format for queries and responses and accessed the ceilometer database?
15:30:16 <mrutkows> OpenStack can brand it sure =)
15:30:44 <mrutkows> dellmann: your my hero
15:30:46 <dhellmann> I need to read more. I'm not sure why the wire format between nova and ceilometer needs to change.
15:31:05 <dragondm> Hmmm... Y'know there is a side project already out there designed to take OS notifications and translate them to different formats...
15:31:13 <sandywalsh> we haven't defined a robust query api for Events yet, might be something worth considering (so long as it's not bloated api-by-committee :)
15:31:15 <mrutkows> dhellmann: would make myself and any info available to you
15:31:21 <jd__> dhellmann: yeah, but I really thinks it's a whole different set of functionnality at this stage
15:31:26 <jd__> audit isn't metering
15:31:30 <dhellmann> jd__: sure
15:31:36 <dragondm> it's called Yagi.
15:31:52 <sandywalsh> dhellmann, I assume it's if the endpoint isn't CM
15:31:56 <dhellmann> but it does seem like it goes towards our revised goal of providing the data we collect for lots of purposes
15:32:10 <dhellmann> this could be our first API extension :-)
15:32:38 <dhellmann> although I don't see an issue with making it a first class part of the API, if it's just returning data in a different format
15:32:41 <jd__> anyway even if we keep this into ceilometer, this is going to need a new collector and/or db schema IMHO
15:32:45 <mrutkows> jd: there is a relationship between metering and auditing
15:33:11 <mrutkows> the cadf spec. acks. this fact if your interested in reading
15:33:13 <dhellmann> jd__: well, no, that was my point -- if they're just mapping events/counters to these new record types, couldn't we do that with the existing data?
15:33:42 <jd__> dhellmann: I've a tendency to think "audit" means "store raw stuff that we are sure nobody messed with"
15:34:00 <dhellmann> such as the notification events sandywalsh is adding support for?
15:34:04 <nijaba> jd__: I tend to agree with your view
15:34:06 <jd__> dhellmann: yes
15:34:07 <mrutkows> dhellmann: perhaps, but the CADF data model assures all info needed for ISO, NIST, COBIT etc are in the record in a normative way
15:34:31 <jd__> or one solution would be to twist sandywalsh's work to use CADF format instead
15:34:36 <nealph> mrutkows: in the the record, or in the api response?
15:34:38 <mrutkows> perhaps we can use this exercise to identify what the essential data is so any format could map
15:34:56 <dhellmann> mrutkows: ok, well, using CADF format throughout does make it seem like a bigger project
15:35:06 <mrutkows> nealph: in the record,the record eventually has to be immutable and even signed
15:35:30 <mrutkows> nealph: audit records cannot be created on demand
15:35:37 <sandywalsh> jd__, how it's stored in the db shouldn't really matter. It's how it appears on the wire and how it's published that really matters.
15:35:39 <dhellmann> notifications aren't signed now, but that would be a straightforward addition
15:36:04 <sandywalsh> dragondm, do you think the right approach would be to translate in Yagi or via a specific notification driver?
15:36:12 <jd__> sandywalsh: for audit purpose, I'm not really sure
15:36:18 <mrutkows> sandy: yes consider it a wire format, but if we can allow it to also be signed and stored in a format a customer needs, that is better
15:36:21 <dragondm> hmm.....
15:36:58 <sandywalsh> mrutkows, customers shouldn't be hitting the CM db directly anyway
15:37:00 <dragondm> it might depend on their requirements.  A driver would be simpler.
15:37:09 <nealph> dragondm: +1
15:37:16 <mrutkows> sandy: auditors want to grab logs eventually and a format has to be one that can be relied upon (and better a standard format)
15:37:37 <sandywalsh> dragondm, that's what I'm thinking. No need to stand up another server
15:38:06 <sandywalsh> mrutkows, but couldn't they do that from the raw events? Why do they have to use the logs?
15:38:10 <mrutkows> sandy: ideally customers should not
15:38:29 <sandywalsh> mrutkows, we wouldn't want to dump raw events in the logs if they contain sensitive info
15:38:32 <mrutkows> sandy: we have lots to talk about i can see =)
15:38:37 <sandywalsh> :)
15:39:04 <mrutkows> sandy: the companies that worked on CADF know that dumping raw events are not enough
15:39:21 <mrutkows> sandy: background and discussion would take many beers
15:39:22 <sandywalsh> I'll reread and give proper feedback. I don't think there's anything in there that worries me. I think it call all be done as drivers/plugins and as optional components.
15:39:42 <sandywalsh> s/call/can/
15:40:15 <mrutkows> sandy: thanks )
15:40:30 <nealph> mrutkows: points for a nice BP though. :)
15:40:50 <mrutkows> nealph: thanks much
15:41:11 <jd__> sounds great :)
15:41:43 <mrutkows> jd: thanks for allowing us to discuss today
15:42:00 <sandywalsh> mrutkows, mailing list or wiki markup for comments? What's best for you?
15:42:15 <mrutkows> wiki markup seems to be more robust
15:42:16 * dhellmann likes the ceilometer architecture diagram in the wiki page
15:42:50 <jd__> dhellmann: yeah me too
15:42:55 <sandywalsh> mrutkows, will do
15:43:20 <jd__> mrutkows: would be a good idea to use your ceilometer diagram from the wiki in our doc actually
15:43:58 <gordc> jd__, was just going to suggest the same
15:44:00 <mrutkows> jd: that would be great
15:44:26 <mrutkows> jd: I can provide the powerpoint originals if needed or tweak as needed
15:44:44 <dhellmann> mrutkows: original files would be good, too, for updates later
15:45:11 <mrutkows> jd: i am actually working on another level of the diagram, to show an overlay of what major files comprosie each box
15:45:15 <mrutkows> comprise
15:45:27 <sandywalsh> you can add the original ppt as an attachment to the wiki page. That's what I usually do.
15:45:32 <mrutkows> dhellmann: ok will post to wiki
15:45:35 <nealph> I think there is a diagram update in the pipeline now...
15:46:08 * nealph searching for the changeid
15:46:29 <eglynn> https://review.openstack.org/#/c/27835/
15:46:42 <eglynn> nealph: ^^^ it that the one you mean?
15:46:44 <mrutkows> Gordon will be helping me to review WIP code and with my first check in
15:46:47 <dhellmann> nealph: there is, but we've been having some trouble communicating the useful bits of the architecture to include
15:46:48 <nealph> yep :)
15:47:28 <mrutkows> nealph: Tong Li works in my group
15:48:04 <mrutkows> nealph: I can work with him to see if he can make use of mine
15:48:11 <nealph> great...maybe merge the text updates from that change and leverage this diagram?
15:49:01 <mrutkows> nealph: will read the patch comments
15:49:08 <mrutkows> and see if I can address with Tong
15:50:08 <nealph> k. will add that suggestion to patch comments.
15:50:47 <jd__> #topic Open discussion
15:50:55 <jd__> if you want to address something else or continue :)
15:51:08 <dhellmann> ttx suggested that we just send the support announcement to the main mailing list, so I will do that
15:51:44 <eglynn> reminder: the freeze for 2013.1.2 is today (release due on June 6th)
15:51:53 <eglynn> so if you've anything you think should be in the second release off stable/grizzly, please tag and/or backport
15:51:59 <eglynn> (I'll do a trawl thru' the git log in any case before EoD today)
15:52:39 <dhellmann> eglynn: is that schedule published somewhere?
15:52:58 <eglynn> dhellmann: one sec, I'll dig it out ...
15:54:14 <nijaba> https://wiki.openstack.org/wiki/Havana_Release_Schedule ?
15:54:30 <eglynn> #link http://lists.openstack.org/pipermail/openstack-stable-maint/2013-April/000479.html
15:54:46 <eglynn> scroll down to "Finally, future stable/grizzly releases will be ..."
15:54:51 <dhellmann> thanks
15:55:10 <dhellmann> I think I joined the stable team after that list was established
15:55:20 <eglynn> a-ha, cool
15:55:37 <dhellmann> I'm still getting set up with notifications and such :-)
15:56:21 <BrooklynChen> jd__: is there a bp for non-admin user authentication?
15:57:27 <jd__> BrooklynChen: there was, this is already implemented
15:59:54 <jd__> #endmeeting