20:01:55 <Rockyg> #startmeeting log_wg
20:01:56 <openstack> Meeting started Wed Apr 22 20:01:55 2015 UTC and is due to finish in 60 minutes.  The chair is Rockyg. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:02:00 <openstack> The meeting name has been set to 'log_wg'
20:02:32 <Rockyg> Now, the question:  do we have anything people want to discuss?  The table is open for new discussions
20:03:41 <nkrinner> not sure if this is worth discussing, but this crossed my mind recently
20:03:42 <jokke_> I really have nothing for  today
20:04:04 <Rockyg> nkrinner: well, better than napping -- go for it
20:04:52 <Rockyg> Also, abviously, I have not gotten the next draft of error codes out yet.  FYI.
20:05:04 <nkrinner> for bugs we have the tag docimpact tag, which automatically notifies the doc team about changes that might affect them. do we want something like this for logs too?
20:05:52 <Rockyg> Oh, yes, please!  I started it, but going project by project, finding a bug that could use the tag, and adding it, was taking lots of time
20:06:09 <Rockyg> If there's a more straightforward way, I'm all for it.
20:06:19 <nkrinner> i was just browsing the code to see how and where to implement that. also trying to find out with who to talk about this.
20:06:49 <nkrinner> so i suggest i will continue investigating this and report on it next week?
20:07:30 <Rockyg> Sounds great
20:07:45 <jokke_> sounds good
20:07:59 <nkrinner> so next week we hopefully see if it is worth the effort then. :-)
20:08:05 <Rockyg> I think I was going the easy route with the tag "log"
20:08:49 <Rockyg> nkrinner: I think it will be.  Once it's available for ops folks, I think they will use it.  Of course, that's also after educating them.
20:08:54 <jokke_> :)
20:09:04 <nkrinner> yeah :-)
20:10:03 <Rockyg> But, summit is soon.  Wondering -- would it make sense to have a working session with ops folks just identifying, tagging and escalating log bugs?  Sort of a log bug triage session?
20:11:40 <Rockyg> I also find it difficult to find "log" bugs because yuu can't do a cross-project search for key words
20:11:52 <jokke_> ++
20:11:52 <nkrinner> sadly i can't make it to vancouver
20:12:02 <Rockyg> Darn!
20:12:25 <Rockyg> If you ever make it to San Jose, I'll buy you a beer.
20:12:33 <nkrinner> :-)
20:13:34 <Rockyg> FYI.  some infra and other folks are looking at phabricator as a possible replacement for storyboard/launchpad.  Take a look and see if it is better
20:14:00 <Rockyg> #topic Rambling
20:14:08 <nkrinner> thanks, will take a look
20:14:12 <Rockyg> Just wanted to get in that ...
20:15:01 <Rockyg> Some more rambling.  We have "log files" scattered across the cloud.  some log "log messages", some log error output, some log HTTP stuff.
20:15:20 <Rockyg> I''ve got a list.  I want to make it a matrix.
20:16:09 <Rockyg> default log name, location, type of messages logged, audience (ops, tenant ops, app admin, etc), etc
20:16:48 <jokke_> ok
20:16:54 <Rockyg> I'm thinking this could help clarify the differences between the various messages for one
20:17:34 <Rockyg> It could also provide a checklist for devs as to what type of message they really want to generate based on classification
20:18:10 <jokke_> yeah I think that would be a good idea
20:18:12 <Rockyg> It would aid everyone from dev to enduser in tracking down information
20:18:14 <nkrinner> it would help making developers more aware about other peoples log needs
20:19:26 <Rockyg> Let's brainstorm on what the columns should be.  Of course the first one is the log file itself.  /there are about 80? I think.
20:19:38 <Rockyg> And location seems to be good for the second
20:20:14 <nkrinner> project/service the log belongs to
20:20:22 <nkrinner> maybe
20:20:59 <Rockyg> https://docs.google.com/spreadsheets/d/1XTncfK_droY8E-Uy2icVuU-z9ya38ZBK_ZIRvGfPOXc/edit?usp=sharing
20:22:18 <Rockyg> #link https://docs.google.com/spreadsheets/d/1XTncfK_droY8E-Uy2icVuU-z9ya38ZBK_ZIRvGfPOXc/edit?usp=sharing
20:22:37 <Rockyg> viewable by?
20:22:41 <nkrinner> i think that is the most important information
20:22:44 <Rockyg> Audience?
20:23:14 <jokke_> I can see it at least
20:23:16 <nkrinner> i think everything is only visible to admins?
20:23:25 <Rockyg> Ops/Domain Ops/Tenant?
20:23:45 <Rockyg> Well, all the vm log files are available to the tenant who spins up the vm
20:24:17 <nkrinner> true
20:24:27 <Rockyg> And if you have nested clouds, then the guy who controls the top of the provisioned cloud owns those cloud files
20:25:18 <Rockyg> And who is supposed to use them, versus who can see them is where security issues happen
20:27:35 <nkrinner> cloud users can do whatever they want with the logs they created i guess
20:27:35 <Rockyg> Somewhere I had found a list that infra put together.  Tryng to find it now.
20:28:27 <Rockyg> Right.  so it's a matter of what they have permission to create.  It's the whole role thing.  So I guess the question is:  What is the scope of the Role?
20:30:11 <nkrinner> i as a developer am mostly interested in what helps me debug issues with services and find all the infos i need in the log files. i have not really thought about user needs so far
20:31:40 <Rockyg> Ture.  Devs think about what they need to develop and Ops/End Users think about what they need to check on health.  and hackers think of what info can be used to break in.
20:32:26 <Rockyg> So, Purpose of the log is a good thing to know.  It helps to inform us who the users will be.
20:32:38 <nkrinner> i see
20:33:13 <nkrinner> purpose alse depends a lot on the log level i guess
20:34:21 <Rockyg> Yes.  Not all logs are for log messages.  Some are for notifications, the housekeeping on RESTful APIs, tracebacks, etc.
20:35:47 <Rockyg> So, for instance, the Apache logs would get most/all API message logs, but then some of the project logs would also capture a log after processing the API response.
20:36:44 <jokke_> yeah and all the wsgi servers should log each request once to the service logs
20:38:47 <Rockyg> So what jokke_ just said leads me to think we should add a field for message type, like wsgi notifications, HTTP, etc?
20:39:06 <nkrinner> ok
20:40:21 <nkrinner> to be honest that is all valuable information i can currently imagine. what else woulde we need to know? maybe a general 'does its job'/'doesn't do its job' rating?
20:40:23 <jokke_> that could work
20:41:27 <nkrinner> for example i am not so happy with some of the logs i have had to look at, but it might be it was only because it were log files of services i am not familiar with
20:41:35 <Rockyg> That's a good idea.  Rating its usefulness
20:42:15 <nkrinner> i think keystone had some quite ugly logs, but then i usually don't touch keystone, so maybe i was simply not educated enought to read them
20:42:45 <jokke_> nkrinner: make a note for us somewhere, example could be good idea
20:43:13 <Rockyg> nkrinner: you can put a comment or note on the spreadsheet, or add to the note I put on it.
20:43:23 <nkrinner> jokke_: true. this is what i remember vaguely
20:44:03 <jokke_> because if the logs on some specific services looks non intuitive, that's perhaps something we could affect
20:44:59 <nkrinner> i have to look at it again, maybe i blamed the wrong service :-/
20:45:07 <Rockyg> Precisely.  Or, more to the point, get them to be more precise;-)
20:47:47 <Rockyg> So, I think getting this started is a good thing.  Let me see if I can find that list of log files again to propagate the first column.  Once we have that, we can announce to the ml for comments, concerns, help filling in.
20:48:36 <nkrinner> yes, i think it is a good start
20:49:02 <Rockyg> I think I'm also going to add a earliest release column for when the file first appears in OpenStack..  Optional, so last.
20:49:47 <Rockyg> Gee.  anything to get out of editing that feakin' spec ;-)
20:51:42 <Rockyg> Jeez, just thought of another tool that ops guys would love for OpenStack.  A log registry.  If a new log is spun up, oslo has a library call that registers the log in a db or some such.
20:52:31 <nkrinner> i think logs should be known and collected automatically anyway
20:53:41 <jokke_> Rockyg: automatic log finder would probably not work with syslog
20:53:42 <Rockyg> nkrinner: you would think, but right now, it doesn't happen.  Each project generates what it thinks it needs in the places it has decided upon.  Wouldn't it be nice to have a central registry?
20:54:05 <Rockyg> jokke_ don't want a finder, want a register
20:54:20 <Rockyg> then devs could find logs they need, too.
20:54:47 <nkrinner> hm. i think we have a tool for that already, let me check
20:54:51 <Rockyg> Kind of like the murano app catalog, but for logs
20:56:08 <nkrinner> it is a custom bash script, i think it is maintained manually
20:56:12 <Rockyg> then that could be fed into logstash or whatever, with rules and create a customized best of collection of the distributed log files.
20:57:04 <Rockyg> And if we know the type of log, that could help us determine how persistent it needs to be, also.
20:57:30 <Rockyg> Some interesting thoughts.  But that said, we're near the end.
20:57:51 <Rockyg> #action nkrinner to look into tagging bugs.
20:58:06 <jokke_> yeah ... I've been thinking also, should we start log aggregator project to the OS or is there something doing that already
20:58:29 <nkrinner> i know someone working on a logstash implementation afaik. maybe i have some news next week
20:58:48 <jokke_> nkrinner: sounds good, keep us posted
20:58:51 <Rockyg> I found a launchpac query for log bugs from Tom Fifield: https://bugs.launchpad.net/openstack/+bugs?field.searchtext=&orderby=-importance&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&assignee_option=any&field.assignee=&field.bug_reporter=&field.
20:58:51 <Rockyg> bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=log&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on&search=Search
20:59:19 <Rockyg> It's supposed to find any bug tagged with "log" ?
20:59:19 <jokke_> ouch
20:59:34 <Rockyg> Yeah.  But, you work with whatch got.
20:59:42 <Rockyg> It's a start.
20:59:55 <Rockyg> so, Thanks everyone.  a working session is a good session.
21:00:00 <Rockyg> 3endmeeting
21:00:03 <jokke_> yeah, and it's time ... thanks both of you ... good discussion today
21:00:07 <Rockyg> #endmeeting