15:00:29 <witek> #startmeeting monasca
15:00:30 <openstack> Meeting started Wed Nov 21 15:00:29 2018 UTC and is due to finish in 60 minutes.  The chair is witek. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:33 <openstack> The meeting name has been set to 'monasca'
15:00:45 <witek> Hello everyone
15:00:49 <koji_n> hi
15:01:05 <witek> hi koji_n
15:01:24 <Fdaisuke_> hello
15:01:29 <dougsz> Hey all
15:01:37 <witek> nice to see you all
15:01:57 <witek> the agenda in the usual place:
15:02:01 <witek> https://etherpad.openstack.org/p/monasca-team-meeting-agenda
15:02:20 <witek> let's start with the quick Summit recap
15:02:32 <witek> #topic Berlin Summit recap
15:03:00 <witek> we had couple presentations in Berlin
15:03:21 <witek> Project Update
15:03:24 <witek> https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22869/monasca-openstack-monitoring
15:03:40 <witek> unfortunately still no videos available
15:04:12 <witek> I have listed our work around events and logs as the focus in Stein release
15:04:47 <witek> the interest was quite big
15:05:09 <witek> 2. Project Onboarding
15:05:26 <witek> http://btw23.de/johannes/talks/monasca-onboarding.tar.bz2
15:05:52 <witek> then we had a presentation from StackHPC
15:06:02 <witek> 3. Monitoring as a Service in the HPC Cloud
15:06:09 <witek> https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22446/monitoring-as-a-service-in-the-hpc-cloud
15:06:23 <witek> very good informative presentation
15:07:04 <witek> presenting Monasca deployed with Kolla Ansible
15:07:43 <joadavis> good to hear the interest in monasca was high
15:07:58 <witek> 4. Monitoring Hands-On Workshop
15:08:03 <witek> https://github.com/martinchacon/monasca-bootcamp
15:08:13 <witek> that one was really packed!
15:08:39 <witek> we had almost 100 people doing hands-on excercises with Monasca
15:08:51 <joadavis> wow
15:08:55 <dougsz> Great job from Martin on that one
15:09:53 <witek> after both, Project Update and Hands-On there were questions about Ceilometer
15:10:08 <witek> if Monasca can/will replace it?
15:10:34 <witek> also, there were questions about deployment methods for the agent
15:11:08 <witek> I have pointed to the monasca-ceilometer plugin
15:11:46 <witek> another very interesting presentation was from Red Hat
15:11:58 <witek> https://www.openstack.org/assets/presentation-media/OpenStack-Summit-2018-Prometheus-Operator-to-monitor-OpenStack.pdf
15:13:00 <witek> they have admitted that Gnocchi is not a viable solution for monitoring at scale
15:13:26 <joadavis> facinating. Scale was one of the main points of Gnocchi
15:13:27 <witek> and proposed an architecture similar to Monasa's
15:14:05 <witek> with Collectd, message bus and Prometheus
15:14:22 <witek> everything deployed on OpenShift
15:15:35 <dougsz> Quite the stack
15:16:43 <witek> in general, my conclusion from the summit is that we should invest some more effort in the agent, to make it more accessible and useful for users
15:17:11 <witek> we should define `core` plugins and support them better, add some testing
15:17:36 <witek> what has come up in the PTG already, we should adevertise our support for Prometheus
15:17:55 <dougsz> I think there are also some issues with some of the detection plugins not working well from inside a container
15:18:31 <witek> yes, we have seen some as well, some fixed, some not
15:19:01 <witek> we should also think about making official packages for the agent
15:19:10 <witek> for Debian/Ubuntu and CentOS
15:20:17 <dougsz> Official Ansible playbook could be nice as well
15:20:39 <witek> have to check if there is one in OSA
15:21:03 <witek> #action check for official Ansible playbook
15:21:21 <joadavis> There is https://docs.openstack.org/openstack-ansible-os_monasca/latest/ but I'm not sure if it is up to date/usable
15:21:34 <dougsz> We have forked one and got it working with Ubuntu/Centos/SuSE
15:22:06 <dougsz> As you say it would be nice to have one under the Monasca namespace
15:22:10 <witek> joadavis: that's for all Monasca, should be functional
15:22:19 <joadavis> long ago we talked internally about contributing changes from Ardana to OSA, but haven't put time in to it
15:23:16 <witek> dougsz: good to hear, perhaps we should move it to OSA?
15:23:26 <witek> can you share the repo?
15:23:56 <dougsz> https://github.com/stackhpc/ansible-role-monasca-agent
15:24:03 <witek> thanks
15:24:35 <witek> another topic which pulled some attention was auto-scaling
15:24:43 <joadavis> https://github.com/ArdanaCLM/monasca-ansible is part of it. There are also repos for monasca-transform-ansible and spark-ansible
15:25:26 <dougsz> 👍
15:26:02 <witek> from what I heard Monasca role in OSA should be working fine
15:26:13 <witek> but does not has logging and monasca-transform
15:26:26 <joadavis> or cassandra support from what I see
15:26:31 <witek> correct
15:27:12 <witek> we could ping the guys if they're interested adding support for these
15:27:51 <witek> anyway, we've side-tracked a little
15:27:59 <joadavis> and add monasca-ceilometer to the list. :)
15:28:20 <witek> I think Ansible is not providing that kind of comfort as distribution packages
15:29:12 <witek> and I think most operators expect to install it from the package
15:29:40 <dougsz> I think we have a slightly different perspective. Galaxy roles have worked well for us.
15:29:56 <witek> agree for your case
15:30:22 <dougsz> Whatever official solution there is, it would be nice to import it to Kolla Ansible. Packages would work, or Galaxy role. Not sure about importing OSA components.
15:30:40 <dougsz> Anyway, sorry for sidetracking!
15:31:40 <witek> another think I wanted to mention is auto-scaling discussions in Forum
15:32:00 <witek> they think about creating a new SIG for auto-scaling
15:32:20 <witek> to unify approaches, identify problems, collect documentation
15:32:38 <joadavis> so a separate SIG from self-healing?
15:32:43 <witek> yes
15:32:59 <witek> ricolin from Heat drives it
15:33:14 <witek> https://etherpad.openstack.org/p/autoscaling-integration-and-feedback
15:33:46 <witek> self-healing had a meeting of course too:
15:33:55 <witek> https://etherpad.openstack.org/p/berlin-self-healing-sig-brainstorm
15:35:07 <witek> another thought after Summit, we should put more attention to upgrading the middleware regularly
15:35:21 <witek> in particular our ELK stack is very outdated
15:35:50 <witek> dougsz: any comments about the summit from you?
15:36:13 <dougsz> Was great to see so many people turn up to the Monasca meetings
15:36:35 <witek> yes, that was indeed very positive
15:36:59 <dougsz> I think you covered it well
15:37:12 <witek> we also had a chat with Fdaisuke_ about plans of monasca-analytics
15:37:47 <Fdaisuke_> yes
15:38:28 <witek> I asked for more work on monitoring related use-cases, like anomaly prediction or forecasting
15:39:21 <Fdaisuke_> We talked about monasca-analytics as follows
15:39:34 <Fdaisuke_> - Update top page of monasca-analytics
15:40:00 <Fdaisuke_> - Submit sample with monasca
15:41:32 <Fdaisuke_> And now, we update existing functions, and add new functions
15:41:41 <Fdaisuke_> for monasca-analytics
15:42:08 <witek> thanks, yes, I would like to have some simple example use case, how the repo can be used with the rest of Monasca
15:42:24 <witek> with easy step-by-step procedure
15:43:13 <witek> oops, I wanted it to be a *short* recap
15:43:26 <joadavis> :)
15:43:35 <joadavis> I'm looking forward to the videos
15:43:40 <witek> if there are no other comments, we can move on
15:44:03 <witek> #topic specs
15:44:50 <witek> https://review.openstack.org/583803
15:45:11 <witek> I have added some small changes and comments yesterday
15:45:45 <joadavis> thanks
15:45:54 <joadavis> I'll take a look
15:46:07 <witek> one question is, should we store all events in one ES index rotated daily?
15:46:17 <witek> for admin project
15:47:34 <witek> https://review.openstack.org/609055
15:47:39 <joadavis> I haven't looked at that side of events yet. And I'm not an ES expert. Anyone else have an opinion?
15:47:45 <dougsz> What are the other options? Classify events and save those to separate indices?
15:47:58 <dougsz> I suppose we still need to rotate them in that case.
15:48:03 <witek> for logs we store them per tenant
15:48:37 <witek> Kibana plugin filters the indices for a given project
15:48:53 <witek> so you cannot see other projects' data
15:49:29 <witek> for events, we could do similarly and spread events across projects
15:49:40 <witek> there are some problems with that approach
15:50:11 <dougsz> Yes - are all events associated with a project id?
15:50:11 <witek> operator(admin) would have to change the index for every project
15:50:42 <witek> dougsz: that's another problem, not all events include project information
15:50:43 <joadavis> some events, like keystone creating a user, are not associated with a project
15:51:59 <dougsz> Hmm. It does seem easier to make them all admin only, at least to begin with.
15:52:36 <dougsz> We could always have something which selectively copies out events to a tenants project if required in the future.
15:52:54 <witek> folks, please think through and leave the comment in review
15:53:16 <dougsz> 👍
15:53:24 <witek> for merging APIs https://review.openstack.org/609055
15:53:37 <witek> someone from our team will polish it this week
15:53:50 <witek> and I think is ready to merge
15:54:15 <witek> there are two others:
15:54:28 <witek> Metrics retention policy enhancement
15:54:33 <witek> https://review.openstack.org/547233
15:54:45 <witek> Spec for Performance metrics of Python Persister
15:54:46 <witek> https://review.openstack.org/605782
15:55:44 <joadavis> I think both of those are ready to merge. We may have more details to add to metrics retention policy, but can add those in a subsequent commit
15:56:16 <witek> OK, I'll try to find time to review
15:56:29 <joadavis> thanks
15:56:34 <witek> #topic self-healing meeting
15:56:43 <witek> one more announcement
15:56:57 <witek> Self-healing SIG has now a regular meeting
15:57:03 <witek> every second Wed
15:57:09 <witek> http://eavesdrop.openstack.org/#Self-healing_SIG_Meeting
15:57:29 <witek> at 9am and 5pm UTC
15:57:39 <witek> the second one in one hour
15:57:53 <witek> I have attended the morning one
15:58:27 <witek> and advertised that we're looking for developers willing to work on Monasca - Vitrage integration
15:58:40 <joadavis> cool
15:59:10 <witek> that's all from me
15:59:18 <witek> the time is also over
15:59:24 <dougsz> thanks all,  bye
15:59:30 <witek> thank you everyone for joining
15:59:37 <koji_n> thx!
15:59:38 <witek> see you next time
15:59:47 <witek> #endmeeting