15:00:32 <witek> #startmeeting monasca 15:00:33 <openstack> Meeting started Wed Oct 17 15:00:32 2018 UTC and is due to finish in 60 minutes. The chair is witek. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:37 <openstack> The meeting name has been set to 'monasca' 15:00:44 <annp_> Hi witek 15:00:49 <witek> Hello everyone 15:00:57 <witek> hi annp_ 15:01:19 <joadavis> hi 15:01:36 <annp_> witek, Do you remember me? 15:01:55 <annp_> :-) 15:02:01 <witek> I guess not from the nickname 15:03:07 <witek> would you introduce yourself? 15:03:25 <annp_> witek, Yes, I'm from fujitsu vietnam. We've had a presentation at Boston summit about logging for security group :-) 15:03:51 <witek> now I know :) 15:04:00 <annp_> :-) 15:04:13 <witek> thanks for joining, let's start with the agenda 15:04:19 <witek> https://etherpad.openstack.org/p/monasca-team-meeting-agenda 15:04:24 <annp_> Thanks please go ahead. 15:04:42 <witek> #topic Merge Monasca APIs 15:05:01 <witek> we have started the discussion last week 15:05:11 <witek> https://review.openstack.org/609055 15:05:30 <witek> dougsz: are you around? 15:05:36 <pandy> witek, would like to know what are the updates in monasca-api ? 15:05:49 <dougsz> hey all - sorry, got distracted 15:06:32 <witek> the spec looks quite good 15:07:55 <witek> dougsz: you had concerns if that's worth the effort, right? 15:08:28 <dougsz> I think it will be hard for us to find the time to push it through alone 15:08:45 <witek> for us, meaning StackHPC? 15:09:28 <dougsz> yeah 15:10:12 <witek> I've talked with the team and I guess we could help 15:10:56 <witek> with the final goal to add `GET /logs` 15:11:07 <dougsz> Hmm, that would be nice :) 15:11:23 <dougsz> I think I will need to talk through how much time I can commit to it 15:11:59 <dougsz> But it would be nice to get all the requirements on the spec so we have a good idea of exactly how big it will be 15:13:02 <witek> yes, like listing the functionality which we actually want to merge 15:13:57 <dougsz> I will try to add some more notes on it this week 15:15:41 <witek> thanks, let's keep active exchange on the spec review 15:16:15 <witek> anything else on this? 15:16:40 <dougsz> Thanks - that's it from me. 15:16:52 <witek> #topic Ceilometer events deprecation 15:17:27 <witek> joadavis: has reported about it last week on #openstack-monasca and in review 15:17:31 <joadavis> So I just recently noticed that Ceilometer has marked the events functionality as deprecated on master 15:18:34 <joadavis> After chatting about it with aagate, we split out a separate spec for a different approach to gathering metrics 15:19:00 <joadavis> Ceilometer deprecating metrics is good and bad for us - 15:19:16 <joadavis> good in that it gives us a chance to be the one place for event monitoring 15:19:30 <joadavis> bad in that long term we can't just use the Ceilometer functions as we had planned 15:20:06 <joadavis> We could still use ceilometer for a few releases. It is only marked as deprecated in Stein so should be available through T 15:20:39 <witek> personally, I'd prefer to go for long-term solution 15:20:43 <joadavis> Thanks for the review feedback on the spec so far 15:21:08 <joadavis> yes, if we are going to spend effort on event gathering we should spend it on the long term solution 15:21:55 <witek> and that means from me, not relying on events collection via Ceilometer 15:22:03 <joadavis> Based on feedback so far, I think another revision of the "events listener" spec will be coming 15:22:45 <witek> let's put the link to review here: 15:22:47 <witek> https://review.openstack.org/583803 15:22:53 <joadavis> I think we may be able to reference the Ceilometer events processing for a pattern to follow, but having independent code will be better 15:24:09 <joadavis> Comments are welcome. 15:24:21 <witek> one idea which is listed in `alternatives` is to overcome the Events API and forward the notifications to Kafka directly 15:24:46 <witek> what do you guys think of it? 15:25:02 <dougsz> Can you do that directly via Oslo now? 15:25:27 <joadavis> I think it would be a more efficient approach. I don't know much about Oslo features yet. 15:25:51 <dougsz> I remember it coming up in Dublin 15:25:58 <witek> https://docs.openstack.org/oslo.messaging/latest/reference/notification_listener.html 15:27:39 <witek> in SUSE Monasca node communicates with RabbitMQ on control node via admin network 15:27:53 <witek> so RabbitMQ is reachable from Monasca 15:28:15 <witek> how is it in e.g. in Kolla? 15:28:20 <joadavis> and it would be good to reuse the Oslo messaging if possible 15:29:16 <dougsz> https://specs.openstack.org/openstack/oslo-specs/specs/queens/update-kafka-support.html 15:30:23 <joadavis> so we may just need a service that listens on rabbit, does some filtering and transformation, then posts on to kafka 15:30:40 <dougsz> witek: No communication between Rabbit and Monasca in Kolla yet, unless you could the Agent plugin 15:30:47 <dougsz> s/could/count 15:31:07 <joadavis> we can configure which events are captured in a configuration file 15:31:28 <witek> dougsz: but is RabbitMQ reachable from Monasca node? 15:31:42 <joadavis> and we will still need a Monasca events api for retrieval and other uses 15:31:43 <dougsz> Yeah, it should be. 15:31:58 <dougsz> I see there is a RabbitMQ input plugin for Logstash 15:33:30 <joadavis> cool 15:33:54 <witek> yes, I've thought about Logstash as well, but I guess there could some reluctance in OpenStack community to use it 15:35:24 <dougsz> It's not particularly operator friendly, although I suppose the config file would need less frequent updating than the log-metrics service. 15:35:26 <joadavis> does Kolla run all the Monasca services in the same container? I suppose if we had different network access requirements for events we could have a separate container 15:35:52 <dougsz> They're all in separate containers, something like 20 in total! 15:36:02 <joadavis> :) 15:36:42 <witek> chaconpiza: asks in review, if we need authentication 15:38:01 <witek> I guess we don't; other opinions? 15:38:05 <joadavis> we always need some level of authentication. I assume we need username/password for rabbitmq. 15:38:33 <dougsz> The use case for me that comes to mind would be tenants using the Event API for monitoring their application 15:39:30 <witek> would they also send events via RabbitMQ, or should we have API for them? 15:39:32 <joadavis> The monasca Events API would still need authentication for a user to retrieve data 15:40:01 <dougsz> In an ideal world, the events would be sent via the Monasca Events API 15:40:28 <joadavis> hmm, for an application... we could explore some options there 15:40:40 <witek> dougsz: for tenants events, or OpenStack infra, or both 15:40:59 <truongnh_> ping annp 15:41:03 <dougsz> witek: Primarily for tenants 15:41:09 <witek> agree 15:42:06 <joadavis> Are you talking about custom events generated by an application that a tenant would want to report on, or are you referencing OpenStack service generated events that a tenant would want to know as they may effect the health of their application? 15:42:37 <witek> custom events 15:42:39 <dougsz> yeah 15:43:14 <witek> OK, please let's stay on ball with the spec review 15:43:24 <witek> I think we have some ideas how to proceed 15:43:44 <witek> anything else for now? 15:43:59 <joadavis> yes, we can work on the spec and continue to discuss if an application event needs to have a keystone token or a static token from config or no auth 15:44:20 <witek> thanks joadavis 15:44:31 <witek> #topic Monasca datasource in Vitrage 15:44:32 <joadavis> I will make another revision to the events listener spec, so keep sending any thoughts and feedback 15:44:48 <witek> short update on Vitrage 15:45:14 <witek> I have joined their remote PTG meeting last week 15:45:46 <witek> we have talked about integrating the projects and adding Monasca datasource to Vitrage 15:46:06 <witek> unfortunately they won't have resources to work on this 15:46:18 <witek> but we discussed what have to be done 15:46:54 <witek> Ifat created stories 15:47:00 <witek> https://storyboard.openstack.org/#!/story/2004064 15:47:29 <witek> they have pretty good documentation about how to add new datasources 15:47:59 <witek> they have also added Prometheus datasource in the last cycle which works with webhook notification 15:48:14 <witek> so we could follow the same implementation pattern 15:48:39 <witek> I have added this story to our Board 15:48:46 <witek> https://storyboard.openstack.org/#!/board/111 15:48:50 <witek> to backlog 15:49:06 <pandy> witek, we are using vitrage, it's great to know monasca integration with vitrage datasource : 15:49:09 <witek> so if anyone has some time, it would be a nice task 15:49:45 <witek> pandy: we need some to implement Monasca datasource in Vitrage 15:50:09 <witek> s/some/someone 15:50:31 <pandy> sure, i can try to help out as implementation point of view :) being having good exposure with vitrage 15:50:50 <witek> that would be fantastic 15:51:19 <witek> as I said, I've added the story to backlog 15:51:33 <pandy> yeah, saw that 15:51:41 <witek> also, if you need more details you can contact me or Ifat from Vitrage 15:51:59 <pandy> sure 15:52:29 <witek> I think it's potentially a good topic for Summit presentation 15:52:52 <dougsz> I'd vote for it 15:53:00 <pandy> common irc channel or way to contact both together ? if Ifat joins monasca channel will be great, else please share me channel and irc name 15:53:48 <pandy> guys, would like to share in india opensource event happened. Where monasca & kolla projects were highlighted 15:54:00 <dougsz> looks like #openstack-vitrage and ifat_afek 15:54:39 <witek> pandy: go ahead 15:54:47 <pandy> thankd dougsz for sharing his details :), witek let me know to discuss 3 node setup 15:55:22 <witek> #topic 3 nodes setup 15:55:35 <pandy> To all shortly, we are handling 1200+ nodes, where currently monasca was installed on 3 node setup 15:56:01 <pandy> challenge is with handling monasca components in particular kafka 15:56:18 <pandy> i have deployed through docker & using docker swarm to handle it 15:56:42 <pandy> would like to know whether monasca-api stores broker_ID any where ? 15:58:00 <pandy> I have used third party kafka-cluster setup to run on docker swarm, which is running fine for other application though i do reboot. But with monasca mainly monasca-api throwing error that not able to kafka and broken 15:58:37 <dougsz> Which version of Kafka are you using in your 3rd party setup? 15:59:03 <pandy> if we have these supported environment variables for monasca/kafka images http://paste.openstack.org/show/732344/ will be great 15:59:20 <pandy> dougsz, am using https://github.com/wurstmeister/kafka-docker 16:00:03 <dougsz> It looks very recent, you'll need to set the messaging version 16:00:41 <dougsz> https://review.openstack.org/#/c/605751/13/doc/source/reference/monasca-guide.rst 16:00:44 <dougsz> ^ See line 55 16:01:02 <witek> I'm sorry, have to wrap up 16:01:06 <pandy> witek suggested to use deploy kafka on each node and point it to KAFKA_URI: node:9092, node2:9092 like that. it will work. But in docker swarm one stack should run fine, which may helps to scale up in future to handle large requests like 1200 nodes datas 16:01:16 <dougsz> thanks all, bye for now 16:01:27 <witek> see you next week and let's continue the work on reviews 16:01:36 <witek> #endmeeting