15:00:51 <witek> #startmeeting monasca
15:00:51 <openstack> Meeting started Wed Aug  7 15:00:51 2019 UTC and is due to finish in 60 minutes.  The chair is witek. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:53 <dougsz> hello all
15:00:55 <openstack> The meeting name has been set to 'monasca'
15:01:04 <witek> hi dougsz
15:01:49 <witek> the agenda as usual:
15:02:01 <witek> https://etherpad.openstack.org/p/monasca-team-meeting-agenda
15:02:28 <witek> #topic IPv6
15:02:50 <witek> not much from me to this one
15:03:05 <witek> just an update on gmann's work
15:03:21 <witek> here the list of current reviews:
15:03:28 <witek> https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+projects:openstack/monasca
15:04:08 <witek> I haven't look into this but we probably will have to change the way we configure Kafka
15:04:24 <witek> to be IPv6 compliant
15:04:28 <joadavis> do we need a new version of kafka, or is it just configuration?
15:04:44 <witek> configuration I assume
15:06:09 <gmann> yeah, seems still issue. I have not checked the recent log though
15:07:07 <gmann> kafka  seems configured with correct uri - https://logs.opendev.org/74/673274/6/check/monasca-tempest-influxdb-ipv6-only/0e53dc2/controller/logs/etc/monasca-api/monasca-api_conf.txt.gz
15:07:08 <witek> the action item is unassigned, it reminds me I should update our priorities list for that
15:09:34 <witek> gmann: we have a change in review upgrading Kafka and updating the configuration file
15:09:47 <witek> https://review.opendev.org/670914
15:10:24 <witek> but I understand it's probably a misconfiguration of clients, right?
15:11:11 <gmann> yeah, i am not seeing all logs there in recent run so cannot confirm exactly what failing
15:11:53 <gmann> may be other failure happening now.
15:12:08 <witek> #action witek Update Train priorities page with IPv6 goal
15:13:15 <witek> thanks gmann, I hope we can pick up on that work
15:13:25 <gmann> thanks witek .
15:13:48 <witek> #topic Confluent Kafka client [notification]
15:14:09 <witek> thanks a log for your reviews in the last week
15:14:44 <witek> I pushed new changes with the implementation in monasca-notification
15:15:19 <witek> https://review.opendev.org/674812
15:15:27 <witek> and https://review.opendev.org/674814
15:15:34 <dougsz> thanks a log :)
15:16:11 <witek> the keys are too close each other :)
15:17:05 <witek> when testing these changes I've noticed problems with kafka-python and newest broker versions
15:17:57 <witek> I think it's related with probing the Kafka API version and the very outdated Kafka client
15:18:17 <witek> anyone there are errors in Kafka log and notification engine crashes
15:18:49 <witek> the latest compatible Kafka version I found was 2.0.1
15:19:01 <witek> https://review.opendev.org/670914
15:19:30 <witek> another bad news is that this is not tested in CI
15:19:46 <witek> I mean notification engine
15:20:56 <witek> I think we should add it to our backlog to add some integration testing here
15:22:09 <witek> do you have any questions to these changes or anything related?
15:23:10 <witek> #topic Faust investigation
15:23:14 <dougsz> Nice work, it seems like a great time to be upgrading the Kafka client given the issues with Kafka 2.3 and other work going on to upgrade ELK
15:24:00 <witek> yes, it's been way too long in the backlog
15:25:21 <witek> for the Faust library investigation, I wanted to share the link to Sumit's repo with some initial code
15:26:33 <witek> one issue which he identified is, that there is no way to dynamically create Tables and Windows after the Application has been initialized
15:27:58 <witek> which means that in the running Faust Application we cannot add alarms with different evaluation periods
15:28:57 <witek> but I think we can work around this by starting new subprocesses
15:29:13 <witek> here is Sumit's repo:
15:29:20 <witek> https://github.com/sjamgade/pythresh-proto
15:29:49 <witek> #topic AOB
15:29:56 <dougsz> So a subprocess per alarm in the most extreme case?
15:30:32 <witek> I think we'd need subprocess per evaluation period
15:31:10 <witek> so, if all alarms have the same period, they'd fit in one Table
15:31:25 <dougsz> I see - that wouldn't be too bad
15:32:00 <witek> but yes, if every alarm would have a different evaluation period, we'll need as many subprocesses with own Tables
15:32:20 <witek> or we'd have to rethink the approach
15:33:13 <joadavis> is there a minimum granularity to the evaluation period?
15:34:00 <joadavis> could you just put all alarms into a table at the minimum period, but have the processing recognize when the per-alarm period wasn't met and just no-op for a cycle?
15:34:50 <joadavis> I may be misunderstanding hwo it works, if the table is used to collect data for just a period and would not carry over to another period
15:36:23 <witek> what we're using here are Windows, being views on the Table
15:36:30 <witek> https://faust.readthedocs.io/en/latest/userguide/tables.html#windowing
15:37:16 <witek> if the window is configured to fit our evaluation period, then we don't have to do much more
15:38:26 <witek> we probably could indeed build in some logic and evaluate alarms for other periods as well though
15:38:59 <dougsz> I forget - do we have any benchmark data yet?
15:39:19 <witek> for Faust? no
15:39:50 <dougsz> Just thinking it would be nice to check that it scales well early on
15:40:25 <witek> yes, good point
15:42:48 <witek> please take a moment to look at our board:
15:42:52 <witek> https://storyboard.openstack.org/#!/board/141
15:43:19 <witek> and add or move the cards to reflect the work you're active on
15:43:41 <witek> I think it can help us all with reviewing
15:44:36 <witek> do we have any other topics for today?
15:45:01 <joadavis> Congratulations to all who had sessions acepted to Shanghai
15:45:16 <witek> oh yes, forgot that one :)
15:45:27 <witek> congratulations joadavis !
15:45:44 <dougsz> yes, well done!
15:45:49 <joadavis> Well, now I have to actually come up with content...
15:46:20 <joadavis> I guess I'll be working card #1165 from our board :)
15:46:49 <dougsz> This is the CloudKitty + Monasca talk right?
15:47:10 <joadavis> https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/23932/transitioning-your-cloud-monitoring-from-ceilometer-to-monasca
15:47:22 <joadavis> quite controversial, I'm sure. ;)
15:48:02 <joadavis> I think Witek's other session will be interesting too - https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/23919/efficient-monitoring-and-root-cause-analysis-in-complex-systems
15:49:03 <witek> the workshop hasn't been accepted this time, right?
15:49:27 <joadavis> I haven't looked for workshops yet...
15:49:44 <joadavis> there are a couple of monitoring in kubernetes talks that may be interesting too
15:52:25 <witek> alright, thanks for joining today
15:52:41 <dougsz> thanks all
15:52:53 <joadavis> thanks, see you later
15:52:53 <witek> have a nice summer week
15:53:09 <witek> and see you next Wednesday
15:53:18 <witek> #endmeeting