15:00:51 #startmeeting monasca 15:00:51 Meeting started Wed Aug 7 15:00:51 2019 UTC and is due to finish in 60 minutes. The chair is witek. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:52 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:53 hello all 15:00:55 The meeting name has been set to 'monasca' 15:01:04 hi dougsz 15:01:49 the agenda as usual: 15:02:01 https://etherpad.openstack.org/p/monasca-team-meeting-agenda 15:02:28 #topic IPv6 15:02:50 not much from me to this one 15:03:05 just an update on gmann's work 15:03:21 here the list of current reviews: 15:03:28 https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+projects:openstack/monasca 15:04:08 I haven't look into this but we probably will have to change the way we configure Kafka 15:04:24 to be IPv6 compliant 15:04:28 do we need a new version of kafka, or is it just configuration? 15:04:44 configuration I assume 15:06:09 yeah, seems still issue. I have not checked the recent log though 15:07:07 kafka seems configured with correct uri - https://logs.opendev.org/74/673274/6/check/monasca-tempest-influxdb-ipv6-only/0e53dc2/controller/logs/etc/monasca-api/monasca-api_conf.txt.gz 15:07:08 the action item is unassigned, it reminds me I should update our priorities list for that 15:09:34 gmann: we have a change in review upgrading Kafka and updating the configuration file 15:09:47 https://review.opendev.org/670914 15:10:24 but I understand it's probably a misconfiguration of clients, right? 15:11:11 yeah, i am not seeing all logs there in recent run so cannot confirm exactly what failing 15:11:53 may be other failure happening now. 15:12:08 #action witek Update Train priorities page with IPv6 goal 15:13:15 thanks gmann, I hope we can pick up on that work 15:13:25 thanks witek . 15:13:48 #topic Confluent Kafka client [notification] 15:14:09 thanks a log for your reviews in the last week 15:14:44 I pushed new changes with the implementation in monasca-notification 15:15:19 https://review.opendev.org/674812 15:15:27 and https://review.opendev.org/674814 15:15:34 thanks a log :) 15:16:11 the keys are too close each other :) 15:17:05 when testing these changes I've noticed problems with kafka-python and newest broker versions 15:17:57 I think it's related with probing the Kafka API version and the very outdated Kafka client 15:18:17 anyone there are errors in Kafka log and notification engine crashes 15:18:49 the latest compatible Kafka version I found was 2.0.1 15:19:01 https://review.opendev.org/670914 15:19:30 another bad news is that this is not tested in CI 15:19:46 I mean notification engine 15:20:56 I think we should add it to our backlog to add some integration testing here 15:22:09 do you have any questions to these changes or anything related? 15:23:10 #topic Faust investigation 15:23:14 Nice work, it seems like a great time to be upgrading the Kafka client given the issues with Kafka 2.3 and other work going on to upgrade ELK 15:24:00 yes, it's been way too long in the backlog 15:25:21 for the Faust library investigation, I wanted to share the link to Sumit's repo with some initial code 15:26:33 one issue which he identified is, that there is no way to dynamically create Tables and Windows after the Application has been initialized 15:27:58 which means that in the running Faust Application we cannot add alarms with different evaluation periods 15:28:57 but I think we can work around this by starting new subprocesses 15:29:13 here is Sumit's repo: 15:29:20 https://github.com/sjamgade/pythresh-proto 15:29:49 #topic AOB 15:29:56 So a subprocess per alarm in the most extreme case? 15:30:32 I think we'd need subprocess per evaluation period 15:31:10 so, if all alarms have the same period, they'd fit in one Table 15:31:25 I see - that wouldn't be too bad 15:32:00 but yes, if every alarm would have a different evaluation period, we'll need as many subprocesses with own Tables 15:32:20 or we'd have to rethink the approach 15:33:13 is there a minimum granularity to the evaluation period? 15:34:00 could you just put all alarms into a table at the minimum period, but have the processing recognize when the per-alarm period wasn't met and just no-op for a cycle? 15:34:50 I may be misunderstanding hwo it works, if the table is used to collect data for just a period and would not carry over to another period 15:36:23 what we're using here are Windows, being views on the Table 15:36:30 https://faust.readthedocs.io/en/latest/userguide/tables.html#windowing 15:37:16 if the window is configured to fit our evaluation period, then we don't have to do much more 15:38:26 we probably could indeed build in some logic and evaluate alarms for other periods as well though 15:38:59 I forget - do we have any benchmark data yet? 15:39:19 for Faust? no 15:39:50 Just thinking it would be nice to check that it scales well early on 15:40:25 yes, good point 15:42:48 please take a moment to look at our board: 15:42:52 https://storyboard.openstack.org/#!/board/141 15:43:19 and add or move the cards to reflect the work you're active on 15:43:41 I think it can help us all with reviewing 15:44:36 do we have any other topics for today? 15:45:01 Congratulations to all who had sessions acepted to Shanghai 15:45:16 oh yes, forgot that one :) 15:45:27 congratulations joadavis ! 15:45:44 yes, well done! 15:45:49 Well, now I have to actually come up with content... 15:46:20 I guess I'll be working card #1165 from our board :) 15:46:49 This is the CloudKitty + Monasca talk right? 15:47:10 https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/23932/transitioning-your-cloud-monitoring-from-ceilometer-to-monasca 15:47:22 quite controversial, I'm sure. ;) 15:48:02 I think Witek's other session will be interesting too - https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/23919/efficient-monitoring-and-root-cause-analysis-in-complex-systems 15:49:03 the workshop hasn't been accepted this time, right? 15:49:27 I haven't looked for workshops yet... 15:49:44 there are a couple of monitoring in kubernetes talks that may be interesting too 15:52:25 alright, thanks for joining today 15:52:41 thanks all 15:52:53 thanks, see you later 15:52:53 have a nice summer week 15:53:09 and see you next Wednesday 15:53:18 #endmeeting