14:00:42 <rhochmuth> #startmeeting monasca 14:00:43 <openstack> Meeting started Wed Aug 23 14:00:42 2017 UTC and is due to finish in 60 minutes. The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:46 <openstack> The meeting name has been set to 'monasca' 14:01:19 <rhochmuth> https://etherpad.openstack.org/p/monasca-team-meeting-agenda 14:01:27 <rhochmuth> Agenda for Wednesday August 23 2017 (14:00 UTC) 14:01:27 <rhochmuth> 1. Mid-cycle planning 14:01:27 <rhochmuth> • https://etherpad.openstack.org/p/monasca_queens_midcycle 14:01:27 <rhochmuth> 2. TSDB 14:01:27 <rhochmuth> • Cassandra update https://etherpad.openstack.org/p/cassandra_monasca 14:01:27 <rhochmuth> • InfluxDB + Kafka 14:01:27 <rhochmuth> • GridDB http://lists.openstack.org/pipermail/openstack-dev/2017-August/121296.html 14:01:34 <rhochmuth> o/ 14:01:39 <witek> hello 14:01:42 <cbellucci> o/ 14:01:42 <rhochmuth> anyone here? 14:01:42 <koji> o/ 14:01:44 <nseyvet_op5> hello 14:01:47 <shinya_kwbt> o/ 14:02:18 <rhochmuth> witek: do you want to drive the meeting? 14:02:47 <witek> we're still in Pike, but I can practice :) 14:03:22 <witek> #topic Mid-cycle planning 14:04:03 <witek> I guess I cannot set the topic, right? 14:04:17 <rhochmuth> interesting, 14:04:24 <rhochmuth> looks good to me 14:04:27 <rhochmuth> let me try 14:04:36 <rhochmuth> #topic Mid-cycle planning 14:04:51 <witek> you're the chair, I don't have rights 14:04:55 <rhochmuth> maybe since i started the meeting 14:05:04 <witek> yes 14:05:04 <rhochmuth> didn't realize that 14:05:29 <witek> I have set up the launchpad to start planning the mid-cycle meeting 14:05:40 <witek> https://etherpad.openstack.org/p/monasca_queens_midcycle 14:05:59 <witek> please put the topics you would like to discuss 14:06:16 <witek> the goal is to plan development for the next release cycle 14:06:43 <rhochmuth> should griddb be added as a topic? 14:07:00 <witek> don't know 14:07:42 <rhochmuth> events? 14:07:55 <witek> yes, we could add events 14:08:20 <akiraY> hi, all. 14:08:30 <witek> hi akiraY 14:08:31 <rhochmuth> grafana update. we've been making lot's of progress on that 14:08:44 <witek> cool, very interested 14:09:25 <rhochmuth> https://github.com/monasca/monasca-grafana 14:10:12 <witek> it does not cover authentication, right? 14:10:37 <rhochmuth> no, it is just the monasca-app that was developed for grafana 14:10:47 <rhochmuth> the one that steve simpson 14:11:13 <Neptu> o/ 14:11:23 <rhochmuth> started 14:12:08 <witek> we also have Grafana fork with authentication in sapcc repo 14:12:20 <rhochmuth> right 14:12:22 <witek> it is not maintained at the moment, right? 14:12:24 <rhochmuth> that is separate 14:12:28 <rhochmuth> right 14:13:51 <rhochmuth> we could also review the state of alarm silencing, inhibition and grouping 14:13:58 <rhochmuth> if that is interesting to folks 14:14:22 <witek> yes, good idea 14:15:30 <witek> we have retention policy appeared in etherpad 14:16:45 <witek> for the dates I proposed 20-21 Sept. 14-18 UTC 14:16:58 <witek> is it OK for everyone? 14:17:31 <jgu_> Yes could we talk about retention policy enhancement? For example, while keeping rolled up metrics longer, expire "regular" monitoring measurements at a shorter time? 14:17:54 <jgu_> Yes the time works for me. 14:18:06 <rhochmuth> i think that date works 14:18:31 <nseyvet_op5> what are plans to send all data to a data lake like HDFS for example? Part or not of Monasca? 14:19:00 <rhochmuth> it isn't part of monasca currently 14:19:13 <rhochmuth> i think it could be done with a kafka connector relatiely easily 14:19:40 <nseyvet_op5> yes, it could. just wondering since there is a data retention topic 14:22:07 <witek> OK, so please put your topics to the etherpad, we can sort them out later 14:22:31 <rhochmuth> thx witek 14:23:23 <rhochmuth> #topic Cassandra update 14:23:36 <rhochmuth> https://etherpad.openstack.org/p/cassandra_monasca 14:23:47 <rhochmuth> jgu_: any updates? 14:24:00 <rhochmuth> or areas that you would like to discuss? 14:24:10 <nseyvet_op5> as a newcomer, how do the result compare to Influx in general? 14:24:38 <rhochmuth> jgu_: that one is for u 14:25:55 <jgu_> Scott is rerunning the test with replication factor of 1 so we can compare with influxdb, apple to apple. I added a performance testing data section to the ether pad. 14:26:23 <rhochmuth> https://etherpad.openstack.org/p/cassandra_monasca 14:26:39 <jgu_> Thanks Roland 14:27:01 <witek> so the worst result is for querying metric definitions by metric name and dimensions 14:27:14 <nseyvet_op5> ok. using the replication factor of 2 (as in the doc), how does it look? 14:27:36 <witek> 0,13 s 14:28:01 <nseyvet_op5> as opposed to how much in Influx? 14:28:40 <jgu_> Witek:yes, that actually uses allow filtering 14:30:09 <witek> after querying metric definitions you still have to query measurements by id, right? 14:30:39 <jgu_> I am implementing and testing the java driver with dev stack deployment at the moment. Will run another performance testing when fully integrated in monasca end to end 14:32:00 <witek> jgu_: after querying metric definitions you still have to query measurements by id, right? 14:32:04 <jgu_> Witek: for measurement retrieval, need up to two queries 14:32:49 <witek> so the total time should be < 0,2 s ? 14:33:13 <witek> does it meet your requirements? 14:34:54 <jgu_> Depends on the GUII workflow, I am wondering if we should add a query measurement by id rest API, if hi workflow let's the user to choose one or multiple metrics first? 14:36:03 <rhochmuth> jgu_: yes, that is something that we proposed too 14:36:11 <nseyvet_op5_> Based on the Cassandra partitioning of the data, how confident are you that the measurements are going to be remain realistic when the Cassandra cluster size increases? 14:36:23 <rhochmuth> since the lookup fomr (name, dimensions) to ID is the bottleneck 14:36:33 <nseyvet_op5_> precisely 14:38:32 <jgu_> Roland: that'd be pretty cool. The Cassandra model query for the look up is pretty reasonable , but combing two queries in one API is doable but a burden 14:40:38 <nseyvet_op5> (sry pb w network) 14:40:54 <witek> jgu_, rhochmuth: how would it work for InfluxDB? 14:41:52 <rhochmuth> for influxdb it would not work 14:42:04 <rhochmuth> unless you stored the ID with the measurement i beleive 14:42:24 <rhochmuth> for example, add another tag in influxdb with the ID 14:44:55 <rhochmuth> should i change the topic? 14:45:21 <rhochmuth> #topic GridDB 14:45:31 <akiraY> :) 14:45:31 <rhochmuth> http://lists.openstack.org/pipermail/openstack-dev/2017-August/121296.html 14:45:45 <rhochmuth> hi akiraY 14:45:55 <witek> nseyvet_op5: for the InfluxDB <-> Cassandra comparison we are more interested on write perf 14:46:11 <akiraY> hi rhochimuth 14:46:44 <akiraY> thank you for reviews 14:47:37 <akiraY> I'll send a new patchset tomorrow 14:48:16 <witek> there is not much information about GridDB in internet 14:48:28 <akiraY> yes. 14:49:10 <witek> I wanted to ask folks what they think, what approach should we take 14:49:28 <akiraY> it was open sourced last year. 14:49:44 <witek> experiment with such new databases, or rather conentrate on some more or less proven solution? 14:49:54 <rhochmuth> the AGPL licensing is a problem for HPE from what I recall 14:50:25 <nseyvet_op5> Does adding support for it in Monasca requires Monasca to commit and support it? 14:50:26 <rhochmuth> is there any possiblitty of making GridDB an Apache 2 license 14:50:45 <jgu> AGPL is a tricky issue 14:50:57 <rhochmuth> i know our legal team/attorneys don't like AGPL 14:51:15 <witek> nseyvet_op5: it means at least reviewing 14:51:16 <rhochmuth> mongodb also had or still has AGPL, and as a result, we can't use it 14:51:19 <nseyvet_op5> I c. It might affect Monasca as a whole. 14:51:19 <akiraY> python binding is under apache license v2. 14:51:29 <witek> nseyvet_op5: testing should also be implemented 14:51:40 <jgu> akiraY: do you have performance testing data? 14:51:48 <akiraY> yes 14:51:57 <rhochmuth> with monasca? 14:52:31 <akiraY> oh, I have no performance data. 14:54:34 <akiraY> I should make a new deployment for benchmarks, but it will take a while. 14:55:13 <akiraY> maybe a month or more 14:55:29 <rhochmuth> if you could that compared to a similar config that jgu_ used for cassandra that would be good 14:56:02 <nseyvet_op5> +1 on that 14:56:04 <akiraY> can I have its detail? 14:56:35 <jgu> https://etherpad.openstack.org/p/cassandra_monasca 14:56:44 <akiraY> thanks 14:56:58 <rhochmuth> • Minimum 3 baremetal nodes 14:56:59 <rhochmuth> • each node: 14:56:59 <rhochmuth> • 256 GB RAM minimum 14:56:59 <rhochmuth> • Two SSDS: commit log disk 256GB; data disk: 2TB 14:56:59 <rhochmuth> • Dedicated VLAN for the replication data link, 10 gb 14:57:19 <akiraY> 256GB... 14:57:20 <rhochmuth> that is listed in the etherpad jgu_ added above 14:57:24 <jgu> yes 14:57:30 <witek> I guess it's not the test setup, jgu right? 14:57:48 <akiraY> hmm.. that's a big problem 14:58:04 <jgu> that's for the production, benchmark for 50 bil metrics and 20 mil unique definitions 14:58:15 <jgu> 200 mil 14:58:42 <akiraY> sorry, I have no such environment. 14:59:09 <rhochmuth> hi folks, i'm going to have to close the meeting 14:59:20 <rhochmuth> witek, any closing comments? 14:59:39 <witek> no, thank you 15:00:04 <rhochmuth> thanks everyone, interesting discussions today 15:00:15 <rhochmuth> will see you all next week 15:00:23 <akiraY> see you. 15:00:28 <rhochmuth> #endmeeting