15:00:27 #startmeeting monasca 15:00:28 Meeting started Wed Mar 16 15:00:27 2016 UTC and is due to finish in 60 minutes. The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:29 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:32 The meeting name has been set to 'monasca' 15:00:32 o/ 15:00:39 hello 15:00:41 o/ 15:00:41 o/ 15:00:43 o/ 15:01:28 i'm sort of out this week on spring break, and haven't been checking up on things 15:01:45 o/ 15:01:52 looks like there are several agenda items at, https://etherpad.openstack.org/p/monasca-team-meeting-agenda 15:01:58 Agenda for Wednesday March 16, 2016 (15:00 UTC) 15:01:58 1. Periodic metrics support 15:01:58 1. https://review.openstack.org/292753 15:01:58 2. https://review.openstack.org/292758 15:01:58 2. monasca-log-api specifiction 15:01:59 1. https://review.openstack.org/273058 15:01:59 3. vertica/api reduction of inner joins patch 15:02:00 3.1 https://review.openstack.org/#/c/287507/ 15:02:46 So, i'll go though the list 15:02:56 #topic period metrics support 15:03:17 i'm assuming that is tomasz 15:03:20 Tomasz has started working on periodic metrics 15:03:46 yes, i see 15:03:55 looks like ben is the only reviewer so far 15:03:57 he doesn't have much experience with monasca-thresh and storm 15:04:18 hi 15:04:26 hi fabiog 15:04:36 so, i think gettign some eyes on that 15:04:39 if someone could take a look, we would be very thankfull 15:04:40 i wont' review this week 15:04:46 but, i'll get back to it next week 15:04:56 rhochmuth: thanks 15:04:57 also our expert in this area craig is alos out htis week 15:05:01 on spring break too 15:05:16 but when he returns next hopefully he can review too 15:05:25 great 15:05:35 unfortuanately, he is a slacker and doesn't show up to meetings when he is on break 15:05:39 :-) 15:05:43 haha 15:06:06 either that or he is smarter than me 15:06:37 no comment 15:07:07 i guess it isn't a lot of code to make those changes 15:07:14 so, the reviews should go prety quickly 15:07:58 #topic monasca-log-api specifiction 15:08:19 So, it looks like we are trying to wrap-up on, https://review.openstack.org/#/c/273058/ 15:08:28 I think is good to go 15:08:40 looks like there are several +1s and 2s, so agree 15:08:44 rhochmuth: press the +1 workflow ;-) 15:08:48 :) 15:09:10 so, then i'm assuming tsv will wrap-up the implementation 15:09:17 i think he was the last one in there 15:10:05 his review is in merge conflict right now, but if we fixes that up, then it should get merged soon 15:10:21 as well as making sure the implementation is correct 15:11:01 i'm going to let witold merge it 15:11:15 you mean implementation? 15:11:47 actually, i meant the spec, but since you started the spec that wouldn't be best 15:11:50 i can +2 it 15:12:01 tsv still needs to wrap the implementation 15:12:07 assiming he is going to do that 15:13:00 i can merge the spec if we agree so here 15:13:05 i agree 15:13:29 one of these days i'll figure out how to do votes in irc 15:13:56 #topic vertica/api reduction of inner joins patch 15:14:02 that's me 15:14:18 yes, i'm assuming you want to get this merged 15:14:20 i think it's ready -- anyone disagree? 15:14:21 i haven't been able to test 15:14:23 yep 15:14:25 but the code looks fine 15:14:35 rbrandt might be able to help 15:14:41 he should be here 15:15:01 rbrndt: u there? 15:15:01 sounds like he pulled in kaiyan to test 15:15:15 I tested it a little myself and kayak ran the tempest tests against it 15:15:28 I was hoping to go over it once more before approving 15:15:37 sure -- np 15:15:48 thx rbrndt 15:15:58 apologies to kaiyan whose name apparently auto corrects to kayak 15:16:04 :) 15:16:09 lol 15:16:40 ok, then i'm assuming that will get merged relatively soon 15:16:46 sweet 15:16:54 #topic https://review.openstack.org/#/c/287507/ 15:17:57 sorry thought that was a new topic the way the indenting worked out 15:18:07 #topic 4. multiple metric and group_by 15:18:28 I started this review last wek https://review.openstack.org/#/c/289675/ 15:18:38 rbrndt is working on it now because i'mout 15:18:59 basically, we would like to add a group_by query parameter to the measurmeents and statistics resources 15:19:31 initially, we would like to only support "group_by=*", to group_by all unique dimensions 15:19:43 this would impact the vertica and influxdb repos 15:19:56 early testing of it was good at TWC -- rbrndt ping me when more baked and i'll retest 15:19:59 the goal is to return multiple metrics in a single query 15:20:08 did you look at performance 15:20:28 no -- just in my sandbox, that'd be tough until it merges and deploys 15:20:41 but it's gonna scream by design :) 15:20:49 so, is everyone ok with this change 15:21:05 i don't think we submitted a blueprint to explicitly cover what this is about 15:21:23 but the review is in progress, although will probably be a couple of weeks until it is complete 15:21:52 does anyone want to discuss further? 15:22:10 the reasons/motiviation for the change, the syntax, ... 15:22:39 implications, … 15:23:17 I haven't take a look before, but please explain 15:23:46 currently, when querying measurments or statistics, only one metric is returned 15:24:05 you can use the merge_metrics, to merge metrics, but in that case, it is still "one metric" 15:24:40 we are starting to see that this is becoming a performance bottleneck 15:25:27 for example, at twc where they are displaying multiple vm metrics per tenant, let's say 10-20 VMs, and 10 separate graphs, that results in 100-220 separte queries of the API 15:25:41 There is also a use case that we are seeing in Ceilosca 15:26:12 the group_by parameter will allow you to get multiple independent metrics back in a single query 15:26:33 we would like to support group_by=region, zone, hostname, ... 15:26:51 where region, zone, hostname are dimension keys/names 15:27:02 once ^^ merges, rbak will make some corresponding grafana2 changes to parse the multiple metrics returned 15:27:05 in the short-term however, we would just like to support group_by=* 15:27:09 rhochmuth: would the group_by allow several params in a single group or is only ine? 15:27:21 where "*" is a wildacard for all dimensions 15:27:39 in other words, return all the unique metrics 15:27:59 that's interesting 15:28:06 it would allow several, fabiog 15:28:15 that is the end-goal, anyway 15:28:20 rhochmuth: ok 15:28:37 so, the review is up and you can follow the progress of it 15:28:44 and comment on it 15:29:03 i'm expecting a very significant performance increase as a result 15:29:12 rhochmuth: Any idea what the timeline for supporting params other than * would be? 15:29:30 rbrndt: u there still? 15:30:02 rhochmuth: is only java, is still WIP or no plans to add Python? 15:30:12 we will add python too 15:30:27 I'm still here 15:30:27 goal is too support java/python influxdb and vertica 15:30:30 all variations 15:30:43 btw, are we supporting Influx 0.10? 15:30:58 rbrndt: now that you've looked at this for a couple of days, do you have a time-line 15:31:13 for the complete java/python vertica/influxdb implementation? 15:31:21 no pressure 15:31:32 I've begun work on the pagination issues we discussed for vertica, so I expect that will be done or close to done today 15:31:47 I have some idea about the influx implementation, but I've yet to test them 15:32:02 so, 2-3 weeks maybe 15:32:03 The python should follow pretty easily once the java is done 15:32:21 I would say 2 would be sufficient to get a pretty good attempt done 15:32:44 so, it will be past the mitaka release date 15:33:10 mostly likely 15:33:13 anymore questions on that topic 15:33:31 witek: did that answer your questions? 15:33:34 what about cassandra support? 15:33:55 yes, I have an idea :) 15:34:08 jobrs: let's talk about next, related to the influxdb discussion 15:34:12 thanks witek 15:34:28 let me know if you have any further questions, and please comment on review if any conerns 15:34:44 #topic infuxdb 0.10.X 15:34:48 you were just mentioning influx and vertica above. influxdb will stop giving away versions with cluster support for free soon. 15:35:11 mhoppal, are you there 15:35:16 yes here 15:35:24 jobrs: oh no 15:35:54 so, we've been doing a lot of influxdb analyis again 15:35:59 definitely 15:36:01 mhoppal can give an update 15:36:06 that too bad ;-( 15:36:21 yes so we run some scale testing with influx 9.5 and the new version of 10 15:36:40 looking to see if there were any differences between the two 15:37:04 we saw a significant improvements in 10 15:37:40 we are still going through the testing but have scaled 10 up to around 9000 measurements per a second 15:37:42 is that clustered -- or single node? 15:37:45 clustered 15:37:47 which storage engine? 15:37:50 tsm 15:37:54 for 10 15:37:59 tsm is default in 10 15:38:02 yes 15:38:15 how is stability? 15:38:24 great 15:38:40 how is disk utilization? 15:38:51 about 10x better then 9.5 15:39:23 so, disk utilization is 1/10 15:39:33 also in 9.5 we run into issues geting past 5000 measurements per a second we were recieving timeouts from influxdb and 10 had no issues with that 15:39:35 and yes 15:40:10 so, summary, performance is looking good, hitting 9000 metrics/sec in test, but we haven't pushed harder yet 15:40:15 stability is great 15:40:25 yes we are going to push it up further 15:40:30 compression is 10X better that 0.9.5 15:40:31 and then start to do ha fail over testing 15:40:35 but it is looking a lot better 15:40:40 did you test failure scenarios like split brain, remove one node and bring it back into the cluster? 15:40:48 additioanlly, compressions is 2X better than vertica 15:40:49 and the memory is a lot better as well 15:41:01 we have not done much failure scenarios yet 15:41:05 in the plan for the next week 15:41:30 christian_: we've done some initial ha testing, but more is planned 15:41:37 so far, we haven't seen any issues 15:41:50 thanks...please keep us up to date :-) 15:42:08 so, based on that, we are rethinking our work on Cassandra 15:42:28 jobrs: do you have a link on the clustering 15:42:58 you mean moving up or pushing out work on cassandra? 15:43:14 https://influxdata.com/blog/update-on-influxdb-clustering-high-availability-and-monetization/ 15:43:21 I was just about to post that... 15:43:30 sry 15:43:40 no problem! :) 15:43:47 it was just to say that we are rethinking our plans around Influx 15:44:13 support is priced around $500/server/month 15:44:14 https://influxdata.com/pricing/#product-subscriptions 15:44:55 cassandra scaleability may be another pro (after looking at what netflix is doing with it) 15:45:26 the reason why we started with cassandra is due to the problems with influxdb 15:45:43 now that influxdb is looking good, we have much less of a reason to do cassandra 15:45:59 are there any blueprints or specs about the Cassandra thoughts? 15:46:04 we still haven't made the final decision thgouth 15:46:17 there is a cassandra blueprint 15:46:33 does the fact that clustering won't be free with influxdb add another dimension to the cassandra support decision rhochmuth? 15:46:43 yes it does 15:46:57 this is what they plan exactly: "Next week we’ll be cutting the first release candidate for the 0.11.0 release. While this release includes significant improvements in the query engine and the clustering code base, it will be the last open source version that includes clustering." 15:47:14 that is really bad news 15:47:58 but two maintenance releases are released before at least (0.10.3 and 0.11) 15:48:10 I've been pointed towards KairosDB, which is built on Cassandra, but I haven't had a chance to look yet 15:48:26 i'll read the post more thoroughly after this meeting 15:48:56 maybe we should fork influxdb? 15:49:57 thanks mhoppal for the update 15:50:05 of course 15:50:10 #topic Shinya's notification methods 15:50:19 that's me 15:50:31 thanks for the review 15:50:45 i just wanted to give you change to let folks know what you are working on 15:50:59 thanks 15:51:23 Im planning to change pagination style to Horizon style. 15:51:48 This needs sort parameter to api. So I added. 15:52:08 and this applies to notification methods? 15:52:09 Im writing selenium integration test. 15:52:35 Alarm Defs and Alarms already have sort parameter. 15:53:33 thanks 15:53:40 It is difficult for me to write selenium test for current implementation of pagination. 15:54:39 So I planned to change pagination style. 15:54:58 the paginiation style changes impact the monaca-ui 15:55:02 correct? 15:55:48 yes Horizon standard style. 15:56:07 It is able to change num of list. 15:56:15 By setting page. 15:56:47 Current implementaion is hard corded 10. 15:57:35 If big screen you have, it is convinient. 15:58:15 ok, will checout out the review 15:58:20 probably need to wrap-up 15:58:40 one other think, i'll be applying tags for the mitaka release in the next week or two 15:58:59 tags need to be applied by March 31st, 15:59:17 this will be the first "official" monasca release in-sync with openstack 15:59:42 so, we need to be careful about stability risky changes 15:59:48 for the next couple of weeks 16:00:07 ok, eveyrone meeting is done 16:00:13 it always creeps up to fast 16:00:15 bye 16:00:20 cya 16:00:21 thanks Roland 16:00:22 bye 16:00:26 bye 16:00:29 thanks & bye 16:00:40 #endmeeting