14:00:25 <rhochmuth> #startmeeting monasca
14:00:26 <openstack> Meeting started Wed Jun  7 14:00:25 2017 UTC and is due to finish in 60 minutes.  The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:29 <openstack> The meeting name has been set to 'monasca'
14:01:10 <rhochmuth> uh oh, no agenda for today's meeting
14:01:19 <rhochmuth> that's a first
14:01:21 <rhochmuth> o/
14:01:33 <rhochmuth> anyone around today?
14:01:36 <koji> o/
14:01:38 <hosanai> o/
14:01:49 <Haruki> o/
14:02:01 <rhochmuth> hi koji and hosanai and haruki
14:02:07 <Fdaisuke> o/
14:02:11 <hosanai> hello
14:02:24 <Haruki> hi
14:02:40 <koji> hello
14:02:43 <rhochmuth> i guess the new time-slot works out a little better for everyone in japan
14:02:51 <rhochmuth> but, it is still a little late
14:03:22 <hosanai> yeah, it works for me :-)
14:03:27 <koji> 11:00pm on our time zone now
14:03:34 <rhochmuth> does anyone have any topics or reviews
14:03:39 <rhochmuth> my wife is in japan
14:03:47 <rhochmuth> she called me at 4:00 AM
14:04:05 <hosanai> hahaha. she tries to kill you :-)
14:04:14 <rhochmuth> she does her best
14:04:34 <hosanai> I would like to share info of analytics status.
14:04:35 <rhochmuth> torture is her specialty though
14:04:50 <rhochmuth> hosanai: you have the floor
14:04:55 <rhochmuth> #topic monasca-analytics
14:05:00 <hosanai> thanks!
14:05:12 <hosanai> brad gave us data for analytics
14:05:12 <Haruki> interesting!
14:05:27 <rhochmuth> brad klein?
14:05:37 <hosanai> yeah
14:05:54 <rhochmuth> very nice
14:06:02 <hosanai> and Fdaisuke checked the data but information is not enough for analytics
14:06:22 <rhochmuth> what data did you get?
14:06:35 <hosanai> We got it.
14:06:50 <rhochmuth> i meant type of data
14:06:55 <rhochmuth> was it metrics, logs?
14:07:20 <hosanai> sorry, i miss understood. Fdaisuke, could you explain it?
14:08:28 <Fdaisuke> ok
14:09:41 <Fdaisuke> there is no metorisc and logs, just only "timestamps" and "value" (100% or not)
14:10:21 <hosanai> my rough understanding is we got timestamp, value and value_meta for each services such as swift, nova and so on
14:10:52 <rhochmuth> do you have the metric name and dimensions?
14:10:55 <Fdaisuke> thanks, hosanai
14:11:30 <koji> isn't the "value" a metric?
14:12:52 <Haruki> memory usage or like that?
14:13:33 <hosanai> in dimensions, there is service and in measurements we have timestamp, value and value_meta. i think.
14:14:13 <hosanai> it looks status of services.
14:14:25 <rhochmuth> and are there any points in which incidents have been labeled?
14:15:10 <rhochmuth> for example, do you have a set of timestamps and label, for an incident and/or type of incident
14:15:22 <hosanai> yeah, he explained us unexpected behavior and the time.
14:16:12 <rhochmuth> so, you have a bunch of metrics, and then a set of incidents and the timestamps for when they occured it sounds like
14:17:04 <hosanai> we don't have metric like memory in the data.
14:17:16 <hosanai> only service status
14:18:08 <hosanai> s/memory/memory usage/
14:18:49 <rhochmuth> oh, so basically if swift goes down, the value of some metric, such as status with dimension service=swift is 1.0
14:19:45 <Haruki> only 1 or 0?
14:20:56 <hosanai> Fdaisuke, could you give us an example? you said the data looks like 100%, 90%...
14:22:31 <Fdaisuke> i saw 100%, 90%, 40%, sometimes 0%
14:23:04 <rhochmuth> ok, so other than the status metric, is there anything else of interest?
14:23:28 <rhochmuth> if that is all you have, then i agree, the dataset is not going to offer a lot of insight
14:23:45 <hosanai> yeah so Fdaisuke sent email to brad to get additional data.
14:23:48 <rhochmuth> or data analysis opportunites
14:24:19 <rhochmuth> i think what you need is a lot of metrics
14:24:36 <rhochmuth> the status metric is already informing you about incidents
14:24:54 <hosanai> i think so. I hope brad keep the data :-)
14:25:37 <rhochmuth> if you had more data, you could potentially identity other metrics that represent the incident
14:25:52 <rhochmuth> or possibly RCA
14:26:15 <hosanai> hopefully :-)
14:26:45 <rhochmuth> it would also be interesting to have the set of alarms that occurred
14:27:19 <rhochmuth> then you could train your data set using the alarms as the response/labels
14:27:49 <rhochmuth> and then try and predict incidents when you don't have the alarms
14:28:39 <hosanai> i see the alarms are useful to create labels.
14:28:43 <rhochmuth> in other works can you develop a model for predicting alarms
14:29:03 <rhochmuth> yeah, i think that would be an interesting problem
14:29:37 <rhochmuth> given a set X of all the metrics and a set y that consists of all the alarms
14:30:10 <hosanai> brad explained us if the status is not 100%, it's a problem from operator point of view.
14:30:35 <rhochmuth> so, status can be used too
14:30:50 <rhochmuth> i guess 0 is no problem
14:31:15 <rhochmuth> 25% could be low status
14:31:23 <rhochmuth> 100% could be there is a problem
14:31:34 <rhochmuth> > 75% could be there is a high probability there is a problem
14:31:46 <rhochmuth> so, that could be useful if you had the other metrics
14:31:55 <rhochmuth> you could use status as the response/label set
14:32:19 <rhochmuth> and then train on that using the metrics as predictors
14:32:30 <hosanai> thanks for the info/advise:-)
14:32:33 <rhochmuth> similar to how alarms would be used
14:32:44 <rhochmuth> your welcom
14:32:50 <hosanai> that's all I would like to share
14:32:51 <rhochmuth> i think you need a lot more data
14:33:06 <hosanai> yeah that's right
14:33:24 <rhochmuth> you could also take the status variables, and treat it as binary
14:33:35 <rhochmuth> for example, anything above 50% is a problem
14:33:54 <rhochmuth> then your problem turns into a binary classification problem
14:34:03 <rhochmuth> but, i'm not sure that is what you want to do
14:34:28 <hosanai> a binary classification means 2 values classification?
14:34:34 <rhochmuth> correct
14:35:06 <rhochmuth> but, you can also treat the predictor as continous, which is what the status is
14:36:14 <hosanai> it's a good way to handle the problem as a binary classification to get good result
14:36:15 <rhochmuth> so, if brad can get you more data, even without the alarms, i think you might have something you can start with
14:37:13 <hosanai> yeah, we can push analytics ahead with brad's help
14:37:23 <rhochmuth> it would also be very interesting to know what metrics correlate well with problems
14:38:03 <hosanai> i think so it's kind of RC
14:38:38 <rhochmuth> that would be good for RCA
14:40:42 <hosanai> yeah, and witec introduced Fujitsu's colleagues who created original algorithm (i forgot the approach) for analytics
14:41:10 <rhochmuth> do you know the names?
14:42:05 <hosanai> i forgot it I got a paper but did't read it yet :(
14:44:02 <hosanai> names for the guys?
14:44:10 <hosanai> you mean
14:44:32 <rhochmuth> well, i was wondering if they were hpe or fujitsu folks
14:46:09 <hosanai> I will send you the names later. I can't access company email now.
14:46:26 <rhochmuth> np
14:46:49 <hosanai> at least 9 hours later :-)
14:47:29 <rhochmuth> i thought you were still at work
14:47:33 <rhochmuth> i'm dissapointed
14:47:38 <rhochmuth> :-)
14:47:50 <hosanai> hahaha
14:49:45 <rhochmuth> so, is that it for today?
14:50:46 <hosanai> i don't have anything now
14:51:02 <rhochmuth> thx hosanai
14:51:13 <rhochmuth> i guess i'll end the meetign a little early
14:51:29 <rhochmuth> #endmeeting