18:01:40 <alazarev> #startmeeting Sahara
18:01:41 <openstack> Meeting started Thu Jul 23 18:01:40 2015 UTC and is due to finish in 60 minutes.  The chair is alazarev. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:42 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:45 <openstack> The meeting name has been set to 'sahara'
18:01:47 <NikitaKonovalov> o/
18:01:50 <alazarev> o/
18:01:54 <vgridnev> hey
18:01:54 <esikachev> hi!
18:02:06 <alazarev> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
18:02:13 <sreshetnyak> o/
18:02:16 <elmiko> thanks alazarev =)
18:02:19 <alazarev> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
18:02:47 <alazarev> elmiko, Sergey has concurrent meeting, I'll char this one
18:02:53 <NikitaKonovalov> I havent got much for this section
18:02:55 <crobertsrh> Not sure I have much to report there.  I have been mostly working on a few other things (manila + sahara) lately.
18:03:08 <crobertsrh> Reviews are sparse, as usual
18:03:13 <NikitaKonovalov> there are still chages to be rebase onto contrib directory
18:03:27 <alazarev> I need to rebase my patch, everything changed after structure change
18:03:53 <alazarev> #topic News / updates
18:04:42 <alazarev> I submitted patches about heat engine refactoring, review plz
18:05:01 <elmiko> i'm working on keytone sessions coding, i could really use reviews on #link https://review.openstack.org/#/c/197743/
18:05:07 <elmiko> i'd like to start pushing code reviews
18:05:22 <elmiko> and i've been experimenting with a microversion api poc for sahara with v2
18:05:26 <vgridnev> https://review.openstack.org/#/c/177280/ is finished, maybe someone will put approved?
18:05:49 <NikitaKonovalov> I've been working on some Sahara benchmarks in Rally project. Now Rally can collect show the Hadoop Counters in its report after each Job Execution completes. https://review.openstack.org/#/c/203324/
18:05:52 <vgridnev> anyway, I've researching spark with swift on cdh plugin
18:06:53 <esikachev> i'm working on adding ability of creating flavor for scenario tests https://review.openstack.org/#/c/203567/
18:07:23 <esikachev> and try fix mapr
18:08:09 <alazarev> NikitaKonovalov, don't we want such functionality in Sahara itself?
18:08:15 <NikitaKonovalov> I'm also working on completing this chain of changes https://review.openstack.org/#/c/177682/
18:08:36 <NikitaKonovalov> @alazarev: well, that's a thing to think about
18:09:18 <NikitaKonovalov> what I've done for rally is simply read the data from hadoop's history server
18:09:36 <NikitaKonovalov> It works for java jobs
18:09:45 <NikitaKonovalov> should fork for pig and hive
18:10:07 <NikitaKonovalov> but I'm not sure if other Job types store anythig similar in History server
18:10:32 <alazarev> NikitaKonovalov, I think about this in more general way, it would be nice to have at least some information about hadoop in horizon, cluster health is the first thing, but it can be extended
18:10:59 <alazarev> #topic Open discussion
18:11:36 <NikitaKonovalov> agree, I thought somoone is working on the spec an implementation for hadoop health checks
18:12:02 <NikitaKonovalov> esikachev: ^^
18:13:04 <elmiko> yea
18:13:08 <esikachev> yes, i am working on this
18:13:12 <NikitaKonovalov> ok
18:13:29 <elmiko> so, vgridnev had asked on the spec about enumerating all the health checks. do we need to do this in the spec?
18:14:06 <elmiko> i'm concerned the list might be long and we'll miss some that will come up during implementation
18:14:21 <elmiko> but i do think it's a good question about how we will present all the checks in the ui
18:14:49 <alazarev> do we want health check returning only red/yelow/green mark? some numbers in addition could be useful too
18:14:58 <elmiko> nice idea
18:15:48 <crobertsrh> colors are nice, maybe numbers or something more specific on a mouseover?
18:15:56 <esikachev> i think be better show what process is failed
18:16:06 <NikitaKonovalov> so that's as always a question of making a generic ui or some fixed layout for fixed metrics, but then we should be sure those are always available
18:16:45 <NikitaKonovalov> btw HDP 2.2 plugin can actually repot provisioning progress in %
18:17:14 <NikitaKonovalov> but that looks more like an exclusive ambari feature now
18:17:43 <elmiko> i think we need this to be more generic as the health might be different for each plugin/version
18:19:06 <NikitaKonovalov> elmiko: I also like that way more
18:20:15 <crobertsrh> +1 to that
18:20:21 <esikachev> +1
18:20:45 <alazarev> I'm a little bit concerned by 3 +2 on https://review.openstack.org/#/c/187978/ after my -1. I think create/delete cluster without errors must not print WARNING in log. What do you think?
18:22:28 <elmiko> hmm, maybe it should be info? or you want to just drop it?
18:23:12 <elmiko> i mean, isn't that catching an error?
18:25:54 <alazarev> elmiko, code assumes that there could be exception even in normal flow
18:26:49 <alazarev> elmiko, so I think exception is not an indicator of failure (this is bad practice, but it is common in python and openstack)
18:27:19 <elmiko> alazarev: fair, that's a good point. so just let the exception continue to raise and if it's unhandled it will be logged there?
18:27:48 <alazarev> elmiko, exactly, like it was before retries implementation
18:27:56 <elmiko> i'd be ok with that
18:28:14 <elmiko> aignatov: ^^
18:28:20 * elmiko doesn't see tmckay
18:29:05 <alazarev> elmiko, it is up to catching code - log or not to log excaption, no intermediate layers needed
18:29:12 <tmckay> hi
18:29:30 <alazarev> tmckay, hi, we are nearly finished :)
18:29:34 <elmiko> alazarev: yea, you are correct. we should let the error keep raising, and if it's caught by something else then good if not it will get logged
18:29:50 <elmiko> tmckay: we're talking about alazarev -1 on https://review.openstack.org/#/c/187978
18:30:05 <elmiko> and if we should log or not on the re-raised error
18:30:09 <tmckay> yeah, I saw that this morning
18:30:19 <elmiko> i'm ok with dropping the log message
18:30:30 <elmiko> well, alazarev convinced me ;)
18:30:41 <tmckay> well, the CR is only changing the level, so I'm fine with it
18:30:58 <tmckay> however, a drop is fine too.  The error is raised.
18:31:11 <tmckay> it will either bubble up or get handled
18:31:16 <alazarev> tmckay, it is changing buggy ERROR to buggy WARNING
18:31:19 <elmiko> yea, the error might be caught and handled somewhere else, so log would be redundant
18:31:22 <tmckay> in the other case, it's just a notification that retries are happening
18:31:39 <elmiko> if we keep the log message, it should be info
18:31:46 <elmiko> or possibly debug
18:31:50 <tmckay> +2 from me either way -- change the level or drop the message :)
18:32:14 <elmiko> alazarev: what do you think about making it debug?
18:32:16 <alazarev> tmckay, it is not about retries, log is about unexpected return code
18:32:49 <alazarev> elmiko, I'm fine with DEBUG, debug is always helpfull to understand things
18:33:00 <tmckay> akazarev, no, I mean in the first part of the else.  My point is that the other log message in this code fragment serves a purpose, so that you know retries are happening
18:33:04 <elmiko> ok. i'm adjusting my review then
18:33:11 <tmckay> the log message in question arguably serves no purpose
18:33:39 <tmckay> yeah, change to DEBUG sounds good to me
18:33:41 <elmiko> right, it may serve no purpose, so better to make it debug
18:34:06 <crobertsrh> Debug seems like the way to go
18:34:18 <elmiko> updated my review
18:34:24 <elmiko> thanks alazarev
18:35:22 <tmckay> me too, so there are not 2 +2 :)
18:35:31 <tmckay> someone might merge ;-)
18:35:34 <elmiko> hehe
18:38:41 <elmiko> cool, we reached an agreement! \o/
18:41:04 <alazarev> anything more to discuss?
18:41:37 <elmiko> nothing here, except to beg for more reviews ;P
18:42:56 <alazarev> elmiko, you are welcome to beg ;)
18:43:09 <alazarev> elmiko, always!
18:43:19 * elmiko shakes cup with change in it
18:43:43 <elmiko> "spare any reviews..."
18:44:41 <alazarev> ok, it looks we are done
18:44:46 <alazarev> #endmeeting