14:00:07 <elmiko> #startmeeting Sahara
14:00:08 <openstack> Meeting started Thu Mar 12 14:00:07 2015 UTC and is due to finish in 60 minutes.  The chair is elmiko. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:12 <openstack> The meeting name has been set to 'sahara'
14:00:34 <elmiko> hey saharans
14:01:06 <elmiko> i'll give folks a few minutes to arrive and then we'll get rolling
14:01:26 <huichun> ^^
14:01:36 <crobertsrh> Hello/
14:02:01 <kchen> hello/
14:02:10 <Nikolay_St> o/
14:02:15 <weiting> O/
14:03:15 <elmiko> ok then
14:03:21 <elmiko> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
14:03:31 <crobertsrh> About 11 items in the review queue.  Good progress though.  I think I've had 4 or 5 merge in the last week.
14:03:46 <sreshetn1> o/
14:03:47 <elmiko> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon
14:03:50 <crobertsrh> Certainly need more eyes on the cluster and job guides
14:04:09 <elmiko> yea, i highly recommend folks give the guides a run-through if you have the chance
14:04:43 <NikitaKonovalov> the new event-log tab for cluster is also on review
14:05:22 <elmiko> col
14:05:27 <elmiko> cool even =)
14:05:32 <NikitaKonovalov> but if you want to try it out, you need a sahara-client patched with this change https://review.openstack.org/#/c/159451/
14:05:49 <elmiko> good to know
14:06:00 <crobertsrh> I'm adding it to the etherpad of reviews
14:06:17 <NikitaKonovalov> and sahara patched with this one https://review.openstack.org/#/c/163812/
14:06:38 <NikitaKonovalov> ^^ is very minor thing so I hope we'll get it merged fast
14:07:39 <elmiko> anything else from horizon land?
14:07:48 <crobertsrh> Not that I can think of.
14:08:27 <elmiko> ok then, moving on
14:08:40 <elmiko> #topic News / updates
14:08:47 <elmiko> folks, please
14:09:40 <tosky> helped pushing two horizon/selenium tests; pending tempest test review to decouple testing from vanilla plugin
14:09:56 <sreshetn1> i'm working on tempest client tests in sahara
14:10:00 <vgridnev> i'm finished work with timeouts patches, work event-log bug-fixes
14:10:26 <egafford> Working with sgotliv on the Spark shell job and generally getting him up to speed on Sahara dev.
14:10:33 <Nikolay_St> uhm.. not sure that it's for this section, but yesterday I attend the LogWorkingGroup meeting. Looks like we are the first team who is implemented a spec for new logging.
14:10:40 <elmiko> i've been doing reviews and working on a bug fix for the key manager usage with swift
14:10:48 <vgridnev> also, we are going to remove sahara.conf.sample, need review https://review.openstack.org/#/c/163779/
14:11:06 <elmiko> Nikolay_St: nice! i was going to cover action items next. we can talk more =)
14:12:14 <Nikolay_St> also, try to figure our what wrong with saharaclient gate job of juno cycle
14:14:12 <tmckay> news updates still?
14:14:12 <elmiko> yea
14:14:13 <tmckay> Okay, I am working on the default-cli CR, it's got an exception for FPF and probably FF if necessary.  Reviews would be great (post link in a second)
14:14:13 <egafford> Nikolay_St: If you're looking into that, I'd like to help.
14:14:13 <tmckay> I need to add a README for context, and still working on polish but the essential thing is there
14:14:18 <huichun_> tmckay: Trevor, hbase common lib is under review
14:14:27 <tmckay> #link https://review.openstack.org/#/c/163649/
14:14:49 <huichun_> tmckay: https://review.openstack.org/#/c/162657/
14:14:56 <tmckay> huichun_, great!  Sorry I couldn't help resolve the oozie libpath issue, maybe during the freeze I can take a look
14:15:15 <elmiko> any more news or updates?
14:15:32 <tmckay> also, job-type-endpoints has stalled a little bit.  Working on that, too.  We need agreement on the spec.
14:15:59 <elmiko> tmckay: is there a burning question or just details?
14:16:00 <tmckay> #link https://review.openstack.org/#/c/157563/
14:16:29 <Nikolay_St> egafford: I appreciate your help. After two days investigation I only can say that it's not devstack fault. pure devstack installation is look ok.
14:16:37 <tmckay> elmiko, spec is not merged yet, and the related client CR has some -1 because we were discussing ouput.
14:16:47 <elmiko> tmckay: ack
14:16:57 <elmiko> ok, moving on
14:16:59 <tmckay> #link https://review.openstack.org/#/c/161448/
14:17:00 <egafford> Nikolay_St: I agree with your assessment. :)
14:17:09 <huichun_> tmckay:hi Trevor, i am now have time to research that issue, will email you if any updates
14:17:10 <tmckay> elmiko, I hope to spin through that stuff again today
14:17:17 <elmiko> #topic Action items from the last meeting
14:17:26 <tmckay> huichun_, excellent, thank you.
14:17:34 <elmiko> Nikolay_St: do you want to brief us on the log meeting?
14:17:44 <Nikolay_St> elmiko: yeap
14:18:24 <Nikolay_St> I don't have much to say - look like community is busy before FF, so no active discussions on Logs.
14:18:44 <huichun_> tmckay: if you have edp job related bp or bugs, you can assign it to me, we have finished the cdh integrtion tests, so we have lots of time now
14:19:10 <elmiko> Nikolay_St: ok, good to know
14:19:23 <Nikolay_St> At the next meeting I will present a short list of how we can improve logging. some notices about where we should|shouldn't have LOG entries
14:19:27 <Nikolay_St> smthg like that
14:19:33 <tmckay> huichun_, excellent.  Take a look through existing [EDP] bugs, and if you see something you would like to fix, just assign yourself (I think you have privilege)
14:19:36 <elmiko> Nikolay_St: great!
14:19:48 <elmiko> ok, other action item was, Nikolay_St to investigate hacking checks for import groups
14:19:53 <huichun_> ok
14:20:03 <Nikolay_St> this guys expect that the more work started after 31march iirc
14:20:14 <Nikolay_St> elmiko: that's done
14:20:22 <elmiko> Nikolay_St: excellent!
14:20:37 <elmiko> that's all for action items from last week
14:20:44 <Nikolay_St> #link https://review.openstack.org/#/c/163021/
14:20:46 <tmckay> huichun_, oh, I will need help soon for default-templates.  I will need a sane default template set for each plugin
14:20:53 <elmiko> #topic Open discussion
14:21:02 <Nikolay_St> #link https://review.openstack.org/#/c/163123/
14:21:06 <tmckay> huichun_, the README will explain it (up soon).  Just JSON files, with some replacement fields
14:21:21 <elmiko> Nikolay_St: thanks
14:21:27 <tmckay> huichun_, so, if you can help me make a CDH set, that would be great
14:21:42 <kchen> tmckay: when you say default templates, do you mean the cluster or node group template?
14:21:43 <tmckay> just a generic CDH cluster that a user would like to launch "out of the box"
14:21:54 <huichun_> tmckay: i have seen that, you just finish HDP plugin, right?
14:21:55 <tmckay> kchen, yes, cluster and node group templates
14:22:27 <tmckay> huichun_, oh, that was job types endpoint.  That's another issue :)
14:23:13 <huichun_> tmckay: sorry, that's edp endpoint bp, does that bp need for every plugin?
14:23:13 <tmckay> kchen, the idea is that you run the default templates CLI and it loads template sets distributed with Sahara into the database (with some config added by user) and then a user can run a cluster without the manual steps of adding templates themselves
14:23:26 <tmckay> kchen, aimed at ease of use and quick introduction to Sahara
14:23:37 <tmckay> huichun_, yes
14:23:48 <kchen> tmckay: it is be/default-templates, right?
14:23:56 <huichun_> tmckay: ok, I will help on CDH set on default-template
14:23:58 <tmckay> huichun_, pretty easy, but I hadn't done the plugin CRs because the spec and main CR are not merged yet
14:24:04 <tmckay> kchen, yes
14:24:56 <weiting> tmckay, so the default template provide a minimal cluster to help the customer launch the cluster, right?
14:25:00 <kchen> so it is only for CLI. Can we use it on Horizon?
14:25:38 <tmckay> weiting, exactly
14:25:52 <tosky> about templates: I had to create cluster/node group templates for Tempest tests, and I noticed that, for cluster configuration, the parameters are like dfs_replication instead of dfs.replication like all the others
14:26:15 <egafford> tmckay: I can pitch in on any default templates that don't have coverage; let me know.
14:26:26 <tmckay> kchen, well, the templates will be available from the UI once they are installed.  But the installation itself will be from a new (small) CLI
14:26:41 <tosky> is it something that depends on CDH internal? Could it be changed in the plugin to have the same parameter as the other Hadoop flavors? Is it worth to file a bug/wishlist?
14:27:05 <tmckay> tosky, yes, file a bug/wishlist and we can discuss on the bug
14:27:15 <tmckay> egafford, thank you
14:28:00 <tmckay> kchen, the idea is that a Sahara admin will install default templates for the desired plugins at Sahara installation time, and then will update them occasionally (when needs change, or a Sahara update, etc)
14:28:14 <tmckay> kchen, they are special because they cannot be deleted or modified, except from the CLI.
14:28:38 <tmckay> kchen, there is an "is_default" field that can only be set through the conductor API, not REST.
14:31:35 <elmiko> anyone have more topics for discussion?
14:32:11 <crobertsrh> Nothing from me
14:32:35 <Nikolay_St> I have
14:32:40 <Nikolay_St> just a request
14:32:58 <kchen> tmckay: so "is_default" means the template cannot be changed by REST, right?
14:33:10 <Nikolay_St> as far as we have a spec for logging style, please, follow it :)
14:33:11 <tmckay> kchen, yes
14:33:20 <kchen> tmckay: i see, thanks
14:33:32 <elmiko> Nikolay_St: can you link the logging spec again please
14:33:38 <tmckay> Nikolay_St, ack, I was in a hurry and just typed what came into my head :)  I will change it.
14:33:44 <Nikolay_St> for example, don't use crazy string formatting
14:33:53 <Nikolay_St> tmckay: ok
14:34:47 <Nikolay_St> #link: https://github.com/openstack/sahara-specs/blob/master/specs/kilo/sahara-log-guidelines.rst
14:34:57 <elmiko> Nikolay_St: thanks
14:34:58 <tmckay> #link https://review.openstack.org/#/c/162209/
14:35:09 <tmckay> kchen, ^^ default template spec
14:35:20 <tmckay> kchen, needs to be adjust slightly (will do that today) based on work
14:37:02 <kchen> tmckay: ok. I will check that
14:39:19 <kchen> Nikolay_St: is it for bp/new-style-logging?
14:39:39 <Nikolay_St> kchen: yeap
14:41:38 <kchen> Nikolay_St: I saw you already have a patch https://review.openstack.org/#/c/154037/, so what need we do other than reviewing it?
14:44:27 <Nikolay_St> kchen: in the spec we have an instructions about what log levels to use in general. also, we decide to use pep3101 for log formats.
14:44:38 <Nikolay_St> #link https://www.python.org/dev/peps/pep-3101/
14:45:48 <kchen> ok
14:47:10 <Nikolay_St> kchen: we try to make it as simple as possible
14:50:11 <elmiko> anything else?
14:51:20 <Nikolay_St> not from my side
14:52:49 <elmiko> ok then, thank you to everyone for attending!
14:53:00 <egafford> Thanks elmiko!
14:53:03 <elmiko> #endmeeting