18:00:02 <SergeyLukjanov> #startmeeting sahara
18:00:03 <openstack> Meeting started Thu Aug 14 18:00:02 2014 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:06 <SergeyLukjanov> sahara folks, ping
18:00:07 <openstack> The meeting name has been set to 'sahara'
18:00:11 <elmiko> o/
18:00:14 <NikitaKonovalov> hi
18:00:21 <SergeyLukjanov> tmckay, are here already?
18:00:37 <SergeyLukjanov> alazarev, sreshetnyak, ping
18:00:39 <SergeyLukjanov> dmitryme
18:00:43 <alazarev> o/
18:00:49 <aignatov> o/
18:00:49 <SergeyLukjanov> elmiko, does croberts already on PTO?
18:01:23 <sreshetnyak> o/
18:01:26 <elmiko> SergeyLukjanov: yes, baby arrived two nights ago :)
18:01:31 <alazarev> SergeyLukjanov: you are fast today, 18:00:02 2014 UTC
18:01:36 <tosky> hi
18:01:42 <aignatov> wow! awesome news!
18:01:46 <SergeyLukjanov> elmiko, yay! my congrats to him if you'll see him :)
18:02:01 <aignatov> my congrats too ;)
18:02:03 <elmiko> i'll pass on the well wishes
18:02:07 <alazarev> grats!
18:02:18 <SergeyLukjanov> okay, ley's start
18:02:25 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda#Next_meetings
18:02:37 <SergeyLukjanov> #topic sahara@horizon status (croberts, NikitaKonovalov)
18:02:44 <SergeyLukjanov> NikitaKonovalov, could you please make an update?
18:02:52 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-juno-post-merge-changes
18:03:01 <SergeyLukjanov> NikitaKonovalov, does bootstrap 3 patch merged?
18:03:40 <SergeyLukjanov> hm
18:03:47 <SergeyLukjanov> it means "nope"?
18:03:48 <SergeyLukjanov> ;)
18:03:55 <NikitaKonovalov> not merget
18:04:21 <NikitaKonovalov> there is on big change from Chad
18:04:35 <NikitaKonovalov> and a few small fixed from me
18:04:45 <SergeyLukjanov> NikitaKonovalov, could you please track them and make them landed?
18:05:03 <NikitaKonovalov> ok, I'l keep a track of them
18:05:20 <alazarev> it has a lot of +1 from sahara guys, but no marks from horizon
18:05:21 <alazarev> https://review.openstack.org/#/c/110680
18:05:44 <SergeyLukjanov> NikitaKonovalov, how about templates copy?
18:06:05 <NikitaKonovalov> the change is also on review
18:06:18 <NikitaKonovalov> but no horizon reviewvers yet
18:06:23 <alazarev> SergeyLukjanov: templates copy are dependant on bootstrap 3 patch
18:06:50 <SergeyLukjanov> alazarev, oh, yeah
18:07:11 <SergeyLukjanov> NikitaKonovalov, anything else on dashboard topic?
18:07:37 <tmckay> here I am
18:07:51 <NikitaKonovalov> I think we should clean the old repo as soon as possible
18:08:01 <SergeyLukjanov> NikitaKonovalov, +1
18:08:02 <NikitaKonovalov> some people put change requests tehre
18:08:06 <NikitaKonovalov> from time to time
18:08:24 <SergeyLukjanov> any objections to remove old dashboard from repo and tweak README to go to the horizon?
18:08:31 <alazarev> SergeyLukjanov: I fixed templates list error handling and it was merged, this is the only sahara patch not in bootstrap3 chain
18:08:43 <tmckay> no objections
18:08:46 <sreshetnyak> no objections
18:09:05 <SergeyLukjanov> #agreed remove old dashboard from repo and tweak README to go to the horizon
18:09:10 <alazarev> devstack has two Saharas now :)
18:09:13 <SergeyLukjanov> any voluteers to do it asap?
18:09:27 <elmiko> SergeyLukjanov: we will still have access to icehouse dashboard though?
18:09:42 <SergeyLukjanov> elmiko, sure, just remove saharadashboard dir in master
18:09:46 <NikitaKonovalov> SergeyLukjanov: I can take that
18:09:48 <SergeyLukjanov> elmiko, and tweak README
18:09:58 <SergeyLukjanov> NikitaKonovalov, okay, thx, please, do it tomorrow ;)
18:10:27 <SergeyLukjanov> #action NikitaKonovalov to remove dashboard code from master of sahara-dasboard and tweak README with instructions
18:10:33 <elmiko> SergeyLukjanov: cool, thanks
18:10:36 <tosky> remove the code?
18:10:40 <alazarev> I can remove dashboard from devstack
18:10:53 <SergeyLukjanov> alazarev, it's done already by myself
18:10:59 <SergeyLukjanov> alazarev, just waiting for review
18:11:10 <alazarev> SergeyLukjanov: oh, ok
18:11:12 <tosky> I mean, are you going to keep the repository but move it somewhere else (not openstack namespace), or just put a big README?
18:11:28 <SergeyLukjanov> tosky, code is already merged to Horizon
18:11:45 <SergeyLukjanov> tosky, only selenium tests and README with instructions will be in sahara-dashboard repo
18:11:56 <alazarev> tosky: delete code from master, maintain branches only
18:11:56 <SergeyLukjanov> tosky, the stable/icehouse branch will not be changed
18:12:13 <elmiko> SergeyLukjanov: forgot to mention, mattf won't be able to make it today
18:12:37 <SergeyLukjanov> elmiko, yeah, I see :)
18:12:49 <SergeyLukjanov> #topic News / updates
18:12:52 <SergeyLukjanov> folks, please
18:13:05 <tmckay> SergeyLukjanov, just so you know it was on purpose, and he mentioned it ahead of time :)
18:13:17 <tosky> SergeyLukjanov, alazarev: ack, thanks
18:13:19 <SergeyLukjanov> tmckay, ack :)
18:13:31 <SergeyLukjanov> I was working on devstack changes (small refactoring and sahara-dashboard removal)
18:13:40 <elmiko> i've been working on chasing down the details needed to update the swift authentication spec. it's complicated and i'd definitely like to talk about the changes we will ask of operators in the name of increased security
18:13:43 <aignatov> just looked at spark integration tests adding, looks nice, I think we can merge it :)
18:13:44 <sreshetnyak> I'm working on various CDH fixes and review
18:14:18 <tmckay> I have a few things to say about Oozie in relation to elmiko swift auth changes in open discussion.  Basically, possible ways to lockdown Oozie (because creds will still be in workflow)
18:14:30 <SergeyLukjanov> aignatov, we should have job that runs it before merge
18:14:45 <tmckay> aignatov, ack, it doesn't do scaling but I want to help with swift auth first, seems like a priority for Juno
18:15:03 <aignatov> i’ve already merged it since it has 2 +2
18:15:05 <alazarev> bug fixing, pushing security group auto creation, ops error handling and antiaffinity for heat
18:15:36 <aignatov> SergeyLukjanov: exactly :)
18:15:38 <alazarev> I also have several topics for open discussion
18:15:43 <tmckay> also, on swift test -- I really want to use spark wordcount, but until we have spark with swift, it's too much trouble to make it use local hdfs (with an upload of a data file, etc etc)
18:15:53 <tmckay> so, I just used SparkPi with no IO
18:16:50 <tmckay> spark with swift may have to wait until K, not sure about Juno.  I think it may ride on top of the hadoop swift changes (I think swift delegates to hadoop for file access) but I don't know
18:16:53 <SergeyLukjanov> elmiko, tmckay, guys, do you think it's possible to migrate to the new security model in J?
18:17:15 <tmckay> it's getting close.
18:17:23 <SergeyLukjanov> elmiko, tmckay, assuming that we should keep backward compat / upgradebility
18:17:43 <elmiko> SergeyLukjanov: code wise, i think we can make it. will be a push, i'm more concerned about the integration issues i outlined in the email.
18:17:45 <tmckay> One reason I want to look at Oozie lockdown.  Even if we miss, Oozie lockdown would improve the situation a lot
18:18:25 <elmiko> SergeyLukjanov: if we switch to the new method(proxy users with trusts), i think we could keep backward compat as needed.
18:18:32 <SergeyLukjanov> tmckay, elmiko, could you please ack that you're planning to switch in backward compat style/
18:18:52 <tmckay> ack,  I agree on backward compat.  We just need a cluster config to say which method
18:19:06 <elmiko> SergeyLukjanov: i'd like to remove it, but i think we can write the hadoop-swift plugin to take both styles of auth into account
18:19:10 <SergeyLukjanov> tmckay, elmiko, I just would like to be sure that we'll be able to upgrade Icehouse sahara to Juno one ;)
18:19:29 <elmiko> basically, if the hadoop-swift see a trust id/domain id with it's user creds, then it will attempt new style, otherwise old style.
18:19:41 <tmckay> I agree.  That should be a goal.  Better to wait a release than break stuff if it comes to that, imho.
18:19:55 <SergeyLukjanov> tmckay, yup
18:20:00 <SergeyLukjanov> elmiko, ++
18:20:17 <SergeyLukjanov> okey, so, we have 2.5 weeks to land it
18:20:17 <tmckay> and the job manager will generate the workflow based on cluster config (or data source object content, or something).  Shouldn't be too hard.
18:20:43 <elmiko> SergeyLukjanov: i think the proposals we are making to the hadoop-swift component will actually make it more functional with keystone v3, we adding features from v3.
18:21:12 <tmckay> For spark, we will have to figure out how to get the auth info into the hadoop config (if indeed spark is delegating to hadoop for file access)
18:21:27 <tmckay> but I think spark swift is probably K
18:21:39 <tmckay> external hdfs will have to be good enough for Juno
18:21:47 <SergeyLukjanov> ++
18:21:50 <alazarev> elmiko: but we should still support keystone v2
18:22:53 <elmiko> alazarev: i think we can by having the hadoop-swift component look to see what parameters it receives. if it receives just username/pass, then v2. if it receives username/pass/trust/domain then v3.
18:23:17 <alazarev> elmiko: sounds good
18:23:34 <elmiko> my big concern though, is that we will need to distribute the new hadoop-swift based images and the packages.
18:23:34 <SergeyLukjanov> elmiko, small correction
18:23:35 <tmckay> elmiko, ack.  I'm guessing it gets a new client for each access
18:23:51 <SergeyLukjanov> elmiko, v3 should be always ued with fallback to v2 with creds
18:23:59 <elmiko> SergeyLukjanov: ack
18:24:19 <SergeyLukjanov> elmiko, I'm working on building images in gate
18:24:36 <SergeyLukjanov> and I'd like to propose moving hadoop-swiftfs to spearated repo
18:24:49 <elmiko> SergeyLukjanov: ++
18:24:54 <alazarev> elmiko: we already distribute hadoop-swift  since upstrean doesn’t support deta locality
18:25:06 <SergeyLukjanov> alazarev, yup
18:25:11 <elmiko> alazarev: ok, great
18:25:21 <elmiko> also, folks, please take a look at https://review.openstack.org/#/c/113591/
18:25:33 <SergeyLukjanov> #topic extract hadoop-swiftfs to the separated repo
18:25:35 <elmiko> i'd like to start making PRs for the change in spec
18:25:38 <SergeyLukjanov> so, the topic is
18:25:45 <tmckay> I +2'd already
18:25:47 <SergeyLukjanov> please, share your thoughts folks
18:25:49 <tmckay> :)
18:26:13 <elmiko> if we move it to a new repo, sahara-extras will be dead?
18:26:22 <SergeyLukjanov> elmiko, I think so
18:26:34 <tmckay> SergeyLukjanov, so edp-examples will be gone, then only thing left in sahara-extra is the hadoop plugin.  Why move it?
18:26:45 <elmiko> i don't mind moving it as part of the work we are doing on swift/auth. since we'll need to make changes anyway.
18:26:49 <tmckay> just a name change at that point, right?
18:27:00 <SergeyLukjanov> tmckay, examples already gone
18:27:22 <SergeyLukjanov> tmckay, good point for renaming
18:27:48 <elmiko> rename and some cleanup to the READMEs
18:27:52 <alazarev> SergeyLukjanov: will it be stackforge, openstack or hadoop-something?
18:27:58 <tmckay> Ideas for new name?
18:28:01 <SergeyLukjanov> tmckay, but probably we'd like to keep the stable/icehouse in -extra
18:28:11 <tmckay> ah, good point
18:28:34 <SergeyLukjanov> tmckay, so, it's better to move hadoop-swiftfs out of extra
18:28:35 <elmiko> are we going to be adding a component for spark-swift access?
18:28:48 <SergeyLukjanov> tmckay, but keep -extra as is with deprecation mark
18:28:53 <tmckay> elmiko, we don't know yet
18:28:57 <tmckay> it might "just work"
18:29:08 * elmiko crosses fingers
18:29:12 <tmckay> elmiko, willb suggested to me a long time ago that swift just delegated to hadoop
18:29:13 <sreshetnyak> imho it's bad idea because hadoop vendors will not support custom hadoop-swiftfs library
18:29:36 <alazarev> tmckay: hdfs is pretty the same in spark, so it should just work
18:29:45 <tmckay> I think so
18:29:51 <elmiko> sreshetnyak: which part is a bad idea?
18:29:53 <tmckay> haven't poked into that yet
18:30:18 <alazarev> now we have hadoop-fs in hadoop repo, but it is outdated
18:31:09 <alazarev> if we manage to make changes fast enough we can use it, this should be a goal
18:31:22 <sreshetnyak> elmiko, extract hadoop-swiftfs library
18:31:27 <SergeyLukjanov> on the other hand the one in -extra works ok, so, no real need to change it now
18:31:39 <SergeyLukjanov> sreshetnyak, good idea
18:31:51 <SergeyLukjanov> we should decide first what to do with code in hadoop
18:32:26 <elmiko> sreshetnyak: are you saying we should leave the hadoop-swiftfs in the extras repo?
18:33:11 <SergeyLukjanov> I think the right approach is to decide where this code should live - in hadoop eco or in openstack
18:33:28 <SergeyLukjanov> and if hadoop eco than in hadoop repo or separatedly as contrib fs
18:33:30 <alazarev> problem with hadoop repo is that changes need to be reviewed by hadoop guys, it is slow and non-trivial for openstack eco
18:33:34 <elmiko> SergeyLukjanov: would hadoop consider accepting it?
18:33:50 <SergeyLukjanov> elmiko, some version of this code is already in hadoop
18:33:52 <elmiko> alazarev: that concerns me too
18:34:04 <SergeyLukjanov> alazarev, yup, it's the main concern
18:34:20 <alazarev> it took 6 months to merge first version
18:34:24 <elmiko> SergeyLukjanov: ok, but the real issue is we need to iterate with changes to the openstack infrastructure. which is must faster than hadoop currently.
18:34:35 <tosky> it could be kept separate until it's integrated, as it happened in the past, I think
18:34:41 <elmiko> that seems like we should keep it separate
18:34:48 <SergeyLukjanov> so, -extra works ok for us now, let's not spend time for useless moving now
18:35:11 <SergeyLukjanov> it
18:35:17 <elmiko> i'm good with less work =)
18:35:23 <alazarev> +1
18:35:29 <SergeyLukjanov> it's easy enough to extract, but we'll still have -extra
18:35:35 <SergeyLukjanov> because of stable branch
18:35:50 <SergeyLukjanov> and so there is not so cool to just add one more repo in fact ;)
18:36:09 <elmiko> if we're going to change the hadoop-swiftfs for this auth stuff, maybe we should leave it in -extra until sometime in K
18:36:20 <SergeyLukjanov> elmiko, yup
18:36:30 <SergeyLukjanov> or we'll have some good ideas on summit
18:36:37 <elmiko> let's hope :)
18:36:48 <sreshetnyak> I think the right approach is to decide where this code should live - in hadoop eco or in openstack
18:36:51 <sreshetnyak> 22:34  SergeyLukjanov@: and if hadoop eco than in hadoop repo or separatedly as contrib fs
18:36:53 <sreshetnyak> 22:34        alazarev : problem with hadoop repo is that changes need to be reviewed by hadoop guys, it is slow and non-trivial for openstack eco
18:36:57 <sreshetnyak> 22:34          elmiko : SergeyLukjanov: would hadoop consider accepting it?
18:37:12 <sreshetnyak> sorry, wrong message
18:37:15 <elmiko> sreshetnyak: it seems like we need to keep it in openstack eco for now
18:37:34 <SergeyLukjanov> yeah, for J at least
18:37:53 <SergeyLukjanov> #agreed no changes needed, at least in J
18:37:57 <SergeyLukjanov> okay, let's move on
18:38:05 <sreshetnyak> elmiko, ok, agree
18:38:10 <SergeyLukjanov> #topic Juno 3 (Sept 4)
18:38:16 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Juno_Release_Schedule
18:38:29 <SergeyLukjanov> so, just reminder - winter is coming!
18:38:34 <elmiko> lol
18:38:55 <SergeyLukjanov> alazarev, what's your topic to discuss?
18:39:11 <SergeyLukjanov> hdfs usage in edp?
18:39:39 <alazarev> SergeyLukjanov: sqlite support, broken pig in hadoop2, EDP dependancy on HDFS
18:39:51 <SergeyLukjanov> #topic sqlite support
18:40:25 <alazarev> I’m Ok with removing sqlite support, but we need to anounce this somehow
18:40:37 <SergeyLukjanov> so, AFAIK other project have same migrations with ALTER COLUMN that isn't supported by SQLit
18:41:33 <SergeyLukjanov> aignatov, looks like neutron has such migrations
18:41:44 <alazarev> we need to clear docs from sqlite in this case
18:41:44 <SergeyLukjanov> aignatov, are you using MySQL for dev?
18:42:01 <dmitryme> alazarev: good point, we need to put that as an item in our update doc
18:42:03 <aignatov> SergeyLukjanov: I think this topic was rised haklfyear ago when we started adding migrations to sahara with alembic
18:42:06 <SergeyLukjanov> and I don't like hacking python/sqla to support sqlite
18:42:08 <tmckay> I use sqlite.  Been meaning to switch.
18:42:15 <aignatov> and decided to get rid of sqlite support
18:42:19 <aignatov> I remember that
18:42:30 <SergeyLukjanov> aignatov, yes, I remember something like this
18:42:47 <SergeyLukjanov> oh yeah, we were having the same migration the whole icehouse
18:42:59 <SergeyLukjanov> until the pre-release squashing of migrations
18:43:06 <dmitryme> aignatov: yep, me too. Then we found a way to make migrations for Icehouse without alter, by sqashing them
18:43:16 <dmitryme> so sqlite worked for Icehouse
18:43:24 <tmckay> I just remove ALTERS :)  works fine
18:43:26 <alazarev> yes, but after that changes were merged for icehouse and incompatibility disappear
18:43:58 <dmitryme> tmckay: don’t do that in any environment you value :-)
18:43:59 <tmckay> just kidding -- not suggesting that as a real solution.  But it does work when you're in a hurry.
18:44:14 <SergeyLukjanov> :)
18:44:18 <alazarev> tmckay: now you can do it in one file, it is Ok with 2-3, nearly impossible with 10+
18:44:30 <SergeyLukjanov> #agreed to drop sqlite support, need to cleanup docs
18:44:44 * tmckay time to switch finally
18:44:47 <alazarev> I can clean up docs and samples
18:45:01 <alazarev> (we have sqlite in samples too)
18:45:02 <SergeyLukjanov> #agreed to not hack migrations by removing ALTER from them in production
18:45:13 <SergeyLukjanov> alazarev, yes, please
18:45:16 <tmckay> lol, did we really need to say that?
18:45:26 <elmiko> tmckay: me too, gotta set up that local mysql server...
18:45:27 <SergeyLukjanov> #action alazarev to cleanup sqlite from docs and samples
18:45:30 <alazarev> to not hack _officially_ :)
18:45:36 <aignatov> the same with neutron to answer on SergeyLukjanov question, you can play with sqlite but it’s not for production and migrations :)
18:45:43 <SergeyLukjanov> tmckay, I'm just kidding ;)
18:46:48 <alazarev> next topic?
18:46:51 <SergeyLukjanov> #topic EDP dependency on HDFS
18:47:48 <alazarev> Jon Maron implemented “mkdir -p” manually because HDFS FsShell API changed in hadoop2
18:48:05 <alazarev> now we have now we have https://bugs.launchpad.net/sahara/+bug/1355114 and https://bugs.launchpad.net/sahara/+bug/1356582
18:48:08 <uvirtbot> Launchpad bug 1355114 in sahara "[HDP] Parallel running EDP jobs failed on HDP1 cluster" [Medium,Confirmed]
18:48:55 <aignatov> ok, is it impossible to fix that?
18:49:18 <alazarev> the main problem is that EDP calls HDFS directly without knowledge what HDFS it is
18:49:35 <alazarev> I see several ways to fix
18:49:50 <SergeyLukjanov> alazarev, and w/o knowledge about existence of HDFS (storm for ex.)
18:50:29 <alazarev> 1. put HDFS (or other storage) communication into plugins (hard to implement, will save from problems with storm)
18:51:01 <alazarev> 2. expose hadoop version (1 or 2) from plugins and add “if” to EDP
18:51:13 * SergeyLukjanov don't like #2
18:51:50 <sreshetnyak> me too
18:52:21 <SergeyLukjanov> any objections against #1?
18:52:26 <alazarev> 3. require clusters to provide unified way for HDFS operations (need to think more)
18:52:37 <aignatov> ca it be fixed that way -> plugin has new SPI method “say to EDP engine which storage you are using for temporary purposes”
18:52:47 <alazarev> *require plugins
18:53:13 <elmiko> i like #1, but sounds complicated
18:53:17 <tmckay> I like #1, especially if we have implementations that can be shared by plugins.  If multiple plugins use hdfs with the same interface, then we should only implement once.
18:53:20 <dmitryme> I actually like #2
18:53:38 <tmckay> #2 is not terrible.
18:53:47 <dmitryme> but I am also ok with tmckay’s correction of #1
18:53:54 <tmckay> Allocate a class that wraps hdfs funtcions based on hadoop version
18:54:10 <alazarev> I don’t like how SPI currently implemented, may be we should switch to multiple interfaces
18:54:12 <tmckay> I think either one can work
18:54:21 <elmiko> tmckay, some sort of storage abstractor?
18:54:38 <alazarev> e.g. we can add “get_storage_manager” method to PSI
18:54:51 <alazarev> *SPI
18:54:59 <aignatov> alazarev: yes, I meant the same
18:55:10 <tmckay> yeah.  Ask the plugin what the hadoop version is, pass it off to a factory, get "MyGreatHDFS" object back, "myobj.makedir(path, "dashp=True)"
18:55:20 * tmckay made that up real quick
18:55:34 <SergeyLukjanov> I like it
18:56:41 <alazarev> tmckay: this is exactly solution with storage manager, plugins will just return shared impl
18:56:54 <tmckay> yes, get_storage_manager, basically the same idea
18:57:03 <tmckay> almost the same as the EDP engine stuff
18:57:17 <tmckay> doesn't have to be a formal plugin, just a factory and common interface
18:57:39 <SergeyLukjanov> tmckay, ++
18:57:43 <tmckay> I think we are all agreeing :)
18:57:47 <aignatov> so we got the decision, looks like nice bluprint to be implemented :)
18:57:52 <alazarev> tmckay: storage manager and EDP engine are hardly tight
18:58:10 <alazarev> do we want this in J?
18:58:30 <tmckay> yes, they are related -- could be the same object
18:58:46 <tmckay> or, at least in the same directory
18:58:55 <SergeyLukjanov> alazarev, I think it'll be cool to fix this issues in J
18:58:55 <tmckay> EDP engine is currently just "status, run, kill"
18:59:06 <SergeyLukjanov> 1 min left
18:59:20 <SergeyLukjanov> #topic Open discussion
18:59:26 <SergeyLukjanov> last seconds :)
18:59:38 <alazarev> with broken pig in oozie I think the best solution is custom oozie build
18:59:41 <tmckay> okay, quick comment on Oozie.  Either with Ooozie config settings, or firewall settings, I think we need to block oozie server access to Sahara and cluster nodes
18:59:47 <elmiko> i have a few concerns about what we will asking operators to change about their keystone configs for the swift/auth stuff. but i will put it an email.
18:59:57 <tmckay> so that people cannot spy on jobs, read configs, and thereby get creds and trust tokens
19:00:07 <SergeyLukjanov> let's continue in our channel
19:00:09 <SergeyLukjanov> thanks folks
19:00:10 <tmckay> I am looking in to both -- firewall setup, and oozie configs
19:00:11 <SergeyLukjanov> #endmeeting