Monday, 2015-03-23

*** mahito has joined #openstack-sahara00:05
*** Longgeek has joined #openstack-sahara01:08
*** oikawa has joined #openstack-sahara01:17
*** _crobertsrh has quit IRC01:24
*** hdd has joined #openstack-sahara01:59
*** Poornima has joined #openstack-sahara02:31
*** Networkn3rd has joined #openstack-sahara02:48
*** coolsvap|afk is now known as coolsvap03:24
*** hdd has quit IRC03:24
*** Longgeek has quit IRC03:30
*** Longgeek has joined #openstack-sahara04:30
*** chandankumar has joined #openstack-sahara04:46
*** Networkn3rd has quit IRC04:59
*** pcaruana has quit IRC05:07
*** Nikolay_St has quit IRC05:08
*** chandankumar has quit IRC05:17
*** raaka has joined #openstack-sahara05:23
*** chandankumar has joined #openstack-sahara05:33
*** Networkn3rd has joined #openstack-sahara06:00
*** tnovacik has joined #openstack-sahara06:03
*** Networkn3rd has quit IRC06:06
*** jamielennox is now known as jamielennox|away06:28
*** chandankumar has quit IRC06:57
*** pcaruana has joined #openstack-sahara06:58
*** sgotliv_ has quit IRC07:01
*** Networkn3rd has joined #openstack-sahara07:02
*** Networkn3rd has quit IRC07:06
*** Nikolay_St has joined #openstack-sahara07:32
*** chandankumar has joined #openstack-sahara07:42
*** chlong has quit IRC07:58
*** sgotliv has joined #openstack-sahara08:21
*** mahito_ has joined #openstack-sahara08:36
*** witlessb has joined #openstack-sahara08:36
*** mahito has quit IRC08:39
openstackgerritlu huichun proposed openstack/sahara: Add integration tests for scaling in Spark  https://review.openstack.org/16564008:45
*** Networkn3rd has joined #openstack-sahara08:51
*** ylobankov__ has quit IRC08:51
openstackgerritMerged openstack/sahara-image-elements: Disable registration on CentOS images  https://review.openstack.org/16623608:52
*** sgotliv has quit IRC08:53
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Add new log messages where it's needed  https://review.openstack.org/16620808:54
*** sgotliv has joined #openstack-sahara08:54
*** ylobankov has joined #openstack-sahara08:55
*** Networkn3rd has quit IRC08:56
*** Nikolay_St has quit IRC09:05
openstackgerritPino Toscano proposed stackforge/sahara-ci-config: Stop using an local diskimage-builder clone  https://review.openstack.org/16674409:05
*** IBerezovskiy has joined #openstack-sahara09:07
*** Nikolay_St has joined #openstack-sahara09:09
*** esikachev has joined #openstack-sahara09:31
openstackgerritMerged stackforge/sahara-ci-config: Stop using an local diskimage-builder clone  https://review.openstack.org/16674409:32
*** sgotliv has quit IRC09:39
*** DenisEgorenko has joined #openstack-sahara09:41
*** sgotliv has joined #openstack-sahara09:52
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: [WIP] Use tox env for building images  https://review.openstack.org/16661809:57
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Add main script for building sahara images  https://review.openstack.org/16660309:57
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Add element dependencies to requirements.txt  https://review.openstack.org/16660109:57
openstackgerritSergey Reshetnyak proposed stackforge/sahara-ci-config: Fix tempest client tests  https://review.openstack.org/16676410:06
*** ylobankov_ has joined #openstack-sahara10:14
*** ylobankov has quit IRC10:15
openstackgerritSergey Reshetnyak proposed stackforge/sahara-ci-config: Fix tempest client tests  https://review.openstack.org/16676410:16
openstackgerritMerged openstack/sahara: Minor - misprint corrected  https://review.openstack.org/16482510:19
openstackgerritMerged openstack/sahara: Fix order of arguments in assertEqual - Part1  https://review.openstack.org/16527110:26
openstackgerritMerged openstack/sahara: Fix order of arguments in assertEqual - Part2  https://review.openstack.org/16534710:26
openstackgerritMerged openstack/python-saharaclient: Add support for show_events parameter  https://review.openstack.org/15945110:30
openstackgerritMerged openstack/python-saharaclient: Updated from global requirements  https://review.openstack.org/15558810:30
openstackgerritMerged openstack/sahara: Fix order of arguments in assertEqual - Part3  https://review.openstack.org/16535010:33
openstackgerritMerged openstack/sahara: Replace direct http requests by sahara client in Quick start guide  https://review.openstack.org/16519210:33
openstackgerritMerged openstack/sahara: Sync with latest oslo-incubator  https://review.openstack.org/16502110:34
*** mahito_ has quit IRC10:38
*** Networkn3rd has joined #openstack-sahara10:40
*** Networkn3rd has quit IRC10:44
openstackgerritMerged openstack/sahara: Add usages of plugin poll - part 1  https://review.openstack.org/16076910:50
openstackgerritMerged openstack/sahara: MapR validation rules fixed  https://review.openstack.org/16538610:53
openstackgerritMerged openstack/sahara: Update MapR plugin docs  https://review.openstack.org/16544410:53
openstackgerritSergey Reshetnyak proposed stackforge/sahara-ci-config: Don't use sudo for dib scripts  https://review.openstack.org/16678010:56
openstackgerritMerged openstack/sahara-dashboard: Updated from global requirements  https://review.openstack.org/16647510:57
*** tosky has joined #openstack-sahara10:57
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Migrate to openjdk - part 2  https://review.openstack.org/14643411:06
*** coolsvap is now known as coolsvap|afk11:11
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Use openjdk by default for java element  https://review.openstack.org/15947311:12
openstackgerritMerged openstack/sahara: Replace empty list with scalable process in scaling  https://review.openstack.org/16384511:13
*** esikachev has quit IRC11:18
openstackgerritSergey Reshetnyak proposed openstack/sahara: Don't use middleware params for client auth  https://review.openstack.org/16387311:22
openstackgerritSergey Lukjanov proposed openstack/sahara-specs: Added dir for image elements specs  https://review.openstack.org/16596511:23
openstackgerritVitaly Gridnev proposed openstack/sahara: Event log supported in new integration tests  https://review.openstack.org/15695611:38
openstackgerritMerged openstack/sahara: Add scenario files for new integration tests  https://review.openstack.org/16464411:42
openstackgerritMerged openstack/sahara: Default version update for vanilla integration test  https://review.openstack.org/16499611:42
openstackgerritMerged openstack/sahara: Install MySQL JDBC driver along with client  https://review.openstack.org/16468511:42
openstackgerritMerged openstack/sahara: Add usages of plugin poll - part 2  https://review.openstack.org/16081811:42
*** Longgeek has quit IRC11:46
*** AndreyPavlov has quit IRC11:53
*** AndreyPavlov has joined #openstack-sahara12:06
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages of poll util for service modules  https://review.openstack.org/16346312:08
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages of poll util for service modules  https://review.openstack.org/16346312:08
*** Poornima has quit IRC12:14
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages of poll util for service modules  https://review.openstack.org/16346312:18
*** sreshetnyak has quit IRC12:18
openstackgerritAndrey Pavlov proposed openstack/sahara: Docs updated with instance locality feature  https://review.openstack.org/16468312:20
*** sreshetnyak has joined #openstack-sahara12:22
openstackgerritArtem Osadchiy proposed openstack/sahara: Fix topology awareness configuration  https://review.openstack.org/16679812:22
openstackgerritVitaly Gridnev proposed openstack/sahara: Minor - allow changing status description of deleting cluster  https://review.openstack.org/16482912:23
*** pino|work has joined #openstack-sahara12:27
* pino|work waves12:27
*** Networkn3rd has joined #openstack-sahara12:28
*** Networkn3rd has quit IRC12:33
*** tnovacik has quit IRC12:40
openstackgerritMerged openstack/sahara: Add integration tests for scaling in Spark  https://review.openstack.org/16564012:51
*** crobertsrh has joined #openstack-sahara12:51
*** chandankumar has quit IRC12:55
*** chandankumar has joined #openstack-sahara12:59
*** tmckay has joined #openstack-sahara13:00
*** chandankumar_ has joined #openstack-sahara13:03
*** chandankumar has quit IRC13:03
*** Networkn3rd has joined #openstack-sahara13:09
*** chandankumar_ has quit IRC13:16
*** coolsvap|afk is now known as coolsvap13:25
*** egafford has joined #openstack-sahara13:35
openstackgerritMartin Kletzander proposed openstack/sahara: Fix common misspellings  https://review.openstack.org/16683413:49
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Migrate to openjdk - part 2  https://review.openstack.org/14643413:54
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Use openjdk by default for java element  https://review.openstack.org/15947313:55
openstackgerritMerged stackforge/sahara-ci-config: Fix tempest client tests  https://review.openstack.org/16676413:58
*** Nikolay_St has quit IRC14:01
*** hogepodge has joined #openstack-sahara14:02
*** esikachev has joined #openstack-sahara14:02
*** chandankumar_ has joined #openstack-sahara14:03
*** coolsvap is now known as coolsvap|afk14:05
openstackgerritMerged openstack/python-saharaclient: Added support of instance locality  https://review.openstack.org/16079514:06
*** tnovacik has joined #openstack-sahara14:15
*** coolsvap has joined #openstack-sahara14:19
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Install jdk instead jre in openjdk element  https://review.openstack.org/16685614:29
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Migrate to openjdk - part 2  https://review.openstack.org/14643414:35
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Use openjdk by default for java element  https://review.openstack.org/15947314:38
*** Nikolay_St has joined #openstack-sahara14:47
*** chandankumar_ has quit IRC14:58
*** Nikolay_St has quit IRC15:03
*** Nikolay_St has joined #openstack-sahara15:04
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Fix pep8 errors  https://review.openstack.org/16687715:07
*** Nikolay_St has quit IRC15:27
*** chandankumar has joined #openstack-sahara15:54
pino|workhello15:54
pino|workcurrently, the fedora & centos images use the redhat-lsb element15:55
pino|workthe only thing it does is installing the "redhat-lsb" package, which drags quite a lot of dependencies (like qt3, qt4, some X libraries, poppler, cups, ghostscript, etc)15:56
pino|workit has been introduced with commit 6b46f637fc45b5d02e2ac0bd4ab0e10063f68166, for "CentOS" support, but not explaining why such package has been used15:57
pino|workis there any strong reason for installing redhat-lsb (and all the lot of dependencies), instead of either installing really needed stuff and/or let other packages drag dependencies as needed?15:57
*** chandankumar has quit IRC16:00
elmikopino|work: i'm trying to remember why we needed redhat-lsb, i feel like there was one small thing. we could definitely excise the extra dependencies.16:01
elmikopino|work: oh, sorry. i was thinking of adding redhat-lsb for a different patch. i'm not sure about this one, it was before i joined the project.16:04
*** chandankumar has joined #openstack-sahara16:10
pino|worksreshetnyak: i hope my last comment in https://review.openstack.org/#/c/166601/ clarifies your concerns16:10
elmikopino|work: for the record i usually at least *attempt* to build images against master dib, it doesn't always work16:13
pino|workone reason more for not blindly allowing newer dib releases16:14
elmikoyea16:14
elmikopino|work: re: the requirements.txt, i think it's nice to specify what we actually use, but s-i-e downloads its dib, will that change in the future?16:17
pino|worksee https://review.openstack.org/166603 & https://review.openstack.org/16661816:18
elmikoi saw 603, that's what was making me curious16:18
elmikoi had not seen 61816:19
*** coolsvap has quit IRC16:19
*** chandankumar has quit IRC16:20
*** sreshetn1 has joined #openstack-sahara16:31
*** chandankumar has joined #openstack-sahara16:36
*** chandankumar has quit IRC16:36
*** chandankumar has joined #openstack-sahara16:37
*** hdd has joined #openstack-sahara16:38
openstackgerritSergey Reshetnyak proposed stackforge/sahara-ci-config: Set public network id for tempest test  https://review.openstack.org/16692516:56
*** sgotliv has quit IRC17:02
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add information about cluster state in test report  https://review.openstack.org/16692817:02
openstackgerritEvgeny Sikachev proposed openstack/sahara: Rename templates in scanerio yaml files  https://review.openstack.org/16693317:20
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add information about cluster state in test report  https://review.openstack.org/16692817:22
*** esikachev has quit IRC17:23
*** hdd has quit IRC17:29
openstackgerritMerged stackforge/sahara-ci-config: Use global settings for dib jobs  https://review.openstack.org/16661617:36
*** sreshetn1 has quit IRC17:38
*** chandankumar has quit IRC17:40
openstackgerritMerged openstack/sahara-image-elements: Bump diskimage-builder version to 0.1.41  https://review.openstack.org/16593017:48
*** Nikolay_St has joined #openstack-sahara18:06
*** hdd has joined #openstack-sahara18:09
openstackgerritArtem Osadchiy proposed openstack/sahara: Set cluster mode on every fileserver node  https://review.openstack.org/16695018:15
openstackgerritArtem Osadchiy proposed openstack/sahara: Set cluster mode on every node  https://review.openstack.org/16695018:15
openstackgerritTrevor McKay proposed openstack/sahara: Add a CLI tool for managing default templates  https://review.openstack.org/16364918:17
openstackgerritArtem Osadchiy proposed openstack/sahara: Set cluster mode on every node  https://review.openstack.org/16695018:24
openstackgerritArtem Osadchiy proposed openstack/sahara: Add missed configs for ThriftJobTrackerPlugin  https://review.openstack.org/16695818:31
tellesnobregatmckay, hey, when a spark job is submitted, is it submitted from the master instance of from the cloud controller?18:32
tmckaytellesnobrega, yes.  It doesn't have to be, but it seemed like a simple thing to do and makes it easy to find the job logs18:48
openstackgerritEvgeny Sikachev proposed openstack/sahara: Rename templates in scenario yaml files  https://review.openstack.org/16693318:57
elmikotmckay: i'm doing some analysis of the config options between vanilla_1 and the hdp stuff now. it's tough lining up these values, there are several that exist in the vanilla stuff that do not in the hdp stuff. i'm a little out of my depth here in terms of which values should belong.19:01
elmikotmckay: when i finish, i'll share these files maybe you can help19:01
tmckayelmiko, thanks, sounds good.  Hopefully I should have some time soon.  I think I'm about done with the CLI stuff, but I believe it needs some unit tests19:03
*** vgridnev has quit IRC19:04
*** vgridnev has joined #openstack-sahara19:04
elmikotmckay: ok, no rush. i'll keep plugging away.19:05
elmikotmckay: i might switch over to the vanilla config hints, they're a little closer to what we have now. should be easier to implement.19:06
openstackgerritPino Toscano proposed openstack/sahara-image-elements: Adopt DIB_DEBUG_TRACE for tracing in elements  https://review.openstack.org/16697219:06
openstackgerritPino Toscano proposed openstack/sahara-image-elements: Add newline at the end of README.md  https://review.openstack.org/16697519:10
openstackgerritPino Toscano proposed openstack/sahara-image-elements: Adopt DIB_DEBUG_TRACE for tracing in elements  https://review.openstack.org/16697219:13
openstackgerritPino Toscano proposed openstack/sahara-image-elements: Add newline at the end of README.md  https://review.openstack.org/16697519:13
*** sreshetn1 has joined #openstack-sahara19:18
elmikopino|work: ping19:22
pino|workelmiko: yo19:22
elmikofor the debug trace, couldn't we add that as a function to the diskimage-create.sh and then just call the function in each element?19:22
pino|workor simply take -x arguments and pass them to disk-image-create19:23
elmikoi was thinkig about reducing the same code over and over, but the dib commit is similar so i guess it's ok19:24
pino|workyeah, and dib-lint "forces" element scripts to use it19:24
elmikogotcha19:25
elmikoyea, i wasn't suggesting removing it. just curious if we could have some sort of `check_debug_trace` function instead19:25
pino|work(but in general having unified switch for tracing is a GoodThingâ„¢)19:25
elmikoagreed19:25
pino|workno we don't, there was no way to enable dib tracing before, and now at least there's DIB_DEBUG_TRACE19:26
elmikoyea19:26
elmikoi was just thinking more about having the same 3 line blurb over and over19:27
pino|workeh, tell dib people :/19:27
elmikoyea, no worries19:27
elmikoi was more curious than anything19:27
pino|workbut honestly better three one/shot lines in a script than no debug at all, and need to manually sparkle set -x19:28
elmiko+119:29
pino|worko/ see you tomorrow19:29
*** pino|work has quit IRC19:29
*** tosky has quit IRC19:29
tmckayelmiko, crobertsrh, what do you think of all the log messages for the template CLI using name and id both, like "blah blah failed for template name (id)"19:46
elmikotmckay: that seems convenient to me19:47
tmckayimho, for human beings, names are much more meaningful19:47
tmckaywhich is why our parents didn't name us a random string of characters :)19:47
tmckayelmiko, okay, thanks.  I'll make a pass and fix them all up.  And then I think I'm really done.  Except for unit tests ;-)19:48
elmikoi like "name (id)"19:48
elmikothen there's no question19:48
tmckayagreed19:48
*** sgotliv has joined #openstack-sahara19:49
tellesnobregatmckay, sorry, i was away, i didnt understand your answer? the deploy is from inside the master?19:50
elmikotmckay: looking at sahara/service/edp/oozie/workflow_creator/workflow_factory.py , do you remember what this bug you mention in the TODO @ L326 was about? (did it merge)19:51
tmckaytellesnobrega, yes.  EDP uses ssh to launch a spark-submit command from a shell on the master node19:51
*** witlessb has quit IRC19:52
tellesnobregatmckay, is the launch_command.py that implements this ssh?19:52
tmckayelmiko, yeah, the UI used to consume the hints, and it expected args to be a dictionary19:52
elmikoah ok19:52
tmckaythat comment can probably go19:52
elmikoi'm fixing that in the root for vanilla, even though i think that function should ultimately disappear19:53
tellesnobregatmckay, i need the same behaviour for storm, deploying from the master19:53
*** chandankumar has joined #openstack-sahara19:54
tmckaytellesnobrega, launch_command.py is a wrapper that I wrote to log the command, save the pid file, redirect IO, etc.  ssh is from sahara/service/edp/spark/engine.py19:55
tmckaytellesnobrega, around line 30119:56
*** witlessb has joined #openstack-sahara19:56
tellesnobregatmckay, thanks19:56
tmckaytellesnobrega, all that stuff is necessary to launch the process from ssh, close all the file descriptors, and get the process id back19:56
crobertsrhsorry, missed the "bing"19:56
tmckayif you don't do it right, ssh will hang19:57
crobertsrhtmckay:  +1 to that format though19:57
tmckaycrobertsrh, okay.  I'm making more work for myself, but I think it will be much nicer in the logs.  Thanks.19:57
crobertsrhless work in the long run when people make garbage for input and come running to you19:58
tellesnobregatmckay, ok, i think i can use your launch_command.py as well19:58
tellesnobregatmckay, got the ssh and deploy command19:58
tellesnobregathanks19:58
tmckaytellesnobrega, lol, I was just going to say "I don't think there is anything spark specific in launch_command.py, I think it will work for you."19:58
crobertsrhYour program will tell them that they suck so you don't have to be the mean one.19:58
tellesnobregatmckay, lol i looks like it can work19:59
tmckaycrobertsrh, yeah, who wants to be mean20:00
*** az__ has joined #openstack-sahara20:26
openstackgerritMerged openstack/sahara-image-elements: Add newline at the end of README.md  https://review.openstack.org/16697520:32
*** chandankumar has quit IRC20:55
*** sgotliv has quit IRC20:59
*** jamielennox|away is now known as jamielennox21:02
*** crobertsrh is now known as _crobertsrh21:08
*** sreshetn1 has quit IRC21:23
openstackgerritAndrew Lazarev proposed openstack/sahara: Switched heat engine from JSON to HOT  https://review.openstack.org/15762321:40
openstackgerritTrevor McKay proposed openstack/sahara: Add a CLI tool for managing default templates  https://review.openstack.org/16364921:51
*** egafford has quit IRC21:52
*** tnovacik has quit IRC23:07
openstackgerritMerged openstack/sahara: Add new log messages where it's needed  https://review.openstack.org/16620823:18
*** chlong has joined #openstack-sahara23:30

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!