Tuesday, 2015-09-08

« Monday, 2015-09-07 Index Wednesday, 2015-09-09 »
*** hdd has joined #openstack-sahara01:14
*** david-ly_ has joined #openstack-sahara01:51
*** david-lyle has quit IRC01:54
*** witlessb has joined #openstack-sahara01:56
*** witlessb has quit IRC02:01
*** macjack has joined #openstack-sahara02:20
*** hdd has quit IRC02:56
*** dave-mcc_ has quit IRC02:58
*** hdd has joined #openstack-sahara03:50
*** jamielennox is now known as jamielennox|away03:56
*** Poornima_K has joined #openstack-sahara04:57
*** macjack has quit IRC05:04
*** vgridnev has joined #openstack-sahara05:45
*** sgotliv_ has quit IRC06:14
*** hdd has quit IRC06:24
*** ashishb has joined #openstack-sahara06:29
*** witlessb has joined #openstack-sahara07:18
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding missed public/protected support for jobs creation  https://review.openstack.org/22099407:38
*** sgotliv_ has joined #openstack-sahara07:49
*** chlong has quit IRC07:56
*** sgotliv_ has quit IRC08:03
*** vgridnev has quit IRC08:12
*** sgotliv_ has joined #openstack-sahara08:23
openstackgerritAndrey Pavlov proposed openstack/sahara: Adding doc about shared and protected resources  https://review.openstack.org/22119208:25
*** ashishb has quit IRC08:43
*** tmckay has joined #openstack-sahara08:51
*** vgridnev has joined #openstack-sahara08:58
*** tosky has joined #openstack-sahara09:06
*** sgotliv_ has quit IRC09:10
openstackgerritVitaly Gridnev proposed openstack/sahara: New doc about autoconfiguration policy  https://review.openstack.org/22102609:11
*** sgotliv_ has joined #openstack-sahara09:40
openstackgerritMerged openstack/sahara-image-elements: Add $JAVA_HOME/bin to PATH  https://review.openstack.org/21626509:47
openstackgerritVitaly Gridnev proposed openstack/sahara: Use custom flavor in gate  https://review.openstack.org/22043509:58
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Use JRE directory for JRE_HOME for RHEL-based os  https://review.openstack.org/21627209:59
openstackgerritVitaly Gridnev proposed openstack/sahara: Update plugin spi docs with new method  https://review.openstack.org/22093310:02
openstackgerritSergey Reshetnyak proposed openstack/sahara: Add keystone and swift url to /etc/hosts  https://review.openstack.org/21668410:15
openstackgerritArtem Osadchiy proposed openstack/sahara: Removed duplicated definition of support Impala in MapR plugin  https://review.openstack.org/22098310:56
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Move JRE_PATH/bin export to separate file  https://review.openstack.org/22125411:04
*** sgotliv_ has quit IRC11:05
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Move JRE_PATH/bin export to separate file  https://review.openstack.org/22125411:10
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding return of updated image for update_image, update_tags  https://review.openstack.org/22125711:16
openstackgerritVitaly Gridnev proposed openstack/sahara: Use custom flavor in gate  https://review.openstack.org/22043511:20
openstackgerritSergey Reshetnyak proposed stackforge/sahara-ci-config: Change path to ambari scenario file  https://review.openstack.org/21878711:21
openstackgerritVitaly Gridnev proposed openstack/sahara: [WIP] Convert True to string for image registry  https://review.openstack.org/22126011:30
openstackgerritMerged stackforge/sahara-ci-config: Change path to ambari scenario file  https://review.openstack.org/21878711:33
openstackgerritAndrey Pavlov proposed openstack/sahara: Adding doc about shared and protected resources  https://review.openstack.org/22119211:33
*** larainema has joined #openstack-sahara11:39
*** tmckay has left #openstack-sahara12:09
*** raildo-afk is now known as raildo12:12
*** egafford has joined #openstack-sahara12:18
*** dave-mccowan has joined #openstack-sahara12:18
*** chlong has joined #openstack-sahara12:22
*** sgotliv_ has joined #openstack-sahara12:23
openstackgerritMerged openstack/sahara: Fix mocks in scenario_unit tests  https://review.openstack.org/21883612:23
openstackgerritMerged openstack/sahara: Remove useless test dependency 'discover'  https://review.openstack.org/21875212:23
*** hdd has joined #openstack-sahara12:29
*** hdd has quit IRC12:52
openstackgerritArtem Osadchiy proposed openstack/sahara: Fixed Hive 1.0 failure on MapR plugin  https://review.openstack.org/22128712:52
*** crobertsrh has left #openstack-sahara12:53
openstackgerritlu huichun proposed openstack/sahara: [sahara doc fix] log guidelines doc  https://review.openstack.org/22029612:54
*** crobertsrh has joined #openstack-sahara12:55
openstackgerritVitaly Gridnev proposed openstack/sahara: Use custom flavor in gate  https://review.openstack.org/22043512:55
*** Poornima_K has quit IRC13:06
*** vgridnev has quit IRC13:07
*** vgridnev has joined #openstack-sahara13:07
openstackgerritVitaly Gridnev proposed openstack/sahara: Convert True to string for image registry  https://review.openstack.org/22126013:30
*** tellesnobrega is now known as tellesnobrega_af13:31
*** tellesnobrega_af is now known as tellesnobrega13:34
*** integralequation has quit IRC13:38
*** tmckay has joined #openstack-sahara13:41
openstackgerritSergey Reshetnyak proposed openstack/sahara: Register SSL cert in Java keystore to access to swift via SSL  https://review.openstack.org/21674813:51
toskycrobertsrh: before you start fixing https://bugs.launchpad.net/sahara/+bug/1493115, see https://review.openstack.org/#/c/221260/13:54
openstackLaunchpad bug 1493115 in Sahara "Can't register tags on images" [High,Confirmed] - Assigned to Chad Roberts (croberts)13:54
crobertsrhtosky:  Yeah, I saw that.  It does seem to fix the problem :)13:55
crobertsrhI will not re-fix it.13:55
crobertsrhThanks for the heads-up!13:55
vgridnevcrobertsrh, any ideas why True became wrong?14:05
crobertsrhvgridnev:  Not yet.  Was planning on looking at the nova client to see what changed (and why)14:06
vgridnevcrobertsrh, it looks like this commit broke that: https://github.com/openstack/nova/commit/5e1fa8b06a860935373915a531fce6e3b0d618fb14:10
crobertsrhIt sure does look like it.14:12
vgridnevoops, nope14:12
vgridnevit was far ago14:12
crobertsrhbut how did we just figure it out?14:13
crobertsrhYeah..was looking at the date14:13
crobertsrhI wonder if a recent commit changed the enum for True14:13
crobertsrhHmm, nope14:14
elmikocandidate for worst compiler switch ever, --set-true-to-false14:15
crobertsrhelmiko:  That's really just a sanity test for the devs.  If that switch ever makes sense to you, you are no longer sane.14:15
elmikolol14:15
*** tosky has quit IRC14:25
*** tosky has joined #openstack-sahara14:31
openstackgerritAndrey Pavlov proposed openstack/sahara: Adding doc about shared and protected resources  https://review.openstack.org/22119214:33
*** raildo is now known as raildo-afk14:34
openstackgerritSergey Reshetnyak proposed openstack/sahara: Register SSL cert in Java keystore to access to swift via SSL  https://review.openstack.org/21674814:35
*** raildo-afk is now known as raildo14:37
openstackgerritVitaly Gridnev proposed openstack/sahara: Use custom flavor in gate  https://review.openstack.org/22043514:38
*** david-ly_ is now known as david-lyle14:53
*** hdd has joined #openstack-sahara14:56
*** crobertsrh has quit IRC14:59
*** crobertsrh has joined #openstack-sahara15:01
elmikoi think we are going to need to go through the documentation again sometime for project name capitalization15:04
pino|workelmiko: Capitalization15:04
*** stanchan has quit IRC15:04
elmikoexactly ;)15:05
*** crobertsrh has quit IRC15:11
*** crobertsrh has joined #openstack-sahara15:16
*** crobertsrh has quit IRC15:22
*** vgridnev has quit IRC15:33
*** macjack has joined #openstack-sahara15:36
*** devlaps has joined #openstack-sahara15:43
tmckayvgridnev, ping15:49
*** crobertsrh has joined #openstack-sahara16:08
*** crobertsrh has quit IRC16:12
*** crobertsrh has joined #openstack-sahara16:16
*** crobertsrh1 has joined #openstack-sahara16:20
*** crobertsrh has quit IRC16:21
rickflareI can help witht that16:30
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/22104916:31
rickflareI am in kind of a holding pattern as I am awaiting assitance to get my RDO manager dev environment setup. I dont have a bare metal system at home that has ipmi.16:31
*** vgridnev has joined #openstack-sahara16:37
*** crobertsrh1 has quit IRC16:39
egaffordrickflare: I'm telling you man, try devstack!16:45
*** crobertsrh has joined #openstack-sahara16:46
openstackgerritVitaly Gridnev proposed openstack/sahara: Add spaces around function params for browser to linewrap on  https://review.openstack.org/19287116:52
openstackgerritTrevor McKay proposed openstack/sahara: Only add current directory to classpath for client deploy mode  https://review.openstack.org/22141417:08
openstackgerritTrevor McKay proposed openstack/sahara: Add SPARK_YARN_HISTORY_SERVER to default templates for cdh  https://review.openstack.org/22142017:22
*** macjack has quit IRC17:39
elmikoneed an opinion from the group17:47
elmikothe spark plugin docs contains this sentence fragment "When a user creates an Hadoop cluster using the Spark plugin,"17:48
elmikodoes "Hadoop cluster" make sense there? i'm thinking maybe "sahara cluster" or even just "cluster"17:48
elmikotmckay: i could use your advice about the swift status in spark, is that available through the job interface?17:51
*** tosky_ has joined #openstack-sahara17:54
*** tosky_ has quit IRC17:54
*** tosky has quit IRC17:57
*** ashishb has joined #openstack-sahara17:58
*** vgridnev has quit IRC18:00
*** vgridnev has joined #openstack-sahara18:06
*** vgridnev has quit IRC18:06
tmckayelmiko, hey18:28
elmikotmckay: hey18:29
elmikoi thought spark had swift support now, but i couldn't remember where we are at with it.18:30
tmckayelmiko, oh.  Yes, spark standalone and spark on cdh both can talk to swift, with the extra hadoop jar and the wrapper job to modify the hadoop config18:31
tmckayspark on ambari ... not sure18:31
openstackgerritMerged openstack/sahara: Adds IPv6 support to auto security group  https://review.openstack.org/21758118:31
*** openstackgerrit has quit IRC18:31
tmckaythat will need hdp 2.2 I think18:31
*** openstackgerrit has joined #openstack-sahara18:31
elmikotmckay: ok, so we can drop the "limitation" portion of the spark plugin doc that says it doesn't work"?18:31
tmckayprobably, where is that?18:32
tmckayI can take a look18:32
elmikosee the bottom of http://docs.openstack.org/developer/sahara/userdoc/spark_plugin.html18:32
tmckaylol, wow, not complicated.  Yes, that can totally go18:33
elmikohehe, awesome18:33
elmikothanks =)18:33
tmckaynp18:33
openstackgerritVitaly Gridnev proposed openstack/sahara: Use custom flavor in gate  https://review.openstack.org/22043518:42
*** vgridnev has joined #openstack-sahara18:48
openstackgerritMerged openstack/sahara: project_name is changed to optional parameter  https://review.openstack.org/21758619:00
elmikotmckay: still around?19:05
tmckayyep19:06
elmikowhat do you think about the second paragraph of this section, http://docs.openstack.org/developer/sahara/userdoc/spark_plugin.html#spark-configuration19:06
elmikoi'm thinking about breaking that off into some sort of "Manual job configuration (optional)" section19:07
*** vgridnev has quit IRC19:09
openstackgerritMerged openstack/sahara: [CDH] Fix problem with launching Spark jobs  https://review.openstack.org/22050019:09
*** ashishb has quit IRC19:09
tmckayelmiko, I don't know, sounds like general information to me19:09
elmikook, i'll leave it alone19:10
tmckayelmiko, I think it's supposed to be the "how" for the second sentence in the 1st paragraph19:10
elmikoit sounded like something that was adivising the way to run jobs on spark by logging in. i thought maybe it was older, from when we didn't have proper job support.19:10
tmckayMaybe change the title?19:10
elmikonah, i can just leave it. if people bring it up on review, i'm ok changing it then.19:11
tmckayIt's a pretty short doc, not too hard to get lost in there :)19:11
elmikohehe, you'd think19:11
tmckayI mean, not too hard not to get lost19:11
elmikolol19:11
openstackgerritMichael McCune proposed openstack/sahara: cleanup spark plugin documentation  https://review.openstack.org/22146819:12
elmikoi'm just trying to nail down the last few docs have yet to review19:12
elmikobut i have to admit, i don't know all the details on some of these.19:12
tmckaygotcha19:12
*** ashishb has joined #openstack-sahara19:17
*** vgridnev has joined #openstack-sahara19:17
openstackgerritEthan Gafford proposed openstack/sahara: Documenting interface map  https://review.openstack.org/21938119:22
*** Ashish has joined #openstack-sahara19:30
*** ashishb has quit IRC19:30
openstackgerritTrevor McKay proposed openstack/sahara: Set the flavor to large for the cdh 5.4.0 name node in template.conf  https://review.openstack.org/22147519:38
elmikoegafford: ping19:39
tmckayvgridnev, interesting bug I think in Spark. So for cdh 53, if you run in yarn-cluster mode on the name node without the spark_yarn_history_server on the same node, spark will fail to upload and distribute the jars and --files because it says "Source and destination filesystems are the same".19:40
tmckayBut, if you run on the spark_yarn_history_server, it works19:41
tmckayAnd, it works in all cases in cdh 5.4.019:41
tmckaySo, your change to run from the history server patches some problem in 5.3 :)19:41
tmckayI looked through spark jira's but didn't find it.  something changed in spark along the way19:41
vgridnevoooh19:41
tmckayvgridnev, so good work :_19:42
tmckay:)19:42
vgridnevprobably we should deprecate cdh5319:42
vgridnevgreat fix19:43
*** Ashish has quit IRC19:43
rickflarethere are still a ton of production clouds running cdh5319:43
tmckayyeah, I don't see a need to kick it yet.  And the change fixes the issue in 53, just an interesting observation19:44
tmckayI ran without vgridnev's patch on 53 and 54, worked on 54, failed on 53, started looking at why.19:44
egaffordelmiko: pong19:46
egaffordWhat's up?19:46
elmikoegafford: just a question about the job interface map doc update19:46
egaffordelmiko: Shoot.19:47
elmikoi added my thoughts on the review, i'm kinda curious about a json example for creating the map. i was just a little confused about the pieces while reading it.19:47
elmikootherwise, solid stuff19:47
egaffordSure; a json example for the interface itself would be sensible.19:48
elmikocool, i think it would help bring the whole thing together19:48
egaffordYup, absolutely.19:48
elmikothanks!19:50
*** crobertsrh has quit IRC19:59
*** crobertsrh has joined #openstack-sahara20:01
openstackgerritEthan Gafford proposed openstack/sahara: Documenting interface map  https://review.openstack.org/21938120:10
*** raildo is now known as raildo-afk20:34
*** tiny-hands has joined #openstack-sahara20:35
*** ekarlso has quit IRC20:38
*** vgridnev has quit IRC20:40
*** raildo-afk is now known as raildo20:44
*** vgridnev has joined #openstack-sahara20:53
*** raildo is now known as raildo-afk20:56
*** ekarlso has joined #openstack-sahara20:57
*** ekarlso has quit IRC20:57
*** ekarlso has joined #openstack-sahara20:57
*** crobertsrh is now known as _crobertsrh21:11
*** vgridnev has quit IRC21:13
*** tmckay has left #openstack-sahara21:16
*** egafford has quit IRC21:22
*** egafford has joined #openstack-sahara21:22
*** tiny-hands has quit IRC21:31
*** vgridnev has joined #openstack-sahara21:31
*** devlaps has quit IRC21:38
*** vgridnev has quit IRC21:44
*** jamielennox|away is now known as jamielennox21:51
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/22104922:06
*** witlessb has quit IRC22:57
*** chlong has quit IRC23:27
*** chlong has joined #openstack-sahara23:28
« Monday, 2015-09-07 Index Wednesday, 2015-09-09 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!