Monday, 2015-08-24

*** japplewhite has joined #openstack-sahara00:11
*** witlessb has quit IRC00:12
*** macjack has quit IRC00:15
*** saneax has quit IRC01:21
*** jamielennox is now known as jamielennox|away02:20
*** jamielennox|away is now known as jamielennox02:32
openstackgerritKen Chen proposed openstack/sahara: Enable YARN ResourceManager HA in CDH plugin  https://review.openstack.org/21300002:38
openstackgerritKen Chen proposed openstack/sahara: Add CDH HDFS HA part in the user doc  https://review.openstack.org/21255202:48
*** macjack has joined #openstack-sahara03:06
*** japplewhite has quit IRC03:19
*** japplewhite has joined #openstack-sahara03:19
*** japplewhite has quit IRC03:23
*** japplewhite has joined #openstack-sahara03:25
*** japplewhite has quit IRC03:28
*** openstackgerrit has quit IRC03:31
*** openstackgerrit has joined #openstack-sahara03:31
*** japplewhite has joined #openstack-sahara03:42
*** japplewhite has quit IRC04:01
*** Nikolay_St has quit IRC04:03
*** coolsvap|away is now known as coolsvap04:06
*** Poornima has joined #openstack-sahara04:23
*** Poornima has quit IRC04:35
*** sgotliv has joined #openstack-sahara04:44
*** sgotliv has quit IRC04:52
*** saneax has joined #openstack-sahara05:06
*** Poornima has joined #openstack-sahara05:17
*** hdd has joined #openstack-sahara05:20
*** nkrinner has joined #openstack-sahara05:36
*** jamielennox is now known as jamielennox|away05:47
*** pino|work has joined #openstack-sahara05:57
*** sgotliv has joined #openstack-sahara06:01
*** macjack has quit IRC06:02
*** macjack has joined #openstack-sahara06:04
*** hdd has quit IRC06:11
*** sgotliv has quit IRC06:16
*** saneax has quit IRC06:17
*** saneax has joined #openstack-sahara06:26
*** vgridnev has joined #openstack-sahara06:47
*** macjack has quit IRC07:13
*** macjack has joined #openstack-sahara07:14
*** macjack has quit IRC07:14
*** macjack has joined #openstack-sahara07:16
*** macjack has quit IRC07:16
*** Nikolay_St has joined #openstack-sahara07:18
*** vgridnev has quit IRC08:23
*** witlessb has joined #openstack-sahara08:34
*** tosky has joined #openstack-sahara08:46
*** tosky has quit IRC08:46
*** sanjayu_ has joined #openstack-sahara08:47
*** tosky has joined #openstack-sahara08:48
*** sanjayu has quit IRC08:50
*** tosky has quit IRC08:51
*** sanjayu_ has quit IRC08:52
*** sanjayu has joined #openstack-sahara08:52
*** tosky has joined #openstack-sahara08:53
*** vgridnev has joined #openstack-sahara09:12
*** DWfuturetec has joined #openstack-sahara09:21
*** macjack has joined #openstack-sahara09:30
*** macjack has quit IRC09:35
*** macjack has joined #openstack-sahara09:35
*** degorenko has joined #openstack-sahara09:36
*** Poornima has quit IRC09:52
*** Poornima has joined #openstack-sahara09:57
*** macjack has quit IRC10:00
*** vgridnev has quit IRC10:08
openstackgerritMerged openstack/sahara: Remove unneeded starting ntp  https://review.openstack.org/21553710:08
*** DWfuturetec has quit IRC10:11
openstackgerritMerged openstack/sahara: Updated from global requirements  https://review.openstack.org/21595910:20
openstackgerritMerged openstack/sahara: Updated from global requirements  https://review.openstack.org/21598910:20
*** ekarlso has quit IRC10:32
*** ekarlso has joined #openstack-sahara10:32
*** DWfuturetec has joined #openstack-sahara10:35
*** jamielennox|away has quit IRC10:50
*** sgotliv has joined #openstack-sahara11:03
*** jamielennox|away has joined #openstack-sahara11:04
*** jamielennox|away is now known as jamielennox11:04
*** vgridnev has joined #openstack-sahara11:05
openstackgerritVitaly Gridnev proposed openstack/sahara: Add open ports for fake plugin  https://review.openstack.org/21605211:17
*** sgotliv has quit IRC11:21
openstackgerritVitaly Gridnev proposed openstack/sahara: Add open ports for fake plugin  https://review.openstack.org/21605211:32
*** Poornima has quit IRC11:48
*** Poornima has joined #openstack-sahara11:50
openstackgerritVitaly Gridnev proposed openstack/sahara: Add open ports for fake plugin  https://review.openstack.org/21605212:15
*** sgotliv has joined #openstack-sahara12:25
*** DWfuturetec has quit IRC12:33
*** hdd has joined #openstack-sahara12:56
*** Poornima has quit IRC13:03
*** tmckay has joined #openstack-sahara13:13
*** japplewhite has joined #openstack-sahara13:25
*** jamielennox has quit IRC13:26
*** _crobertsrh is now known as crobertsrh13:28
*** links has joined #openstack-sahara13:31
*** egafford has joined #openstack-sahara13:37
*** japplewhite has quit IRC13:38
*** hdd has quit IRC13:41
egaffordHello all! Please see https://etherpad.openstack.org/p/sahara-liberty-bug-fix-day for bugs. If we possibly can, let's hit high-prios first.13:42
*** jamielennox|away has joined #openstack-sahara13:43
egafford(Not sure how many of us are about yet.)13:43
*** jamielennox|away is now known as jamielennox13:43
* elmiko salutes egafford13:46
elmikosir, yes sir!13:46
egaffordelmiko: Having trouble finding an appropriate smiley to express the mix of sincere amusement and :/ that deserves.13:47
*** saneax has quit IRC13:47
elmikohaha13:47
egaffordThat absurd happy fistbump emote from a few weeks ago would work probably.13:47
egaffordelmiko: This one is interesting: https://bugs.launchpad.net/sahara/+bug/148665313:49
openstackLaunchpad bug 1486653 in Sahara "Create big cluster failed because token expired" [High,In progress] - Assigned to Sergey Reshetnyak (sreshetniak)13:49
egafford(And very related to work we've both done.)13:50
elmikoyea, it's probably the type of thing where we should attempt to re-acquire the token when we see a failure13:50
tmckayworking on the swift/spark integration bug, like I was on Friday13:52
egaffordtmckay: Woot.13:52
tmckaycurrent problem is CDH cluster install fails13:52
tmckaytells me it can't install the Oozie sharelib13:52
egaffordtmckay: That is a significant problem, indeed.13:52
tmckayany ideas? anyone ever seen this?13:52
egaffordDon't think so, no. tmckay: what's the bug # on that? I want to make sure I get it on the sheet early.13:53
elmikoi've only seen that during image creation, not cluster creation13:53
tmckayhttps://bugs.launchpad.net/sahara/+bug/148654413:54
openstackLaunchpad bug 1486544 in Sahara "[Spark1.3] Spark jobs doesn't work with Swift" [Critical,Triaged] - Assigned to Trevor McKay (tmckay)13:54
tmckayso the spark standalone stuff is fixed, but yarn scheduler is still broken13:54
elmikotmckay: so actually i was wrong about the cdh stuff. i was able to get a 2-node cluster working last week with the spark scenario tests13:55
vgridnevtmckay, have you tried to rebuild image?13:55
tmckayjust made a new image on Friday13:55
crobertsrhegafford:  Oh...thanks for the bug fix day reminder :)13:56
tmckayvgridnev, elmiko, do you know of a known good image somewhere that I can pull down?  Just to make sure?13:57
egaffordcrobertsrh: No problem.13:57
elmikotmckay: i would share mine, but the internet here is very slow13:57
vgridnevtmckay, do you use cents image?13:58
vgridnevcentos*13:58
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Set $JAVA_HOME/bin directory to PATH  https://review.openstack.org/21626514:01
tmckayvgridnev, yes14:02
SergeyLukjanovegafford, thx for organising the bug fix day ;)14:02
egaffordSergeyLukjanov: NP.14:02
*** jamielennox has quit IRC14:08
*** jamielennox|away has joined #openstack-sahara14:10
*** jamielennox|away is now known as jamielennox14:11
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Use JRE directory for JRE_HOME for RHEL-based os  https://review.openstack.org/21627214:11
*** DWfuturetec has joined #openstack-sahara14:13
elmikoegafford: fyi, i'm picking up https://bugs.launchpad.net/sahara/+bug/148649314:21
openstackLaunchpad bug 1486493 in Sahara "Tests return 0 exit code after fail" [High,Triaged] - Assigned to Michael McCune (mimccune)14:21
elmikoi marked the pad14:21
egaffordelmiko: I noticed!14:22
* egafford gives elmiko the "first non-egafford pad update' badge.14:23
egaffordHighly coveted.14:23
elmikohaha14:23
elmikonice, just wanted to keep you in the loop ;)14:23
egaffordelmiko: Thanks very much; more communication is better.14:24
vgridnevegafford, SergeyLukjanov was first14:27
egaffordvgridnev: :014:27
* elmiko hands his badge back to egafford 14:27
vgridnevegafford, see line 22314:28
* egafford hands the badge to SergeyLukjanov, with all due apologies.14:28
egaffordThe committee for distribution of bugfix-day-related badges sincerely regrets this error, and reaffirms its commitment to transparency and justice.14:29
elmikolol14:29
SergeyLukjanov:)14:30
*** macjack has joined #openstack-sahara14:35
egaffordSergeyLukjanov: Where does sreshetnyak work (in terms of TZ)?14:35
openstackgerritMerged openstack/sahara-image-elements: Update CentOS-6.6 image version  https://review.openstack.org/21568614:35
egaffordvgridnev: ^ You will probably also know this.14:38
vgridnevegafford, right now all mirantis-sahara folks are in saratov14:39
egaffordsreshetnyak has (reasonably!) signed up for a bunch of reasonably high-prio bugs before bugfix week; I want to get an idea of what he thinks he'll get done and which bugs he wouldn't mind someone else grabbing.14:39
egaffordvgridnev: Ooh, ouch.14:39
egaffordThought you were largely after us (in SF) rather than before (in Saratov). My apologies.14:40
sreshetnyakegafford: I'm working on #1488101 and #148665314:43
sreshetnyakand #1478059 on review14:44
openstackgerritMerged openstack/sahara-image-elements: Fix centos7 building image  https://review.openstack.org/20995114:44
openstackgerritMerged openstack/sahara-image-elements: Added support of MapR 5.0.0 to sahara-image-elements  https://review.openstack.org/20786514:45
egaffordsreshetnyak: What about https://bugs.launchpad.net/sahara/+bug/1478059 and https://bugs.launchpad.net/sahara/+bug/1475263?14:46
openstackLaunchpad bug 1478059 in Sahara "Sahara uses publicURL by default" [High,In progress] - Assigned to Sergey Reshetnyak (sreshetniak)14:46
openstackLaunchpad bug 1475263 in Sahara kilo "Borders in requirements.txt not sorted as in global" [High,In progress] - Assigned to Sergey Reshetnyak (sreshetniak)14:46
openstackgerritMerged openstack/sahara-image-elements: Remove libevent installation from MapR plugin  https://review.openstack.org/21452614:46
*** bapalm- has joined #openstack-sahara14:46
sreshetnyakttps://bugs.launchpad.net/sahara/+bug/1478059 fix on review https://review.openstack.org/#/c/206017/14:47
egaffordsreshetnyak: Good then; in terms of 1475263, do you want to keep that in addition to the two you're currently working on, or release it back into the pool?14:50
sreshetnyakI think need to move this bug to invalid or incomplete because it's a problem in ci-build-package system.14:57
egaffordsreshetnyak: Reasonable.15:02
*** Erming_ has quit IRC15:03
*** nkrinner has quit IRC15:14
*** Nikolay_St has quit IRC15:35
*** tellesnobrega has quit IRC15:37
*** IBerezovskiy has joined #openstack-sahara15:37
*** egafford has quit IRC15:40
*** egafford has joined #openstack-sahara15:41
*** tellesnobrega has joined #openstack-sahara15:43
*** vgridnev has quit IRC16:02
SergeyLukjanovelmiko, I've just release python-barbicanclient 3.3.016:04
SergeyLukjanovAFAIR you've been waiting for it :)16:04
*** links has quit IRC16:08
*** hdd has joined #openstack-sahara16:18
*** hdd has quit IRC16:27
*** hdd has joined #openstack-sahara16:37
*** IBerezovskiy has quit IRC16:39
elmikoSergeyLukjanov: yea, that was the precursor to castellan release, thanks!16:42
*** Nikolay_St has joined #openstack-sahara16:44
pino|workelmiko: of course, first a barbican, then a castellan around it, makes sense16:47
elmikopino|work: lol, yes ;)16:47
*** sgotliv has quit IRC16:47
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add workaround for POST image updating  https://review.openstack.org/21633616:50
*** vgridnev has joined #openstack-sahara16:50
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add workaround for POST image updating  https://review.openstack.org/21633616:51
openstackgerritMerged openstack/sahara: Add support of SSL in scenario tests  https://review.openstack.org/21247616:51
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add workaround for POST image updating  https://review.openstack.org/21633616:52
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add workaround for POST image updating  https://review.openstack.org/21633616:56
SergeyLukjanovelmiko, pino|work, ok cool16:56
pino|workSergeyLukjanov: mine was just a joke... :)16:56
elmikoSergeyLukjanov: for me, it means i will probably start pushing the improved secret storage reviews soon™16:57
*** melaks has joined #openstack-sahara17:19
egaffordPicking up https://bugs.launchpad.net/sahara/+bug/1436425.17:19
openstackLaunchpad bug 1436425 in Sahara "[CDH] Too many connection" [High,Confirmed] - Assigned to Ethan Gafford (egafford)17:19
elmikogood luck egafford, that one looked interesting...17:19
egaffordelmiko: Yeah, looks like a probable SQL misconfig, but it might be deeper.17:20
elmikothat would be cool if it's just sql config17:20
egaffordYeah, either that or we're doing something really quite awful with connection pooling. Or CDH is.17:21
egaffordelmiko: How are our overconfident tests coming along?17:22
egaffordManage to repro?17:22
elmikoyes, i can repro. i'm digging through the testr and script stuff now17:22
elmikojust trying to figure out what return code is not being picked up17:23
egaffordCool. Glad that got picked up early; very worrisome really.17:23
elmikoi actually noticed this last week when i was running the scenario tests, mainly why i picked this one up17:23
openstackgerritMerged stackforge/sahara-ci-config: Add workaround for POST image updating  https://review.openstack.org/21633617:30
*** vgridnev has quit IRC18:10
*** bapalm- is now known as bapalm_18:19
*** vgridnev has joined #openstack-sahara18:22
elmikoegafford: so, this is weird with #148649318:24
elmikoi created a special fail test, that does an assertEqual(True, False)18:24
elmikoif i run that through the scenario tests, then i get a fail on the command line18:24
elmikobut if i run an actual scenario test, that fails, i get success on the cli18:25
* elmiko scratches head18:25
egaffordelmiko: Yeah, I'm absolutely seeing it too (cluster bombs out with resource error, scenario tests give me a thumbs-up.)18:27
elmikoyea, same18:27
elmikoi'm digging in  a little more here, i do see a difference in the pretty_tox.sh return18:28
elmikothis is so weird18:28
egaffordWonder if we're catching a specific exception type, logging it, and not reraising?18:28
elmikoi think that might be possible18:28
egaffordYour assertion exception got through, but.18:28
elmikowell, if i do the assertEqual, then yea, i get a true failure from testr18:29
elmikoand even if i raise and exception i'm seeing a true fail18:29
elmikobut, the scenario test i'm running, for spark, raises an assertion, and it says i have 1 failed test, but tests says "yay you win!"18:30
elmikooh well, back to the drawing board18:30
*** macjack has quit IRC18:42
openstackgerritMerged openstack/sahara: Remove libevent installation from MapR plugin  https://review.openstack.org/21452018:49
egaffordelmiko: In scenario.base.BaseTestCase.tearDown, I added "self.fail("Scenario tests failed")" before the check for whether any(c['status'] == CHECK_FAILED_STATUS), and the tests passed. I also tried raising an exception there.18:50
egaffordelmiko: I'm uncertain that the tearDown method is being called.18:50
elmikoegafford: good call, i can confirm that a self.fail will produce an error on the cli18:50
elmikoi'm starting to be curious about the fixtures that we use in the _poll_job_status18:51
elmikoi wonder if it's doing something, as my jobs are the ones that are failing18:51
egaffordelmiko: So you're seeing self.fail fail the tests? Curious.18:51
egaffordWhere did you inject self.fail to test?18:52
elmikoegafford: yea, i have a scenario harness setup, and i just call self.fail('oops') and i see it happen18:52
elmikocurrently, i'm tracing through the scenario base class to see if i can figure out why it's eating the self.fail calls, because i can see those happen in the test that doesn't report a failure18:53
openstackgerritMerged openstack/sahara: Actually install Heat for the Hest-based jobs  https://review.openstack.org/21595219:01
openstackgerritMerged openstack/sahara: Fail if FAILED in the scenario tests run log  https://review.openstack.org/21596219:01
elmikoegafford: wonder if we need to do that ^19:03
elmikoSergeyLukjanov: on https://review.openstack.org/215962 did you look into why the scenario tests don't report properly to the shell?19:03
elmikoegafford: ok, i can fully reproduce this with my custom code now. i am calling self.fail and i get success on the cli19:05
openstackgerritMerged openstack/sahara: Implement ability of creating flavor for scenario tests  https://review.openstack.org/20356719:05
openstackgerritMerged openstack/sahara: Update plugin version for transient tests to vanilla 2.7.1  https://review.openstack.org/21362219:05
openstackgerritMerged openstack/sahara: Fix HBase config name when using HA with HDP 2.0.6  https://review.openstack.org/21286519:07
openstackgerritMerged openstack/sahara: Removed confusing typos in utils/openstack/base.py file  https://review.openstack.org/21246019:07
openstackgerritMerged openstack/sahara: Add validation rules about IMPALAD  https://review.openstack.org/21553419:07
vgridnevelmiko, egafford tmckay sreshetnyak crobertsrh I need your help in reviewing 1 line long bug fix for horizon19:10
vgridnevhttps://review.openstack.org/#/c/216384/1/openstack_dashboard/contrib/sahara/content/data_processing/jobs/workflows/create.py,cm19:11
vgridnevhttps://review.openstack.org/#/c/216384/119:11
egaffordelmiko: I think we're relying on tearDown to report failure status, though; if you look at the track_result decorator, we're creating a custom test_info dict for each test, and failing only if any of those dicts' statuses == CHECK_FAILED_STATUS. We reraise for each individual test, but the test driver may eat that exception to keep running further tests. I'm not actually seeing the table tearDown prints in the positive or negative19:11
crobertsrhwill do19:11
vgridnevthanks19:11
elmikoegafford: i don't think the issue is with the test class19:12
elmikoi think it has more to do with the way os.system reports the exit status on unix systems19:12
egaffordelmiko: Cool.19:12
elmikoegafford: and i have a very simple test that shows this19:13
egaffordvgridnev: I'm thinking we're going to need a lot of patch sets on this one. :)19:13
egaffordelmiko: Nice! If you've confirmed a cause, I'll stop poking your bug's bear.19:13
vgridnevegafford, around 5019:14
egaffordvgridnev: Yeah. Honestly, I don't see that we're going to be able to merge this without approval from at least a simple majority of the TC.19:14
elmikoegafford: here is the essence of our issue, http://paste.openstack.org/show/426307/19:15
egaffordvgridnev: (I will wait for CI, but obviously this looks fine. :) )19:15
vgridnevCI? you mean jenkins?19:16
egaffordvgridnev: Yes, I suppose that is what I mean, looking at my own Horizon patch for reference.19:17
openstackgerritVitaly Gridnev proposed openstack/sahara: Use internalURL endpoint by default in Sahara  https://review.openstack.org/20601719:21
openstackgerritMichael McCune proposed openstack/sahara: Changing scenario runner to use subprocess  https://review.openstack.org/21639619:33
egaffordelmiko: Nice!19:34
elmikoegafford: for all that it was a 2 line change lol19:34
elmikobut, i learned something, so that's cool19:35
egaffordYeah, awesome is not measured in LOC.19:35
elmiko+119:36
elmikoegafford: i feel like that quote would make a good motivational poster for an engineering office19:42
egaffordelmiko: Huh; os.system reports 512 vs subprocess' 2 on one error case. Is there a modulo going on in sys.exit?19:42
elmikoegafford: it has to do with how os.system returns the exit code19:43
egafford(They both return a plain old int, type-wise.)19:43
elmikoos.system returns a 16-bit number, high byte is the exit code, low byte is the signum that killed19:43
elmikoso, 512 would make sense for a 219:43
elmikoi kept getting 256 for a 119:44
egaffordRight, absolutely. There seemed to be a byteshift going on.19:44
elmikoand since the sys.exit only takes an 8bit, then boom19:44
egafford...Ah. Rockin'.19:44
elmikobut, i guess it only works that way on unixen, so for windows it might work fine19:44
egaffordIt just truncates it down to 0 byte, huh.19:44
elmikoright19:44
egaffordWow, okay then.19:45
elmikoyea, on unix exit status can only be a byte19:45
egaffordFrom msdn: System Error Codes (0-499)19:45
elmikointeresting...19:46
egaffordhttps://msdn.microsoft.com/en-us/library/windows/desktop/ms681382%28v=vs.85%29.aspx19:46
elmikobut i have a feeling os.system works different on windows anyways19:46
egaffordI guess single bytes just cramped MS' style. Quite possible, yeah.19:46
elmikolol19:46
vgridnevtellesnobrega, let's don't nitpick on commit message on big change: https://review.openstack.org/#/c/195568/920:03
tellesnobregasure20:03
tellesnobregavgridnev, sorry for that20:04
*** sgotliv has joined #openstack-sahara20:36
*** Nikolay_St has quit IRC20:40
*** DWfuturetec has quit IRC20:59
*** DWfuturetec has joined #openstack-sahara21:17
*** hdd has quit IRC21:22
*** tmckay has left #openstack-sahara21:24
*** crobertsrh is now known as _crobertsrh21:26
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add API methods for call verifications, DB table  https://review.openstack.org/19671321:35
*** sgotliv has quit IRC21:42
*** egafford is now known as _egafford21:48
*** pino|work has quit IRC21:56
*** vgridnev has quit IRC21:59
*** _egafford has quit IRC22:07
*** tosky has quit IRC22:49
*** vgridnev has joined #openstack-sahara22:50
*** tosky has joined #openstack-sahara22:50
*** tosky has quit IRC22:51
*** vgridnev has quit IRC22:52
*** tosky has joined #openstack-sahara22:53
*** hdd has joined #openstack-sahara22:53
*** tosky has quit IRC22:55
*** tosky has joined #openstack-sahara22:57
*** tosky has quit IRC22:58
*** tosky has joined #openstack-sahara22:59
*** tosky has quit IRC23:01
*** DuncanT has quit IRC23:38
*** DuncanT has joined #openstack-sahara23:38
*** DWfuturetec has quit IRC23:42
*** witlessb has quit IRC23:43

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!