Monday, 2015-08-17

*** hdd has joined #openstack-sahara00:25
*** hdd has quit IRC00:30
openstackgerritKen Chen proposed openstack/sahara: Enable YARN ResourceManager HA in CDH plugin  https://review.openstack.org/21300001:22
*** rodrigod` has joined #openstack-sahara01:54
*** rodrigod` is now known as rodrigods01:57
*** macjack has joined #openstack-sahara02:07
*** logan2 has quit IRC02:18
*** logan2 has joined #openstack-sahara02:57
openstackgerritweiting-chen proposed openstack/sahara-specs: Add new spec for NFS-as-a-data-source blueprint  https://review.openstack.org/21083903:05
*** saneax has joined #openstack-sahara03:58
*** saneax has quit IRC04:03
*** openstack has joined #openstack-sahara04:18
*** saneax has joined #openstack-sahara04:54
*** Poornima has joined #openstack-sahara05:17
*** nkrinner has joined #openstack-sahara05:27
*** coolsvap|away is now known as coolsvap05:32
*** macjack has quit IRC05:38
*** hdd has joined #openstack-sahara05:38
*** macjack has joined #openstack-sahara05:42
*** coolsvap is now known as coolsvap|away05:55
*** hdd has quit IRC06:24
*** saneax has quit IRC06:29
*** macjack has quit IRC06:30
*** witlessb has joined #openstack-sahara06:32
openstackgerritwilly lin proposed openstack/sahara: Rename "get_job_status" to "get_job_info"  https://review.openstack.org/21361106:34
*** macjack has joined #openstack-sahara06:36
*** macjack has quit IRC06:53
*** macjack has joined #openstack-sahara06:55
*** esikachev has joined #openstack-sahara06:57
*** macjack has quit IRC07:06
-openstackstatus- NOTICE: Gerrit is currently under very high load and may be unresponsive. infra are looking into the issue.07:07
*** macjack has joined #openstack-sahara07:07
*** macjack has quit IRC07:19
*** macjack has joined #openstack-sahara07:22
*** Nikolay_St has joined #openstack-sahara07:35
openstackgerritEvgeny Sikachev proposed openstack/sahara: Update plugin version for transient tests to vanilla 2.7.1  https://review.openstack.org/21362207:39
*** vgridnev has joined #openstack-sahara07:59
*** sgotliv has joined #openstack-sahara08:09
*** sgotliv has quit IRC08:38
*** sgotliv has joined #openstack-sahara08:52
*** degorenko has joined #openstack-sahara08:55
*** sgotliv has quit IRC09:02
*** sgotliv has joined #openstack-sahara09:14
*** tosky has joined #openstack-sahara09:21
*** macjack has quit IRC09:31
openstackgerritDenis Egorenko proposed openstack/puppet-sahara: Configure rpc options separately from ceilometer notifications  https://review.openstack.org/19874409:52
*** macjack has joined #openstack-sahara10:14
openstackgerritDenis Egorenko proposed openstack/puppet-sahara: Remove Sqlite validation for database_connection  https://review.openstack.org/21365010:14
openstackgerritDenis Egorenko proposed openstack/puppet-sahara: Configure rpc options separately from ceilometer notifications  https://review.openstack.org/19874410:16
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add force options for apt-get upgrade  https://review.openstack.org/21365110:21
-openstackstatus- NOTICE: review.openstack.org (aka gerrit) is going down for an emergency restart10:22
*** ChanServ changes topic to "review.openstack.org (aka gerrit) is going down for an emergency restart"10:22
openstackgerritDenis Egorenko proposed openstack/puppet-sahara: Configure rpc options separately from ceilometer notifications  https://review.openstack.org/19874410:27
*** macjack has quit IRC10:29
openstackgerritLuigi Toscano proposed openstack/sahara: doc, sahara-templates: fix typo  https://review.openstack.org/21365310:33
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add force options for apt-get upgrade  https://review.openstack.org/21365110:44
*** ashishb has joined #openstack-sahara10:44
openstackgerritEvgeny Sikachev proposed stackforge/sahara-ci-config: Update version of vanilla plugin for transient check  https://review.openstack.org/21365710:44
*** ChanServ changes topic to "OpenStack Sahara // IRC Meetings - http://eavesdrop.openstack.org/#OpenStack_Data_Processing_(Sahara)_Team_Meeting"10:49
-openstackstatus- NOTICE: Gerrit restart has resolved the issue and systems are back up and functioning10:49
*** sgotliv has quit IRC11:01
openstackgerritSergey Lukjanov proposed openstack/sahara: Make infra engine configurable in devstack plugin  https://review.openstack.org/21354511:03
openstackgerritSergey Lukjanov proposed openstack/sahara: Run scenario tests for the fake plugin in gate  https://review.openstack.org/21354611:03
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding updates for clusters, jobs, job binary internals  https://review.openstack.org/21366511:06
openstackgerritMerged stackforge/sahara-ci-config: Add force options for apt-get upgrade  https://review.openstack.org/21365111:06
*** esikachev has quit IRC11:11
openstackgerritSergey Lukjanov proposed openstack/sahara: Add sample usage info for pre_test_hook  https://review.openstack.org/21366711:11
*** esikachev has joined #openstack-sahara11:19
openstackgerritEvgeny Sikachev proposed stackforge/sahara-ci-config: Update version of vanilla plugin for transient check  https://review.openstack.org/21365711:21
openstackgerritDenis Egorenko proposed openstack/puppet-sahara: Update parameters for Sahara  https://review.openstack.org/20598811:21
*** sgotliv has joined #openstack-sahara11:23
openstackgerritEvgeny Sikachev proposed stackforge/sahara-ci-config: Update version of vanilla plugin for transient check  https://review.openstack.org/21365711:28
openstackgerritDenis Egorenko proposed openstack/puppet-sahara: Configure rpc options separately from ceilometer notifications  https://review.openstack.org/19874411:30
openstackgerritAndrey Pavlov proposed openstack/sahara: Formatting and mounting methods changed for ironic  https://review.openstack.org/20048311:42
*** sgotliv has quit IRC11:42
*** sgotliv has joined #openstack-sahara11:57
*** Poornima has quit IRC11:59
*** chlong has quit IRC11:59
*** sgotliv has quit IRC11:59
*** sgotliv has joined #openstack-sahara12:00
*** witlessb has quit IRC12:07
*** witlessb has joined #openstack-sahara12:08
openstackgerritAndrey Pavlov proposed openstack/sahara: Adding shared and protected resources support  https://review.openstack.org/19556812:15
openstackgerritAndrey Pavlov proposed openstack/sahara: Adding is_public and is_protected fields support  https://review.openstack.org/19506512:15
*** DWfuturetec has joined #openstack-sahara12:15
openstackgerritLuigi Toscano proposed openstack/sahara: Scenario tests: store ssh key if resources are retained  https://review.openstack.org/21369012:20
openstackgerritSergey Lukjanov proposed openstack/sahara: Add sample usage info for pre_test_hook  https://review.openstack.org/21366712:21
openstackgerritSergey Lukjanov proposed openstack/sahara: Run scenario tests for the fake plugin in gate  https://review.openstack.org/21354612:21
*** tellesnobrega_af has quit IRC12:28
SergeyLukjanovhey folks, does anybody wants to chair the bug triage day?12:29
*** tellesnobrega has joined #openstack-sahara12:30
openstackgerritVitaly Gridnev proposed stackforge/sahara-ci-config: Start using edp.yaml.mako template  https://review.openstack.org/21369312:31
*** ekarlso has quit IRC12:44
*** ekarlso has joined #openstack-sahara12:44
tmckayhmm, is CI broken?12:51
vgridnevtmckay, right now seems not https://sahara.mirantis.com/zuul/12:54
tmckayvgridnev, okay, thanks.  I had a rebase several days ago which did not refire for some reason12:55
openstackgerritMerged stackforge/sahara-ci-config: Update version of vanilla plugin for transient check  https://review.openstack.org/21365712:55
tmckayjust added a recheck12:56
openstackgerritMerged stackforge/sahara-ci-config: Start using edp.yaml.mako template  https://review.openstack.org/21369312:56
*** shikel has joined #openstack-sahara12:58
*** ashishb has quit IRC13:02
*** esikachev has quit IRC13:03
*** elmiko has joined #openstack-sahara13:07
*** DWfuturetec has quit IRC13:31
*** egafford has joined #openstack-sahara13:35
elmikodid anyone sign up to run bug triage day?13:49
elmikoSergeyLukjanov: ^13:49
SergeyLukjanovelmiko, not yet :)13:49
SergeyLukjanovelmiko, do you want to do it?13:50
tmckaywell, we can all just start triaging bugs :)13:50
elmikoSergeyLukjanov: i can help, we mainly need to check all the new untriaged stuff13:50
elmikoi'll setup an etherpad13:51
elmikohttps://etherpad.openstack.org/p/liberty-3-sahara-bug-triage13:51
openstackgerritMerged openstack/sahara: Add recommendation support to Cloudera plugin  https://review.openstack.org/19309813:54
openstackgerritMerged openstack/sahara: Support placeholders in args of job for i/o  https://review.openstack.org/20609413:54
SergeyLukjanovelmiko, oh, I've missed your message13:57
openstackgerritMerged openstack/sahara: Add scenario gate testing placeholders  https://review.openstack.org/21354413:57
SergeyLukjanovelmiko, I've already mailed the etherpad to openstack-dev13:57
SergeyLukjanovhttp://etherpad.openstack.org/p/sahara-liberty-bug-triage-day13:57
SergeyLukjanovprev. week13:57
SergeyLukjanovlet's use this one to avoid people flustrating on different etherpads in mail list and here ;)13:58
SergeyLukjanovI've added link to the wiki page with useful queries and order of the bugs checking by prio13:58
openstackgerritVitaly Gridnev proposed stackforge/sahara-ci-config: Revert "Update version of vanilla plugin for transient check"  https://review.openstack.org/21371614:00
*** chlong has joined #openstack-sahara14:03
elmikoSergeyLukjanov: np, i'll copy what i have into the new pad14:11
vgridnevI marked https://bugs.launchpad.net/sahara/+bug/1479666 as duplicate of https://bugs.launchpad.net/sahara/+bug/143642514:14
openstackLaunchpad bug 1436425 in Sahara "duplicate for #1479666 [CDH 5.3.0] Too many connection" [Undecided,New]14:14
uvirtbotLaunchpad bug 1479666 in sahara "Too many connections  (dup-of: 1436425)" [Undecided,New]14:14
openstackLaunchpad bug 1436425 in Sahara "[CDH 5.3.0] Too many connection" [Undecided,New]14:14
uvirtbotLaunchpad bug 1436425 in sahara "[CDH 5.3.0] Too many connection" [Undecided,New]14:14
uvirtbotLaunchpad bug 1436425 in sahara "[CDH 5.3.0] Too many connection" [Undecided,New] https://launchpad.net/bugs/143642514:14
elmikoSergeyLukjanov: ok, copied all "undecided" bugs into the new pad14:15
elmikohttps://etherpad.openstack.org/p/sahara-liberty-bug-triage-day14:16
*** egafford has quit IRC14:16
*** _crobertsrh is now known as crobertsrh14:17
elmikovgridnev: thanks14:17
vgridnevelmiko, what I should do with that bug in ether pad?14:17
elmikomark it with strike-through (ctrl+s) once it's been triaged14:18
*** egafford has joined #openstack-sahara14:18
elmikooops, thats (ctrl+5)14:20
elmikovgridnev: yea, thanks!14:20
*** DWfuturetec has joined #openstack-sahara14:22
vgridnevWe have several bugs that filed for Hadoop 1 in Vanilla and HDP, so I suppose that it should be marked as Invalid or Won't Fix ?14:22
elmikovgridnev: i think won't fix makes sense, with a comment about deprecating hadoop114:23
SergeyLukjanovIMO Hadoop 1 issues are won't fix14:23
SergeyLukjanovelmiko ++14:23
toskySergeyLukjanov, elmiko, vgridnev: can we can first kill Hadoop 1 and HDP standalone and then kill the bugs14:24
toskyorder is important sometime :)14:24
vgridnevI think that invalid bug: https://bugs.launchpad.net/sahara/+bug/141696814:24
openstackLaunchpad bug 1416968 in Sahara "scaling test failed in cdh integration test" [Undecided,New] - Assigned to lu huichun (lhcxx0508)14:24
uvirtbotLaunchpad bug 1416968 in sahara "scaling test failed in cdh integration test" [Undecided,New]14:25
uvirtbotLaunchpad bug 1416968 in sahara "scaling test failed in cdh integration test" [Undecided,New] https://launchpad.net/bugs/141696814:25
elmikotosky: i'm ok with that, but we should at least mark the bugs with a comment if we are going to leave them as "undecided"14:25
toskyelmiko: sure, even group them; not sure how to do it in launchpad14:25
toskya tag maybe so that they can be found easily?14:25
elmikohmm, vgridnev what is an example of a hadoop1 bug?14:26
vgridnev1min14:26
vgridnevhttps://bugs.launchpad.net/sahara/+bug/143815314:27
openstackLaunchpad bug 1438153 in Sahara "[HDP 1.3.2] With enabled auto security grup EDP jobs do not pass" [Undecided,New]14:27
uvirtbotLaunchpad bug 1438153 in sahara "[HDP 1.3.2] With enabled auto security grup EDP jobs do not pass" [Undecided,New]14:27
uvirtbotLaunchpad bug 1438153 in sahara "[HDP 1.3.2] With enabled auto security grup EDP jobs do not pass" [Undecided,New] https://launchpad.net/bugs/143815314:27
elmikothanks14:27
vgridnevhttps://bugs.launchpad.net/sahara/+bug/143637214:28
openstackLaunchpad bug 1436372 in Sahara "Java job has "KILLED" state" [Undecided,New]14:28
uvirtbotLaunchpad bug 1436372 in sahara "Java job has "KILLED" state" [Undecided,New]14:28
uvirtbotLaunchpad bug 1436372 in sahara "Java job has "KILLED" state" [Undecided,New] https://launchpad.net/bugs/143637214:28
elmikotosky, SergeyLukjanov, vgridnev, so for hadoop1 maybe could mark them as "triaged" with a comment about deprecation, and when we have deprecated we can remark the bugs as "won't fix"?14:29
vgridnevelmiko, I propose to move them to Won't fix and add target to liberty-314:29
elmikovgridnev: yea, but tosky is asking that we not mark them as "won't fix" until we actually deprecate the hadoop1 stuff. since it is still technically in the codebase.14:30
vgridnevelmiko, spec for drop already merged14:31
toskyvgridnev, elmiko: exactly; first remove the code, then close the bugs, otherwise it's a bit confusing14:31
toskyoh14:31
toskyI missed the merge14:31
elmikotosky: so, are you ok with us marking them as "won't fix" then?14:32
toskywell, if the spec is approved, they are technically zombies14:32
toskyso...14:32
elmikok, thanks14:32
*** Nikolay_St has quit IRC14:33
elmikook then, i'm gonna mark some of these and leave comments14:33
toskyif you remember, please add a link to the spec when closing them14:33
elmikogood idea14:33
elmikotosky: take a look at https://bugs.launchpad.net/sahara/+bug/143637214:35
openstackLaunchpad bug 1436372 in Sahara "Java job has "KILLED" state" [Undecided,Won't fix]14:35
uvirtbotLaunchpad bug 1436372 in sahara "Java job has "KILLED" state" [Undecided,New]14:35
uvirtbotLaunchpad bug 1436372 in sahara "Java job has "KILLED" state" [Undecided,New] https://launchpad.net/bugs/143637214:35
elmikoand see my comment14:35
toskyelmiko: right, make sense14:35
elmikok, thanks14:36
toskySergeyLukjanov, vgridnev, elmiko: the spec is only for Vanilla 1 and HDP 1; I think also one of the MapR plugins is Hadoop 1, isn'it ?14:36
elmikogood question, /me looks14:37
vgridnevtosky, Mapr plugin is some kind of magic in our sahara code14:37
elmikolol14:38
toskyO.o14:38
vgridnevno one knows what happens there14:38
elmikoi actually really like how the mapr plugin is structured14:38
toskycan this be raised with MapR developers/contributors? 3.1.1 is the Hadoop 1 iirc14:39
toskyor maybe not, let me recheck14:40
elmikotosky: i think it's written to be somewhat flexible with regards to hadoop version14:40
toskyuhm, it's not really clear for me from MapR release notes14:42
elmikohence vgridnev's comment ;)14:43
elmikoi think you are correct though, 3.1.1 is hadoop 114:44
elmikobut 4.0.0 looks like it supports both14:44
crobertsrhHas anyone else managed to reproduce https://bugs.launchpad.net/sahara/+bug/147753014:46
openstackLaunchpad bug 1477530 in Sahara "Cluster stays in waiting state and then goes into error" [Undecided,New]14:46
uvirtbotLaunchpad bug 1477530 in sahara "Cluster stays in waiting state and then goes into error" [Undecided,New]14:46
uvirtbotLaunchpad bug 1477530 in sahara "Cluster stays in waiting state and then goes into error" [Undecided,New] https://launchpad.net/bugs/147753014:46
crobertsrhwow, attack of the bots14:47
vgridnevelmiko, tosky I'm ping Mapr guy, he will verify bugs for Mapr plugin14:47
elmikovgridnev: awesome, thank you14:47
elmikocrobertsrh: i have seen that issue before, but i think we might have fixed the indefinite "waiting" status with the timeout stuff.14:47
crobertsrhOk.  I see it's with nova networks.  I haven't used nova in quite awhile and was hoping to avoid doing so.14:48
egaffordelmiko: Yeah, the bug specifies that it goes into Error, though, so I think the bug is a follow-up on "configuring forever".14:48
elmikoshould be easy to test, just turn off the sec groups and auto-config sec. groups, then try to spin a cluster14:48
elmikoseems like not a bug to me...14:49
vgridnevDo we have one job with nova-network + heat engine on our CI?14:51
elmikonot sure about that14:52
vgridnevseems that HDP 2.0.6 is testing on that environment14:52
*** nkrinner has quit IRC14:53
egaffordvgridnev: And HDP 2.0.6 has been failing for a while... hm...14:54
elmikoouch14:55
vgridnevegafford, not so much14:56
tmckayhi folks.  https://bugs.launchpad.net/sahara/+bug/1470525, changing the output in the CLI for the job list14:56
openstackLaunchpad bug 1470525 in Sahara "Sahara CLI does not show start_time in job-list" [Wishlist,New] - Assigned to Henrique Truta (henriquetruta)14:56
uvirtbotLaunchpad bug 1470525 in sahara "Sahara CLI does not show start_time in job-list" [Wishlist,New]14:56
uvirtbotLaunchpad bug 1470525 in sahara "Sahara CLI does not show start_time in job-list" [Wishlist,New] https://launchpad.net/bugs/147052514:56
vgridnevIt's passed there: https://review.openstack.org/#/c/207039/14:56
tmckaychanged it to wishlist, what about a target?  I guess we still have 2 weeks before the freeze.  Would we require a spec for this?  Should it be posted as a bluepring instead of a bug?14:57
AndreyPavlovWhat do you suppose to do with CLI-related bugs (wich sounds more like wishes)? Spec for new CLI, integrated with openstackclient, is on review. Should we fix the old one?14:57
htrutatmckay: I think someone has suggested to put it as a bug14:57
egaffordtmckay: Honestly, order by UUID is kind of so unuseable that filing it as a bug makes some sense to me.14:57
tmckayI'm going to suggest that it be closed wontfix and redone as a blueprint14:57
egaffordtmckay: There are UX nice-to-haves, but this does strike me as a UX oversight / bug.14:58
tmckayhtrutra, hey there :)  yeah, gray area14:58
htrutatmckay: hey :) let me see here14:58
tmckayalright, I'm okay leaving it as a bug.  Only thought was, since there is a suggestion about how to sort the list, it sounds more like a spec/blueprint to me14:59
elmikoAndreyPavlov: i think we should mark those as wishlist and add a comment to the bug about migrating to openstackclient14:59
egaffordtmckay: Yeah, totally valid that it could be treated as a feature.14:59
tmckayeither way.  If it's a bug, we should target to liberty 3, and if it doesn't get in we can bump to M14:59
egaffordtmckay: +1.14:59
tmckayhtrutra, okay, carry on.  Wishlist/confirmed/l315:00
tmckaythanks15:00
elmikoAndreyPavlov brings up a good point about sahara cli related stuff. if we are moving to the openstackclient we should reassess how we handle these new features15:00
htrutatmckay: found it: https://review.openstack.org/#/c/191875/3//COMMIT_MSG15:00
htrutaguess I was too slow15:00
vgridnevtmckay, it should affects only saharaclient15:01
vgridnevnot sahara15:01
tmckayhtruta, heh, aignatov shot me down :)15:01
egaffordFolks good with targetting https://bugs.launchpad.net/sahara/+bug/1485624 to L3? Oversight related to Main Class and other required config keys on interface map.15:01
openstackLaunchpad bug 1485624 in Sahara "Main Class required in configs even when mapped in interface" [Undecided,New]15:01
aignatovtmckay: how? :)15:01
uvirtbotLaunchpad bug 1485624 in sahara "Main Class required in configs even when mapped in interface" [Undecided,New]15:01
uvirtbotLaunchpad bug 1485624 in sahara "Main Class required in configs even when mapped in interface" [Undecided,New] https://launchpad.net/bugs/148562415:01
tmckayokay, I stand corrected15:01
tmckayaignatov, I told htruta his bug should be a bp, you told him in the commit that it should be a bug :)15:02
aignatov:)15:02
tmckayand it already merged, so it should be fix committed15:02
egaffordAlso, hruta, tmckay: It's worth real thought as to whether that ordering should be put into the service layer (otherwise we get into weird places on pagination.)15:02
aignatovsorry for that htruta tmckay15:02
egaffordAh, never mind.15:02
htrutatmckay: it was a patch before the one you showed... but it should follow the same line15:02
tmckayhmm, htrutra, why only "partial bug" on that commit? Is there more?15:02
tmckayaignatov, lol, completely np15:02
elmikoegafford: +1 on 148562415:03
aignatovactually this change looks really simple :)15:03
egaffordelmiko: Cool.15:03
htrutatmckay: the first: https://review.openstack.org/#/c/19187515:03
aignatovso I’ve thought that it could look like bug15:03
htrutathe second: https://review.openstack.org/#/c/197606/15:03
htrutaI thought about doing this in Horizon, as well15:03
htrutawould make my life a lot easier15:03
elmikoegafford: are you gonna mark that one as confirmed?15:04
tmckayhtrutra, hmm, still "closes bug"  though, wonder why launchpad didn't pick it up15:04
egaffordelmiko: Well, in theory, Confirmed is "someone other than the reporter."15:04
egaffordIt's pretty definitely happening, though.15:04
tmckayhtruta, anything else to do on that, or is it really closed?  I think we want "fix committed" on the bug15:05
tmckaymaybe because the ":" was missing in the commit message?15:05
htrutatmckay: was intending to take it to horizon, but didn't have the time for it15:06
htrutaI'm ok with closing it for now15:07
tmckayhtrutra, that's okay.  that can be a separate issue.  Let's close this one, maybe it didn't close because it was against sahara and not the client?15:07
egaffordhtruta: If the order by is in place in the client, won't that propagate to Horizon without a specific change?15:07
tmckayegafford, client or CLI? /me looks at the change15:08
htrutaI don't think so... I've only changed the CLI part15:08
tmckaythis is just the shell ... not the client15:08
egaffordtmckay: Ah, yeah, okay, if it's just in the shell, we still need work there.15:08
egaffordhtruta: Cool; thanks for the clarification.15:08
tmckayI picked the simplest, and also the most discussed, bug to triage :)15:09
egaffordtmckay: Nice place to start. :)15:09
htrutaegafford: np15:09
elmikoall i ask is that people update the etherpad once they have triaged a bug15:12
tmckayquestion now is milestone for the client ...15:12
tmckaySergeyLukjanov, unsure how to set milestone on https://bugs.launchpad.net/sahara/+bug/1470525, it's already merged, I moved it against the client instead of Sahara15:16
openstackLaunchpad bug 1470525 in Python client library for Sahara "Sahara CLI does not show start_time in job-list" [Wishlist,Fix committed] - Assigned to Henrique Truta (henriquetruta)15:16
uvirtbotLaunchpad bug 1470525 in sahara "Sahara CLI does not show start_time in job-list" [Wishlist,New]15:16
uvirtbotLaunchpad bug 1470525 in sahara "Sahara CLI does not show start_time in job-list" [Wishlist,New] https://launchpad.net/bugs/147052515:16
egaffordtmckay: I remember you had all manner of trouble with VM sizing on the Cloudera default templates. In your expert opinion on that point, is there anything we can actually do about https://bugs.launchpad.net/sahara/+bug/1416969?15:16
openstackLaunchpad bug 1416969 in Sahara "big flavor cost too resource when running cdh integration test" [Undecided,New] - Assigned to lu huichun (lhcxx0508)15:16
uvirtbotLaunchpad bug 1416969 in sahara "big flavor cost too resource when running cdh integration test" [Undecided,New]15:16
uvirtbotLaunchpad bug 1416969 in sahara "big flavor cost too resource when running cdh integration test" [Undecided,New] https://launchpad.net/bugs/141696915:16
tmckayegafford, nope15:17
egaffordtmckay: That was my thought.15:17
openstackgerritSergey Lukjanov proposed openstack/sahara: [WIP] Run scenario tests for the fake plugin in gate  https://review.openstack.org/21354615:17
tmckaywithout the flavors outlined in the default templtes, cdh will hang15:17
egaffordCDH: Putting the big in big data.15:17
vgridnevegafford, I think it's another invalid bug15:18
egaffordvgridnev: Yeah, agreed; just wanted to confirm. Any objection to invalid status on that one?15:18
*** chlong has quit IRC15:18
egafford(Or perhaps just a comment to Lu Huichun, for politeness' sake?)15:18
elmikoso, are we marking this one as not a bug? https://bugs.launchpad.net/sahara/+bug/147753015:19
openstackLaunchpad bug 1477530 in Sahara "Cluster stays in waiting state and then goes into error" [Undecided,New]15:19
uvirtbotLaunchpad bug 1477530 in sahara "Cluster stays in waiting state and then goes into error" [Undecided,New]15:19
uvirtbotLaunchpad bug 1477530 in sahara "Cluster stays in waiting state and then goes into error" [Undecided,New] https://launchpad.net/bugs/147753015:19
elmikoimo it seems more like a config/operator issue than a bug15:19
egaffordWell, is anyone successfully using nova-net in their devstack setups atm?15:20
elmikoeither that or we should change sahara to ensure that there are no blocks to using ssh before starting a cluster15:20
elmikoegafford: i just don't see the bug here. the operator doesn't have a security group to allow ssh traffic, of course sahara will fail15:21
elmikothe one thing we could do, is check sec.groups to ensure that ssh is available, but that might be a bit much.15:21
egaffordelmiko: I think the bug is that the auto sec groups aren't being assigned through Nova.15:21
elmikoegafford: maybe we should mark as incomplete and ask whether auto-groups were used?15:22
vgridnevbtw, do we care, that we use some kind of icehouse quichstart?15:22
egaffordelmiko: Or at least, that's my read on what the issue is, assuming a generous stance toward the reporter.15:22
egaffordelmiko: Yeah, that's fair.15:22
elmikovgridnev: that's a question for doc days ;)15:22
elmikovgridnev: but i think we should update the quickstart, so yes, i think we should care15:23
crobertsrhquickstart should probably be updated15:23
vgridnevelmiko, do we want to file bug for that? Just to ensure that it will be fixed in docs-days15:23
elmikohmm, good question vgridnev. i suppose it wouldn't hurt15:24
vgridnevelmiko, https://bugs.launchpad.net/sahara/+bug/148564815:26
openstackLaunchpad bug 1485648 in Sahara "Quickstart user guide is too old" [Undecided,New]15:26
uvirtbotLaunchpad bug 1485648 in sahara "Quickstart user guide is too old" [Undecided,New]15:26
uvirtbotLaunchpad bug 1485648 in sahara "Quickstart user guide is too old" [Undecided,New] https://launchpad.net/bugs/148564815:26
tmckayhey, alright, I am the only one that I see that did a strikethrough as elmiko requested.15:26
egaffordelmiko, tmckay: Marked https://bugs.launchpad.net/sahara/+bug/1416969 incomplete pending notation of a specific flavor that does work for the CDH namenode. Seemed the most polite thing.15:26
openstackLaunchpad bug 1416969 in Sahara "big flavor cost too resource when running cdh integration test" [Undecided,Incomplete] - Assigned to lu huichun (lhcxx0508)15:26
uvirtbotLaunchpad bug 1416969 in sahara "big flavor cost too resource when running cdh integration test" [Undecided,Incomplete]15:26
uvirtbotLaunchpad bug 1416969 in sahara "big flavor cost too resource when running cdh integration test" [Undecided,New] https://launchpad.net/bugs/141696915:26
tmckayHas anyone actually triaged anything else?  Where are the strikethroughs?15:26
tmckayam I on the wrong sheet?15:26
elmikotmckay: https://etherpad.openstack.org/p/sahara-liberty-bug-triage-day15:26
tmckayhttps://etherpad.openstack.org/p/liberty-3-sahara-bug-triage15:26
elmikothere are a bunch on that page15:26
tmckaydoh15:27
elmikoyea, SergeyLukjanov had sent out the link to his page on the ML15:27
elmikoegafford: ack15:27
tmckayelmiko, saw that conversation and still picked the wrong one15:29
elmikotmckay: no worries =)15:30
elmikothere needs to be a way to delete an etherpad, ooh wait i know15:30
elmikook, fixed15:31
*** chlong has joined #openstack-sahara15:31
vgridnevthat should be also won't fix: https://bugs.launchpad.net/sahara/+bug/1416992 - we use new scenario tests15:32
openstackLaunchpad bug 1416992 in Sahara "need two parameters added in the config.py in cdh integration test" [Undecided,New] - Assigned to lu huichun (lhcxx0508)15:32
uvirtbotLaunchpad bug 1416992 in sahara "need two parameters added in the config.py in cdh integration test" [Undecided,New]15:32
uvirtbotLaunchpad bug 1416992 in sahara "need two parameters added in the config.py in cdh integration test" [Undecided,New] https://launchpad.net/bugs/141699215:32
elmikovgridnev: why won't fix?15:32
vgridnevwe don't use old integrations tests15:33
elmikoah, ok15:33
vgridnevOr invalid, not so sure15:33
elmikoso, this should be fixed for the tempest tests then?15:33
vgridnevwhy?15:34
egaffordHm: https://bugs.launchpad.net/sahara/+bug/1416968 is likely about the old integration tests, but I'm noting that the current CI pipelines for CDH don't run scaling tests. I can see adapating this bug to be a test coverage bug for scaling for CDH in the scenario tests.15:34
openstackLaunchpad bug 1416968 in Sahara "scaling test failed in cdh integration test" [Undecided,New] - Assigned to lu huichun (lhcxx0508)15:34
uvirtbotLaunchpad bug 1416968 in sahara "scaling test failed in cdh integration test" [Undecided,New]15:34
uvirtbotLaunchpad bug 1416968 in sahara "scaling test failed in cdh integration test" [Undecided,New] https://launchpad.net/bugs/141696815:34
elmikomaybe i misunderstood what you are saying. are we not checking the cdh plugin during tempest testing?15:34
elmikovgridnev, egafford, sounds like we should mark 1416968 as incomplete and ask a few questions?15:36
egaffordelmiko: Yup, sounds about right.15:36
*** chlong has quit IRC15:38
vgridnevelmiko, agreed15:39
egaffordvgridnev, elmiko: Added a few comments there.15:39
elmikoegafford: thanks15:39
egaffordelmiko: How does one make text background red in etherpad? I may be being dense, but I see no interface for that fanciness.15:40
*** chlong has joined #openstack-sahara15:40
* egafford needs his triaged bugs to be fancier!15:40
elmikoegafford: its based on your user color15:40
egaffordelmiko: Ah. This makes sense.15:40
egaffordThought it signified something entirely different.15:41
elmikonah, AndreyPavlov just happened to be red color for this pad15:41
egaffordInstead, it signifies apavlov.15:41
egafford(A great thing to signify, really.) :)15:41
elmikoso, the comments you made to 1416968, do those apply to https://bugs.launchpad.net/sahara/+bug/1416992 as well?15:42
openstackLaunchpad bug 1416992 in Sahara "need two parameters added in the config.py in cdh integration test" [Undecided,New] - Assigned to lu huichun (lhcxx0508)15:42
uvirtbotLaunchpad bug 1416992 in sahara "need two parameters added in the config.py in cdh integration test" [Undecided,New]15:42
uvirtbotLaunchpad bug 1416992 in sahara "need two parameters added in the config.py in cdh integration test" [Undecided,New] https://launchpad.net/bugs/141699215:42
elmikowell, the first question at least15:42
*** hdd has joined #openstack-sahara15:43
egaffordelmiko: Yeah, these are definitely old. I'll check the scenario tests quick to see if there's feature parity with what this is asking for, though.15:43
elmikoegafford: thanks!15:43
vgridnevwant is the purpose of this bug?15:44
vgridnevI don't clearly understand that15:45
egaffordvgridnev: It's not clear. :)15:45
elmikosounds like a good case to mark it incomplete and ask questions =)15:45
vgridnevelmiko, +115:45
tmckayvgridnev, anything to add on https://bugs.launchpad.net/sahara/+bug/1466876? Looks "incomplete" to me, no info on how to rerpoduce.  Have you seen it since?15:45
openstackLaunchpad bug 1466876 in Sahara "Arguments dropped when creating context" [Undecided,New]15:45
uvirtbotLaunchpad bug 1466876 in sahara "Arguments dropped when creating context" [Undecided,New]15:45
uvirtbotLaunchpad bug 1466876 in sahara "Arguments dropped when creating context" [Undecided,New] https://launchpad.net/bugs/146687615:45
vgridnevtmckay, I have some https://sahara.mirantis.com/logs/39/207039/4/check/gate-sahara-neutron-heat-vanilla_2.6.0-u14/cf5f283/15:47
vgridnevtmckay, it's reproducable on all envs with api and 2 engines15:48
tmckayah, okay, multiple engines.  Would you kindly add the above information to the bug, and then someone can confirm it?15:49
tmckayvgridnev, also, a note on whether it breaks anything (if not, it can be "low")15:49
vgridnevtmckay, I suppose it should invalid, btw15:50
egaffordHm; this one is interesting (and has bit me a bit in the past): https://bugs.launchpad.net/sahara/+bug/141964315:50
openstackLaunchpad bug 1419643 in Sahara "saharaclient should check the input param when cluster-create" [Undecided,Triaged] - Assigned to warewang (wangguangcai)15:50
uvirtbotLaunchpad bug 1419643 in sahara "saharaclient should check  the  input param when cluster-create" [Undecided,Triaged]15:50
uvirtbotLaunchpad bug 1419643 in sahara "saharaclient should check  the  input param when cluster-create" [Undecided,Triaged] https://launchpad.net/bugs/141964315:50
elmikoegafford: i just triaged that one15:50
tmckayvgridnev, why invalid?15:50
egaffordIs the stdin JSON CLI feature worth potential hanging?15:50
vgridnevI was added to setup logging on devstack15:50
egaffordelmiko: Ah, okay. I don't see Milestone or Importance, though...15:50
vgridnevor to handle it's correctly15:50
elmikoegafford: i left those open as we need to discuss what should happen. i added it to the meeting agenda for this week, based on alazarev's comments in the bug.15:51
egaffordelmiko: Sensible.15:52
elmikoegafford: i'm not sure the proper response, i suppose if we determine that stdin is sending nothing then we should fail early.15:52
egaffordelmiko: I think it's supposed to wait for the user to type JSON.15:53
elmikoit's also partially a pbkac issue as i've seen many other commands just hang when you pipe stdin to them15:53
elmikoegafford: right... how can we know what they are doing at the shell15:53
egaffordelmiko: But I'm totally unconvinced that's a realistically useful feature, or worth the hanging failure.15:53
*** vgridnev has quit IRC15:54
elmikoegafford: agreed, if stdin.read() == 0 then we should probably fail15:54
egaffordelmiko: +1.15:54
egafford(Mind if I note the addition to the agenda and this discussion on the bug, for memory's sake?)15:55
egaffordAh, you've done so since I last checked. Bully for you!15:55
egaffordThis one (https://bugs.launchpad.net/sahara/+bug/1431460) is a real oddity (CDH uses _ rather than . to separate tokens in config keys.) Does anyone know of a reason in the bowels of CDH why this is appropriate?15:56
openstackLaunchpad bug 1431460 in Sahara "Different naming pattern for Cluster template parameters in CDH plugin" [Undecided,New]15:56
uvirtbotLaunchpad bug 1431460 in sahara "Different naming pattern for Cluster template parameters in CDH plugin" [Undecided,New]15:56
uvirtbotLaunchpad bug 1431460 in sahara "Different naming pattern for Cluster template parameters in CDH plugin" [Undecided,New] https://launchpad.net/bugs/143146015:57
*** egafford is now known as egafford|afk15:58
openstackgerritSergey Lukjanov proposed openstack/sahara: [WIP] Run scenario tests for the fake plugin in gate  https://review.openstack.org/21354616:04
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Install xfsprogs for ability to formatting volumes in XFS FS  https://review.openstack.org/21376816:04
openstackgerritMerged openstack/sahara-image-elements: Added ability to specify exact package versions for MapR  https://review.openstack.org/20209016:08
tmckayegafford|afk, taking a peek at 14316016:14
*** egafford|afk is now known as egafford16:16
egaffordelmiko: We planning on doing an Importance field review later, once we've assigned status and milestone? Ideally, importance will guide next week's bugfix day.16:25
elmikoegafford: i'm ok with making a second pass later in the week for importance. or even reviewing that we start grabbing bugs16:26
egaffordelmiko: (Being as we're in semi-headless mode, I suppose anyone could just start doing being the change they want to see in the world, but.) Yeah, sounds good.16:26
egaffordelmiko: I'll definitely want it to be filled and team-approved in the overwhelming majority of cases by Monday, one way or another.16:27
elmikoegafford: fair, i'll make sure to followup at the end of each day on the etherpad. i will attempt to make guesses at importance for the bugs.16:27
elmikowe can always re-assess if/when necessary16:28
egaffordelmiko: Yeah. Some lightweight process might be nice there. Maybe recording tentative importance on the etherpad, lazy consensus until Thursday or so?16:28
elmikoegafford: +1, i'll make some notes as we go16:29
egafford(Where anyone can make a first stab at importance, for discussion?)16:29
egaffordelmiko: Solid.16:29
elmikoyea, i'll just put importance beneath the link, and if you have an idea put it there. i'll also note this in the pad16:29
egaffordelmiko: +1.16:30
elmikook, updated the pad16:30
egaffordelmiko: \o/16:30
egaffordelmiko: What's the Importance enum, for those of us who are +2-challenged?16:33
elmikosec16:33
egaffordelmiko: (Thanks!)16:33
elmikocritical, high, medium, low, wishlist16:33
elmikoadded this to the pad as well16:35
*** ashishb has joined #openstack-sahara16:39
*** vgridnev has joined #openstack-sahara16:44
elmikoegafford: thinking about this more, it makes sense to do two pass.16:46
egaffordTriage, then prio?16:46
elmikofirst we determine which are valid bugs, ie not incomplete or invalid16:46
elmikoyea, then prio16:46
egaffordAgreed.16:46
elmikofair, i'm gonna target for starting the prio pass by wednesday evening16:47
egaffordI think a first pass as we go (marking a starting point on the pad) is still a good call, though, to give us a starting point.16:47
elmikounless we finish triage before then16:47
elmikoegafford: +116:47
elmikoif you have an idea about prio, definitely add a comment on the pad16:47
*** DWfuturetec has quit IRC16:47
egaffordelmiko: +1 right back at you, buddy. :)16:47
elmikohehe16:47
*** sgotliv has quit IRC16:51
*** DWfuturetec has joined #openstack-sahara17:04
*** hdd has quit IRC17:07
*** hdd has joined #openstack-sahara17:09
egaffordvgridnev: Have you seen https://bugs.launchpad.net/sahara/+bug/1436425 happen? Noted that you changed the tag from [CDH 5.3.0] to [CDH].17:13
openstackLaunchpad bug 1436425 in Sahara "[CDH] Too many connection" [Undecided,New]17:13
uvirtbotLaunchpad bug 1436425 in sahara "[CDH] Too many connection" [Undecided,New]17:13
uvirtbotLaunchpad bug 1436425 in sahara "[CDH] Too many connection" [Undecided,New] https://launchpad.net/bugs/143642517:13
vgridnevsame for cdh https://bugs.launchpad.net/sahara/+bug/1479666 egafford17:15
openstackLaunchpad bug 1436425 in Sahara "duplicate for #1479666 [CDH] Too many connection" [Undecided,New]17:15
uvirtbotLaunchpad bug 1479666 in sahara "Too many connections  (dup-of: 1436425)" [Undecided,New]17:15
uvirtbotLaunchpad bug 1436425 in sahara "[CDH] Too many connection" [Undecided,New]17:15
openstackLaunchpad bug 1436425 in Sahara "[CDH] Too many connection" [Undecided,New] https://launchpad.net/bugs/143642517:15
uvirtbotLaunchpad bug 1436425 in sahara "[CDH] Too many connection" [Undecided,New] https://launchpad.net/bugs/143642517:15
egaffordOkay, so effectively the two bugs confirm one another. :)17:16
*** degorenko has quit IRC17:16
*** IBerezovskiy has quit IRC17:19
*** hdd has quit IRC17:23
*** hdd has joined #openstack-sahara17:25
egaffordvgridnev, SergeyLukjanov, et al.: Is this a Sahara issue or an MOS 7.0 issue? I'm seeing check_cinder pass in our CI, in, say, https://sahara.mirantis.com/logs/22/213622/1/check/gate-sahara-neutron-heat-vanilla_2.7.1-u14/8646c44/console.html. https://bugs.launchpad.net/sahara/+bug/148453517:29
openstackLaunchpad bug 1484535 in Sahara "In scenario tests cinder check fail with trace" [Undecided,New] - Assigned to Evgeny Sikachev (esikachev)17:29
uvirtbotLaunchpad bug 1484535 in sahara "In scenario tests cinder check fail with trace" [Undecided,New]17:29
uvirtbotLaunchpad bug 1484535 in sahara "In scenario tests cinder check fail with trace" [Undecided,New] https://launchpad.net/bugs/148453517:29
egafford(Very odd issue; 'module' object has no attribute 'poll' in select, deep in tempest_lib.)17:30
vgridnevegafford, i reproduce same on my macbook, btw it's not reproduced on ci17:32
egaffordvgridnev: Weird... are you running the upstream codebase (sahara from our tox) or a packaged MOS 7.0 product version? (Just wondering about the "on MOS 7.0" comment in the bug report, and how that would work in CI but not manually with the same code, especially given that the error seems to be pretty definitely saying that the select module, itself, is missing one of its key members.)17:35
toskyvgridnev: if you can reproduce, can you please add the details about tempest-lib version etc? I asked something on the bug17:36
egaffordSo odd... they'd have to overwrite select somehow...17:37
vgridnevegafford, on upstream codebase, of cource17:37
egaffordvgridnev: Okay, good to know, just trying to imagine how this could even happen. Thanks very much for clarifying.17:37
*** DWfuturetec has quit IRC17:38
egaffordvgridnev: If you could mark the bug as Confirmed and mark a possible Importance on the etherpad, that'd be really great.17:38
egaffordvgridnev: http://stackoverflow.com/questions/19740471/cannot-use-python-select-poll-in-mac-os17:41
egaffordNeat.17:41
egaffordOr at least, a very real possibility.17:41
vgridnevok, egafford could you please that as invalid with that link?17:42
egaffordSure; at the very least, it's a bug in tempestlib that seems likely to only effect macs (and thus, not something to fix in Sahara itself.)17:43
vgridnevagreed17:43
egaffordvgridnev: One imagines the CI lab is running on Ubuntu?17:43
toskyvgridnev: do you know if esikachev use a Mac too?17:43
vgridnevyep17:44
egaffordtosky, always with the reasonable QE questions...17:44
vgridnevegafford, ci lab installed with ubuntu17:44
egaffordvgridnev: Cool; this diagnosis makes a lot of sense then. Thanks very much.17:45
tmckayinteresting devstack tidbit, glance failing to upload more images, although there is space.  Restarted swift-proxy service, upload works.17:47
tmckayI've run into this before but had not found a solution without ./unstack.sh ^^17:48
toskyegafford: oh, needs to head out in 2 minutes, but please keep an eye on https://review.openstack.org/#/c/212865/ - it escaped my testing17:52
egaffordtosky: Ack; do we want to make a RH bug about it now so we remember? I can provide that service. :)17:53
toskyegafford: let's talk tomorrow!17:54
egaffordtosky: +1!17:54
*** tosky has quit IRC17:54
egaffordelmiko: You're all Sparky. I'm not! What are your immediate thoughts on https://bugs.launchpad.net/sahara/+bug/1452127?17:55
openstackLaunchpad bug 1452127 in Sahara "Spark plugin does not pass JAVA_OPTS and configurations" [Undecided,New]17:55
uvirtbotLaunchpad bug 1452127 in sahara "Spark plugin does not pass JAVA_OPTS and configurations" [Undecided,New]17:55
uvirtbotLaunchpad bug 1452127 in sahara "Spark plugin does not pass JAVA_OPTS and configurations" [Undecided,New] https://launchpad.net/bugs/145212717:55
*** crobertsrh has left #openstack-sahara17:56
tmckayegafford, never implemented17:58
tmckayjust a change to the Spark EDP engine17:58
egaffordtmckay: So is this more of a feature request than a stark bug?17:58
tmckayI would say so.  It's not the case that it was meant to be there and is broken.17:59
tmckayprobably not too hard to add, could potentially land in L317:59
egaffordtmckay: Yeah, we have the technology to add two key-mapped configuration types, certainly.18:00
tmckayit would be great if the reporter gave us an example -- I am unclear on what java_opts or configs to actually set for spark18:00
tmckayor, how to verify that they actually work18:00
egaffordtmckay: You're core; could you set the milestone and status on that if you'd like to approve it for L3?18:01
egaffordtmckay: Also, what would be the fun in knowing what you're coding before you code it? ;)18:02
egafford(I kid; I kid.)18:02
*** crobertsrh has joined #openstack-sahara18:02
tmckayack, I'll set it and ask for an example18:02
egaffordtmckay: Perfect; thanks.18:02
elmikoegafford: did you get everything sorted out?18:15
openstackgerritSergey Reshetnyak proposed openstack/sahara-image-elements: Install xfsprogs for ability to formatting volumes in XFS FS  https://review.openstack.org/21376818:16
egaffordelmiko: Think so, yeah. Thanks. Need to take a triage-break for a little while and wrestle with TripleO. Are we aiming for total triage / prio by Wednesday?18:19
elmikoegafford: goal is total triage/prio by friday18:20
elmikoit would be cool if we could get total triage by wednesday eve., then prio on thurs/fri18:21
egaffordelmiko: Cool. Looks like we're 20/32 triaged now. :)18:21
*** ashishb has quit IRC18:21
elmikoegafford: awesome, thanks for keeping an eye on that18:21
egaffordelmiko: If we can get total triage and prio by Wednesday, though, I can get the bugfix etherpad sorted sensibly for our team meeting Thursday, and we can give everyone a little time to sign up for fixes with prio already assigned. Good stretch goal if we find that we're that awesome. :)18:22
elmikoegafford: ack, that's a good goal to work towards18:23
egaffordelmiko: Seems like we're on track so far, though we may well lose some steam.18:23
elmikoi figure we will, but i'll make a pass at trying to prio some of the stuff we have done18:24
egaffordelmiko: Happy to ride shotgun when you do.18:24
elmikoack18:24
elmikoif we can make good use of the etherpad to take first stabs at the prio, we should be in good shape18:25
egaffordelmiko: Absolutely. We can firm up Thurs/Fri.18:25
elmikonice, apparently Alan Moore is teaming up for a new HP Lovecraft inspired comic book series18:26
egaffordelmiko: <318:26
tmckayhey guys, need an opinion on "Spark cannot connect to separate HDFS cluster" (skipping bug number because I want the bots to be quiet)18:36
egaffordtmckay: :)18:36
elmikoshoot18:36
elmiko(i looked at that one a little too)18:36
tmckaybug is against spark to hadoop 2.4.1 cluster18:37
tmckayI just did spark to spark, and spark to Fedora vanilla hadoop 2.6, works fine18:37
elmikoyea, i was curious about that as well18:37
tmckayvenza noted that cdh vs non-cdh might be an issue18:37
tmckayalso, this was spark 1.3.118:37
elmikohmm18:37
tmckayso, given that hadoop 2.4.1 is deprecated, and spark version is new, and I didn't reproduce, I'm tempted to say "invalid"18:38
elmikomaybe mark as incomplete with a request for more recent versions?18:38
elmikoor yea, invalid with a note about versions18:38
egaffordIn this case, if tmckay actively failed to repro with the newer, supported version, I think invalid works. The bug was actually really good about noting versions, so I'm not sure what other info it could provide if incomplete.18:39
tmckayI suppose it could still be an issue for juno or kilo -- spark 1.3.1 is newish18:39
elmikoyea18:39
tmckayhmm, only question might be kilo18:39
tmckaythis could be a user that does not want to run off the tip18:40
egaffordtmckay: Ack. This feature was intended to function for Spark in Kilo, yes?18:40
elmikomakes good sense18:40
tmckayyeah, should theoretically always function18:40
tmckayhdfs is hdfs18:40
egaffordtmckay: Invalid in master and moving it to kilo/stable makes sense to me then.18:41
tmckaythere was a bug with updating the /etc/hosts file, but if you reference by ip it should work18:41
tmckayok, I'll try a kilo devstack18:41
tmckayuser also notes though that the foreign hdfs didn't seem to be listening on 8020 or 9000, which sounds like the foreign hdfs was messed up to me ...18:42
tmckayalso, he tried to use floating ips18:43
tmckayuser could have messed up networking, in which case it's not going to work either18:43
tmckayfrom January, I'm going to kill it18:44
*** DWfuturetec has joined #openstack-sahara18:44
DWfuturetecdoes anyone have proper nodegroup templates (master + worker) for Vanilla Apache Hadoop 2.6.0? I used the old templates from the quickstart guide (actually for 1.2.1), but I think that the master node doesn’t have „jobtracker“ as a process anymore18:46
elmikoDWfuturetec: have you tried the default templates?18:47
elmikoDWfuturetec: this may be of some help, http://docs.openstack.org/developer/sahara/userdoc/installation.guide.html#optional-installation-of-default-templates18:48
DWfuturetecelmiko, thanks for the help18:48
DWfutureteci will try this18:48
elmikoDWfuturetec: also, i have a utility in my github that i use to create 2.6.0 clusters. here is the relevant code for make them (it's python btw)18:49
elmikohttps://github.com/elmiko/psychic-dromedary/blob/master/psydr/cmds/cluster.py#L3818:49
openstackgerritMerged openstack/sahara: Support manila shares as binary store  https://review.openstack.org/20469018:49
DWfuturetecelmiko, Thanks - looks like a handy snippet18:50
elmikoDWfuturetec: i use the python-saharaclient for creating the templates, but you could extract that dictionary into a json object if necessary18:51
tmckayDWfuturetec, default templates should help, I just used them today :)  Let us know if you have any trouble.18:55
DWfuturetecelmiko, tmckay - thanks for now .. I’m on it - i will let you know if it worked18:56
* tmckay hopes so18:56
tmckayI wrote it :) default templates18:57
DWfuturetectmckay .. then there is no doubt, that it won’t work ;-)18:57
tmckaywith help on the actual templates, that is18:57
tmckayheh, thanks for the vote of confidence18:57
elmikolol18:59
DWfuturetecthe floating_ip placeholder is replaced with e.g. the ext-net network ID ? (floating ip pool)19:00
elmikoyes19:00
tmckayyes, it is actually the uuid of the network19:00
elmikoeven though tmckay wishes it was the name ;)19:00
tmckaywe would like to find a nicer way to specify that19:00
elmikohehe19:00
tmckayjinx19:00
elmikoi owe you a beverage19:00
DWfuturetecanother thing: does to floating_ip pool need to be external IPs (let’s say „genuine ipv4“ addresses) or can I use an internal network for my cluster (simple 192.168.1.0/24 network)19:04
elmikoyou can use an internal network, it just needs to be something that sahara can request floating ip addresses on19:05
elmikoit should be a network configured through neutron(or nova-net) as a floating ip pool network19:05
DWfuturetecyes it is a neutron-based network with floating ips19:05
elmikook, then you can just supply the uuid of that network19:06
*** hdd has quit IRC19:12
*** hdd has joined #openstack-sahara19:15
vgridnevegafford, is this bug https://bugs.launchpad.net/sahara/+bug/1436425 just some problems with configuration of OpenStack?19:22
openstackLaunchpad bug 1436425 in Sahara "[CDH] Too many connection" [Undecided,Confirmed]19:22
uvirtbotLaunchpad bug 1436425 in sahara "[CDH] Too many connection" [Undecided,Confirmed]19:22
uvirtbotLaunchpad bug 1436425 in sahara "[CDH] Too many connection" [Undecided,Confirmed] https://launchpad.net/bugs/143642519:22
DWfuturetecelmiko, I’m a little bit confused .. I’m trying to write my own .json templates without the python script or „sahara-templates“ cli tool .. in your cluster.py, you add „'net_id': mgmt_net,“ to the cluster template, the default template contains „"neutron_management_network": "{neutron_management_network}“,“ instead … which one is the right one and which network UUID do I use for a19:23
DWfutureteca mgmt network?19:23
elmikoDWfuturetec: 1sec19:24
DWfuturetecno hurry19:24
egaffordvgridnev: It could certainly be a MySQL misconfiguration.19:25
egaffordvgridnev: (It looks to me like it is a MySQL misconfiguration, in fact.)19:25
elmikoDWfuturetec: so yea, the json template should be "neutron_management_network", "net_id" is something that the python client accepts19:26
egaffordvgridnev: But, it's been confirmed to happen by multiple sources, so getting to the bottom of root cause seems to still be a good idea, and that might be a non-trivial investigation, so it makes sense to me to keep it as a bug for next week.19:26
vgridnevegafford, I didn't find anything except just increasing size of pool in mysql19:26
DWfuturetecelmiko, ok thx .. and the mgmt network is just another floating-ip network?19:27
vgridnevegafford, that's make sense, agreed19:27
egaffordvgridnev: I agree that that's probably going to be the fix. :)19:27
elmikoDWfuturetec: the management network should be a network that will allow the controller to talk with the cluster19:27
elmikoDWfuturetec: usually, in devstack, the management network is fixed IPs (or the "private" net), and the floating_ip network is the "public" network19:28
egaffordvgridnev: Might be an easy one; might spiral out into crazy puppet module madness. We'll see.19:28
DWfuturetecelmiko, well I used the example architecture for my test envorinment, which basically means there is this 10.0.0.0/24 management network between all nodes - but this (of course) won’t show up in neutron, because it’s manually configured within the distribution19:29
elmikoDWfuturetec: hmm, not sure how to deal with that. usually the mgmt net would be something defined with the networking service in openstack19:30
DWfuturetecelmiko, i will have a look in the documentation - maybe I missed something during the network setup19:31
elmikoDWfuturetec: have you played around with devstack?19:31
DWfuturetecelmiko, just started it up once .. i soon went to build my own test environment (example architecture on 5 machines, controller/network/compute/block/object1+2)19:33
*** hdd has quit IRC19:33
elmikoDWfuturetec: ok, i'll share a screenshot of what my networks look like19:33
DWfuturetecelmiko, thanks that would be great!19:33
*** hdd has joined #openstack-sahara19:33
elmikoDWfuturetec: https://mimccune.fedorapeople.org/openstack_networks.png19:38
elmikoDWfuturetec: that's what my networks look like, the "public" network is what i use for managment and the "private" network is what i use for floating_ip_pool19:38
elmikothe private network is defined only in my demo project19:38
DWfuturetecelmiko: okay I will try to configure neutron like this19:39
elmikosame with the router too19:39
elmikoDWfuturetec: good luck!19:39
elmikodoes this, https://bugs.launchpad.net/sahara/+bug/1426398 , look like a wishlist to anyone else?19:49
openstackLaunchpad bug 1426398 in Sahara "Current anti-affiity only allows instances equals to number of hypervisors" [Undecided,New]19:49
uvirtbotLaunchpad bug 1426398 in sahara "Current anti-affiity only allows instances equals to number of hypervisors" [Undecided,New]19:49
uvirtbotLaunchpad bug 1426398 in sahara "Current anti-affiity only allows instances equals to number of hypervisors" [Undecided,New] https://launchpad.net/bugs/142639819:49
egaffordelmiko: Hard to call it a stark bug, but it does sound like we can do better (esp. in small private clouds where hypervisor count might not  be huge.)19:51
elmikoegafford: i'm just trying to figure out what hints we observer now for the schedulers. seems like we have something, also this sounds like a request for an improvement as opposed to an outright bug19:52
egaffordAlso, elmiko, added tag-style statuses to bugs a few minutes ago (tried to reorg bugs into headings, but that trashed all pad history.) Feel free to remove or ask me to do so if it feels cluttering. Yeah, seems like a feature request to me, regardless of impl, unless what we claim we're doing in our docs or specs is Just Wrong.19:53
elmikoegafford: ack, i'll look at the pad19:54
elmikoegafford: looks good, thanks19:55
elmikoman bug triage pwned my day =(19:59
egaffordelmiko: Oh, totally.19:59
egaffordI have done almost nothing else.19:59
egaffordComplete pwnage.20:00
egaffordAt least we've been pwnt for The Community. :)20:00
crobertsrhbah, horizon github is down20:16
egaffordcrobertsrh: Bah!20:17
crobertsrhI did spend 10 min thinking it was something I did, but I doubt I took out the whole repo.20:17
egaffordcrobertsrh: If you did, it's an impressive feat.20:18
crobertsrhtrue20:18
DWfutureteci just tried to start my cluster, but it failed with „floating ip pool not found“ .. just to be clear -> as floating_ip_pool I use the ID of the network or the subnet ID? how are floating and non-floating networks defined anyway? DHCP + allocation pool disabled/enabled?20:35
elmikoDWfuturetec: the uuid as listed by neutron20:35
egaffordDWfuturetec: The floating_ip_pool should be a public network uuid (not subnet, or private network.)20:36
DWfuturetecelmiko: e.g. neutron net-list outputs both .. ID for the network itself + ID for the subnet20:36
elmikoi think you just want the ID for the network20:37
egaffordDWfuturetec: +1 to elmiko on that.20:37
DWfuturetecegafford: well I could use my ext-net .. problem is my provider won’t give me a bigger subnet .. I just have 3 floating IPs available in the pool20:37
DWfuturetecmaybe I could define another „external“ network with floating-ips which doesn’t use public accessible IPv4 addresses20:38
egaffordDWfuturetec: ...Ah, I see. You can designate one node as a public gateway to your cluster, I know. I haven't worked with that setup extensively, but it's probably a good way to get around your limitation.20:40
elmikoDWfuturetec: +1, that would be one way to go. make a truly private network that you control20:40
DWfuturetecegafford: elmiko: thanks .. i think i just need to create the already existing networks (10.0.0.0/24 for management, etc.) within neutron again, so I can access them from within - the nodes itself already are communicating over this network and are using another vm as gateway to „outer space“20:42
*** crobertsrh is now known as _crobertsrh21:33
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/21228121:44
elmikoegafford: is this something we would likely port to juno? https://bugs.launchpad.net/sahara/+bug/143340121:46
openstackLaunchpad bug 1433401 in Sahara "In stable/juno branch, cluster launch failed" [Undecided,New]21:46
uvirtbotLaunchpad bug 1433401 in sahara "In stable/juno branch, cluster launch failed" [Undecided,New]21:46
uvirtbotLaunchpad bug 1433401 in sahara "In stable/juno branch, cluster launch failed" [Undecided,New] https://launchpad.net/bugs/143340121:46
*** uvirtbot has quit IRC21:50
egaffordelmiko: If it is actionable, it may well be something that can only be fixed by a non-cherry-pick commit to juno.21:56
egaffordelmiko: Depends on the root cause and whether it's still active in master.21:57
*** DWfuturetec has quit IRC21:57
egaffordelmiko: I don't know that it's necessarily a candidate for the L bugfix cycle, but we may want to shove it in there anyway, just so it gets some attention.21:59
elmikoegafford: ok, thanks. i just wanted to make sure we are still in the window for fixing juno stuff22:04
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/21228122:06
egaffordelmiko: Yeah, I don't believe Juno's EOL; it's only at 2014.2.3. By precedent, it's got at least 6 months and 1 release left in it.22:07
elmikoegafford: ack, i'll try to replicate this22:08
openstackgerritSergey Lukjanov proposed openstack/sahara: [WIP] Run scenario tests for the fake plugin in gate  https://review.openstack.org/21354622:10
openstackgerritSergey Lukjanov proposed openstack/sahara: Remove never executable code from devstack plugin  https://review.openstack.org/21389522:18
*** vgridnev has quit IRC22:23
*** vgridnev has joined #openstack-sahara22:23
*** vgridnev has quit IRC22:24
*** chlong has quit IRC22:34
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/21228122:41
openstackgerritMerged openstack/puppet-sahara: Remove Sqlite validation for database_connection  https://review.openstack.org/21365022:59
*** AndreyPavlov has quit IRC23:19
*** AndreyPavlov has joined #openstack-sahara23:19
*** tiny-hands has joined #openstack-sahara23:20
openstackgerritSergey Lukjanov proposed openstack/sahara: [WIP] Run scenario tests for the fake plugin in gate  https://review.openstack.org/21354623:27
*** egafford has quit IRC23:40

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!