14:03:17 <SergeyLukjanov> #startmeeting sahara
14:03:19 <openstack> Meeting started Thu Sep 24 14:03:17 2015 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:03:22 <openstack> The meeting name has been set to 'sahara'
14:03:23 <AndreyPavlov> hi
14:03:28 <Venkatesh> Hi
14:03:31 <elmiko> hi
14:03:31 <vgridnev> hey
14:03:32 <weiting> hi
14:03:33 <SergeyLukjanov> I've been selecting right channel for it :)
14:04:06 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
14:04:12 <SergeyLukjanov> #topic sahara@horizon status (crobertsrh, vgridnev)
14:04:20 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon
14:04:32 <crobertsrh> almost everything seems to be merged
14:04:35 <vgridnev> It looks like we will not have new changes, that will be landed in liberty-rc1
14:04:45 <crobertsrh> the moving cluster config to a separate tab would be nice to get in
14:05:09 <vgridnev> cores don't want to merge that because it looks like enhancement
14:05:16 <egafford> crobertsrh: Did that get an FFE?
14:05:20 <crobertsrh> No
14:05:28 <vgridnev> it filed as bug
14:05:51 <crobertsrh> Be sure to think of UI ideas for the M cycle.
14:06:24 <vgridnev> I have several things that we can do in M
14:07:15 <SergeyLukjanov> #topic News / Updates
14:07:43 <vgridnev> this week I tried to fix several bugs that we have in sahara
14:08:03 <elmiko> working on the improved secret store, talks for tokyo, and bug cleanup. also trying to diagnose some issues with the hadoop-swift fs stuff.
14:08:55 <sreshetnyak> working on fix for saharaclient gate
14:08:58 <AndreyPavlov> working on client and CLI stuff, fortunately gate has been fixed and patches can be merged
14:10:26 <tosky> sreshetnyak: sorry for that, I didn't notice it broke (not sure when it happened)
14:10:43 <egafford> Summit prep mostly: presentations (EDP UX stuff, Manila,) and trying to get small-scale POCs for Zaqar integration and alternative image gen solutions before summit. Other than that, TripleO patches on review and out of WIP.
14:12:11 <sreshetnyak> tosky: no problem ;)
14:12:33 <tellesnobrega> i'm also working on the summit prep (sahara+storm talk)
14:14:21 <SergeyLukjanov> okay, thx folks, let's move on
14:14:29 <SergeyLukjanov> #topic Liberty release handling
14:14:36 <SergeyLukjanov> Release critical CRs: https://etherpad.openstack.org/p/sahara-liberty-release-patches Call for reviews!
14:14:54 <SergeyLukjanov> please, add CRs to the etherpad if you think that they are release critical
14:15:08 <SergeyLukjanov> so, we're going to have RC1 by the end of this week
14:16:00 <SergeyLukjanov> and please review rest of the patches from etherpad
14:18:23 <SergeyLukjanov> any questions regarding release handling?
14:18:40 <SergeyLukjanov> do we have stuff to backport to liberty client?
14:18:46 <SergeyLukjanov> (0.11.0)
14:19:08 <AndreyPavlov> i guess we have
14:19:55 <AndreyPavlov> just for example https://review.openstack.org/#/c/220994/
14:21:19 <AndreyPavlov> and this one https://review.openstack.org/#/c/225956/ or even better this one https://review.openstack.org/#/c/221714/
14:21:28 <SergeyLukjanov> 220994 agreed
14:22:02 <SergeyLukjanov> 220956 agreed
14:22:24 <SergeyLukjanov> 221714 is a new feature, shouldn't be backported
14:22:40 <SergeyLukjanov> sorry about this
14:22:44 <AndreyPavlov> and what do you think about https://review.openstack.org/#/c/221257/ ?
14:23:04 <AndreyPavlov> not really sure it should be backported
14:23:31 <SergeyLukjanov> no, it shouldn't, AndreyPavlov
14:23:45 <AndreyPavlov> ok then
14:23:54 <SergeyLukjanov> AndreyPavlov, could you please backport 220994 and 220956 asap?
14:24:18 <AndreyPavlov> sure
14:24:55 <AndreyPavlov> and the last one - should we backport documentation updates?
14:25:57 <SergeyLukjanov> AndreyPavlov for client?
14:26:02 <AndreyPavlov> yep
14:26:09 <SergeyLukjanov> I would say no
14:26:27 <AndreyPavlov> ok)
14:26:53 <SergeyLukjanov> anything else on release handling?
14:29:45 <SergeyLukjanov> okay
14:29:56 <SergeyLukjanov> #topic Tokyo summit working group topic discussion
14:30:28 <elmiko> #link https://etherpad.openstack.org/p/mitaka-sahara-session-plans
14:30:30 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-liberty-proposed-sessions
14:30:39 <SergeyLukjanov> oops, wrong link :)
14:30:41 <elmiko> hehe
14:30:58 <SergeyLukjanov> elmiko, thx
14:31:04 <SergeyLukjanov> #undo
14:31:04 <openstack> Removing item from minutes: <ircmeeting.items.Link object at 0xa250710>
14:31:55 <elmiko> i need to fill out the apiv2 and sec. plans still
14:32:10 <elmiko> i have a few bullet points i'd like to add
14:32:19 <SergeyLukjanov> I've still not put my ideas to the list
14:32:35 <SergeyLukjanov> but we'll have internal planning early October, so, should add more stuff after
14:32:42 <elmiko> cool
14:32:43 <SergeyLukjanov> AFAIR the same for Red Hat
14:32:47 <elmiko> yea
14:33:08 <tmckay> oct 1, first meeting
14:33:41 <tmckay> I'll read the mitaka session plan pad again today
14:37:55 <SergeyLukjanov> #topic Open discussion
14:38:07 <SergeyLukjanov> but seems like not too many questions :)
14:38:32 <elmiko> i have a couple topics here
14:38:53 <elmiko> 1. i think we should be using our hadoop-swift jar in the vanilla images
14:38:54 <crobertsrh> go for it elmiko
14:39:09 <elmiko> should we also be replacing the that jar in the other plugin images?
14:39:20 <elmiko> we are currently only using it in the spark images
14:39:48 <elmiko> i have a patch to update the vanilla image creation to include the swift_hadoop element, but i wasn't sure about doing this to the vendor provided images (cdh, etc)
14:39:59 <elmiko> thoughts?
14:40:47 <pino|work> speaking of that... something elmiko and me talked about yesterday in #-sahara
14:41:09 <egafford> elmiko: Hm... from a CDH/HDP/MapR support perspective, that's really their call. Certainly muddies the waters a bit.
14:41:20 <pino|work> the swift jar is currently hosted on the mirantis website, and rebuilt from the content of the sahara-extras repo
14:41:29 <elmiko> egafford: right, that's why i was only targetting the vanilla images
14:42:06 <pino|work> wouldn't it be better to do a post-merge job for sahara-extras, taking care of rebuild that jar and publish it somewhere on openstack.org?
14:42:20 <elmiko> +1
14:42:23 <tosky> elmiko: if some features requires it, it should go in all images I think
14:42:42 <elmiko> tosky: yea, it is required for proxy domain users
14:42:42 <SergeyLukjanov> I would saythat eventually we'll host it on tarballs.o.o (/me need more time to debug issues of building it in gate)
14:42:53 <SergeyLukjanov> and after that we probably should use everywhere
14:42:58 <crobertsrh> I don't think we should necessarily force it on the vendor plugins
14:43:25 <elmiko> are there any objections to me creating a patch for the vanilla images?
14:43:34 <crobertsrh> No objection here
14:44:27 <egafford> For vanilla, absolutely.
14:44:42 <elmiko> k, cool
14:44:47 <elmiko> next question
14:45:00 <tosky> for cloudera, it should be already: https://bugs.launchpad.net/sahara/+bug/1474128
14:45:02 <openstack> Launchpad bug 1474128 in Sahara "[CDH] Unable to launch EDP Spark jobs with Swift" [High,Fix released] - Assigned to Oleg Borisenko (al-foo)
14:45:02 <tosky> \
14:45:26 <elmiko> tosky: yea, it already is included when creating a spark image
14:45:47 <weiting> Any one tried Sahara and Trove integration? I got one customer case, they would like to use Storm and Cassandra together.
14:46:08 <elmiko> i am attempting to create a project for the outreachy mentor program, and i am curious if anyone has advice about this bug/feature https://bugs.launchpad.net/sahara/+bug/1426398
14:46:09 <openstack> Launchpad bug 1426398 in Sahara "Current anti-affinity only allows instances equals to number of hypervisors" [Wishlist,Triaged]
14:46:09 <tosky> given that Ambari is rewritten "internally" and not by HWX, I would say to start using on all images
14:46:43 <tosky> weiting: which kind of integration exactly? An Hadoop application accessing a DB executed by Trove? Not sure what is needed on Sahara and/or Trove side
14:46:57 <elmiko> tosky: i would be ok with doing the work on the other images, but it sounds like we don't have consensus about that
14:47:04 <tosky> afaik Trove datastores are all marked experimental, with the exception of mysql one
14:47:11 <tosky> elmiko: oki
14:47:17 <SergeyLukjanov> elmiko, it's about we couldn't say "no more that 2 instances of specific role per node"
14:48:05 <egafford> weiting: I think that if you had a Cassandra DB and and a Sahara Hadoop cluster in the same network (or two connected networks), they could talk to each other, but I don't know that there's any more explicit integration (planned or implemented).
14:48:16 <SergeyLukjanov> I'm for using version from sahara-extra in all plugins, I think spec should be written for it, tosky
14:48:18 <elmiko> SergeyLukjanov: ok, so we would need to create some new data in the cluster creation request to allow this type of affinity?
14:48:28 <weiting> tosky: Ok, got it. I am not sure, just would like to know any one tried this case.
14:48:50 <egafford> weiting: Trove doesn't really have any equivalent of our EDP engine; it's a provisioner.
14:48:50 <tmckay> elmiko, we have suggested a feature for extra elements as a command line option to DIB, mabye we should include the swift jar as an element and make it optional that way
14:49:11 <weiting> egafford: Yep, I agree.
14:49:17 <elmiko> tmckay: we already have it as an element, it just needs a little work for the other images
14:49:43 <tmckay> k. then I would still go with an optional include for the others, maybe.
14:49:47 <elmiko> imo, i'd rather just add it from a bug. i don't think it will be difficult to add
14:49:58 <elmiko> i'm already experimenting with it locally
14:50:27 <pino|work> how big would this additional jar(s) be?
14:50:41 <elmiko> its not much bigger than the current jar
14:51:09 <elmiko> i think the current hadoop-openstack.jar is like 120k and the new one is like 130k
14:51:16 <elmiko> iirc
14:51:54 <weiting> Another quesiton from our customer is about version control, current Sahara usually maintain the latest one version. But some of our customers usually use the old one.
14:51:56 <pino|work> sounds like could be just added unconditionally in every image then
14:52:13 <pino|work> it isn't like sahara images strive for being slim anyway...
14:52:20 <elmiko> thats kinda what i was thinking pino|work, just make a bug and start updating the images
14:52:56 <elmiko> i mean, currently, if an operator tried to use proxy domain users with swift, it would fail on every image except spark...
14:53:49 <tosky> weiting: the latest version for Hadoop distributions?
14:54:22 <weiting> tosky: yep, just like CDH5.4 in cloudera plugin. But our customer needs CDH5.3
14:54:42 <weiting> Another case is storm v0.9.2. But current version is v0.9.3
14:55:46 <tosky> uhm, I would add another point to the this: our deprecation policy
14:56:10 <tosky> for example, vanilla 2.6 was supported in Kilo, now it's deprecated, but deprecated means that the operator can't run clusters
14:56:16 <weiting> Some customers, they are evaluating Sahara, but finally they don't use Sahara that is because the version support.
14:56:38 <tosky> thinking more about this, I would expect one cycle were deprecated means "you know, it will be removed" and then it's removed, but still working in the deprecation time
14:56:54 <tosky> do we have some discussion for this scheduled during Summit/
14:56:55 <tosky> ?
14:57:47 <elmiko> add it to the etherpad =)
14:57:48 <weiting> I would like to propose some feedback from our customers in this Summit. But I am afraid I may not join this Summit since the budget limit from our team.
14:57:57 <elmiko> add it to the etherpad =)
14:59:17 <vgridnev> one more question to think there: ok, we support many plugins of many versions: how we can handle that plugin X version Y is not broken by some reason?
15:00:21 <tmckay> tosky, +1 on the meaning of deprecated
15:00:22 <carl_baldwin> hi
15:00:26 <pavel_bondar> hi
15:00:26 <egafford> vgridnev: This was the main concern when we discussed separating plugins out to separate repos; unless we have really good (and really big) CI, we can't really know.
15:00:28 <mlavalle> elmiko: hi, sorry to interrupt
15:00:33 <tosky> I think we are out of time
15:00:36 <elmiko> mlavalle: no worries
15:00:43 <regXboi> moo
15:00:47 <tmckay> tosky, I have myself gone in and removed the exception in Sahara code to keep running "deprecated" plugin versions
15:00:49 <tosky> SergeyLukjanov: time to close?
15:00:49 <vikram> hi
15:00:54 <pc_m> hi
15:00:55 <mlavalle> elmiko: ready for next meeting
15:00:56 <tmckay> the code is still there, it should be a warning imho
15:00:59 <elmiko> #endmeeting
15:01:02 <Swami> hi
15:01:05 <elmiko> i don't think i can do it
15:01:08 <mlavalle> elmiko: thanks!
15:01:11 <elmiko> SergeyLukjanov: *cough*
15:01:12 <carl_baldwin> Thanks, elmiko!
15:01:12 <egafford> elmiko: No chair powers.
15:01:28 * regXboi notes why a meeting should always have more than one chair
15:01:32 <openstack> mlavalle: Error: Can't start another meeting, one is in progress.  Use #endmeeting first.
15:01:35 <elmiko> regXboi: +1
15:01:37 <regXboi> mlavalle: previoud meeting hasn't ended
15:01:42 <carl_baldwin> Who is chair?
15:01:47 <elmiko> carl_baldwin: SergeyLukjanov
15:01:59 <elmiko> we failed to HA our chair duties
15:02:01 * haleyb plays jeopardy song
15:02:05 <elmiko> sorry...
15:02:41 <regXboi> haleyb: I always liked "syncopated clock"
15:02:48 <SergeyLukjanov> #endmeeting