14:04:45 <SergeyLukjanov> #startmeeting sahara
14:04:46 <openstack> Meeting started Thu Jan 29 14:04:45 2015 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:04:47 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:04:49 <openstack> The meeting name has been set to 'sahara'
14:04:56 * mattf waves
14:05:06 <elmiko> ahh, there we go
14:05:13 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
14:05:18 * mattf may have gone to -alt first, oops
14:05:20 <alazarev> it's pain to wake up at 6am :)
14:05:33 <SergeyLukjanov> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
14:05:40 <egafford> Hello.
14:05:49 <NikitaKonovalov> ok
14:05:50 <mattf> alazarev, not if you just got to PT from ET and you're 3hrs ahead
14:05:53 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon
14:05:57 <elmiko> lol
14:06:14 <NikitaKonovalov> as you can see the changes now are getting some reviews
14:06:26 <mattf> for later - let's drop "ex Savanna"
14:06:32 <SergeyLukjanov> I've added an action item to the next cross-project meeting to discuss slow-merging patches issues
14:06:38 <NikitaKonovalov> mostly -1 for code style or similar stuff
14:07:13 <NikitaKonovalov> but that's a progress anyway
14:07:24 <alazarev> no progress on my horizon patchea
14:07:45 <NikitaKonovalov> alazarev: you've got a +2 here https://review.openstack.org/#/c/140518/
14:08:20 <NikitaKonovalov> so I thinks we might get some of those merged soon
14:08:35 <alazarev> NikitaKonovalov, and here https://review.openstack.org/#/c/140485/
14:09:04 <crobertsrh> thanks for updating NikitaKonovalov....sorry I was late
14:09:53 <NikitaKonovalov> so that's all the update from me
14:10:02 <SergeyLukjanov> I hope we'll get faster reviews someday...
14:10:27 <SergeyLukjanov> #topic News / updates
14:10:29 <SergeyLukjanov> folks, please
14:10:54 <elmiko> still cranking away on the security docs, fixed some bugs, and a bunch o reviews
14:11:00 <SergeyLukjanov> mattf, thanks for the oslo sync :) you're still the only how authorised to do it :)
14:11:01 <sreshetnyak> i'm working on new integration tests and bux fixing
14:11:04 <tosky> working on tempest to remove the hardcoded dependencies on some plugins
14:11:26 <vgridnev_> i'm working with hdp bug and event-log
14:11:32 <kchen> please help review the cdh version management patch https://review.openstack.org/#/c/147933/
14:11:57 <weiting> working on cdh plugin impala service testing
14:11:59 <kchen> to separate the codes for different cdh versions
14:12:01 <NikitaKonovalov> as the new release of stable/jun approaches there is a list of backports that should be done. You can find a chain of changes here https://review.openstack.org/#/c/150825/
14:12:33 <egafford> Working on the unified job interface map impl now that the spec is merged (thanks SergeyLukjanov.)
14:12:34 <huichun> working on cdh service testing
14:13:06 <alazarev> I created a few of specs: auto clean up, placeholders in data sources
14:13:07 <mattf> SergeyLukjanov, it's muscle memory at this point. tho my plans are to obsolete myself. we can prune more and more these days.
14:13:38 <crobertsrh> Here's a fairly early version of the guided cluster creation ("wizard") page.  Feel free to take a peek and comment.  https://review.openstack.org/#/c/147677/
14:13:39 <SergeyLukjanov> mattf, heh
14:14:24 <SergeyLukjanov> crobertsrh, nice, /me need to to try
14:15:23 <SergeyLukjanov> #topic Kilo release schedule
14:15:28 <SergeyLukjanov> 3
14:15:33 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Kilo_Release_Schedule
14:15:38 <SergeyLukjanov> kilo-2 is next week
14:15:53 <SergeyLukjanov> and 2014.2.2 is next week two if I remember correctly
14:16:08 <SergeyLukjanov> I'd like to enlarge stable-maint core team for sahara
14:16:25 <tmckay> I should have spark-swift stuff in today, and I'm trying to verify cdh5 version for spark (from venza)
14:16:27 <SergeyLukjanov> and so, I need volunteers to review and help with support of stabe/juno branch
14:16:48 <tmckay> I volunteer
14:16:58 <tmckay> are there docs somewhere on stable maint?
14:17:18 <SergeyLukjanov> yup, I'll share it
14:17:31 <egafford> SergeyLukjanov: I'm pretty involved in stable maintenance downstream at Red Hat, so I'd like to volunteer too; fits in nicely with a lot of my work.
14:17:36 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/StableBranch
14:18:13 <SergeyLukjanov> tmckay, egafford, ack
14:18:49 <SergeyLukjanov> #topic some breaking change in saharaclient Python API to align with other clients (find vs findall, see first comment in https://review.openstack.org/#/c/150347/) (pas-ha, won't be there)
14:20:08 <SergeyLukjanov> currently I don't see any issues with it
14:20:15 <SergeyLukjanov> it just adds the find_unique
14:20:30 <elmiko> as we're talking about the client, it might be nice to watch this as well https://review.openstack.org/#/c/145579/
14:20:41 <elmiko> could be something we can implement in the future
14:21:06 <SergeyLukjanov> elmiko, ++
14:21:10 <elmiko> granted, it's an api change, but the client could use this for sorting
14:21:23 <SergeyLukjanov> alazarev, do you have any issues with https://review.openstack.org/#/c/150347/ ?
14:21:53 <alazarev> SergeyLukjanov, not issues
14:22:57 <alazarev> A question: do we want to make our client in line woth other openstack clients? If yes this will change client API
14:23:07 <alazarev> *with
14:23:22 <elmiko> i think we should be in line with the other clients
14:23:29 <mattf> alazarev, yes, apiclient is gone from oslo too
14:23:36 <crobertsrh> +1 for being in-line
14:23:42 <mattf> some potential pruning...
14:23:54 <alazarev> +1 for being inline
14:23:58 <SergeyLukjanov> +1 for being inline
14:24:24 <SergeyLukjanov> this change doesn't break anything, it just adds one more func
14:24:44 <SergeyLukjanov> so, currently there is no compatibility issues
14:24:58 <mattf> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/apiclient/base.py#L23
14:24:59 <alazarev> this clould break old versions of client users
14:25:27 <mattf> alazarev, how old?
14:25:38 <alazarev> e.g. juno sahara will not work with new client
14:25:51 <elmiko> mattf: makes me wonder, do we need to have some start porting stuff to the common openstack client?
14:25:55 <mattf> know that for sure, or speculating?
14:26:40 <alazarev> if we change method names - it will definitly be broken
14:27:20 <alazarev> I'm not about proposed change
14:27:27 <SergeyLukjanov> oh
14:27:28 <SergeyLukjanov> ok :)
14:27:34 <mattf> alazarev, sounds like direction is to get on the new client lib, but we'll have to be careful or plan breaks at appropriate times
14:27:36 <alazarev> I'm about message in the patch
14:28:03 <alazarev> we now use "find" as "findall"
14:28:06 <mattf> elmiko, dunno
14:28:10 <alazarev> do we want to rename?
14:28:26 <mattf> that's independent of the lib we use tho
14:28:34 <mattf> we could break that now and change lib later
14:28:56 * mattf resets
14:29:02 <SergeyLukjanov> alazarev, we could start using findall by adding alias while keeping find available too
14:29:06 <mattf> alazarev, gotcha. i'm all set.
14:29:36 <alazarev> SergeyLukjanov, but other clients use "find" as "find one"
14:29:44 <mattf> there's a semantic change going in already that allows for finding just one element, get an exception if not found
14:30:20 <mattf> alazarev, is find/findall the only break that'd happen to get us inline?
14:30:46 <SergeyLukjanov> IMO it should be investigated and a spec should be proposed for it
14:30:55 <mattf> +2
14:30:56 <elmiko> +1
14:31:01 <alazarev> mattf, I don't know, this is what Pavlo pointed
14:31:02 <tmckay> +1
14:31:04 <SergeyLukjanov> to make us able to discuss it and probably share with TC and some WGs
14:31:20 <mattf> +1 don't design now
14:31:43 <alazarev> what is the usual practice? should old versions work with any version of client?
14:31:46 <tmckay> And I'm okay with multiple names mapping to the same functionality for compat, maybe with a deprecate message
14:32:13 <mattf> alazarev, may be appropriate to go to findall for v2
14:32:20 <SergeyLukjanov> mattf, ++
14:32:22 <alazarev> tmckay, it is not possible since current name is used for other purpose
14:32:35 <tmckay> ah, I see.  nevermind :)
14:33:09 * tmckay is watching child drive while attending meeting
14:33:19 <alazarev> v2 could be a solution
14:33:29 <alazarev> do we have plans for v2?
14:33:34 <mattf> hehe
14:33:48 <mattf> -2 on changing semantics w/o bumping version
14:33:53 <elmiko> i'm still interesting in v2....
14:34:01 <elmiko> and i know pecan much better now ;)
14:34:03 <SergeyLukjanov> alazarev, it will be the most huge spec ever for sahara
14:34:13 <alazarev> elmiko, everyone is interested :)
14:34:44 <tmckay> maybe by now we should call it v3 <wink>
14:34:50 <elmiko> lol
14:35:18 <crobertsrh> sahara "millenium edition"
14:35:38 <mattf> next topic?
14:35:39 <alazarev> crobertsrh, millenium was in 2000 :)
14:35:51 <SergeyLukjanov> #topic Open discussion
14:35:52 <SergeyLukjanov> :)
14:35:55 <crobertsrh> No...I mean the NEXT millenium :)
14:36:02 <mattf> let's drop "ex Savanna"
14:36:06 <elmiko> i'd like to talk about DIB briefly
14:36:09 <egafford> crobertsrh: Then we'd have to quickly replace it with Sahara XP, and no one wants that.
14:36:13 <tmckay> +1
14:36:15 <SergeyLukjanov> mattf, already done for meeting page
14:36:16 <elmiko> mattf: +1
14:36:17 <crobertsrh> heh
14:36:32 <mattf> open season on "ex Savanna", if you see it, remove it
14:36:46 <mattf> and no, not from old docs
14:36:50 <elmiko> is DIB supposed to work with openSuse? because we have clauses for it in the script but it doesn't work on the latest version of openSuse....
14:36:58 <SergeyLukjanov> fixed for https://launchpad.net/sahara
14:37:32 <mattf> fyi, i added sahara to openhub yesterday - https://www.openhub.net/p/openstack-sahara
14:37:55 <elmiko> mattf: nice!
14:38:24 <mattf> openhub, ex ohloh
14:38:33 <elmiko> is anyone using openSuse to run diskimage-create?
14:38:44 <mattf> elmiko, i'm not
14:38:50 <elmiko> is there a reason for keeping compatibility?
14:39:04 <mattf> try git blame to find the author...
14:39:37 <elmiko> thought i'd bring it up here first, just to see if anyone knew about it
14:39:40 <elmiko> but yea
14:39:42 <mattf> Thomas Bechtold
14:39:46 <crobertsrh> There was a guy asking about OpenSuse about 6 months ago
14:39:52 <alazarev> I want to talk about cm_api one more time sinse I don't see ways to support indirect access for it; we need really few things from it, don't we want to create our own lib?
14:39:58 <mattf> commit 7dfcbe3a62813001376ecd71864e1bc947c7a7e6
14:39:58 <mattf> Author: Thomas Bechtold <tbechtold@suse.com>
14:39:58 <mattf> Date:   Tue Nov 18 07:06:38 2014 +0100
14:39:58 <mattf> Add openSUSE support for diskimage-create.sh
14:40:02 <mattf> kinda on purpose
14:40:12 <tosky> elmiko: afaik SuSE ships with their OpenStack version, not sure about Sahara
14:40:16 <SergeyLukjanov> alazarev, I'm ok with it
14:40:38 <tosky> do you mean another lib which replaces cm_api?
14:40:44 <tmckay> alazarev, more sahara-extra?
14:40:44 <elmiko> maybe i should send him an email asking him to fix compat for suse lol
14:40:55 <elmiko> or just file bugs i suppose
14:40:58 <SergeyLukjanov> nope, just a very simple client embeded to plugin
14:41:00 <tosky> a different wrapper for the proprietary CDH console?
14:41:05 <mattf> elmiko, very reasonable. contributed and should be maintained.
14:41:06 <SergeyLukjanov> like it was done in intel plugin
14:41:16 <tmckay> SergeyLukjanov, +1
14:41:17 <alazarev> tosky, just class in sahara, like we did with HDP
14:42:03 <mattf> SergeyLukjanov, alazarev, cm_api has many functions, we use a couple, and the couple we use are primarily to do the CDH Console REST calls? so the proposal is to drop cm_api and have our own impl of the REST calls?
14:42:25 <alazarev> mattf, exactly
14:42:44 <crobertsrh> It is tempting to "make our own" rather than try to wedge in the cm_api.
14:42:50 <mattf> +1 w/ regrets
14:42:50 <SergeyLukjanov> mattf, yeah, mostly because cm_api isn't packaged on any platforms and it's difficult to use it for some features like indirect access
14:43:27 <alazarev> CDH guys, what do you think?
14:43:38 <egafford> SergeyLukjanov: +1 on that as well; unless Cloudera starts packaging, it's an overall maintenance win. (Also +1 to mattf's regrets, but so it goes.)
14:43:57 <kchen> I think maybe we can define a subset of current cm_api, and only use those APIs.
14:44:07 <tmckay> if it's packaged in the future, of course, we can always use the "real thing"
14:44:14 <tmckay> and back out the home-brew
14:44:46 <egafford> tmckay: Indeed; that would be the ideal.
14:45:00 <weiting> Actually it's about maintenance. I mean if we do it we need to do the maintain job if there is any update from Cloudera.
14:45:21 <weiting> For long term plan, it should be a good idea to have it without cm_api.
14:45:50 <alazarev> weiting, is REST changed often?
14:45:55 <mattf> weiting, any info on likelihood of cloudera making incompatible changes to cm_api that we'd have to mirror?
14:46:07 <mattf> alazarev, good use of fewer words
14:46:09 <kchen> if we only maintain a subset of cm_api, maybe we can decrease the job of maintenance?
14:46:45 <tmckay> and does CDH use any kind of api versioning?  Maybe that would help, too
14:46:47 <alazarev> we need really few REST calls from it
14:46:59 <weiting> CM usually support the older version cm_api
14:47:13 <kchen> Yes. currently released cm_api is v8
14:47:25 <SergeyLukjanov> oh, and cm_api doesn't support py33 => no way to global requirements
14:47:37 <weiting> And v8 can support v7 and v6
14:47:50 <SergeyLukjanov> alazarev, ++
14:47:58 <mattf> weiting, what's the release cadence?
14:48:42 <weiting> We don't know. There is no cadence currently.
14:49:16 <tmckay> sounds to me like maintenance would be reasonable, with api versions and a subset of REST calls
14:49:21 <weiting> It depends on CM and CDH update.
14:49:33 <mattf> ok
14:49:39 <kchen> maybe we can stay on v8, until changes are required.
14:49:47 <mattf> we can always decide later that this was a bad decision and back it out
14:49:56 <tmckay> +1
14:50:12 <mattf> openstack bot isn't holding any stone tablets
14:50:38 <elmiko> yea, given the talk seems like we need to make our own version instead of using cm_api
14:50:59 <elmiko> the py33 support is a total deal breaker for gobal req., as SergeyLukjanov said
14:51:09 <alazarev> do we have volunteers to make our small version of cm_api?
14:51:22 <egafford> weiting: Our other options are to independently maintain cm_api itself in packages on n OSes, beg Cloudera to package it on n OSes themselves, and not support Cloudera, right? Failing Cloudera packaging, the subset seems like the smallest to me.
14:53:28 <weiting> Ok, let us propose a bp for a subnet of cm_api.
14:54:26 <SergeyLukjanov> ack
14:54:30 <SergeyLukjanov> anything else?
14:54:33 <SergeyLukjanov> 5 mins left
14:54:45 <crobertsrh> nothing from me
14:54:52 <alazarev> nothing from me
14:55:02 <kchen> no from me.
14:55:10 <tmckay> intel guys, I will try to address Oozie mail thread questions today or tomorrow.  Trying to validate cdh5 for spark
14:55:20 <mattf> nothing here
14:55:55 <kchen> hope we can finish the subset of cm_api patch by kilo3
14:56:37 <SergeyLukjanov> kchen, it'll be great, but don't forget that spec is needed earlier
14:56:47 <kchen> tmckay: thanks
14:56:57 <egafford> kchen: +1. I'd like to volunteer to help with impl to get that done, once the spec is up; full Cloudera support by Kilo would be wonderful.
14:57:04 <SergeyLukjanov> FPF is March 5
14:57:09 <kchen> SergeyLukjanov: ok
14:57:10 <SergeyLukjanov> (feature proposal freeze)
14:58:20 <kchen> egafford: thanks
14:58:41 <kchen> I will finish the specs asap
14:58:47 <SergeyLukjanov> thanks folks
14:58:51 <SergeyLukjanov> #endmeeting