18:08:02 <SergeyLukjanov> #startmeeting sahara
18:08:03 <openstack> Meeting started Thu Mar 27 18:08:02 2014 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:08:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:08:06 <openstack> The meeting name has been set to 'sahara'
18:08:20 <mattf> hello SergeyLukjanov
18:08:52 <crobertsrh> hello
18:08:57 * SergeyLukjanov completely forgot about meeting
18:09:33 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SavannaAgenda
18:09:34 <SergeyLukjanov> ookay
18:09:47 <SergeyLukjanov> #topic Project renaming status
18:10:17 <SergeyLukjanov> so, I think that we could say that renaming successfully completed except several very nit places
18:10:30 * SergeyLukjanov going to update wiki page about meetings after the meeting
18:10:40 <SergeyLukjanov> #info enaming successfully completed except several very nit places
18:11:04 <SergeyLukjanov> #undo
18:11:05 <openstack> Removing item from minutes: <ircmeeting.items.Info object at 0x29461d0>
18:11:11 <SergeyLukjanov> #info renaming successfully completed except several very nit places
18:11:22 <SergeyLukjanov> any more thoughts on it?
18:11:28 <alazarev> are we going to rename savanna-ci?
18:11:46 <SergeyLukjanov> alazarev, yup, after rc1 I think to not break dev process
18:12:07 <SergeyLukjanov> oh, as mattf suggested I think that we should have 0.7.0 release of client w/o backward compat aliaces
18:12:12 <SergeyLukjanov> aliases*
18:12:35 <SergeyLukjanov> I'm thinking about tomorrow if everything will work good
18:12:45 <SergeyLukjanov> so, it's a good time to propose fixes to the client ;)
18:13:07 <SergeyLukjanov> ok, let's move on
18:13:11 <SergeyLukjanov> #topic RC1 status
18:13:33 <SergeyLukjanov> so, as planned we postpone RC1 cut for one week and it'll be next week
18:14:00 <SergeyLukjanov> we're now waiting for rest of Hadoop 2 changes to be merged for both vanilla and idh
18:14:22 <SergeyLukjanov> + I hope to include doc updates to the RC1 too to avoid a big number of backports
18:14:41 <SergeyLukjanov> #info RC1 next week, ~ April 3
18:14:46 <SergeyLukjanov> any objections?
18:15:18 <SergeyLukjanov> is there anyone here?
18:15:23 <SergeyLukjanov> :)
18:15:26 <elmiko> here :)
18:15:28 <jspeidel> no objection
18:15:30 <elmiko> no objection
18:15:38 <SergeyLukjanov> good, cool
18:15:38 <alazarev> no objections :)
18:15:40 <ErikB1> Ok w/ me
18:15:42 <crobertsrh> no objection
18:15:50 <jspeidel> how much work do we have left for IDH 2.0?
18:16:00 <jspeidel> I ask because of this
18:16:02 <jspeidel> #info http://gigaom.com/2014/03/27/intel-ditches-its-hadoop-distro-and-puts-millions-behind-cloudera/
18:16:03 * SergeyLukjanov fighting cold and just wake up :)
18:16:22 <ErikB1> just change the I to C
18:16:22 * mattf is here now
18:16:31 <SergeyLukjanov> jspeidel, it's ready and on review, just waiting for stable passing os savanna-ci
18:16:36 <mattf> jspeidel, good point
18:16:39 <SergeyLukjanov> ErikB1, :)
18:17:12 <SergeyLukjanov> let's discuss idh integration later, I hope alazarev will share some thoughts / details on it
18:17:43 <SergeyLukjanov> [22:12:08]  <SergeyLukjanov>	 oh, as mattf suggested I think that we should have 0.7.0 release of client w/o backward compat aliaces
18:17:43 <SergeyLukjanov> [22:12:12]  <SergeyLukjanov>	 aliases*
18:17:43 <SergeyLukjanov> [22:12:36]  <SergeyLukjanov>	 I'm thinking about tomorrow if everything will work good
18:17:47 <SergeyLukjanov> matt ^^
18:17:50 <SergeyLukjanov> mattf*
18:18:17 <SergeyLukjanov> ok, moving on
18:18:27 <SergeyLukjanov> #topic Bug / doc days
18:18:29 <mattf> SergeyLukjanov, ack, tomorrow, check
18:18:53 <alazarev> jspeidel: integration with IDH 2.0 is done, integration tests are on review
18:18:59 <SergeyLukjanov> we should have a "testing" day after the RC1 to verify sahara
18:19:06 <SergeyLukjanov> and re doc day - we need it
18:19:10 <SergeyLukjanov> really need
18:19:16 <mattf> alazarev, jspeidel is suggesting no one will care... intel has dropped idh
18:19:47 <SergeyLukjanov> so, I'm proposing to choose a day early next week to write some docs
18:20:05 <SergeyLukjanov> and we need volunteer for tracking docs, ruhe is unavailable
18:20:26 <SergeyLukjanov> I'm proposing Monday for doc day
18:20:37 <ErikB1> what does that exactly mean?
18:21:10 <SergeyLukjanov> coordinate/collaborate to write docs, team will work on it the whole day
18:21:19 <SergeyLukjanov> like we have bug triage day in the mid cycle
18:21:35 * ErikB1 thinks that sounds fine
18:21:49 <SergeyLukjanov> it'll help to push missing docs faster
18:22:00 <SergeyLukjanov> so, here is a blueprint https://blueprints.launchpad.net/sahara/+spec/update-docs-icehouse
18:22:04 <crobertsrh> +1 for Monday
18:22:13 <SergeyLukjanov> any volunteers to track doc activity and status?
18:22:25 <SergeyLukjanov> update blueprint, assign writers, erc.
18:22:44 <SergeyLukjanov> :(
18:23:10 <elmiko> i will volunteer to help, but i don't have the most experience :/
18:23:58 <SergeyLukjanov> elmiko, you'll need to set find volunteers for doc updates and know their status + find missed items
18:24:21 <elmiko> that sounds... challenging
18:24:51 <elmiko> if other's are willing to help though, i don't mind trying to collate everything
18:24:52 <jspeidel> elmiko, thanks for volunteering
18:25:07 <elmiko> jspeidel: :)
18:25:15 <crobertsrh> I'll certainly help update docs wherever I can
18:25:28 <mattf> i say the next person to join does it
18:25:37 <aignatov> elmiko: I'll help you if i have time on Mon
18:25:39 <SergeyLukjanov> I really hope that we will write most of the docs at Monday
18:25:48 <elmiko> lol
18:25:50 <mattf> themistymay, winner!
18:26:06 <themistymay> yeah?
18:26:10 <SergeyLukjanov> themistymay, congratulations!
18:26:15 <elmiko> just to be clear, based on the blueprint SergeyLukjanov posted i need to make sure we get all those items finished?
18:26:25 <themistymay> oh god, what did i get myself into
18:26:27 <mattf> themistymay, by being the first person to join (last), you get the option to drive a docs day on monday
18:26:34 <aignatov> themistymay: you will coordinate doc updating on monday
18:26:36 <SergeyLukjanov> themistymay, you're the chief doc writer of openstack!
18:26:40 * elmiko backs aways slowly
18:26:49 <themistymay> oh man
18:26:59 <themistymay> that sounds, like a blast
18:27:04 <SergeyLukjanov> :)
18:27:24 <mattf> sounds like no one is jumping at the chance
18:27:28 <themistymay> i get my opensouce approvals back and this is what i get
18:27:36 <themistymay> I'd be happy to help
18:27:39 <SergeyLukjanov> elmiko, yup, and probably add some more items
18:27:43 <themistymay> but leading may be dangerous
18:28:00 <SergeyLukjanov> so, I'm assigning bp to elmiko
18:28:11 <elmiko> i'll give it my best :)
18:28:11 <SergeyLukjanov> and hope that we all help him on coordination
18:28:19 <SergeyLukjanov> and hope all folks will write docs ;)
18:28:33 <jspeidel> elmiko, I can help with HDP related docs
18:28:56 <SergeyLukjanov> it doesn't mean that elmiko will write all docs :)
18:29:06 <mattf> or even any
18:29:12 <SergeyLukjanov> I hope some ;)
18:29:16 <elmiko> my only concern is my lack of familiarity with sahara
18:29:22 <alazarev> elmiko: probably I;m the right person for IDH stuff
18:29:24 <themistymay> same here
18:29:37 <SergeyLukjanov> so, I'm expecting that component "owners" will cover their components like plugins, etc.
18:29:42 <elmiko> cool
18:29:47 <aignatov> actually, do we have any new text to be added to docs?
18:29:53 <jspeidel> SergeyLukjanov, +1
18:29:58 <elmiko> i am logging this, so beware, i will find people on monday!
18:29:59 <SergeyLukjanov> we should update all docs in fact
18:30:10 <SergeyLukjanov> especcially dev / installation / user guides
18:31:04 <SergeyLukjanov> docs are very important for us - it's our first real OpenStack aligned release
18:31:33 <SergeyLukjanov> so, I'm asking all of you to write/check/update docs
18:31:40 <themistymay> can do!
18:31:56 <SergeyLukjanov> any objects for doc day @ Monday?
18:32:18 <alazarev> no objections :)
18:32:22 <themistymay> none here
18:32:30 <crobertsrh> no objections
18:32:35 <aignatov> native writers are welcome to update mistakes in the docs :)
18:32:57 <themistymay> aignatov :P
18:33:02 <SergeyLukjanov> yeah, that's a real issue for Russians ;)
18:33:06 <tmckay> late to the party, sure, I'll sign up
18:33:21 <SergeyLukjanov> awesome
18:33:25 <tmckay> We missed the meeting last week and I forgot about it :)
18:33:32 <SergeyLukjanov> tmckay, me too :)
18:33:33 <tmckay> Short memory
18:33:35 <crobertsrh> it was cancelled
18:33:52 <tmckay> ack, missed == cancelled
18:35:02 * SergeyLukjanov probably we'll unavailable on Tuesday - kitchen furniture and electronics "installation" :)
18:35:13 <SergeyLukjanov> ok, let's move on
18:35:31 <SergeyLukjanov> #topic Action items from the last meeting
18:35:41 <SergeyLukjanov> #info nope
18:35:48 <SergeyLukjanov> #topic News / updates
18:35:51 <SergeyLukjanov> folks, please!
18:36:45 <aignatov> few of renaming, heat bug fixing were on me
18:36:49 <aignatov> last week
18:37:10 <crobertsrh> Mostly just small bugs for me on the dashboard recently.  I'm also trying to come up to speed on disk image building.
18:37:47 <themistymay> I am back to available for now.  My new company is reviewing the paperwork for intellectual property still, until they get back to me, I am free to commit to open source.  If things change, I will update and drop off again (*sigh*).  Docs seem like a good step.
18:37:50 <mattf> <jspeidel> #info http://gigaom.com/2014/03/27/intel-ditches-its-hadoop-distro-and-puts-millions-behind-cloudera/ <- news, raises question about future of idh plugin
18:38:09 <jspeidel> mattf, yes
18:38:29 <jspeidel> is this the correct time to discuss IDH?
18:38:33 <ErikB1> Still working on Blueprints. Figuring out when we can integrate with HDP Plugin.
18:39:12 <ErikB1> Also looking at logging and determining what the short comings are on Sahara when *certain* things fail and what is logged. Essentially a usability issue.
18:39:39 <SergeyLukjanov> yup, we could try (re <jspeidel>	 is this the correct time to discuss IDH?)
18:39:59 <alazarev> IDH integration is done, no sense in canceling it in Icehouse
18:40:09 <SergeyLukjanov> alazarev, agreed
18:40:11 <jspeidel> alazarev, agreed
18:40:28 <jspeidel> I would vote to remove it after Icehouse
18:40:40 <alazarev> we should consider adding cloudera in J
18:41:11 <aignatov> ErikB1: did you think about demo examples?
18:41:31 <alazarev> and it would be great to have cloudera integration done by cloudera guy
18:41:31 <SergeyLukjanov> so, re IDH - we should discuss removing of it with Intel folks first
18:42:01 <SergeyLukjanov> and then make a decision on summit with the default answer to remove it in Juno release
18:42:07 <SergeyLukjanov> alazarev, ++
18:42:31 <aignatov> what's about integration tests for idh?
18:42:53 <jspeidel> aignatov, I wouldn't spend any more time on them
18:43:25 <SergeyLukjanov> aignatov, we should merge and run them for Icehouse due to the fact we're including plugin into the release
18:43:44 <jspeidel> aignatov, yes keep whatever we have
18:43:53 <aignatov> ok
18:44:02 <SergeyLukjanov> the funny thing is that we have a good enough coverage :)
18:44:17 <SergeyLukjanov> s/funny/sad/
18:44:30 <aignatov> I see, I meat that we can make idh job is non-voting
18:45:00 <dmitryme> aignatov: I think we should keep it voting for Icehouse
18:45:05 <alazarev> aignatov: why? if we support plugin it should work
18:45:21 <dmitryme> if we ship it in the release, then lets make sure it is working
18:45:26 <SergeyLukjanov> yup, if we're including it in I then it should work
18:45:27 <jspeidel> aignatov, +1 to keep it as voting
18:45:30 <alazarev> aignatov: unless we remove plugin completely
18:46:00 <SergeyLukjanov> so, let's keep idh job voting till the Icehouse release and than will see
18:46:02 <mattf> sounds business as usual w/ a slant to removing idh in J
18:46:13 <jspeidel> lets just not spend any more time on it then we need to get existing tests stable
18:46:23 <aignatov> ok, ok, I'm surrendering, it was just an idea :)
18:46:26 <mattf> but we should get feedback from idh plugin users -- how do we do that?
18:46:52 <jspeidel> mattf, you need to find them first
18:47:20 <SergeyLukjanov> #action alazarev contact intel folks to clarify idh drop off
18:47:31 <alazarev> probably there will be some migration plan from Intel
18:47:33 <mattf> in a world where you don't know your users the best you can do is public execute a deprecation plan
18:47:57 <alazarev> I don't believe they will just stop IDH support
18:48:00 <mattf> and doing that takes time, so something for J
18:48:27 <SergeyLukjanov> mattf, yup, release in I and decide on summit for J sounds good for me
18:48:59 <mattf> we should also discuss HDP1 and Vanilla1
18:49:10 <aignatov> mattf: agree
18:49:49 <SergeyLukjanov> so, agreed on "IDH plugin release as part of Icehouse, decide the fate of IDH on summit and probably deprecate / remove in Juno"
18:49:51 <aignatov> as I see Hadoop 2.X is the trend now, so for me, we don't need to expand features on Hadoop 2.x
18:49:52 <SergeyLukjanov> any objections?
18:50:08 <aignatov> SergeyLukjanov: +1
18:50:09 <SergeyLukjanov> aignatov, you mean 1.x?
18:50:15 <aignatov> yes
18:50:28 <alazarev> SergeyLukjanov: +1
18:50:54 <jspeidel> aignatov, that seems to be right
18:50:57 <tmckay> do we have Hadoop 2 images pre-generated anywhere?
18:51:12 <tmckay> I was playing with HDP plugin but I used centos with 1.3.2
18:51:24 <SergeyLukjanov> #agreed IDH plugin release as part of Icehouse, decide the fate of IDH on summit and probably deprecate / remove in Juno
18:51:25 <alazarev> hadoop2 is a trend for customers who build new hadoop clouds
18:51:30 <mattf> ErikB1, bob_nettleton ^^
18:51:52 <aignatov> jspeidel, alazarev: well, it should be discussed on the summit
18:52:06 <SergeyLukjanov> I'm afraid that there are still too many users of Hadoop 1, so, probably, it's a bit early to drop of 1.x
18:52:07 <bob_nettleton> tmckay, we have HDP 2.0.6 images generated in sahara-image-elements
18:52:11 <alazarev> customers who have hadoop clouds already just don't look into sahara's side now
18:52:16 <SergeyLukjanov> anyway, it's a good topic for summit
18:52:30 <jspeidel> SergeyLukjanov, not drop just not spending time to enhance
18:52:41 <tmckay> bob_nettleton, okay, great
18:52:59 <bob_nettleton> tkmckay, if you use "diskimage-create.sh -p hdp", the script will generate images for both HDP versions.
18:53:01 <mattf> now that we have multiple plugins and the tech is moving, we should have a discussion about plugin lifecycle
18:53:33 <SergeyLukjanov> jspeidel, sounds reasonable IMO
18:53:34 <tmckay> bob_nettleton, thank you
18:53:35 <alazarev> jspeidel: It is strange to add features to hadoop2 and don't support them in 1.x
18:53:36 <mattf> something prematurely kicked off before hong kong
18:53:54 <bob_nettleton> tmckay, you're welcome.
18:54:01 <aignatov> tmckay: IvanBerezovsky told me that he upload vanilla hadoop 2 images to the savanna-files
18:54:02 <mattf> alazarev, yeah, feature parity / support from plugins... complex.
18:54:20 <tmckay> aignatov, that's great, too, thanks
18:54:41 <SergeyLukjanov> mattf, yup, especially while we're growing our base features
18:55:10 <mattf> 5 min warning
18:55:13 <tmckay> aignatov, did we make any progress on picking a demo to do?  I forgot to look in my training manuals ...
18:55:25 * mattf is very interested in ^^
18:55:33 <mattf> VERY
18:55:52 <aignatov> tmckay: no, i'm looking for ErikB1's ideas
18:55:54 <tmckay> mattf, missing the antecedent of ^^
18:56:01 <tmckay> which ^^ is that?
18:56:03 <jspeidel> we have recently seen issue with openstack rate limits when trying to provision cluster > 20
18:56:06 <mattf> tmckay, demo
18:56:11 <jspeidel> is this an issue with Heat integration?
18:56:12 <tmckay> thought so :)
18:56:42 <aignatov> jspeidel: did you use heat engine for your cluster?
18:56:46 <tmckay> mattf, I meant to ping jvyas and see if he has ideas
18:56:57 <aignatov> actually it's not turned on by default
18:57:02 <SergeyLukjanov> #topic Open discussion
18:57:05 <dmitryme> jspiedel: you can tune rate limits, is there any problem with that?
18:57:06 <SergeyLukjanov> 3 mins left
18:57:18 <jspeidel> ^^ bnettleton
18:57:45 <tosky> bob_nettleton: were you able to generate proper centos-based images? I've seen few issues ( elmiko too ), maybe selinux
18:57:50 <aignatov> one idea from the books it's a weather analyser job
18:58:02 <SergeyLukjanov> jspeidel, anyway, it depends on OpenStack conf and yes, it could fail if rates are very low
18:58:20 <bob_nettleton> tosky, I have been able to generate HDP images on Centos.  I've been using Ubuntu to generate the images without any problems.
18:58:27 <SergeyLukjanov> jspeidel, but in the heat provisioning engine, it should b handled by heat
18:58:57 <bob_nettleton> SergeyLukjanov, is heat turned on by default in Sahara now?
18:59:03 <tmckay> aignatov, an accurate weather predictor would be awesome
18:59:06 <jspeidel> Sergeyukjanov, appears that the limits are very low by default
18:59:08 <SergeyLukjanov> 1 min left
18:59:11 <tosky> bob_nettleton: so, there is an issue when generating them from Fedora or CentOS (see https://bugs.launchpad.net/sahara/+bug/1292614 )
18:59:14 <SergeyLukjanov> bob_nettleton, nope
18:59:26 <SergeyLukjanov> oh, good catch
18:59:34 <tosky> bob_nettleton: and also, the CentOS image that I generated from Ubuntu needed a selinux relabeling
18:59:40 <aignatov> tmckay: one concern is that canonical example from the book
18:59:52 <SergeyLukjanov> looks like we should postpone enabling heat by default to the time when heat will gate on sahara to avoid random issues
18:59:54 <aignatov> it's kinda of boring for hadoopers :)
18:59:58 <tmckay> aignatov, ack
19:00:01 <SergeyLukjanov> like we see last month
19:00:04 <mattf> SergeyLukjanov, +1
19:00:08 <SergeyLukjanov> #endmeeting