14:00:48 <tellesnobrega> #startmeeting sahara
14:00:49 <openstack> Meeting started Thu Nov 30 14:00:48 2017 UTC and is due to finish in 60 minutes.  The chair is tellesnobrega. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:50 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:52 <openstack> The meeting name has been set to 'sahara'
14:01:19 <tellesnobrega> waiting for people to show hands
14:01:20 <shuyingya> o/
14:01:42 <shuyingya> Wait me 3 minutes please
14:01:52 <tellesnobrega> sure
14:02:01 <jeremyfreudberg> o/
14:04:52 <tellesnobrega> waiting until 5 after so we can start
14:05:07 <jeremyfreudberg> sure
14:05:15 <tellesnobrega> and that is time
14:05:22 <tellesnobrega> #topic News/Updates
14:05:24 <shuyingy_> Hi  I am comming
14:05:39 <tosky> here, sorry
14:05:52 <jeremyfreudberg> o/
14:06:22 <tellesnobrega> I'm currently finishing up the work on decommission of specific node. I just need to check shuyingy_'s comment on the order of deletion, and write tests
14:06:39 <tellesnobrega> other than that, I need to get back to mapr image generation
14:06:53 <tellesnobrega> and try to squeeze some time to work on plugin outside sahara code
14:07:03 <tellesnobrega> not sure it will happen, the last one, but I will try
14:07:41 <tellesnobrega> Oh, I also need to do the decommission work on the sahara client and UI
14:07:56 <tellesnobrega> UI should be a pain, but it is a necessary evil
14:09:08 <jeremyfreudberg> so, I'm working on revising s3_hadoop element, finishing up s3 job binary mostly
14:09:37 <tellesnobrega> good, do you believe we will have the feature in queens?
14:09:41 <shuyingy_> Today is the last day of business travel. this week mostly working on deploying sahara on k8s by helm. If you are interesting, please review https://review.openstack.org/#/c/477718/. and will continue to work on HBase support next week. hopefully it can be finished next week.
14:09:53 <jeremyfreudberg> tellesnobrega, s3 job binary should defniitely be done
14:10:21 <jeremyfreudberg> data source is another story, those are always a bit trickier since they have to work on the cluster itself, it's not just an abstraction
14:10:22 <tellesnobrega> that is great shuyingy_, did it all work well:
14:10:23 <tellesnobrega> ?
14:10:50 <tellesnobrega> jeremyfreudberg, I understand, we still have some time, let me know if you need help
14:10:58 <jeremyfreudberg> tellesnobrega, sure
14:11:35 <shuyingy_> I am try to review s3 patch. but doesn't find any problem because of lack of knowledge about s3
14:12:12 <jeremyfreudberg> yes, it would be good have someone who really knows S3 to say whether or not we are exposing enough boto options
14:12:20 <jeremyfreudberg> i made some nodes about s3 region and such
14:12:25 <jeremyfreudberg> there are lots of fancy kwargs that can go there
14:12:31 <jeremyfreudberg> but who knows what users actually need
14:12:36 <jeremyfreudberg> *notes
14:12:47 <tellesnobrega> that might be tricky to find
14:13:03 <tellesnobrega> s3 knowledge interested in OpenStack, even more on sahara
14:13:04 <shuyingy_> yes, it works well. I need try to find more frequently changed configurations for user to configure sahara.
14:14:14 <tellesnobrega> tosky, any news/updates?
14:14:33 <tosky> I'm swamped in internal testing, but I'm still keeping an eye on the zuul changes (and nagging zuul developers)
14:14:52 <tellesnobrega> cool
14:15:01 <tosky> so yes, we will have proper native jobs way before queens (hopefully before the end of the year)
14:15:13 <tellesnobrega> nice :) that is great to hear
14:16:07 <tosky> I think I already wrote it: I could have pushed some changes, and then fixed them; but infra is not pushing to remove the old jobs, so I'm lazily waiting to avoid some loops of write/fix
14:16:27 <tellesnobrega> makes sense
14:16:31 <jeremyfreudberg> tosky, i see
14:16:32 <shuyingy_> now is in middle of Queen, right? I am sorry that i didn't finished HBase support and keypair injecting util now
14:16:52 <jeremyfreudberg> shuyingy_, that's ok
14:16:58 <jeremyfreudberg> i have the same schedule problems
14:17:02 <tellesnobrega> shuyingy_, yes, we are approaching q-2. It is on December 7th
14:17:33 <jeremyfreudberg> feature freeze is between Jan 22 - Jan 26, so you still have time after that
14:17:39 <tosky> up to the PTL, but as long as the new features are additional and do not impact the existing code, it should not be a problem if they land a bit later
14:17:39 <tellesnobrega> yes
14:18:00 <tellesnobrega> and we can always get FFE if needed
14:18:11 * jeremyfreudberg reminds everyone he has to log off today by 14:27 UTC
14:18:18 <shuyingy_> that is great
14:18:20 <tellesnobrega> it won't be a problem I believe. But we can't wait too much
14:19:36 <tellesnobrega> lets move on, we have a important topic for today
14:19:44 <jeremyfreudberg> every topic is important
14:19:47 <jeremyfreudberg> ;)
14:19:51 <shuyingy_> :)
14:19:51 <tellesnobrega> true
14:19:54 <shuyingy_> aha
14:19:57 <tellesnobrega> critical one
14:20:07 <tellesnobrega> #topic plugins update
14:20:26 <jeremyfreudberg> indeed, this is is a critical topic
14:20:35 <tellesnobrega> At the PTG we decided that we should try to have all updates by Q2
14:20:46 <tellesnobrega> that means we have about 7 days to finish them all
14:21:02 <tosky> or defines exceptions :)
14:21:09 <tellesnobrega> this isn't a must, and if we fail, nobody will die.
14:21:35 <tellesnobrega> My plan here is for us to have a list of the plugins that needs to be updated and we can start working on it asap
14:21:42 <jeremyfreudberg> yes, nobody will die, and i put a provision here https://releases.openstack.org/queens/schedule.html#q-sahara-plugin-deadline that telles can basically suspend this rule if he wants
14:21:59 <tosky> and an estimated rough complexity, if you can
14:21:59 <jeremyfreudberg> yes, let's figure out what can get updated
14:22:04 <jeremyfreudberg> yes
14:22:08 <tellesnobrega> so we can have it finished close to our schedule
14:22:33 <jeremyfreudberg> so, there is a finally as of a few weeks ago a stable Hadoop 2.8.x release
14:22:37 <jeremyfreudberg> it ended up 2.8.2
14:22:45 <jeremyfreudberg> there is no stable 2.9 or 3.0 yet
14:22:51 <tellesnobrega> yes
14:23:42 <jeremyfreudberg> so vanilla plugin can be upgraded if we want
14:23:51 <jeremyfreudberg> along with pig and hive and oozie(?) on that image
14:24:06 <tosky> and will 2.9 be close 2.8, before the big breakage with 3.0?
14:24:16 <tellesnobrega> sounds about right
14:24:28 <tosky> I wonder why they still plan full development on the old series, but well...
14:24:41 <jeremyfreudberg> tosky, 2.9 is close to 2.8 yes
14:24:46 <jeremyfreudberg> and there will be parallel development
14:24:58 <jeremyfreudberg> not quite like Python 2 / Python 3, but something like that :(
14:25:15 <jeremyfreudberg> so, vanilla
14:25:24 <jeremyfreudberg> spark plugin can can be upgraded too
14:25:31 <shuyingy_> cdh too
14:25:42 <shuyingy_> it has cdh 5.13 right now
14:25:44 <jeremyfreudberg> cdh maybe not, i had some problem with a package no longer available
14:25:58 <jeremyfreudberg> we had to cancel our 5.12 upgrade because we didn't have time to figure it out
14:26:09 <tellesnobrega> #action update vanilla to 2.8.2 also pig, hive and oozie(?)
14:26:27 <jeremyfreudberg> for spark there is 2.1.2 and 2.2.0 so far
14:26:52 <jeremyfreudberg> the cdh thing was navigator trustee if i recall correctly
14:26:55 <tellesnobrega> shuyingy_, jeremyfreudberg is right, we have to take a deeper look into CDH, they changed some packages to enterprise only, so we may have hit a roadblock there
14:27:07 <jeremyfreudberg> btw, i have to sign off basically now
14:27:13 <jeremyfreudberg> but tellesnobrega can figure out storm
14:27:17 <tellesnobrega> thanks jeremyfreudberg
14:27:20 <tosky> bye jeremyfreudberg o/
14:27:29 <jeremyfreudberg> mapr and hdp i'm sure have something new
14:27:30 <shuyingy_> jeremy byebye
14:27:35 <jeremyfreudberg> not sure if that's absolutely essential
14:27:37 <jeremyfreudberg> bye all!
14:27:42 <jeremyfreudberg> send me an email if you have urgent questions
14:27:44 <tellesnobrega> see ya
14:27:48 <tellesnobrega> will do
14:27:50 <jeremyfreudberg> i will read the trasncript of ths meeting asap
14:28:06 <tellesnobrega> do we want both new spark versions?
14:28:36 <shuyingy_> I don't think so.
14:28:39 <tosky> uhm, do you mean basic spark and cdh/spark?
14:28:41 * tosky confused
14:28:46 <tellesnobrega> I think spark 2.1.0 should not be deprecated yet, we can add the newest one
14:28:51 <tellesnobrega> upstream spark
14:28:55 <tosky> oh, I see
14:29:12 <tellesnobrega> and later, if needed we replace 2.1.0 with 2.1.2
14:29:24 <tosky> isn't 2.1.2 just a minor update for 2.1? I think we didn't solve the problem of minor compatibile versions
14:29:40 <shuyingy_> we don't need update minor version I think
14:29:43 <tosky> maybe we could just allow the code to define multiple tags for each plugin/version
14:29:54 <tosky> so can we register it as 2.1.0 and it should work?
14:30:05 <tellesnobrega> that would be nice
14:30:56 <tellesnobrega> to be sure, spark is going to 2.2.0
14:31:22 <tellesnobrega> and we keep 2.1.0 and see if we can work tags to have minor versions
14:31:32 <tellesnobrega> all agreed?
14:32:14 <shuyingy_> do you mean we can build an image with spark 2.1.2 and tag it as 2.1.0?
14:32:30 <tosky> I would say that it should be fine for n ow
14:32:32 <tosky> now
14:32:47 <tellesnobrega> shuyingy_, yes
14:32:51 <shuyingy_> from code side, that's is not difficult. sound nice
14:33:01 <shuyingy_> \o/
14:33:12 <tellesnobrega> #action update spark to 2.2.0
14:33:20 <tellesnobrega> lets move on to storm
14:33:27 <tellesnobrega> storm newest version is 1.1.1
14:33:43 <tellesnobrega> we have 1.1.0, so we can take the same approach we did for spark
14:34:24 <tellesnobrega> sounds ok?
14:35:14 <shuyingy_> sure. it is really convenient to user
14:35:41 <tellesnobrega> cool
14:35:46 <tellesnobrega> MapR
14:35:57 <tellesnobrega> I check their website, seems like the newest version is 5.2
14:36:00 <tellesnobrega> which we already have
14:36:30 <tosky> it would be nice to understand why they disappeared also from sahara
14:36:37 <tosky> but no new upstream versions too, uhm
14:37:19 <tellesnobrega> wait, seems like there is a mapr 6
14:38:21 <tellesnobrega> but in tr mapr the newest version is 5.2
14:40:00 <tellesnobrega> yeah, seems like 5.2 is the last one on downloads
14:40:19 <tellesnobrega> Ambari has a new version 2.6
14:41:21 <tellesnobrega> https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.6.0
14:41:22 <tosky> I can't find the release date for 6, but there is the documentaiton
14:41:35 <shuyingy_> we may not have too much energy to update them all. MapR used to update by MapR themself, right?
14:41:43 <tosky> and which HDP versions can be deployed by ambari 2.6?
14:41:56 <tellesnobrega> tosky, checking that now
14:42:00 <shuyingy_> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_support-matrices/content/ch_matrices-ambari.html#ambari_stack
14:42:15 <tellesnobrega> 6.0.0	November 21, 2017	N/A
14:42:19 <tellesnobrega> tosky, ^
14:42:44 <tosky> tellesnobrega: argh, one week
14:42:51 <tosky> I would say we can push it to Rocky
14:42:57 <tellesnobrega> tosky, for sure
14:43:17 <tellesnobrega> seems like hdp doesn't support ambari 2.6 yet
14:43:25 <tosky> that would be weird
14:43:46 <tosky> I remember that Ambari x.y should support at least HDP x.y
14:43:54 <tosky> but I may remember incorrectly
14:44:15 <tellesnobrega> the link is looking was wrong
14:45:29 <tellesnobrega> on HDP website there is only the matrix up to ambari 2.5
14:46:05 <tellesnobrega> https://hortonworks.com/blog/hdp-2-6-3-dataplane-service/
14:46:14 <tellesnobrega> tosky, shuyingy_  ^
14:46:27 <tosky> uhm uhm
14:46:47 <tellesnobrega> both ambari 2.6.0 and HDP 2.6.3
14:46:50 <tosky> at least we are not the only one with documentation that requires some updates :)
14:46:58 <tellesnobrega> lol
14:47:01 <tellesnobrega> true
14:47:13 <shuyingy_> haha
14:47:16 <shuyingy_> :)
14:47:18 <tellesnobrega> so we want to update ambari to 2.6.0 with HDP to 2.6.3
14:47:22 <tellesnobrega> all in favor?
14:48:33 <shuyingy_> up
14:48:36 <shuyingy_> agree
14:49:17 <tellesnobrega> #action update ambari to 2.6 with HDP to 2.6.3
14:49:41 <shuyingy_> Now we should decide how to divide the work
14:49:45 <tellesnobrega> I think we covered all plugins
14:49:50 <tellesnobrega> yes
14:49:52 <shuyingy_> yes
14:50:10 <tellesnobrega> I believe that spark and storm should be fairly quick work, I can take them both
14:50:33 <tosky> only HDP 2.6, or should we try to switch to ambari 2.6 to deploy also the older versions?
14:50:36 <shuyingy_> btw, I will share you the performance comparison of vanilla to aws emr next week
14:50:51 <tellesnobrega> cool
14:50:52 <tosky> it may be complicated, but it could also simplify part of the matrix
14:51:17 <tellesnobrega> tosky, not sure, ambari/hdp has always confused me
14:51:28 <shuyingy_> I will update vanilla if I can. otherwise I can invite my colleague to help us.
14:51:31 <tosky> Ambari can deploy few versions
14:51:50 <shuyingy_> ambari/hdp has always confused me too
14:52:06 <shuyingy_> the image name of ambari let me crazy
14:53:02 <tosky> yeah
14:53:29 <tellesnobrega> tosky, what do you think is the best way to go with ambari?
14:53:32 <tosky> from the previous document it seems that Ambari 2.5 was able to deploy all the old versions
14:53:44 <tosky> so it may be the same with Ambari 2.6
14:54:16 <tosky> right now, if I'm not mistaken, we deploy two slightly different versions of ambari for the two version of hdp
14:54:43 <tosky> but if ambari 2.6 can deploy 2.6, 2.4 (and 2.3, if we don't want to deprecated it), maybe it's easier to handle
14:55:03 <tosky> otherwise we would need different codepaths on our side if the various ambari versions differs
14:55:15 <tosky> but if ambari hides the differences between HDP versions, well
14:55:18 <tellesnobrega> yeah, I think we have 2.2.0.0 and 2.2.1.0
14:55:52 <tellesnobrega> our ambari version is very old at this point
14:57:35 <tellesnobrega> so we have ambari and vanilla to update
14:57:36 <tosky> I think we support ambari 2.4 too
14:57:44 <tellesnobrega> yeah
14:58:03 <tellesnobrega> not sure
14:58:04 <shuyingy_> 2.4 and 2.5 both support
14:58:27 <tellesnobrega> on image gen the versions are  2.2.0.0 and 2.2.1.0
14:58:39 <tosky> but again, if HDP 2.6 can be deployed with ambari 2.6 only (ok, also 2.5), we need to update to support that ambari
14:58:59 <tosky> if, on the other side, that ambari can deploy all hdp versions, why not kill the olders ambari?
14:59:07 <tellesnobrega> sounds good, lets take a closer look on that and get back to it
14:59:18 <tellesnobrega> that would be good too
14:59:34 <tellesnobrega> and maybe we should take a class on ambari/hdp
14:59:36 <tellesnobrega> lol
15:00:14 <tellesnobrega> shuyingy_, do you have time to update any plugin?
15:00:17 <shuyingy_> so I will update vanilla
15:00:31 <shuyingy_> you may miss my message
15:00:37 <shuyingy_> :)
15:00:42 <tellesnobrega> great
15:00:54 <tellesnobrega> I will try to check ambari and see if jeremy can work on it
15:00:58 <shuyingy_> at 22:51, I said it :)
15:01:02 <tellesnobrega> otherwise I will work on it was well
15:01:04 <tellesnobrega> yes
15:01:07 <shuyingy_> haha
15:01:08 <tellesnobrega> we have to close it
15:01:11 <tellesnobrega> thanks guys
15:01:15 <shuyingy_> bye
15:01:17 <tellesnobrega> see you next week
15:01:20 <tellesnobrega> bye
15:01:22 <shuyingy_> see you next week
15:01:28 <tosky> o/
15:01:39 <tellesnobrega> #endmeeting