openstackgerrit | OpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements https://review.openstack.org/354322 | 00:25 |
---|---|---|
*** sfelix has joined #openstack-sahara | 01:22 | |
*** sfelix has quit IRC | 01:23 | |
*** zhiyan has quit IRC | 02:55 | |
*** zhiyan has joined #openstack-sahara | 03:03 | |
*** links has joined #openstack-sahara | 03:33 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements https://review.openstack.org/354322 | 04:56 |
*** gokhan has joined #openstack-sahara | 05:17 | |
*** nkrinner_afk is now known as nkrinner | 05:45 | |
*** rcernin has quit IRC | 05:51 | |
*** witlessb has joined #openstack-sahara | 06:15 | |
*** rcernin has joined #openstack-sahara | 06:21 | |
*** pgadiya has joined #openstack-sahara | 06:30 | |
*** pgadiya has quit IRC | 06:31 | |
*** nkrinner has quit IRC | 06:41 | |
*** nkrinner has joined #openstack-sahara | 06:42 | |
*** nkrinner has quit IRC | 06:42 | |
*** nkrinner has joined #openstack-sahara | 06:42 | |
*** nkrinner is now known as nkrinner_afk | 06:49 | |
*** vgridnev has joined #openstack-sahara | 06:58 | |
*** tesseract- has joined #openstack-sahara | 06:59 | |
openstackgerrit | Merged openstack/sahara-dashboard: Updated from global requirements https://review.openstack.org/354323 | 07:03 |
*** nkrinner_afk is now known as nkrinner | 07:30 | |
*** akuznetsov has joined #openstack-sahara | 07:31 | |
*** jamielennox|away has quit IRC | 07:38 | |
*** abehl has joined #openstack-sahara | 07:40 | |
*** abehl has quit IRC | 07:41 | |
*** akuznetsov has quit IRC | 07:43 | |
*** pgadiya has joined #openstack-sahara | 07:44 | |
*** pgadiya has quit IRC | 07:44 | |
*** amoralej|off is now known as amoralej | 07:44 | |
*** jamielennox|away has joined #openstack-sahara | 07:47 | |
*** jamielennox|away is now known as jamielennox | 07:48 | |
openstackgerrit | Merged openstack/python-saharaclient: Updated from global requirements https://review.openstack.org/354316 | 08:05 |
openstackgerrit | Merged openstack/sahara: Updated from global requirements https://review.openstack.org/354322 | 08:10 |
*** sfelix has joined #openstack-sahara | 08:24 | |
*** sfelix has left #openstack-sahara | 08:25 | |
*** anshul has joined #openstack-sahara | 08:43 | |
*** shuyingya has quit IRC | 08:57 | |
*** zemuvier has joined #openstack-sahara | 08:59 | |
openstackgerrit | Michael Ionkin proposed openstack/sahara-dashboard: Designate integration https://review.openstack.org/347411 | 09:41 |
*** tellesnobrega has quit IRC | 09:47 | |
*** Poornima has joined #openstack-sahara | 10:06 | |
*** tosky has joined #openstack-sahara | 10:15 | |
*** tellesnobrega has joined #openstack-sahara | 10:22 | |
*** tellesnobrega has quit IRC | 11:02 | |
*** zhugx has joined #openstack-sahara | 11:10 | |
openstackgerrit | Vitaly Gridnev proposed openstack/sahara: more safe handling verification https://review.openstack.org/354653 | 11:13 |
openstackgerrit | Vitaly Gridnev proposed openstack/sahara: more safe handling verification https://review.openstack.org/354653 | 11:15 |
*** Poornima has quit IRC | 11:57 | |
*** amoralej is now known as amoralej|lunch | 12:23 | |
tosky | vgridnev: about the change to spark pi example on https://review.openstack.org/#/c/351920/ - apart from the failures on vanilla, cdh, ambari, it would be good to see the pass at least for the 4 spark | 12:24 |
vgridnev | tosky, at least 4 spark passed one | 12:24 |
vgridnev | once* | 12:24 |
vgridnev | before recheck | 12:24 |
vgridnev | there where 2 cdh jobs (which are also consume spark jobs) + 3 sparks | 12:25 |
tosky | 3 over 4 in the first recheck (only failing spark 1.3 on mitaka, DONEWITHERROR) | 12:25 |
tosky | ah, right, cdh also use them | 12:26 |
vgridnev | tosky, is it right that with https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/sahara.yaml#L231 we will not run any tempest jobs? | 12:26 |
tosky | but cdh consume the wordcount only, so it should not be affected | 12:26 |
tosky | vgridnev: I'm not a granade expert but I think yes, that should disable tempest | 12:27 |
vgridnev | but it's still running smoke | 12:28 |
vgridnev | brr | 12:29 |
tosky | unless granade overrides it | 12:30 |
tosky | oh | 12:30 |
*** zhugx has quit IRC | 12:30 | |
vgridnev | http://logs.openstack.org/41/352441/5/check/gate-grenade-dsvm-sahara/438e5ab/console.html | 12:32 |
*** chlong|mtg has quit IRC | 12:33 | |
tosky | vgridnev: looking in granade source code, there is a TARGET_RUN_SMOKE variable, but I don't see it enabled or disabled in project-config | 12:35 |
tosky | I mean, by any project there | 12:35 |
vgridnev | tosky, https://review.openstack.org/#/c/354700/ | 12:41 |
tosky | ah, even more generic | 12:44 |
vgridnev | https://github.com/openstack-dev/grenade/blob/16cb2c8b5726b2c908780df27bfe70250bab41fa/grenaderc | 12:44 |
vgridnev | RUN_SMOKE var | 12:45 |
*** shuyingya has joined #openstack-sahara | 12:45 | |
tosky | vgridnev: so do you think that the Spark PI could be merged based on those previous results, or is there something that we can do for the Sahara CI? | 12:48 |
*** anshul has quit IRC | 12:51 | |
*** links has quit IRC | 12:53 | |
openstackgerrit | Vitaly Gridnev proposed openstack/sahara: testcommit https://review.openstack.org/354707 | 12:53 |
*** amoralej|lunch is now known as amoralej | 12:59 | |
*** _elmiko is now known as elmiko | 13:00 | |
tosky | vgridnev: and also, about the jars: the spark/wordcount, as you wrote it's a combination of pieces from spark examples and a class from you, do you think we could simply add the code? | 13:01 |
vgridnev | spark word count had some modification regarding storing results into output directory, I think I will add that code soon | 13:05 |
tosky | so if that is added, we will miss only the source code (and later build procedure) for edp-mapreduce.jar (oozie examples, but we use only one) and hadoop-mapreduce-examples-2.6.0.jar (standard hadoop examples, but we use only one from them) | 13:07 |
vgridnev | About merging python example, I think that we can proceed, there is no issues with launching simple spark examples written on python. There some issues about python jobs that requires access to swift | 13:07 |
vgridnev | edp-mapreduce | 13:08 |
vgridnev | one minute | 13:08 |
tosky | so the spark wordcount should stay java | 13:08 |
tosky | fine by me :) | 13:08 |
* tosky bbl | 13:09 | |
vgridnev | https://github.com/yahoo/oozie/blob/master/examples/src/main/java/org/apache/oozie/example/SampleMapper.java | 13:09 |
vgridnev | tosky, spark word count is in scala | 13:10 |
tosky | yes, I know, I put the link of the oozie edp-mapreduce code | 13:10 |
tosky | on the etherpad | 13:10 |
tosky | isn't it the same? | 13:10 |
tosky | https://github.com/apache/oozie/tree/master/examples/src/main/java/org/apache/oozie/example | 13:10 |
vgridnev | yep, seems it's same | 13:11 |
tosky | the other problematic one is udf.jar, as the original license is not clear (it comes from the examples of a book), but I sent an email to the book author | 13:12 |
vgridnev | can we do some alternative example? | 13:13 |
tosky | that's an other option | 13:14 |
tosky | we already have another one (top-todoers) but a) I didn't check the origin and b) it does not exercise the libs part (like the other do) | 13:15 |
vgridnev | I mean alternative example with lib for sure | 13:18 |
vgridnev | where is current udfs are located? | 13:20 |
*** itisha has joined #openstack-sahara | 13:20 | |
*** tellesnobrega has joined #openstack-sahara | 13:21 | |
*** akuznetsov has joined #openstack-sahara | 13:23 | |
vgridnev | probably we can use something like https://github.com/apache/pig/blob/27b153dbd688d8328e00d2d4bead84f3c879b2ae/src/org/apache/pig/builtin/TRIM.java | 13:24 |
vgridnev | but package that as user defined function | 13:24 |
vgridnev | I think that will be ok | 13:25 |
vgridnev | the code is almost ok | 13:25 |
vgridnev | almost same I mean | 13:25 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: copying oozie.warden to prevent failure https://review.openstack.org/350566 | 13:25 |
*** akuznetsov has quit IRC | 13:28 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: copying oozie.warden to prevent failure https://review.openstack.org/350566 | 13:29 |
*** tellesnobrega has quit IRC | 13:33 | |
tosky | vgridnev: the source code of the current udf is https://github.com/tomwhite/hadoop-book/blob/master/ch16-pig/src/main/java/com/hadoopbook/pig/Trim.java | 13:38 |
tosky | vgridnev: I kept https://etherpad.openstack.org/p/sahara-embedded-jars updated :) | 13:38 |
*** rodrigods has quit IRC | 13:39 | |
*** rodrigods has joined #openstack-sahara | 13:40 | |
*** nkrinner is now known as nkrinner_afk | 13:50 | |
*** gokhan has quit IRC | 13:51 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Fix wrong owner setting for config files https://review.openstack.org/354741 | 14:07 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Fix wrong hue-livy process name and move installation https://review.openstack.org/354748 | 14:20 |
*** tellesnobrega has joined #openstack-sahara | 14:22 | |
*** tellesnobrega has quit IRC | 14:29 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Remove MAPR_USER variable https://review.openstack.org/354753 | 14:36 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Fix wrong instance count in provision events https://review.openstack.org/354763 | 14:55 |
tosky | vgridnev: is that "remove MAPR_USER variable" really a bug? I mean, can't it change in the future? | 15:00 |
vgridnev | I not sure, mapr always uses mapr user | 15:01 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Refactor service home dir owner setting https://review.openstack.org/354771 | 15:02 |
tosky | but uhm, you usually add variables, not remove them | 15:03 |
tosky | anyway, -1 for the lack of description in the commit message, but I'm not core there so it can be overruled :) | 15:03 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Remove ssl config for Hue https://review.openstack.org/354776 | 15:06 |
vgridnev | tosky, better commit messages are always good | 15:07 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Remove support for Spark standalone https://review.openstack.org/354779 | 15:11 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Replace mfs.exchange with g.copy_file where it is possible https://review.openstack.org/354786 | 15:18 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Rename all ClusterContext variables to 'cluster_context' https://review.openstack.org/354789 | 15:22 |
*** pcaruana has quit IRC | 15:23 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Add custom health check for MapR plugin https://review.openstack.org/354792 | 15:30 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Refactor service home dir owner setting https://review.openstack.org/354771 | 15:33 |
tosky | vgridnev: uhm, do you think that it really needed to open a bug for each of one of those mapr changes? | 15:36 |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Replace mfs.exchange with g.copy_file where it is possible https://review.openstack.org/354786 | 15:39 |
vgridnev | I'm not against that | 15:41 |
tosky | I don't know, for some changes it seems overkill | 15:43 |
tosky | it seems _to me_ | 15:43 |
*** rcernin has quit IRC | 15:47 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Add Sentry service v1.6 to MapR plugin https://review.openstack.org/354808 | 15:48 |
openstackgerrit | Merged openstack/sahara-tests: embedded jars: replace spark-example.jar https://review.openstack.org/351920 | 15:51 |
*** vgridnev has quit IRC | 16:00 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Add new version pack for services https://review.openstack.org/354820 | 16:12 |
*** shuyingya has quit IRC | 16:15 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Add Sentry service v1.6 to MapR plugin https://review.openstack.org/354808 | 16:18 |
*** tmckay has joined #openstack-sahara | 16:21 | |
openstackgerrit | Grigoriy Rozhkov proposed openstack/sahara: Add MapR core 5.2 https://review.openstack.org/354829 | 16:28 |
*** tesseract- has quit IRC | 16:35 | |
*** tellesnobrega has joined #openstack-sahara | 16:49 | |
*** amoralej is now known as amoralej|off | 17:27 | |
*** tellesnobrega has quit IRC | 18:25 | |
*** agireud has quit IRC | 18:47 | |
*** agireud has joined #openstack-sahara | 18:55 | |
*** tellesnobrega has joined #openstack-sahara | 18:57 | |
*** tosky has quit IRC | 18:58 | |
openstackgerrit | Andreas Jaeger proposed openstack/puppet-sahara: Move other-requirements.txt to bindep.txt https://review.openstack.org/355004 | 19:12 |
*** tellesnobrega has quit IRC | 19:12 | |
openstackgerrit | Andreas Jaeger proposed openstack/sahara-dashboard: Move other-requirements.txt to bindep.txt https://review.openstack.org/355022 | 19:15 |
*** tellesnobrega has joined #openstack-sahara | 20:17 | |
*** agireud has quit IRC | 20:48 | |
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org is restarting for a scheduled upgrade, but should return to service momentarily: http://lists.openstack.org/pipermail/openstack-dev/2016-August/101394.html | 20:50 | |
*** tmckay has quit IRC | 20:51 | |
*** agireud has joined #openstack-sahara | 20:55 | |
-openstackstatus- NOTICE: The Mediawiki service at wiki.openstack.org will be offline from 21:00 UTC until approximately 23:00 UTC for a planned upgrade http://lists.openstack.org/pipermail/openstack-dev/2016-August/101395.html | 20:59 | |
*** ChanServ changes topic to "The Mediawiki service at wiki.openstack.org will be offline from 21:00 UTC until approximately 23:00 UTC for a planned upgrade http://lists.openstack.org/pipermail/openstack-dev/2016-August/101395.html" | 20:59 | |
*** elmiko is now known as _elmiko | 22:12 | |
*** witlessb has quit IRC | 22:15 | |
-openstackstatus- NOTICE: ok https://wiki.openstack.org/ is now running Mediawiki 1.27.0; please let us know in #openstack-infra if anything seems wrong | 23:01 | |
*** ChanServ changes topic to "ok https://wiki.openstack.org/ is now running Mediawiki 1.27.0; please let us know in #openstack-infra if anything seems wrong" | 23:01 | |
*** ChanServ changes topic to "OpenStack Sahara // IRC Meetings - http://eavesdrop.openstack.org/#OpenStack_Data_Processing_(Sahara)_Team_Meeting" | 23:07 | |
-openstackstatus- NOTICE: https://wiki.openstack.org/ is now running Mediawiki 1.27.0; please let us know in #openstack-infra if anything seems wrong | 23:07 | |
*** agireud has quit IRC | 23:44 | |
*** agireud has joined #openstack-sahara | 23:53 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!