| openstackgerrit | Andrew Lazarev proposed openstack/sahara: Fixed error on attempt to delete job execution several times https://review.openstack.org/136932 | 00:14 |
|---|---|---|
| *** tnovacik has quit IRC | 00:17 | |
| *** tellesnobrega_ has joined #openstack-sahara | 00:51 | |
| *** tellesnobrega_ has quit IRC | 01:01 | |
| *** tellesnobrega_ has joined #openstack-sahara | 01:04 | |
| openstackgerrit | Ken Chen proposed openstack/sahara: Add CDH plugin documents https://review.openstack.org/136697 | 01:06 |
| *** tellesnobrega_ has quit IRC | 01:28 | |
| *** tellesnobrega_ has joined #openstack-sahara | 01:28 | |
| *** Samos123 has joined #openstack-sahara | 01:38 | |
| *** Poornima has joined #openstack-sahara | 01:40 | |
| *** tellesnobrega_ has quit IRC | 01:45 | |
| *** tmckay has quit IRC | 02:31 | |
| *** chandankumar has joined #openstack-sahara | 02:32 | |
| *** jodah has quit IRC | 02:44 | |
| *** tellesnobrega_ has joined #openstack-sahara | 02:49 | |
| *** chandankumar has quit IRC | 03:02 | |
| *** tmckay has joined #openstack-sahara | 03:02 | |
| *** chandankumar has joined #openstack-sahara | 03:03 | |
| *** Poornima has quit IRC | 03:05 | |
| *** tmckay has left #openstack-sahara | 03:18 | |
| *** chandankumar has quit IRC | 03:28 | |
| *** ViswaV has quit IRC | 03:53 | |
| *** ViswaV has joined #openstack-sahara | 04:14 | |
| *** ViswaV_ has joined #openstack-sahara | 04:16 | |
| *** ViswaV has quit IRC | 04:18 | |
| *** jodah has joined #openstack-sahara | 04:27 | |
| openstackgerrit | Kazuki OIKAWA proposed openstack/sahara: Added ability to access a swift from vanilla-1 hive https://review.openstack.org/133157 | 04:28 |
| *** ViswaV_ has quit IRC | 04:29 | |
| *** ViswaV has joined #openstack-sahara | 04:34 | |
| *** ViswaV has quit IRC | 04:54 | |
| *** chandankumar has joined #openstack-sahara | 05:25 | |
| *** tnovacik has joined #openstack-sahara | 05:45 | |
| *** k4n0 has joined #openstack-sahara | 06:24 | |
| *** Longgeek has joined #openstack-sahara | 06:42 | |
| *** Longgeek has quit IRC | 06:42 | |
| *** Longgeek has joined #openstack-sahara | 06:42 | |
| *** Longgeek has quit IRC | 07:01 | |
| *** Longgeek has joined #openstack-sahara | 07:02 | |
| *** Longgeek has quit IRC | 07:06 | |
| *** Longgeek has joined #openstack-sahara | 07:14 | |
| openstackgerrit | Kazuki OIKAWA proposed openstack/sahara: Added hive support for vanilla2 https://review.openstack.org/133186 | 07:47 |
| *** stannie has joined #openstack-sahara | 08:31 | |
| *** tnovacik has quit IRC | 08:42 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: And new endpoint and periodic task with events ops https://review.openstack.org/134504 | 09:03 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: And new endpoint and periodic task with events ops https://review.openstack.org/134504 | 09:05 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add provision steps for direct engine https://review.openstack.org/134490 | 09:09 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara-image-elements: Image Elements for MapR https://review.openstack.org/126148 | 09:17 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add provision steps to Vanilla 1 plugin https://review.openstack.org/136748 | 10:01 |
| openstackgerrit | Sergey Lukjanov proposed openstack/sahara-specs: Add missed blueprint linl to auth policy https://review.openstack.org/137006 | 10:16 |
| openstackgerrit | Sergey Lukjanov proposed openstack/sahara-specs: Add missed image elements bp link to mapr spec https://review.openstack.org/137013 | 10:40 |
| openstackgerrit | Merged openstack/sahara-specs: Add missed blueprint linl to auth policy https://review.openstack.org/137006 | 10:44 |
| openstackgerrit | Merged openstack/sahara-specs: Add missed image elements bp link to mapr spec https://review.openstack.org/137013 | 10:45 |
| *** tnovacik has joined #openstack-sahara | 11:01 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add provision steps to Vanilla 2 Plugin https://review.openstack.org/137018 | 11:05 |
| *** tosky has joined #openstack-sahara | 11:09 | |
| openstackgerrit | Abbass Marouni proposed openstack/sahara: Enable HDFS NameNode High Availability with HDP 2.0.6 plugin https://review.openstack.org/132051 | 11:14 |
| openstackgerrit | Abbass Marouni proposed openstack/sahara: Enable HDFS NameNode High Availability with HDP 2.0.6 plugin https://review.openstack.org/132051 | 11:16 |
| *** k4n0 has quit IRC | 11:20 | |
| openstackgerrit | Kazuki OIKAWA proposed openstack/sahara-image-elements: Add openstack-hadoop.jar build/deploy script and vanilla2 hive support https://review.openstack.org/137027 | 11:50 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: And new endpoint and periodic task with events ops https://review.openstack.org/134504 | 11:58 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara-specs: Adding unique identificators to Exceptions https://review.openstack.org/136360 | 12:10 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Fix broken unit tests https://review.openstack.org/137035 | 12:17 |
| *** tellesnobrega_ has quit IRC | 12:25 | |
| *** Longgeek_ has joined #openstack-sahara | 12:44 | |
| *** Longgeek has quit IRC | 12:44 | |
| *** sreshetnyak has joined #openstack-sahara | 12:56 | |
| *** sreshetnyak has quit IRC | 12:56 | |
| *** sreshetnyak has joined #openstack-sahara | 12:56 | |
| openstackgerrit | Merged openstack/sahara-specs: CDH HBase Support https://review.openstack.org/132159 | 13:07 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding uuids to exceptions https://review.openstack.org/136826 | 13:07 |
| *** stannie has quit IRC | 13:12 | |
| openstackgerrit | Merged openstack/sahara-specs: Specification Repository Backlog Refactor https://review.openstack.org/135313 | 13:20 |
| *** chandankumar has quit IRC | 13:21 | |
| *** Longgeek has joined #openstack-sahara | 13:24 | |
| *** Longgeek_ has quit IRC | 13:27 | |
| *** Networkn3rd has quit IRC | 13:41 | |
| *** IvanBerezovskiy has quit IRC | 13:45 | |
| *** IvanBerezovskiy has joined #openstack-sahara | 13:46 | |
| *** _crobertsrh is now known as crobertsrh | 13:55 | |
| *** egafford has joined #openstack-sahara | 14:00 | |
| openstackgerrit | Abbass Marouni proposed openstack/sahara: Enable HDFS NameNode High Availability with HDP 2.0.6 plugin https://review.openstack.org/132051 | 14:15 |
| openstackgerrit | Chad Roberts proposed openstack/sahara-specs: Updating query filtering support for Sahara https://review.openstack.org/136865 | 14:16 |
| *** tmckay has joined #openstack-sahara | 14:28 | |
| *** yanndegat has joined #openstack-sahara | 14:31 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add provision steps for direct engine https://review.openstack.org/134490 | 14:48 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add provision steps to Vanilla 1 plugin https://review.openstack.org/136748 | 14:56 |
| *** tosky has quit IRC | 14:56 | |
| *** tosky has joined #openstack-sahara | 14:57 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add provision steps to Vanilla 2 Plugin https://review.openstack.org/137018 | 14:59 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding uuids to exceptions https://review.openstack.org/136826 | 15:00 |
| tellesnobrega | hi, i just ran my first storm job on sahara storm cluster :) | 15:15 |
| elmiko | yay! | 15:16 |
| tellesnobrega | will be submtting patches soon | 15:16 |
| elmiko | nice, glad to hear =) | 15:16 |
| *** samuelms has joined #openstack-sahara | 15:19 | |
| tmckay | tellesnobrega, great | 15:21 |
| *** egafford has quit IRC | 15:30 | |
| openstackgerrit | Andrew Lazarev proposed openstack/sahara: Fixed error on attempt to delete job execution several times https://review.openstack.org/136932 | 15:36 |
| openstackgerrit | Andrew Lazarev proposed openstack/sahara: Fixed error on attempt to delete job execution several times https://review.openstack.org/136932 | 15:37 |
| *** Networkn3rd has joined #openstack-sahara | 15:46 | |
| *** skolekonov has quit IRC | 15:53 | |
| openstackgerrit | Merged openstack/sahara-specs: [EDP] Improve Java type compatibility https://review.openstack.org/110576 | 16:01 |
| *** egafford has joined #openstack-sahara | 16:04 | |
| openstackgerrit | Julien Danjou proposed openstack/sahara: Remove py26 from tox https://review.openstack.org/137122 | 16:08 |
| tellesnobrega | thanks. I just need to run some more tests, see it working more | 16:11 |
| *** chandankumar has joined #openstack-sahara | 16:12 | |
| SergeyLukjanov | tellesnobrega, yay! | 16:12 |
| tellesnobrega | SergeyLukjanov, i saw that you are involved with blazar as well, :q | 16:13 |
| tellesnobrega | this last :q was a vim command that went to a different window | 16:13 |
| tellesnobrega | i'm very interested in that | 16:14 |
| tellesnobrega | SergeyLukjanov, how about the tempest, devstack problem? any ideas? | 16:15 |
| SergeyLukjanov | tellesnobrega, unfortunately I'm not looking on it for about 5 month because of lack of free time :( | 16:15 |
| tellesnobrega | i see | 16:15 |
| tellesnobrega | it has some interesting concepts, the idea of reserving resource is very useful to me | 16:16 |
| SergeyLukjanov | tellesnobrega, actually, I've been involved in it because my wife (Dina Belova) was working on it and than when she switched to Ceilometer, we both eventually stopped working on it | 16:16 |
| SergeyLukjanov | tellesnobrega, yeah, I've been actively participating in architecture and concept design phase for blazar, I really like this idea too | 16:17 |
| *** tnovacik has quit IRC | 16:17 | |
| SergeyLukjanov | tellesnobrega, but unfortunately it wasn't blessed by tc and there are bunch of issues in reserving non-compute resources | 16:17 |
| tellesnobrega | hum, what im working on (masters) is to provision opportunistic instances with some SLOs guarantees to enhance data processing | 16:18 |
| SergeyLukjanov | tellesnobrega, wow! | 16:19 |
| tellesnobrega | so, reserving resource would be a great place to get resources that cant be used for spawning an instance for the user, but we could use as long as we let it go when needed | 16:19 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Fix broken unit tests https://review.openstack.org/137035 | 16:28 |
| *** IvanBerezovskiy has left #openstack-sahara | 16:32 | |
| *** tnovacik has joined #openstack-sahara | 16:33 | |
| *** chandankumar has quit IRC | 16:51 | |
| *** tnovacik has quit IRC | 16:55 | |
| tmckay | SergeyLukjanov, alazarev, I approved the spec for the Java EDP job type and commented on the review https://review.openstack.org/#/c/115884/ | 16:56 |
| tmckay | If we need to revise the spec, we can. | 16:57 |
| tmckay | when you have a chance, let me know what you think of the alternative of using edp.java.adapt_for_oozie (or something) as a flag instead of a new job type | 16:57 |
| tmckay | I think it answers alazarev concern | 16:58 |
| *** Longgeek has quit IRC | 16:58 | |
| *** tnovacik has joined #openstack-sahara | 17:06 | |
| tmckay | You know, it occurs to me that a plain Hadoop EDP job engine could be built using ssh and Linux commands, just like the Spark standalone engine. It might be interesting to try. It could be an alternative to Oozie (or maybe even a replacement). | 17:14 |
| tmckay | we would just pass things directly to the "hadoop" command. As long as we can implement run, cancel, status, we have something that works | 17:15 |
| tmckay | it might help remove some of the oozie-centric things we have in Sahara | 17:15 |
| tmckay | the Spark engine would be a model | 17:16 |
| *** afaranha has joined #openstack-sahara | 17:28 | |
| *** Uian has joined #openstack-sahara | 17:31 | |
| *** ViswaV has joined #openstack-sahara | 17:38 | |
| *** ViswaV_ has joined #openstack-sahara | 17:44 | |
| *** ViswaV has quit IRC | 17:44 | |
| *** russellb has quit IRC | 18:11 | |
| *** russellb has joined #openstack-sahara | 18:12 | |
| elmiko | tmckay: doesn't oozie provide us some status type endpoint? | 18:13 |
| *** ViswaV_ has quit IRC | 18:15 | |
| openstackgerrit | Emilien Macchi proposed stackforge/puppet-sahara: Refactorise Keystone resources management https://review.openstack.org/136394 | 18:20 |
| *** _mattf is now known as mattf | 18:23 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Fix broken unit tests https://review.openstack.org/137035 | 18:24 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Fix broken unit tests https://review.openstack.org/137035 | 18:27 |
| tmckay | elmiko, yes, oozie does through the client, but I was thinking about a pure hadoop command line EDP engine. It might be desirable in some cases. | 18:36 |
| elmiko | tmckay: ok, i thought so. it would just mean that the pure hadoop wouldn't have as nice status updating? | 18:36 |
| tmckay | elmiko, for instance, the whole issue with the Java wrapper to iron out issues between jobs on another big data system and a job in oozie from Sahara | 18:37 |
| tmckay | yes | 18:37 |
| tmckay | It would be basic stuff, like Spark. Not sure what status api the hadoop CLI has, but worst case it could use OS level commands to check on the process id, etc | 18:37 |
| elmiko | yea | 18:38 |
| elmiko | i just wanted to make sure i understood all the parts in motion | 18:38 |
| *** ViswaV has joined #openstack-sahara | 18:38 | |
| tmckay | and the launcher I used for Spark could easily wrap a hadoop command | 18:38 |
| tmckay | ack | 18:38 |
| tmckay | it has a certain "less is more" appeal | 18:38 |
| tmckay | the best part is, it would just be another plugin in the toolbox | 18:38 |
| elmiko | yea | 18:39 |
| elmiko | i'd be curious to hear if users would want that type of plugin, although i agree it would be easy to package. | 18:41 |
| tmckay | elmiko, can you review https://review.openstack.org/#/c/137035/ | 18:55 |
| tmckay | I think that is part of what is breaking CI | 18:55 |
| tmckay | nice solution | 18:56 |
| * elmiko takes a look | 18:56 | |
| SergeyLukjanov | tmckay, sounds fair | 19:01 |
| alazarev | tmckay, regarding https://review.openstack.org/#/c/115884/ I like your suggestions | 19:04 |
| tmckay | alazarev, okay, thanks | 19:05 |
| *** tnovacik has quit IRC | 19:05 | |
| tmckay | elmiko, okay, great, two +2 so as soon as we pass CI we can merge. This should fix up the py27 tests for Jenkins | 19:07 |
| elmiko | tmckay: would be nice to have a non-rh +2, but i will defer to your seniority ;) | 19:12 |
| tmckay | elmiko, heh, I think this one is critical. It's mucking everything up | 19:12 |
| elmiko | yea | 19:12 |
| tmckay | besides, we have an implicit +2 from sreshetnyak ;-) | 19:13 |
| elmiko | lol, nice =) | 19:13 |
| SergeyLukjanov | tmckay, elmiko, +2 from me too | 19:25 |
| tmckay | yay! | 19:25 |
| elmiko | SergeyLukjanov: do you know who generates specs.openstack.org? | 19:26 |
| SergeyLukjanov | elmiko, jenkins ;) | 19:33 |
| *** samuelms has quit IRC | 19:33 | |
| elmiko | lol, of course... | 19:33 |
| elmiko | SergeyLukjanov: here's the thing, if i change the layout of our specs repo i'm concerned it will break specs.oo.o | 19:34 |
| elmiko | i guess i just need to propose a change against the source for that page, just not sure where it lives | 19:35 |
| SergeyLukjanov | elmiko, do you mean index page on specs.o.o? | 19:39 |
| SergeyLukjanov | elmiko, it's a static page | 19:39 |
| elmiko | SergeyLukjanov: more concerned about this page, http://specs.openstack.org/openstack/sahara-specs/ | 19:39 |
| SergeyLukjanov | it's just an index.html generated by sahara-specs repo's docs tox env | 19:39 |
| elmiko | ahh ok, cool. thanks! | 19:40 |
| *** tosky has quit IRC | 19:40 | |
| tmckay | okay, back later tonight or tomorrow (even though I'm officially on PTO). I have some chores to do :) | 19:41 |
| SergeyLukjanov | elmiko, but if you're moving specs to new urls, you need to add redirects using the redirects file | 19:43 |
| SergeyLukjanov | elmiko, example: specs/juno/redirects | 19:43 |
| elmiko | SergeyLukjanov: i'm just going to move them out of the implemented directory. i think if i just fix the index it will work. | 19:44 |
| *** tmckay has quit IRC | 19:44 | |
| SergeyLukjanov | elmiko, nope, you'll need to add redirects to avoid broken links | 19:44 |
| elmiko | SergeyLukjanov: ack | 19:46 |
| *** tnovacik has joined #openstack-sahara | 19:47 | |
| elmiko | SergeyLukjanov: i'm brushing up the language in the spec README, is it the case for Sahara that specs not accepted by the second milestone must get an exception for inclusion? | 20:04 |
| openstackgerrit | Shilla Saebi proposed openstack/sahara: changes to quickstart https://review.openstack.org/136231 | 20:22 |
| *** tnovacik has quit IRC | 20:27 | |
| openstackgerrit | Uian proposed openstack/sahara-image-elements: Making variables in uppercase. Rename variable to uppercase for consistency. Closes-bug: #1373071 https://review.openstack.org/137199 | 20:38 |
| openstackgerrit | Andrew Lazarev proposed openstack/sahara: Fixed auto security group for nova network https://review.openstack.org/137201 | 20:40 |
| SergeyLukjanov | elmiko, I think we should chat about it on the next meeting | 20:42 |
| SergeyLukjanov | elmiko, but it's a very good practise | 20:42 |
| *** Uian has quit IRC | 20:49 | |
| elmiko | SergeyLukjanov: ok, cool. i won't put anything in the README about it. | 20:51 |
| tellesnobrega | :q | 20:57 |
| tellesnobrega | sorry | 20:57 |
| tellesnobrega | SergeyLukjanov, any solution for the devstack-tempest dependency? | 20:57 |
| *** ViswaV has quit IRC | 20:59 | |
| openstackgerrit | Michael McCune proposed openstack/sahara-specs: Adding backlog to specs repo https://review.openstack.org/137211 | 21:00 |
| SergeyLukjanov | tellesnobrega, not yet | 21:03 |
| openstackgerrit | Michael McCune proposed openstack/sahara-specs: Specification Repository Backlog Refactor https://review.openstack.org/137211 | 21:03 |
| *** ViswaV has joined #openstack-sahara | 21:13 | |
| *** egafford has quit IRC | 21:20 | |
| *** crobertsrh is now known as _crobertsrh | 21:29 | |
| SergeyLukjanov | tellesnobrega, I think I've find the way to migrate | 21:39 |
| SergeyLukjanov | tellesnobrega, are you if I'll drive the devstack change? | 21:39 |
| *** Networkn3rd has quit IRC | 21:40 | |
| *** ViswaV has quit IRC | 22:07 | |
| *** ViswaV_ has joined #openstack-sahara | 22:16 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!