| *** bandarji has quit IRC | 00:49 | |
| *** alazarev has joined #openstack-sahara | 01:08 | |
| *** ViswaV_ has joined #openstack-sahara | 01:35 | |
| *** alazarev has quit IRC | 01:37 | |
| *** ViswaV has quit IRC | 01:38 | |
| *** bradd1 has joined #openstack-sahara | 01:59 | |
| *** zhiyan_ is now known as zhiyan | 02:44 | |
| *** bradd1 has quit IRC | 03:13 | |
| *** sunrenjie6 has joined #openstack-sahara | 03:33 | |
| *** sunrenjie6 has quit IRC | 03:39 | |
| *** sunrenjie6 has joined #openstack-sahara | 03:40 | |
| *** sunrenjie6 has quit IRC | 03:44 | |
| *** rhodgin has quit IRC | 04:38 | |
| openstackgerrit | Alexander Ignatov proposed a change to openstack/sahara: Fixed H405 pep8 style check https://review.openstack.org/100092 | 05:02 |
|---|---|---|
| *** alazarev has joined #openstack-sahara | 05:25 | |
| *** akuznetsov has joined #openstack-sahara | 05:43 | |
| *** alazarev has quit IRC | 05:45 | |
| *** alazarev has joined #openstack-sahara | 05:52 | |
| *** alazarev has quit IRC | 06:12 | |
| *** openstackgerrit has quit IRC | 06:19 | |
| *** openstackgerrit has joined #openstack-sahara | 06:23 | |
| *** alazarev has joined #openstack-sahara | 06:33 | |
| *** miqui has quit IRC | 06:36 | |
| *** alazarev has quit IRC | 06:57 | |
| *** sunrenjie6 has joined #openstack-sahara | 07:00 | |
| *** sunrenjie6 has quit IRC | 07:01 | |
| *** sunrenjie6 has joined #openstack-sahara | 07:01 | |
| *** sunrenjie6 has quit IRC | 07:17 | |
| *** sunrenjie6 has joined #openstack-sahara | 07:18 | |
| *** tnovacik|leaving has joined #openstack-sahara | 07:18 | |
| *** sunrenjie6 has quit IRC | 07:22 | |
| *** julienvey has joined #openstack-sahara | 07:25 | |
| *** tnovacik|leaving is now known as tnovacik | 07:40 | |
| openstackgerrit | A change was merged to openstack/sahara: Handle remote driver not loaded situation https://review.openstack.org/100184 | 07:57 |
| openstackgerrit | A change was merged to openstack/sahara: Fixed E265 pep8 https://review.openstack.org/100091 | 07:57 |
| *** jpich has joined #openstack-sahara | 07:57 | |
| *** witlessb has joined #openstack-sahara | 08:12 | |
| *** julienvey has quit IRC | 08:18 | |
| *** julienvey has joined #openstack-sahara | 08:18 | |
| *** witlessb has quit IRC | 08:24 | |
| openstackgerrit | Denis Egorenko proposed a change to openstack/sahara: Fix Sahara CI links https://review.openstack.org/100566 | 08:24 |
| *** witlessb has joined #openstack-sahara | 08:28 | |
| *** sunrenjie6 has joined #openstack-sahara | 08:29 | |
| *** IvanBerezovskiy has joined #openstack-sahara | 08:41 | |
| *** julienve_ has joined #openstack-sahara | 08:45 | |
| *** julienvey has quit IRC | 08:49 | |
| openstackgerrit | Daniele Venzano proposed a change to openstack/sahara: Implement scaling for Spark clusters https://review.openstack.org/99295 | 09:23 |
| *** sunrenjie6 has quit IRC | 09:35 | |
| *** sunrenjie6 has joined #openstack-sahara | 09:35 | |
| *** julienve_ has quit IRC | 09:38 | |
| *** julienvey has joined #openstack-sahara | 09:39 | |
| *** sunrenjie6 has quit IRC | 09:39 | |
| *** zhiyan is now known as zhiyan_ | 09:51 | |
| *** tosky has joined #openstack-sahara | 09:55 | |
| openstackgerrit | Denis Egorenko proposed a change to stackforge/sahara-ci-config: Add ZMQ parameter to gate-savanna-pep8-trunk https://review.openstack.org/100845 | 10:30 |
| *** jpich has quit IRC | 10:51 | |
| *** jpich has joined #openstack-sahara | 11:04 | |
| *** bogdando has quit IRC | 11:16 | |
| *** bogdando has joined #openstack-sahara | 11:26 | |
| *** jpich has quit IRC | 11:29 | |
| openstackgerrit | Dmitry Mescheryakov proposed a change to openstack/sahara: [test commit] my test commit https://review.openstack.org/100862 | 11:34 |
| openstackgerrit | Dmitry Mescheryakov proposed a change to openstack/sahara: [test commit] my test commit https://review.openstack.org/100862 | 11:35 |
| *** jpich has joined #openstack-sahara | 11:42 | |
| *** jpich has quit IRC | 11:49 | |
| *** jpich has joined #openstack-sahara | 12:02 | |
| *** bogdando has quit IRC | 12:02 | |
| *** bogdando has joined #openstack-sahara | 12:03 | |
| *** jpich_ has joined #openstack-sahara | 12:04 | |
| *** jpich has quit IRC | 12:06 | |
| *** bogdando has quit IRC | 12:09 | |
| *** bogdando has joined #openstack-sahara | 12:13 | |
| *** _tmckay has quit IRC | 12:15 | |
| *** jpich_ is now known as jpich | 12:16 | |
| *** ylobankov has joined #openstack-sahara | 12:26 | |
| *** ylobankov has joined #openstack-sahara | 12:26 | |
| *** tellesnobrega has quit IRC | 12:56 | |
| *** _elmiko is now known as elmiko | 12:58 | |
| *** tellesnobrega has joined #openstack-sahara | 13:02 | |
| *** _crobertsrh is now known as crobertsrh | 13:12 | |
| *** tmckay has joined #openstack-sahara | 13:17 | |
| openstackgerrit | Denis Egorenko proposed a change to openstack/sahara: Fix Sahara CI links https://review.openstack.org/100566 | 13:27 |
| openstackgerrit | Denis Egorenko proposed a change to openstack/sahara: Fix Sahara CI links https://review.openstack.org/100566 | 13:27 |
| openstackgerrit | A change was merged to openstack/python-saharaclient: Updated from global requirements https://review.openstack.org/100580 | 13:28 |
| tellesnobrega | tmckay: I just read your spec and blueprint and looked into the code | 13:30 |
| tellesnobrega | looks good, just small things to change | 13:30 |
| tellesnobrega | but looks cool | 13:30 |
| *** _mattf is now known as mattf | 13:30 | |
| tmckay | tellesnobrega, thanks. The next big thing to decide is how we map job executions to engines. Kind of fuzzy. For example: | 13:37 |
| tmckay | I have heard that it is possible to run mapreduce jobs on spark. It is also possible to deploy Oozie on a cluster running spark. | 13:37 |
| tmckay | So, when we submit a Hive job, let's say, how do we decide how to run it? Job type is not enough. We need to know node processes at least, I think, and maybe how spark is installed and configured. | 13:38 |
| openstackgerrit | A change was merged to openstack/sahara: Updated from global requirements https://review.openstack.org/100581 | 13:39 |
| tmckay | aignatov, SergeyLukjanov, ^^ I should add this discussion to the etherpad. Not sure what to do here. | 13:40 |
| tellesnobrega | i see | 13:40 |
| tmckay | I have an idea to how to ask the plugin, but it might be too much ... I'll post it on the etherpad | 13:41 |
| tellesnobrega | ok | 13:44 |
| *** miqui has joined #openstack-sahara | 13:46 | |
| tellesnobrega | so, for storm, the idea is to implement a new plugin for it, create a engine that will manage its jobs | 13:47 |
| *** bradd1 has joined #openstack-sahara | 13:49 | |
| *** sballe has joined #openstack-sahara | 13:54 | |
| tmckay | tellesnobrega, yes. | 13:58 |
| tmckay | tellesnobrega, as with spark, it may be possible to wrap storm jobs in Oozie, and use that engine, but it may be more trouble than it's worth. That is our current thought for spark | 13:59 |
| tellesnobrega | ok. I will keep that in mind as well | 14:00 |
| openstackgerrit | A change was merged to openstack/sahara: Sync up oslo log module https://review.openstack.org/100015 | 14:03 |
| openstackgerrit | A change was merged to openstack/sahara: Migrated integration tests to testtools https://review.openstack.org/96981 | 14:04 |
| openstackgerrit | A change was merged to openstack/sahara: Make deleting transient clusters safe https://review.openstack.org/97167 | 14:04 |
| *** rhodgin has joined #openstack-sahara | 14:21 | |
| openstackgerrit | A change was merged to openstack/sahara: Fix docs for configuring authentication https://review.openstack.org/100262 | 14:23 |
| *** bradd1 has quit IRC | 14:32 | |
| aignatov | tmckay: great, I’ve looked at your patch, good start on this activity :) | 14:32 |
| tmckay | aignatov, thanks! I need your brilliance on the job engine selection problem. I'm adding to the pad now ... | 14:32 |
| tmckay | aignatov, we can do simple things, but I want to get it right so it is extensible -- so maybe more complexity is needed | 14:33 |
| aignatov | yes, I thought before that each plugin may have it’s own edp engine (and even more than one engine) | 14:34 |
| aignatov | and it’s not dependent on job types | 14:34 |
| tmckay | aignatov, I have an idea to ask the plugin to return a job engine | 14:40 |
| tmckay | with some defaults, for instance we can provide a default Oozie engine since we already have it, and a default spark command line engine (which I hope we will have :) ) or a default Ooyala engine | 14:41 |
| tmckay | aignatov, I was thinking of passing a dictionary of constructor methods for the defaults, so the plugin could just allocate and return a default object or make its own | 14:41 |
| tmckay | aignatov, current vanilla and vanilla 2 would just return the default Oozie engine. Spark would return the spark one. HDP, which looks like it might support both (at the same time, or one or the other?) could decide | 14:42 |
| tmckay | aignatov, and if someone wants to be fancy they can derive their own JobEngine and return it | 14:43 |
| tmckay | aignatov, I think the plugin needs to look at the cluster too and examine what node processes are running | 14:43 |
| tmckay | okay, on the phone, have to go for now | 14:43 |
| openstackgerrit | Dmitry Mescheryakov proposed a change to openstack/python-saharaclient: Make Cluster Template accept default_image_id field https://review.openstack.org/100933 | 14:59 |
| *** bradd1 has joined #openstack-sahara | 15:08 | |
| aignatov | tmckay: yes, about plugin needs, recently sreshetnyak added validate_edp method to plugin SPI | 15:14 |
| aignatov | it could check all node processes for edp job executions | 15:14 |
| aignatov | moreover it could check all needed stuff to decide if it can run job with provided engine | 15:15 |
| *** tnovacik has quit IRC | 15:19 | |
| *** IvanBerezovskiy has left #openstack-sahara | 15:21 | |
| *** jpich has quit IRC | 15:22 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/sahara: [DO NOT MERGE] Decrease samples for PI job in IT https://review.openstack.org/100944 | 15:36 |
| *** jpich has joined #openstack-sahara | 15:39 | |
| *** ViswaV has joined #openstack-sahara | 15:45 | |
| *** ViswaV_ has quit IRC | 15:47 | |
| *** sballe_ has joined #openstack-sahara | 15:53 | |
| *** bandarji has joined #openstack-sahara | 15:55 | |
| *** jpich has quit IRC | 15:55 | |
| *** sballe has quit IRC | 15:56 | |
| *** bradd1 has quit IRC | 15:57 | |
| *** bradd1 has joined #openstack-sahara | 16:03 | |
| *** ViswaV has quit IRC | 16:07 | |
| *** ViswaV has joined #openstack-sahara | 16:11 | |
| tmckay | aignatov, ping | 16:28 |
| aignatov | pong | 16:28 |
| aignatov | tmckay: pong :) | 16:28 |
| tmckay | aignatov, I updated the pad, starting at line 55 "Determing a JobEngine for a Job". I particularly like option 3. It would be easy to add for existing plugins. | 16:29 |
| tmckay | https://etherpad.openstack.org/p/sahara_spark_edp | 16:29 |
| tmckay | Can you take a look when you have a minute? | 16:29 |
| tmckay | I think I'll try an experiment, maybe post it as a dependent CR | 16:30 |
| tmckay | SergeyLukanov, too. ^^ I don't see alazarev around. | 16:30 |
| tmckay | dmitryme, maybe you are interested in this too ^^ and venza, but I think venza is on a different schedule :) | 16:31 |
| * tmckay gets some lunch | 16:31 | |
| tmckay | tellesnobrega ^^ | 16:31 |
| tellesnobrega | thanks | 16:32 |
| aignatov | tmckay: ack | 16:32 |
| aignatov | I’ll have a look | 16:32 |
| aignatov | but later, time to go home :) | 16:33 |
| tmckay | gotcha, thanks. I'll put up a sample CR, I think it could work very nicely | 16:34 |
| *** mattf is now known as _mattf | 16:40 | |
| dmitryme | tmckay: yep, I will take a look | 16:49 |
| *** sballe__ has joined #openstack-sahara | 17:19 | |
| *** sballe_ has quit IRC | 17:23 | |
| *** themistymay has joined #openstack-sahara | 17:24 | |
| *** bradd1 has quit IRC | 17:26 | |
| *** bradd1 has joined #openstack-sahara | 17:29 | |
| SergeyLukjanov | tmckay, alazarev is on vacation till Monday | 17:29 |
| tmckay | okay, thanks | 17:30 |
| SergeyLukjanov | tmckay, I'll be traveling tomorrow the whole day (for visa) | 17:31 |
| SergeyLukjanov | tmckay, I'll review your specs/patches at friday | 17:31 |
| tmckay | SergeyLukjanov, okay, I'm here tomorrow then gone on vacation until June 30 | 17:31 |
| SergeyLukjanov | unfortunately I need to wake up for the plane in 6 hours... | 17:31 |
| tmckay | heh, go to sleep! | 17:31 |
| SergeyLukjanov | tmckay, okay, I think I'll make review till the end of your vacation :) | 17:32 |
| tmckay | sounds good | 17:32 |
| SergeyLukjanov | tmckay, the problem is that now is 9:32 PM and I need to wake up at 3 AM :) | 17:32 |
| SergeyLukjanov | but usually I'm going to sleep at 1-2 AM | 17:32 |
| SergeyLukjanov | :) | 17:32 |
| tmckay | SergeyLukjanov, oh, that's terrible | 17:34 |
| SergeyLukjanov | tmckay, the pros is that it's a one day trip for visa | 17:34 |
| tmckay | yes, that's nice. | 17:35 |
| *** _mattf is now known as mattf | 17:35 | |
| *** themistymay has quit IRC | 17:47 | |
| tmckay | crobertsrh, ping with a weird problem | 17:49 |
| crobertsrh | tmckay: pong with an odd response | 17:50 |
| tmckay | I login in to horizon (local), and immediately get sent back to the login screen, but log says it was successful | 17:50 |
| tmckay | Login successful for user "admin". | 17:50 |
| tmckay | [18/Jun/2014 17:49:25] "POST /auth/login/ HTTP/1.1" 302 0 | 17:50 |
| tmckay | [18/Jun/2014 17:49:25] "GET / HTTP/1.1" 200 1719 | 17:50 |
| tmckay | [18/Jun/2014 17:49:25] "GET /static/dashboard/css/6bf08293a8e0.css HTTP/1.1" 200 151321 | 17:50 |
| tellesnobrega | tmckay: i would also need an image with storm stuff as well right? | 17:50 |
| crobertsrh | tmckay: That is weird. Maybe clear your cookies for localhost? | 17:50 |
| tmckay | tellesnobrega, yes, unless the provisioning plugin for storm could install it all | 17:51 |
| crobertsrh | or whatever address you're using to hit your horizon | 17:51 |
| tellesnobrega | it can, but isn't it better to have a predefined image to make it quicker | 17:56 |
| tmckay | tellesnobrega, yes | 17:56 |
| tmckay | crobertsrh, it's getting a 302 on the post, POST /auth/login/ HTTP/1.1" 302 0 -- not found, huh? | 17:56 |
| tmckay | it worked yesterday, of course | 17:57 |
| crobertsrh | of course | 17:57 |
| tellesnobrega | i think so too | 17:57 |
| *** akuznetsov has quit IRC | 17:58 | |
| tmckay | crobertsrh, still running my old setup with local horizon and sahara-dashboard linked in. Anything weird there? | 18:02 |
| tmckay | or, anything different? | 18:03 |
| crobertsrh | Shouldn't be anything different | 18:03 |
| crobertsrh | Not many changes merged there lately | 18:03 |
| crobertsrh | any errors in the horizon log? | 18:03 |
| *** bradd1 has quit IRC | 18:04 | |
| crobertsrh | oh...log was above | 18:04 |
| crobertsrh | try a different browser maybe? | 18:04 |
| crobertsrh | that would eliminate any cookie related stuff | 18:04 |
| crobertsrh | Or even a "private" window of firefox | 18:04 |
| *** bradd1 has joined #openstack-sahara | 18:07 | |
| tmckay | same. So, for some reason it can't resolve /auth/login it looks like | 18:07 |
| crobertsrh | Are you sure there were no errors at horizon startup? | 18:12 |
| tmckay | crobertsrh, only this | 18:13 |
| tmckay | DeprecationWarning: check_for_test_cookie is deprecated; ensure your login view is CSRF-protected. | 18:13 |
| tmckay | WARNING:py.warnings:DeprecationWarning: check_for_test_cookie is deprecated; ensure your login view is CSRF-protected. | 18:13 |
| crobertsrh | When did you last update your horizon bits? | 18:14 |
| tmckay | just now, I was getting failures so I upated everything | 18:14 |
| crobertsrh | Hmm. I suppose there could be something in their latest that is hosing things down. I haven't updated my dashboard "classic" env in awhile | 18:15 |
| crobertsrh | Maybe grab an earlier horizon commit from icehouse? | 18:16 |
| tmckay | yeah, I could try. Only thing I think should have changed from yesterday to today was devstack, though | 18:16 |
| tmckay | I wonder if authentication is failing for some reason | 18:17 |
| tmckay | and causing a redirect | 18:17 |
| crobertsrh | Oh, I thought you meant you changed your horizon bits, not devstack | 18:18 |
| tmckay | crobertsrh, I was getting failures earlier, so then I updated horizon and sahara-dashboard and still got the same failures, then I pinged you | 18:19 |
| crobertsrh | ah, ok. | 18:19 |
| tmckay | Only thing that should have changed before my first login was devstack. And my Sahara edp bits, but that shouldn't matter | 18:19 |
| tmckay | gonna try pdb in process_request() | 18:20 |
| crobertsrh | Right. anything in keystone logs? | 18:20 |
| *** akuznetsov has joined #openstack-sahara | 18:22 | |
| *** mattf is now known as _mattf | 18:23 | |
| tmckay | I should like. hmm, looks like "302 Found" from the debugger which is just a redirect | 18:26 |
| *** akuznetsov has quit IRC | 18:31 | |
| *** bradd1 has quit IRC | 18:41 | |
| *** tmckay has quit IRC | 18:45 | |
| *** tmckay has joined #openstack-sahara | 18:49 | |
| *** tosky has quit IRC | 19:00 | |
| *** ViswaV has quit IRC | 19:05 | |
| *** openstackgerrit has quit IRC | 19:06 | |
| *** openstackgerrit_ has joined #openstack-sahara | 19:07 | |
| *** openstackgerrit_ is now known as openstackgerrit | 19:08 | |
| *** bradd1 has joined #openstack-sahara | 19:09 | |
| *** ViswaV has joined #openstack-sahara | 19:22 | |
| tmckay | crobertsrh, still hosed. I wonder if some update to an external module is to blame. Falling back to cli. | 19:23 |
| crobertsrh | Not sure. When I get a minute, I can try a "latest everything" with the classic dashboard and see what I get. | 19:24 |
| tmckay | cool, thanks. as long as I can get this latest CR tested and pushed, I'm happy. PTO starts Friday :) | 19:25 |
| crobertsrh | taking the family to camp again? Or is that later in the summer? | 19:25 |
| tmckay | weeha, my "get the job engine from the plugin with default selection based on job type" patch works. | 19:39 |
| *** bandarji has quit IRC | 19:50 | |
| *** julienvey has quit IRC | 19:57 | |
| *** bandarji has joined #openstack-sahara | 20:02 | |
| *** julienvey has joined #openstack-sahara | 20:28 | |
| *** julienve_ has joined #openstack-sahara | 20:29 | |
| *** bandarji has quit IRC | 20:31 | |
| *** julienvey has quit IRC | 20:32 | |
| *** julienve_ has quit IRC | 20:35 | |
| openstackgerrit | Trevor McKay proposed a change to openstack/sahara: Allow plugins to choose the EDP implementation https://review.openstack.org/101016 | 20:42 |
| crobertsrh | tmckay: looks like a couple other guys in #openstack-horizon are seeing your logging in to dashboard error | 20:51 |
| tmckay | awesome | 20:52 |
| *** miqui has quit IRC | 20:54 | |
| crobertsrh | tmckay: https://bugs.launchpad.net/horizon/+bug/1331406 | 20:58 |
| tmckay | crobertsrh, thank you, I figured it was devstack :) | 21:02 |
| crobertsrh | I'm glad it was devstack | 21:03 |
| *** bandarji has joined #openstack-sahara | 21:04 | |
| *** crobertsrh is now known as _crobertsrh | 21:07 | |
| openstackgerrit | Trevor McKay proposed a change to openstack/sahara: Refactor the job manager to allow multiple execution engines https://review.openstack.org/100622 | 21:08 |
| openstackgerrit | Trevor McKay proposed a change to openstack/sahara: Allow plugins to choose the EDP implementation https://review.openstack.org/101016 | 21:09 |
| *** tmckay has quit IRC | 21:25 | |
| *** julienvey has joined #openstack-sahara | 21:30 | |
| *** julienvey has quit IRC | 21:31 | |
| *** julienvey has joined #openstack-sahara | 21:31 | |
| *** julienvey has quit IRC | 21:36 | |
| *** witlessb has quit IRC | 22:21 | |
| *** bandarji has quit IRC | 22:34 | |
| *** bradd1 has quit IRC | 22:49 | |
| *** sballe__ has quit IRC | 22:52 | |
| *** elmiko is now known as _elmiko | 22:54 | |
| *** rhodgin has quit IRC | 22:54 | |
| openstackgerrit | Robert Levas proposed a change to openstack/sahara: Upgrades the HDP plug-in to install Hue https://review.openstack.org/101050 | 23:31 |
| *** rlevas has joined #openstack-sahara | 23:46 | |
| *** roblevas has joined #openstack-sahara | 23:47 | |
| *** roblevas has quit IRC | 23:49 | |
| *** rlevas has quit IRC | 23:49 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!