| *** jamielennox|away is now known as jamielennox | 01:07 | |
| *** chen12 has joined #openstack-sahara | 01:26 | |
| *** Networkn_ has joined #openstack-sahara | 02:05 | |
| *** Networkn3rd has quit IRC | 02:06 | |
| openstackgerrit | Ken Chen proposed openstack/sahara: Transform configuration values into int or float when needed https://review.openstack.org/187444 | 03:38 |
|---|---|---|
| *** coolsvap|afk is now known as coolsvap | 03:44 | |
| openstackgerrit | Ken Chen proposed openstack/sahara-image-elements: Add installing keytrustee-keyprovider package https://review.openstack.org/189183 | 03:57 |
| openstackgerrit | Ken Chen proposed openstack/sahara-image-elements: Add installing keytrustee-keyprovider package https://review.openstack.org/189183 | 04:04 |
| openstackgerrit | Ken Chen proposed openstack/sahara-image-elements: Add installing keytrustee-keyprovider package https://review.openstack.org/189183 | 04:28 |
| *** hdd has joined #openstack-sahara | 04:49 | |
| *** sgotliv has quit IRC | 05:50 | |
| openstackgerrit | Ken Chen proposed openstack/sahara-image-elements: Add installing keytrustee-keyprovider package https://review.openstack.org/189183 | 06:08 |
| *** hdd has quit IRC | 06:20 | |
| *** hdd has joined #openstack-sahara | 06:28 | |
| *** hdd has quit IRC | 06:40 | |
| openstackgerrit | li,chen proposed openstack/sahara-specs: Expose supported elements for clusters https://review.openstack.org/178892 | 07:22 |
| *** witlessb has joined #openstack-sahara | 07:25 | |
| *** sgotliv has joined #openstack-sahara | 07:31 | |
| *** pino|work has joined #openstack-sahara | 07:46 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-specs: Adding custom scenario tests https://review.openstack.org/188807 | 07:49 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-specs: Adding custom scenario tests https://review.openstack.org/188807 | 07:52 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Added method for connect to node and run command https://review.openstack.org/185419 | 07:55 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Added method for connect to node and run command https://review.openstack.org/185419 | 07:56 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add support of custom scenario to scenario tests https://review.openstack.org/187150 | 07:57 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add support of custom scenario to scenario tests https://review.openstack.org/187150 | 07:57 |
| *** sgotliv has quit IRC | 08:03 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add method for geting ip address node with process https://review.openstack.org/187232 | 08:10 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add method for geting ip address node with process https://review.openstack.org/187232 | 08:14 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Added method for connect to node and run command https://review.openstack.org/185419 | 08:15 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add support of custom scenario to scenario tests https://review.openstack.org/187150 | 08:15 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add support of custom scenario to scenario tests https://review.openstack.org/187150 | 08:15 |
| *** sgotliv has joined #openstack-sahara | 08:20 | |
| *** sgotliv has quit IRC | 08:30 | |
| *** sgotliv has joined #openstack-sahara | 08:44 | |
| *** tosky has joined #openstack-sahara | 09:30 | |
| *** degorenko has quit IRC | 09:55 | |
| *** openstackgerrit has quit IRC | 10:09 | |
| *** openstackgerrit has joined #openstack-sahara | 10:09 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Increase flavors for MAPR testcase https://review.openstack.org/189256 | 10:23 |
| *** esikachev has joined #openstack-sahara | 10:39 | |
| *** coolsvap is now known as coolsvap|afk | 10:47 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 11:03 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 11:03 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 11:04 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 11:09 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 11:09 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add custom check of impala https://review.openstack.org/188777 | 11:14 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add custom check of impala https://review.openstack.org/188777 | 11:15 |
| *** esikachev has quit IRC | 11:40 | |
| *** egafford has joined #openstack-sahara | 11:59 | |
| *** esikachev has joined #openstack-sahara | 12:03 | |
| *** raildo has joined #openstack-sahara | 12:11 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 12:19 |
| *** tmckay has joined #openstack-sahara | 12:22 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add custom check of impala https://review.openstack.org/188777 | 12:26 |
| *** crobertsrh has joined #openstack-sahara | 12:33 | |
| *** raildo has quit IRC | 12:50 | |
| *** tellesnobrega has quit IRC | 12:50 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check sqoop2 in custom scenario https://review.openstack.org/189288 | 13:04 |
| *** tellesnobrega has joined #openstack-sahara | 13:06 | |
| *** raildo has joined #openstack-sahara | 13:06 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-image-elements: Add dib-lint checks for elements https://review.openstack.org/166602 | 13:26 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 13:29 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add custom check of impala https://review.openstack.org/188777 | 13:34 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add custom check of impala https://review.openstack.org/188777 | 13:42 |
| *** htruta has joined #openstack-sahara | 13:44 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check sqoop2 in custom scenario https://review.openstack.org/189288 | 13:47 |
| *** hdd has joined #openstack-sahara | 14:12 | |
| *** logan2 has quit IRC | 14:12 | |
| *** logan2 has joined #openstack-sahara | 14:13 | |
| *** mattf is now known as _mattf | 14:25 | |
| *** hdd has quit IRC | 14:29 | |
| *** jamielennox is now known as jamielennox|away | 14:38 | |
| *** hdd has joined #openstack-sahara | 14:38 | |
| openstackgerrit | Merged openstack/sahara: Update list of supported API versions https://review.openstack.org/186180 | 14:39 |
| openstackgerrit | Merged openstack/sahara: Minor improvement of validation rules https://review.openstack.org/185924 | 14:40 |
| openstackgerrit | Merged openstack/sahara-image-elements: Add missing hadoop version check for CDH 5.4 https://review.openstack.org/187274 | 14:44 |
| *** sgotliv has quit IRC | 15:00 | |
| *** sgotliv has joined #openstack-sahara | 15:02 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add check for cinder in scenario tests https://review.openstack.org/187560 | 15:10 |
| *** esikachev has quit IRC | 15:19 | |
| crobertsrh | tmckay: Is there a config I can pass that will change where spark tosses the output of a job? | 15:30 |
| openstackgerrit | Merged openstack/sahara: Added checking of event-log in scenario tests https://review.openstack.org/184440 | 15:35 |
| openstackgerrit | Merged openstack/sahara: Add "null" for fields in cluster and node group template JSON schemas https://review.openstack.org/181477 | 15:35 |
| openstackgerrit | Merged openstack/sahara: Don't use reduce for python3 compatibility https://review.openstack.org/187076 | 15:35 |
| openstackgerrit | Merged openstack/sahara: Disable neutron DEBUG logs https://review.openstack.org/187776 | 15:35 |
| openstackgerrit | Merged openstack/sahara: Remove WritableLogger wrapper https://review.openstack.org/188130 | 15:35 |
| tmckay | crobertsrh, no | 15:36 |
| tmckay | crobertsrh, I don't think so. Maybe I made /tmp/spark-edp configurable | 15:37 |
| tmckay | crobertsrh, what are you trying to do? | 15:37 |
| *** Networkn_ has quit IRC | 15:54 | |
| *** sgotliv has quit IRC | 16:02 | |
| *** degorenko has joined #openstack-sahara | 16:07 | |
| *** egafford has quit IRC | 16:17 | |
| openstackgerrit | Ken Chen proposed openstack/sahara-image-elements: Add installing keytrustee-keyprovider package https://review.openstack.org/189183 | 16:17 |
| *** hdd has quit IRC | 16:17 | |
| *** hdd has joined #openstack-sahara | 16:19 | |
| crobertsrh | tmckay: I'm just playing with manila. Looking to see what I can do with my spark cluster quickly. | 16:38 |
| tmckay | crobertsrh, ok. You should be able to find a Spark example that writes (or reads) from a filesystem. If, I can help you make one. | 16:39 |
| crobertsrh | Nah, I'll come up with something awesome | 16:39 |
| tmckay | simple wordcount that ingests from local storage, or writes the results out | 16:39 |
| crobertsrh | Should just be gravy. I have the manila share mounted on the cluster vm | 16:40 |
| tmckay | most of the examples unfortunately write to stdout, but several I saw read from a file | 16:40 |
| tmckay | awesome | 16:40 |
| *** hdd has quit IRC | 16:46 | |
| *** hdd has joined #openstack-sahara | 16:54 | |
| *** hdd has quit IRC | 17:16 | |
| *** hdd has joined #openstack-sahara | 17:17 | |
| tellesnobrega | SergeyLukjanov, tmckay, elmiko have you guys given any thought about the multiple clusters creation implementation? where should it be done? i'm more inclined to do it in the sahara side, but it has more changes(API, quotas checking, probably gonna have to change some return to a list of cluster) | 17:22 |
| openstackgerrit | Ken Chen proposed openstack/sahara: Enable Java Keystore KMS service in CDH5.4 https://review.openstack.org/189373 | 17:22 |
| elmiko | tellesnobrega: i need to read it again, i'm not sure just yet | 17:23 |
| tellesnobrega | changing in the clients and horizon is easier i think, won't change the API and quotas will be checked the same ways as today (we wont guarantee that all cluster can be created upfront) | 17:23 |
| tellesnobrega | from my point of view is a matter of want we want to do as a feature, if we want it more complete and robust or simpler and quick | 17:24 |
| elmiko | yea, it sounded like the proper way was to manipulate at the API level, from outside the server. | 17:26 |
| *** egafford has joined #openstack-sahara | 17:27 | |
| openstackgerrit | Gael Chamoulaud proposed stackforge/puppet-sahara: Move rabbit/kombu settings to oslo_messaging_rabbit section https://review.openstack.org/173891 | 17:28 |
| *** jodah has joined #openstack-sahara | 17:30 | |
| tellesnobrega | elmiko, I'm ok with that, as long as we are clear that we won't check quotas for the whole thing, just cluster by cluster as of today (we would loop in the number of cluster and send the requests or we can send it in parallel, which may cause race condition problems) | 17:32 |
| elmiko | tellesnobrega: hmm, the quota question is a good one. this is why i'm not sure yet, i need to think about it a little more. | 17:32 |
| elmiko | tellesnobrega: what happened to the count= parameter we talked about in the meeting the other day? | 17:33 |
| * tellesnobrega tries to remember what exactly that was | 17:34 | |
| tellesnobrega | elmiko, just a second, i'm going for the logs | 17:35 |
| elmiko | i thought we had a count parameter that was being passed to the cluster creation, like maybe there would be the possibility for requesting multiple clusters. but of course, this would mean they would all be the same config clusters. | 17:35 |
| elmiko | if the user has the capability to request different types of clusters to be spawned during a multi-provision operation then that would be more complicated and require a pretty large change. | 17:36 |
| tellesnobrega | for sure, the idea is to create clusters with the same configuration | 17:38 |
| tellesnobrega | if we pass the count parameter than the creation would be handled on the server side | 17:38 |
| elmiko | tellesnobrega: yea, which would be helpful for quota stuff and helping with race conditions. i dunno, i'm still not sure about the best approach lol | 17:39 |
| tellesnobrega | lol | 17:39 |
| elmiko | doing it from the client is easier, but doing it from the server is probably better | 17:39 |
| elmiko | i guess that's my opinion, for now ;) | 17:39 |
| tmckay | to me, it seems like the same case as just hitting launch and filling out the form multiple times. You may find out along the way that you are out of resources. Oh well. | 17:40 |
| elmiko | which brings up a great question, what happens if you request N clusters, and it fails during the creation, do they all get killed, or ? | 17:41 |
| tmckay | I see it as a mouse-click-saving device. Just fire em off, submit them to the engines, and let them go. | 17:41 |
| tmckay | I think individual clusters would fail. | 17:41 |
| elmiko | i suppose you could do some resource checking before attempting the creation of the entire operation, then report back. | 17:42 |
| elmiko | i'm just not sure, there are some error cases we would need to define | 17:42 |
| tmckay | I don't know, we could make this complicated. What if a prof, for instance, wanted to create clusters for a class, each student has a tenant? | 17:42 |
| tmckay | what he would want at that point is N clusters with a tenant map. Too crazy? Still fewer mouse clicks. Maybe the prof writes his own client wrapper :) | 17:42 |
| elmiko | good point, i wasn't even thinking that complicated yet. just one project, N cluster requested, then how would that flow | 17:43 |
| * tmckay steps away for a bit | 17:43 | |
| elmiko | woud you check ahead of time to ensure they could all be created? | 17:44 |
| elmiko | would you fail part way if things started to error? | 17:44 |
| elmiko | if one cluster failed but the rest were created is that ok? | 17:44 |
| elmiko | i think you can see where this is going... | 17:44 |
| tellesnobrega | my idea is the following | 17:45 |
| tellesnobrega | if we implement on the server side, we check the dict, loop on how many clusters we want to create and do it sequentially, checking quotas by clusters | 17:46 |
| tellesnobrega | if we don't have enough quota left, the last cluster that was being created will fail and we break the loop | 17:47 |
| tellesnobrega | can be this way from the client and horizon side as well, we loop and send creation requests, since we are looping we wont be having a race condition | 17:48 |
| elmiko | makes sense. i think server side might be better. but i don't have a strong opinion on this, yet. | 17:49 |
| elmiko | tellesnobrega: i gotta go afk for a little bit, i'll be back later | 17:50 |
| tellesnobrega | elmiko, ok thanks | 17:50 |
| *** chen12 has quit IRC | 17:51 | |
| *** pino|work has quit IRC | 17:55 | |
| *** tosky has quit IRC | 17:56 | |
| *** openstackgerrit has quit IRC | 18:09 | |
| *** openstackgerrit has joined #openstack-sahara | 18:09 | |
| *** sgotliv has joined #openstack-sahara | 18:34 | |
| *** hdd has quit IRC | 18:46 | |
| *** sgotliv has quit IRC | 18:46 | |
| *** hdd has joined #openstack-sahara | 18:58 | |
| *** egafford1 has joined #openstack-sahara | 19:12 | |
| *** egafford has quit IRC | 19:15 | |
| openstackgerrit | Sergey Kolekonov proposed openstack/sahara-image-elements: Add quotes to unsafe if conditions in mapr element https://review.openstack.org/189404 | 19:21 |
| crobertsrh | Any of you know of a way I can increase my swift storage size without re-stacking? | 20:42 |
| elmiko | not me, i usually have to up that size and restack =( | 20:43 |
| * crobertsrh hopes to avoid that | 20:44 | |
| elmiko | understandable | 20:45 |
| crobertsrh | Of course, I do love the smell of a fresh stack in the morning. | 20:46 |
| *** raildo has quit IRC | 21:00 | |
| openstackgerrit | Colleen Murphy proposed stackforge/puppet-sahara: Unpin beaker-rspec https://review.openstack.org/189450 | 21:02 |
| *** crobertsrh is now known as _crobertsrh | 21:12 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements https://review.openstack.org/189464 | 21:20 |
| *** Networkn3rd has joined #openstack-sahara | 21:22 | |
| openstackgerrit | Merged stackforge/puppet-sahara: Unpin beaker-rspec https://review.openstack.org/189450 | 21:44 |
| *** egafford1 has quit IRC | 22:10 | |
| *** chlong has joined #openstack-sahara | 23:36 | |
| *** witlessb has quit IRC | 23:40 | |
| *** hdd has quit IRC | 23:46 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!