| *** rcernin has joined #openstack-sahara | 00:19 | |
| *** rcernin has quit IRC | 00:23 | |
| *** rcernin has joined #openstack-sahara | 00:24 | |
| *** aolwas has quit IRC | 00:57 | |
| *** brad[] has quit IRC | 00:59 | |
| *** brad[] has joined #openstack-sahara | 01:00 | |
| *** elmiko has quit IRC | 01:01 | |
| *** elmiko has joined #openstack-sahara | 01:25 | |
| *** caowei has joined #openstack-sahara | 01:31 | |
| *** rcernin has quit IRC | 01:32 | |
| *** rcernin has joined #openstack-sahara | 01:32 | |
| *** rcernin has quit IRC | 01:33 | |
| *** rcernin has joined #openstack-sahara | 01:55 | |
| *** kiennt26 has joined #openstack-sahara | 02:03 | |
| *** dave-mccowan has quit IRC | 02:59 | |
| *** caowei has quit IRC | 03:25 | |
| *** kiennt26 has quit IRC | 04:07 | |
| *** rcernin has quit IRC | 04:21 | |
| *** caowei has joined #openstack-sahara | 04:24 | |
| *** rcernin has joined #openstack-sahara | 04:45 | |
| *** kiennt26 has joined #openstack-sahara | 06:31 | |
| *** tosky has joined #openstack-sahara | 06:36 | |
| *** pcaruana has joined #openstack-sahara | 06:39 | |
| *** tesseract has joined #openstack-sahara | 06:54 | |
| *** pgadiya has joined #openstack-sahara | 07:02 | |
| *** spectr has joined #openstack-sahara | 07:13 | |
| *** anshul has joined #openstack-sahara | 07:34 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: WIP Native Zuul v3 jobs https://review.openstack.org/512058 | 07:43 |
|---|---|---|
| *** hoonetorg has quit IRC | 08:07 | |
| *** hoonetorg has joined #openstack-sahara | 08:24 | |
| *** tosky has quit IRC | 08:31 | |
| *** tosky has joined #openstack-sahara | 08:57 | |
| *** rcernin has quit IRC | 08:59 | |
| openstackgerrit | Kevin Lefevre proposed openstack/sahara-dashboard master: Fix issue creating NGT using dashboard https://review.openstack.org/500193 | 08:59 |
| *** anshul has quit IRC | 09:18 | |
| *** anshul has joined #openstack-sahara | 09:22 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara stable/ocata: Updated from OpenStack Ansible Tests https://review.openstack.org/514755 | 09:49 |
| *** kiennt26 has quit IRC | 10:07 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-dashboard master: Initial Zuul v3 jobs: unittests https://review.openstack.org/511501 | 10:15 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-dashboard master: Initial Zuul v3 jobs: unittests https://review.openstack.org/511501 | 10:19 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-dashboard master: Initial Zuul v3 jobs: unittests https://review.openstack.org/511501 | 10:22 |
| *** pcaruana has quit IRC | 10:37 | |
| *** pgadiya has quit IRC | 11:02 | |
| *** caowei has quit IRC | 11:09 | |
| *** pgadiya has joined #openstack-sahara | 11:10 | |
| *** dave-mccowan has joined #openstack-sahara | 11:30 | |
| *** pcaruana has joined #openstack-sahara | 11:43 | |
| *** pgadiya has quit IRC | 12:00 | |
| *** pcaruana has quit IRC | 12:03 | |
| *** pcaruana has joined #openstack-sahara | 12:04 | |
| *** masuberu has joined #openstack-sahara | 12:06 | |
| openstackgerrit | Kevin Lefevre proposed openstack/sahara-dashboard master: Fix issue creating NGT using dashboard https://review.openstack.org/500193 | 12:07 |
| *** masber has quit IRC | 12:09 | |
| *** pgadiya has joined #openstack-sahara | 12:09 | |
| openstackgerrit | Kevin Lefevre proposed openstack/sahara-dashboard master: Initial Zuul v3 jobs: unittests https://review.openstack.org/511501 | 12:18 |
| openstackgerrit | Kevin Lefevre proposed openstack/sahara-dashboard master: Fix issue creating NGT using dashboard https://review.openstack.org/500193 | 12:18 |
| *** pgadiya has quit IRC | 12:21 | |
| openstackgerrit | Kevin Lefevre proposed openstack/sahara master: Use non corrupted libext from image. https://review.openstack.org/515042 | 12:27 |
| *** pgadiya has joined #openstack-sahara | 12:35 | |
| *** pgadiya has quit IRC | 12:41 | |
| *** pgadiya has joined #openstack-sahara | 12:53 | |
| *** pgadiya has quit IRC | 12:56 | |
| *** ukaynar has joined #openstack-sahara | 14:13 | |
| tosky | tellesnobrega: anything against https://review.openstack.org/#/c/511501/ so that we can unlock further reviews like https://review.openstack.org/#/c/500193/ ? | 14:13 |
| tellesnobrega | both look good to me | 14:14 |
| tellesnobrega | tosky, approved your patch, you can approve the other one | 14:15 |
| tosky | thanks! | 14:15 |
| tosky | converting the sahara-dashboard integration test would be a bit more complex, but for now we can use the legacy one | 14:16 |
| tosky | I think that the Jeremy's remarks for https://review.openstack.org/#/c/500193/ have been addressed, but I will wait few hours just in case | 14:18 |
| openstackgerrit | Merged openstack/sahara-dashboard master: Initial Zuul v3 jobs: unittests https://review.openstack.org/511501 | 14:27 |
| *** pcaruana has quit IRC | 15:49 | |
| masuberu | good afternooon all, can sahara deploy cloudera manager or ambari? | 16:19 |
| masuberu | or is it just deploying the cluster with the services? | 16:19 |
| masuberu | by services I mean HDFS, hadoop, spark, etc | 16:19 |
| *** ukaynar has quit IRC | 16:24 | |
| *** tesseract has quit IRC | 16:25 | |
| tellesnobrega | masuberu, it deploys both cloudera manager and ambari | 16:28 |
| tellesnobrega | depending on the plugin | 16:28 |
| *** anshul has quit IRC | 16:30 | |
| *** jeremyfreudberg has joined #openstack-sahara | 17:04 | |
| masuberu | tellesnobrega, will it provide a web interface and provide monitoring etc for the big data cluster? | 17:06 |
| jeremyfreudberg | (sorry to interrupt) hey tosky, when we convert the "legacy-sahara-dsvm-scenario-multinode-spark" to v3, do you think it's possible to disable nested virtualization there? this patch https://review.openstack.org/#/c/513948/ is the first I heard of doing that -- but it seems useful | 17:06 |
| tellesnobrega | masuberu, I don't think we plan to provide monitoring for the cluster on the sahara side | 17:07 |
| tosky | jeremyfreudberg: the tested job (which almost works) inherits from the devstack job, which iirc already disables it | 17:07 |
| tellesnobrega | masuberu, you are able to access CDH manager web interface and Ambari web interface as well | 17:07 |
| tosky | so we are already covered | 17:07 |
| jeremyfreudberg | tosky: aha, I see | 17:08 |
| tosky | in fact the scenario part of https://review.openstack.org/#/c/512058/ works! | 17:08 |
| tosky | almost | 17:08 |
| tosky | in the multinode test I'm not sure why I see the logs of sahara and other services on both nodes | 17:08 |
| tosky | I shouldn't see them on the compute node | 17:08 |
| tosky | I will ask later | 17:08 |
| tellesnobrega | masuberu, if you click on the cluster, you can see its info, with manager url and username and password | 17:08 |
| tosky | but basically the scenario test works | 17:08 |
| masuberu | tellesnobrega, then would it be possible to deploy the cluster using cloudera manager instead of sahara? that way I can have the agents on the cluster nodes? | 17:09 |
| tosky | I may split the patch so that we can approve the converted scenario tests while waiting for the tempest tests | 17:09 |
| masuberu | and see the metrics through the web interface? | 17:09 |
| tosky | masuberu: but sahara uses the cloudera manager | 17:09 |
| jeremyfreudberg | tosky: not my area of expertise, but regarding the compute node, do you have the equivalent of this https://review.openstack.org/#/c/447215/3/jenkins/jobs/sahara.yaml in the new jobs | 17:09 |
| tellesnobrega | masuberu, when you deploy with sahara, the manager is also up and you can its metric on the manager web interface | 17:09 |
| tellesnobrega | no need to redeploy, the agents are already there | 17:10 |
| masuberu | ok, that is what I ment by monitoring | 17:10 |
| masuberu | that is perfect then! | 17:10 |
| masuberu | fantastic | 17:10 |
| tosky | jeremyfreudberg: that's probably the thing I'm missing; the complication here is that devstack-gate is going to be less useful, so if it's a devstack-gate thing, I need to replace it | 17:11 |
| tosky | let me chec | 17:11 |
| tosky | check* | 17:11 |
| masuberu | and can I also let's say provide the IPs and keys for the nodes and sahara will deploy (through cdh) in a bare metal flavor? | 17:11 |
| openstackgerrit | Merged openstack/sahara-dashboard master: Fix issue creating NGT using dashboard https://review.openstack.org/500193 | 17:11 |
| masuberu | by bare metal I mean the nodes will have the OS already installed | 17:11 |
| jeremyfreudberg | tosky: I need to log off now, but I get what you mean. good luck. when you finish that you can also get the py3 scenario working (swift does not support py3 has been the culprit) | 17:12 |
| tellesnobrega | masuberu, yes, we do support baremetal cluster. You have to set up the nodegroup template using the baremetal flavors, but it should work | 17:12 |
| *** jeremyfreudberg has quit IRC | 17:12 | |
| masuberu | is sahara integrated with ironic? | 17:13 |
| tosky | yep, I need to ask whether it's possible to disable py3 for switch, which I'm not sure about - let's see! | 17:13 |
| tellesnobrega | masuberu, yes | 17:15 |
| masuberu | another question, how bad is neutron latency affecting sahara cluster? I believe hadoop and spark needs a low latency network? | 17:15 |
| tellesnobrega | masuberu, https://docs.openstack.org/sahara/pike/user/sahara-on-ironic.html | 17:15 |
| masuberu | is it recommended to use provider network instead of SDN? | 17:16 |
| tosky | uhm, I don't think that we have specific recommendation about this; it is more on the real of Neutron; what Neutron gives us, we use it | 17:21 |
| tosky | it's not different from manually deploying Hadoop on some instances/bare metal machines deployed using OpenStack | 17:21 |
| tosky | I will discuss this more tomorrow during the meeting, but I tried to capture the status of the zuulv3 migration for Sahara: https://etherpad.openstack.org/p/zuulv3-sahara | 17:43 |
| tosky | at least from my point of view | 17:43 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: WIP Basic Zuul v3 jobs https://review.openstack.org/510579 | 17:49 |
| *** tosky has quit IRC | 17:50 | |
| *** tosky has joined #openstack-sahara | 18:22 | |
| *** catintheroof has joined #openstack-sahara | 18:47 | |
| *** catintheroof has quit IRC | 18:49 | |
| *** catintheroof has joined #openstack-sahara | 18:51 | |
| openstackgerrit | Merged openstack/sahara master: Use non corrupted libext from image. https://review.openstack.org/515042 | 19:22 |
| tellesnobrega | tosky, looks like set_pipelines helps | 20:21 |
| tellesnobrega | not too much | 20:21 |
| tellesnobrega | but it should work for now | 20:22 |
| tellesnobrega | it reduced from 4 to 3 minutes | 20:22 |
| tosky | uhm, maybe not enough for really big files | 20:22 |
| tosky | what was the size tested? | 20:23 |
| tellesnobrega | 115MB | 20:33 |
| tellesnobrega | tosky, ^ | 20:34 |
| tellesnobrega | maybe it is this fast because it is all in one place here | 20:34 |
| tosky | is the reduction enough to not trigger the timeout? | 20:34 |
| tellesnobrega | default timeout is 2 minutes | 20:34 |
| tellesnobrega | I will check with put later | 20:47 |
| *** masuberu has quit IRC | 20:51 | |
| *** rcernin has joined #openstack-sahara | 21:16 | |
| *** catintheroof has quit IRC | 22:18 | |
| *** ukaynar has joined #openstack-sahara | 22:39 | |
| *** tosky has quit IRC | 23:02 | |
| *** masuberu has joined #openstack-sahara | 23:05 | |
| *** ukaynar has quit IRC | 23:13 | |
| *** ukaynar has joined #openstack-sahara | 23:14 | |
| *** ukaynar has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!