| openstackgerrit | Nguyen Hung Phuong proposed openstack/sahara-image-elements master: fix tox python3 overrides https://review.openstack.org/581176 | 00:15 |
|---|---|---|
| *** Bhujay has joined #openstack-sahara | 04:53 | |
| *** links has joined #openstack-sahara | 05:03 | |
| *** Bhujay has quit IRC | 06:59 | |
| *** Bhujay has joined #openstack-sahara | 07:05 | |
| *** Bhujay has quit IRC | 07:09 | |
| *** tesseract has joined #openstack-sahara | 07:12 | |
| *** Bhujay has joined #openstack-sahara | 07:13 | |
| *** tosky has joined #openstack-sahara | 07:37 | |
| *** rcernin has quit IRC | 08:03 | |
| *** Bhujay has quit IRC | 08:11 | |
| openstackgerrit | Tuan Do Anh proposed openstack/sahara master: Trivial fix typo of description https://review.openstack.org/581633 | 09:04 |
| *** Bhujay has joined #openstack-sahara | 09:29 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara master: Revert "Disable keystone_authtoken/service_token_roles_required for now" https://review.openstack.org/581674 | 11:01 |
| openstackgerrit | Merged openstack/sahara master: Final fixup to APIv2 responses https://review.openstack.org/581072 | 11:12 |
| openstackgerrit | Vu Cong Tuan proposed openstack/sahara master: Remove testrepository https://review.openstack.org/581680 | 11:13 |
| *** Bhujay has quit IRC | 11:24 | |
| openstackgerrit | Merged openstack/python-saharaclient master: Reflect response fixes for jobs/jobs templates https://review.openstack.org/581077 | 11:31 |
| *** openstack has joined #openstack-sahara | 13:04 | |
| *** ChanServ sets mode: +o openstack | 13:04 | |
| *** Bhujay has joined #openstack-sahara | 14:25 | |
| *** Bhujay has quit IRC | 14:30 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara master: Switch make_json_error back to being a function https://review.openstack.org/581773 | 15:06 |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara master: Give the illusion of microversion support https://review.openstack.org/581774 | 15:06 |
| *** jeremyfreudberg has joined #openstack-sahara | 15:08 | |
| jeremyfreudberg | tellesnobrega: one problem with your spark image-- it is installing too many packages. i see it installing cloudera manager, which is wrong. it should only be these: https://github.com/openstack/sahara-image-elements/blob/master/elements/hadoop-cloudera/package-installs.yaml | 15:10 |
| *** links has quit IRC | 15:16 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: Add basic S3 job binaries API tests https://review.openstack.org/581781 | 15:51 |
| *** tesseract has quit IRC | 16:01 | |
| tellesnobrega | jeremyfreudberg, hmm, thanks | 16:02 |
| *** jeremyfreudberg has quit IRC | 16:32 | |
| tellesnobrega | tosky, HDP 2.3 with ambari 2.2.0.0 passes scenarios locally | 16:46 |
| tosky | tellesnobrega: that's good to hear, thanks | 16:49 |
| tosky | so maybe the issue showed only with 2.4.something, and 2.2.x were too "primitives" to be affected | 16:49 |
| tellesnobrega | should I test 2.2.1.0 as well? | 16:50 |
| tosky | tellesnobrega: if you can, yes please | 17:01 |
| tellesnobrega | no problem at all | 17:03 |
| tellesnobrega | tosky, do you know why we install cloudera stuff on spark? | 17:23 |
| tellesnobrega | trying spark 2.2 with sahara-image-pack Operation with name 'Executing "apt-get -y install cloudera-manager-agent cloudera-manager-daemons cloudera-manager-server cloudera-manager-server-db-2"'' timed out after 300 second(s) | 17:23 |
| tellesnobrega | this is during cluster creation | 17:23 |
| tosky | tellesnobrega: do we have a new-type image generator for Spark? | 17:25 |
| tosky | oh, your patch maybe | 17:25 |
| tellesnobrega | I'm working on it | 17:25 |
| tosky | in fact I don't know | 17:27 |
| tosky | cloudera-manager is a bit too much, if only few libraries are needed | 17:27 |
| tellesnobrega | from the SIE stuff, we only install hadoop-hdfs-namenode and hadoop-hdfs-datanode | 17:28 |
| tosky | so some variable is missing, because the hadoop-cloudera element is used for spark | 17:28 |
| tosky | but it does not do all the magic that it usually does | 17:28 |
| tellesnobrega | it is used, but only for those packages | 17:29 |
| tellesnobrega | https://github.com/openstack/sahara-image-elements/blob/master/elements/hadoop-cloudera/install.d/50-install-cloudera#L15 | 17:29 |
| tellesnobrega | this is set to 1 for spark | 17:29 |
| tellesnobrega | so the packages are not installed | 17:29 |
| tosky | thanks to DIB_CDH_HDFS_ONLY, yeah | 17:30 |
| tosky | the name is not really helpful | 17:30 |
| tellesnobrega | true | 17:30 |
| tellesnobrega | but why would we need to install those during cluster creation? | 17:30 |
| tosky | but I don't see any reference to those names in the current code | 17:31 |
| tellesnobrega | me neither | 17:31 |
| tellesnobrega | not sure why this is happening | 17:31 |
| tosky | something in the validation code? :) | 17:31 |
| tellesnobrega | probably | 17:31 |
| tellesnobrega | by validation you are saying from image-pack validation? | 17:38 |
| *** jeremyfreudberg has joined #openstack-sahara | 17:39 | |
| tellesnobrega | tosky, apparently it is something from sahara-image-pack, not sure why it is trying to install the packages, they are not on the images.yaml | 17:44 |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara master: Give the illusion of microversion support https://review.openstack.org/581774 | 17:44 |
| tosky | tellesnobrega: the code should be there somewhere | 17:46 |
| tosky | tellesnobrega: I mean, even if they are not listed, the code should be there; if you want, please share the patch | 17:53 |
| jeremyfreudberg | yes, i'd like to see the patch | 17:54 |
| tellesnobrega | sure | 17:54 |
| tellesnobrega | I will put the patch for review in a bit | 17:54 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Adding Spark to sahara-image-pack https://review.openstack.org/581808 | 17:59 |
| tellesnobrega | tosky, jeremyfreudberg ^ | 18:00 |
| tosky | tellesnobrega: can you please share the output of the image generation? | 18:04 |
| tosky | I suspect that one of the two packages installed brings cloudera-manager as dependency, but that can be checked only comparing the full log with the yaml | 18:13 |
| tellesnobrega | hm | 18:37 |
| tellesnobrega | tosky, I guess this is the relevant part | 18:40 |
| tellesnobrega | http://paste.openstack.org/show/725594/ | 18:40 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Adding Spark to sahara-image-pack https://review.openstack.org/581808 | 18:49 |
| tosky | tellesnobrega: I think that those packages have either strict dependencies on cloudera-manager, or, more likely, a semi-strict dependency: recommends, which is enabled by default on Debian and Ubuntu | 19:30 |
| tellesnobrega | tosky, shouldn't it be installed during image creation then? | 19:30 |
| tosky | tellesnobrega: uh, isndo you have the log for the image creation to compare? | 19:31 |
| tosky | yes, I would expect them to behave in the same way | 19:31 |
| *** jeremyfreudberg has quit IRC | 19:46 | |
| tellesnobrega | that part I sent is the part that install the hdfs part | 19:51 |
| tellesnobrega | and all cloudera, so cloudera is not being installed | 19:58 |
| tellesnobrega | I'm checking if it is really necessary | 19:59 |
| tosky | tellesnobrega: I totally misread the log that you sent; it does not mention cloudera-manager | 20:05 |
| tosky | tellesnobrega: isn't there any reference of cloudera-manager in the logs? | 20:06 |
| tellesnobrega | nope | 20:09 |
| tellesnobrega | I think it might have been residual from previous run | 20:09 |
| tellesnobrega | I'm testing again | 20:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Revert "Disable keystone_authtoken/service_token_roles_required for now" https://review.openstack.org/581674 | 20:10 |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara-dashboard master: [WIP] APIv2 in dashboard https://review.openstack.org/581878 | 20:16 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: fix tox python3 overrides https://review.openstack.org/580064 | 20:27 |
| tosky | tellesnobrega: I tried too, and the execution fails with 'Error during command execution: "sudo chwon -R $USER:$USER /opt/spark"' | 20:38 |
| tosky | tellesnobrega: added few comments | 20:56 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: Trivial: Update Zuul Status Page to correct URL https://review.openstack.org/570634 | 22:00 |
| *** rcernin has joined #openstack-sahara | 22:15 | |
| openstackgerrit | Merged openstack/sahara-specs master: fix tox python3 overrides https://review.openstack.org/581193 | 23:01 |
| openstackgerrit | Merged openstack/sahara master: S3 data source https://review.openstack.org/578967 | 23:02 |
| openstackgerrit | Merged openstack/sahara-dashboard master: Fix the coverage tox target https://review.openstack.org/579108 | 23:18 |
| openstackgerrit | Merged openstack/sahara-image-elements master: fix tox python3 overrides https://review.openstack.org/581176 | 23:18 |
| openstackgerrit | Merged openstack/sahara-image-elements master: Adding Spark 2.3 https://review.openstack.org/578085 | 23:22 |
| openstackgerrit | Merged openstack/sahara-dashboard master: Change approach to saharaclient authentication https://review.openstack.org/581088 | 23:23 |
| *** tosky has quit IRC | 23:35 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!