| *** masber has joined #openstack-sahara | 00:10 | |
| *** masuberu has quit IRC | 00:13 | |
| *** masuberu has joined #openstack-sahara | 00:26 | |
| *** masber has quit IRC | 00:29 | |
| openstackgerrit | Merged openstack/sahara master: Improve force delete https://review.openstack.org/572455 | 01:04 |
|---|---|---|
| openstackgerrit | Zhuang Changkun proposed openstack/sahara master: Replace the deleted keypair in clusters for API v2 https://review.openstack.org/539856 | 01:55 |
| *** dave-mccowan has quit IRC | 02:57 | |
| *** hoonetorg has joined #openstack-sahara | 03:28 | |
| *** Bhujay has joined #openstack-sahara | 04:04 | |
| *** links has joined #openstack-sahara | 04:24 | |
| *** pcaruana has joined #openstack-sahara | 04:50 | |
| *** pcaruana has quit IRC | 05:18 | |
| *** pcaruana has joined #openstack-sahara | 06:06 | |
| *** Bhujay has quit IRC | 06:13 | |
| *** masber has joined #openstack-sahara | 06:35 | |
| *** masuberu has quit IRC | 06:38 | |
| *** tosky has joined #openstack-sahara | 06:40 | |
| *** masuberu has joined #openstack-sahara | 06:41 | |
| *** masber has quit IRC | 06:45 | |
| *** masuberu has quit IRC | 06:54 | |
| *** masuberu has joined #openstack-sahara | 06:54 | |
| *** masuberu has quit IRC | 06:55 | |
| *** masuberu has joined #openstack-sahara | 06:55 | |
| *** masuberu has quit IRC | 06:57 | |
| *** masuberu has joined #openstack-sahara | 06:57 | |
| *** masuberu has quit IRC | 06:58 | |
| *** masuberu has joined #openstack-sahara | 06:59 | |
| *** rcernin has quit IRC | 07:00 | |
| *** masuberu has quit IRC | 07:00 | |
| *** masuberu has joined #openstack-sahara | 07:00 | |
| *** masuberu has quit IRC | 07:01 | |
| *** masuberu has joined #openstack-sahara | 07:01 | |
| *** masuberu has quit IRC | 07:02 | |
| *** masuberu has joined #openstack-sahara | 07:03 | |
| *** tesseract has joined #openstack-sahara | 07:07 | |
| openstackgerrit | Merged openstack/puppet-sahara master: fix tox python3 overrides https://review.openstack.org/573563 | 08:33 |
| *** links has quit IRC | 09:08 | |
| *** mtsv has joined #openstack-sahara | 09:21 | |
| *** links has joined #openstack-sahara | 09:25 | |
| *** masuberu has quit IRC | 10:10 | |
| *** links has quit IRC | 10:12 | |
| *** links has joined #openstack-sahara | 10:25 | |
| mtsv | Hi guys, I'm trying to set up a Cloudera CDH 5.11.0 cluster on openstack pike using the horizon UI and the cluster creation guide. My cluster passes validation but then fails with the following error https://hastebin.com/rugigoxaxo.sql Any ideas what I might be missing? | 11:25 |
| tosky | uhm, most likely a configuration problem; how did you configure the cluster and the node groups? | 11:54 |
| mtsv | Attempting to create the cluster using the cli interface resulted in the following error: "internal endpoint for data-processing service in RegionOne region not found" I guess this could be the problem. | 12:09 |
| tosky | that may be a different problem - I guess that the first creation happened through the web UI? | 12:10 |
| tosky | anyway, that error about internal endpoint missing points to something missing in the configuration of the service | 12:11 |
| tosky | how was sahara (and openstack) deployed? | 12:11 |
| mtsv | The first creation was through the web UI. I deployed openstack using kolla. | 12:14 |
| tosky | oh, my practical knowledge of kolla is basically 0 | 12:14 |
| tosky | I can only suggest you to ask on kolla channels about this error | 12:14 |
| tosky | regarding the first part, I was asking about the setting of the cluster | 12:15 |
| tosky | but in fact there may be other configuration issues | 12:15 |
| mtsv | I followed this guide https://blog.cloudera.com/blog/2015/05/how-to-get-started-with-cdh-on-openstack-with-sahara/ to set up the cluster | 12:18 |
| tosky | did you create the image yourself with sahara-image-elements? | 12:20 |
| mtsv | Yes, I also tried downloading one of the prebuilt images from cloudera but it resulted in the same error. | 12:20 |
| tosky | the prebuilt images are up until ocata, we don't distribute them anymore | 12:21 |
| mtsv | The endpoint error is confusing as the endpoint does appear when I do "openstack endpoint list" "| 1002fc0e21d14b58975b291df3a78497 | RegionOne | sahara | data_processing| True| internal| http://192.168.137.86:8386/v1.1/%(tenant_id)s |" | 12:21 |
| tosky | data_processing vs data-processing | 12:21 |
| mtsv | ah | 12:22 |
| tosky | so kolla has some issues :) | 12:22 |
| mtsv | As for the prebuilt image I figured I'd give it a try on pike anyway. I used version 5.9 of the plugin with a 5.9 image. Would that not work? | 12:22 |
| tosky | I advise to rebuild the image anyway | 12:23 |
| tosky | but the guide is not correct | 12:23 |
| tosky | apart from the main address of the repository is on git.openstack.org, not github, you should switch to the stable/pike branch before building the image | 12:23 |
| mtsv | Okay, I'll give it a try, thanks. | 12:23 |
| tosky | and no need to export a variable to select the version of CDH, there is a switch for the CLI command | 12:23 |
| tosky | see https://docs.openstack.org/sahara/latest/user/cdh-imagebuilder.html | 12:24 |
| tosky | also, if you have some time, and the generated image does not work, try to build it with the other operating system (ubuntu vs centos 7) | 12:25 |
| *** hoonetorg has quit IRC | 12:37 | |
| *** dave-mccowan has joined #openstack-sahara | 12:41 | |
| *** hoonetorg has joined #openstack-sahara | 12:47 | |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Adding CDH 5.13 https://review.openstack.org/573802 | 13:03 |
| *** dave-mccowan has quit IRC | 13:21 | |
| *** dave-mccowan has joined #openstack-sahara | 13:59 | |
| tellesnobrega | jeremyfreudberg, are you around? | 14:03 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Adding CDH 5.13 https://review.openstack.org/573802 | 14:06 |
| tellesnobrega | tosky, about boot from volume | 14:07 |
| tosky | yes? | 14:08 |
| tellesnobrega | the client part, it worked setting to None, but I'm not convinced that is the right way to go | 14:08 |
| tellesnobrega | looking at other properties, is_protected and some other | 14:09 |
| tellesnobrega | others | 14:09 |
| tellesnobrega | they set default to False | 14:09 |
| tosky | they didn't have the problem of keeping compatibility with the older sahara | 14:10 |
| tosky | maybe | 14:10 |
| tellesnobrega | huum, they are probably there from the start | 14:11 |
| tosky | my understanding is this one of the things that can be solved with microversioning, but not on v1 | 14:11 |
| tellesnobrega | makes sense | 14:13 |
| *** jeremyfreudberg has joined #openstack-sahara | 14:31 | |
| jeremyfreudberg | hey guys -- does my comment on the cdh 5.13 make sense? i was having trouble finding the right way to explain it | 14:31 |
| tosky | oh, yes, it makes sense | 14:34 |
| tosky | it's pretty straightforward | 14:34 |
| tosky | aka: read only values, extracted by magic from the plugin, used to present the keys and the defaults in the UI | 14:34 |
| jeremyfreudberg | yep, actually the concept is simple enough -- but the code which implements that magic is... magic | 14:35 |
| *** hoonetorg has quit IRC | 14:35 | |
| jeremyfreudberg | actually, what is interesting that in the cdh validation of node processes hive metastore is not a requirement of sentry | 14:40 |
| jeremyfreudberg | actually, let me think about that, it might not be as interesting/not interesting as i thought | 14:41 |
| jeremyfreudberg | yeah, it doesn't matter | 14:43 |
| tosky | do you think that we could validate the validity of a sentry service against that configuration option? | 14:46 |
| jeremyfreudberg | short answer: yes. long answer: | 14:48 |
| jeremyfreudberg | remember this patch: https://review.openstack.org/#/c/572198/1/sahara/service/api/v2/clusters.py | 14:48 |
| jeremyfreudberg | first step is "recommend configs" | 14:48 |
| tosky | yep | 14:48 |
| jeremyfreudberg | second step is "validate plugin" (i.e. node processes) | 14:48 |
| jeremyfreudberg | so there is a choice | 14:49 |
| jeremyfreudberg | A) in the validation of sentry, see if the user has passed in the special hive config value explicitly | 14:49 |
| jeremyfreudberg | or | 14:49 |
| jeremyfreudberg | B) | 14:49 |
| jeremyfreudberg | change the recommedation step to emit the special hive config value | 14:50 |
| tellesnobrega | I think B is the safest choice | 14:50 |
| jeremyfreudberg | yeah, B is what i'm looking into now | 14:51 |
| jeremyfreudberg | the code is a bit convoluted | 14:51 |
| tellesnobrega | if the config is a must, we should not give the user the option to screw it up | 14:51 |
| jeremyfreudberg | agreed | 14:51 |
| tosky | so now the question is: follow-up patch? The basic CDH 5.13 patch is big enough | 14:51 |
| jeremyfreudberg | a followup patch is fine by me | 14:52 |
| jeremyfreudberg | i do have one more thought, though | 14:52 |
| tellesnobrega | go ahead | 14:53 |
| jeremyfreudberg | should we emit hive_enable_db_notification=true only when sentry is present? or should we emit it as true always? | 14:53 |
| jeremyfreudberg | (By emit, I mean, the config-recommending step that is part of the plugin code) | 14:53 |
| tosky | so far, I only know that it's a new requirement for sentry | 14:54 |
| tosky | I don't know how to easily check if it's a requirement for more cases | 14:54 |
| tellesnobrega | does it affect anything else? | 14:54 |
| tosky | apart from downloading the cloudera documentation and running a diff | 14:54 |
| jeremyfreudberg | err, i guess it uses a little more disk space to be storing those notifications... but i'm not sure if it really matters | 14:56 |
| jeremyfreudberg | let's make it true always, for simplicity | 14:58 |
| jeremyfreudberg | because then i can edit the json, too, to reflect it nicely in the ui | 14:58 |
| jeremyfreudberg | otherwise there are too many branches of options | 14:58 |
| jeremyfreudberg | if that makes sense | 14:58 |
| tellesnobrega | it works for me | 14:59 |
| jeremyfreudberg | cool | 14:59 |
| tellesnobrega | and if we see that is causing trouble later we can always revisit it | 14:59 |
| jeremyfreudberg | so i guess it's good to merge the patch then | 15:00 |
| jeremyfreudberg | and i will do a follow-up once i figure things out | 15:00 |
| tellesnobrega | thanks | 15:00 |
| tellesnobrega | you can do the honors and +A it please | 15:00 |
| jeremyfreudberg | tosky: do you want to wait for the experimntal job to finish again? or do we get the idea at this point | 15:00 |
| tosky | jeremyfreudberg: I tested it manually (image generation and cluster creation) with the patch 4, which is basically the same as the last version | 15:01 |
| tellesnobrega | heading out for lunch | 15:02 |
| jeremyfreudberg | tosky: sounds good | 15:02 |
| jeremyfreudberg | tellesnobrega: enjoy it | 15:02 |
| tellesnobrega | I will be back for the downstream meeting | 15:02 |
| tellesnobrega | thanks | 15:02 |
| tellesnobrega | see you in a bit | 15:02 |
| tosky | thanks! | 15:02 |
| *** links has quit IRC | 15:36 | |
| *** tesseract has quit IRC | 16:33 | |
| openstackgerrit | Merged openstack/sahara master: Adding CDH 5.13 https://review.openstack.org/573802 | 17:35 |
| jeremyfreudberg | tosky: if you're still around, i have some doubts about this https://github.com/openstack/sahara/blob/master/sahara/plugins/cdh/v5_9_0/resources/images/common/install_extjs | 17:46 |
| tosky | jeremyfreudberg: yes, it's buggy | 17:46 |
| tosky | «if [ ! -f "${EXTJS_DESTINATION_DIR}/${extjs_archive}" ]; then» does not work when extjs_archive=/tmp/$(basename $EXTJS_DOWNLOAD_URL) | 17:47 |
| tosky | it should be a bit different | 17:47 |
| jeremyfreudberg | so in my case what i'm seeing is that when this code gets run again during cluster validation(?) the zip file extraction hangs because it is prompting the user if it wants to overwrite some file | 17:48 |
| tosky | yes, exactly | 17:54 |
| tosky | one of my pending task is to open the story for it | 17:54 |
| tosky | feel free to open it of course :) | 17:54 |
| jeremyfreudberg | i'll let you do the honors | 17:55 |
| tosky | did you generate the image with sahara-image-elements? | 17:55 |
| jeremyfreudberg | tosky: i'm being lazy and using the cdh 5.9 ocata image, but it is supposed to be almost the same as the 5.9 image pack | 17:55 |
| tosky | almost | 17:56 |
| jeremyfreudberg | have you seen this zip file problem when launching image pack images? | 17:57 |
| tosky | a double error makes that working with image pack, because the content is not uncompressed | 17:57 |
| tosky | only with sie images | 17:57 |
| jeremyfreudberg | hmm, i see | 17:57 |
| jeremyfreudberg | to be honest i don't really use image pack images ever, because i rarely use vendor plugins | 17:58 |
| tosky | yeah, we need pack for vanilla too | 18:00 |
| *** jeremyfreudberg has quit IRC | 18:04 | |
| *** openstackgerrit has quit IRC | 18:04 | |
| *** dave-mccowan has quit IRC | 18:15 | |
| *** hoonetorg has joined #openstack-sahara | 19:04 | |
| *** dave-mccowan has joined #openstack-sahara | 20:07 | |
| *** openstackgerrit has joined #openstack-sahara | 20:15 | |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Updating plugins status for Rocky https://review.openstack.org/566130 | 20:15 |
| tellesnobrega | tosky, on boot from volume, should we have it only for api v2 or do we want on v1 as well? | 20:30 |
| tosky | tellesnobrega: aka "let's fix the problem with the client"? | 20:33 |
| tosky | :D | 20:33 |
| tellesnobrega | a little more than that | 20:33 |
| tellesnobrega | should we add new features to v1 at this point? | 20:33 |
| tellesnobrega | but I just tested locally and it is creating templates with boot from volume flag | 20:34 |
| tellesnobrega | using CLI | 20:34 |
| tellesnobrega | trying UI now | 20:34 |
| tosky | uhm, it depends on the feature and the expected time before we can make v2 stable | 20:34 |
| tosky | if it's Stein, maybe | 20:34 |
| tosky | we can talk about it tomorrow | 20:35 |
| tellesnobrega | I will keep the code in for now | 20:35 |
| tellesnobrega | and we talk more tomorrow | 20:35 |
| tellesnobrega | good to see it working :) | 20:35 |
| tellesnobrega | UI is looking good to | 20:37 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara-dashboard master: Boot from volume https://review.openstack.org/575227 | 20:49 |
| *** openstackgerrit has quit IRC | 21:04 | |
| *** openstackgerrit has joined #openstack-sahara | 21:31 | |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara-dashboard master: Boot from volume https://review.openstack.org/575227 | 21:31 |
| *** dave-mccowan has quit IRC | 21:43 | |
| openstackgerrit | Merged openstack/python-saharaclient master: Reflect change to multiple clusters creation https://review.openstack.org/574861 | 21:49 |
| openstackgerrit | Merged openstack/python-saharaclient master: Support of the improved force-delete in client https://review.openstack.org/574854 | 21:51 |
| *** rcernin has joined #openstack-sahara | 22:26 | |
| *** rcernin is now known as rcernin|brekkie | 22:27 | |
| *** rcernin|brekkie is now known as rcernin | 22:27 | |
| *** dave-mccowan has joined #openstack-sahara | 23:22 | |
| *** tosky has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!