| openstackgerrit | Emilien Macchi proposed openstack/puppet-sahara stable/pike: Prepare for Pike post release https://review.openstack.org/534533 | 00:41 |
|---|---|---|
| *** dave-mccowan has joined #openstack-sahara | 00:49 | |
| *** tosky has quit IRC | 00:54 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-sahara stable/ocata: Prepare for Ocata post release https://review.openstack.org/534568 | 00:59 |
| *** shuyingya has joined #openstack-sahara | 01:17 | |
| *** shuyingy_ has joined #openstack-sahara | 01:27 | |
| *** shuyingya has quit IRC | 01:30 | |
| *** dave-mccowan has quit IRC | 01:42 | |
| *** dave-mccowan has joined #openstack-sahara | 01:43 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara-dashboard master: Support of S3 binaries in dashboard https://review.openstack.org/530672 | 02:08 |
| *** dave-mccowan has quit IRC | 02:19 | |
| openstackgerrit | Merged openstack/sahara-dashboard master: Fix the quickstart url and participate url https://review.openstack.org/534141 | 02:30 |
| *** shuyingya has joined #openstack-sahara | 02:47 | |
| *** shuying__ has joined #openstack-sahara | 02:48 | |
| *** shuyingy_ has quit IRC | 02:49 | |
| *** shuyingya has quit IRC | 02:51 | |
| *** dave-mccowan has joined #openstack-sahara | 02:52 | |
| openstackgerrit | Merged openstack/sahara master: [APIv2]Enable APIv2, experimentally https://review.openstack.org/524934 | 03:05 |
| openstackgerrit | Merged openstack/sahara master: add bugs link in README.rst https://review.openstack.org/533443 | 03:05 |
| openstackgerrit | sunqingliang proposed openstack/sahara-dashboard master: fix wrong url in bindep.txt, replate http with https https://review.openstack.org/534606 | 03:30 |
| *** caowei has joined #openstack-sahara | 03:46 | |
| *** caowei has quit IRC | 04:25 | |
| *** caowei has joined #openstack-sahara | 04:25 | |
| *** dave-mccowan has quit IRC | 04:32 | |
| *** links has joined #openstack-sahara | 04:49 | |
| *** caowei has quit IRC | 04:55 | |
| *** alinanesterova has joined #openstack-sahara | 05:32 | |
| *** caowei has joined #openstack-sahara | 05:34 | |
| *** caowei has quit IRC | 05:59 | |
| *** caowei has joined #openstack-sahara | 06:01 | |
| *** caowei has quit IRC | 06:03 | |
| openstackgerrit | sunqingliang proposed openstack/sahara-dashboard master: lack of comma https://review.openstack.org/534636 | 06:11 |
| *** caowei has joined #openstack-sahara | 06:32 | |
| openstackgerrit | ShangXiao proposed openstack/sahara-dashboard master: Update url links in doc file of sahara-dashboard https://review.openstack.org/534641 | 06:33 |
| *** pgadiya has joined #openstack-sahara | 06:39 | |
| *** caowei has quit IRC | 07:17 | |
| *** caowei has joined #openstack-sahara | 07:21 | |
| *** caowei has quit IRC | 07:25 | |
| *** gokhan_ has quit IRC | 07:29 | |
| *** gokhan_ has joined #openstack-sahara | 07:46 | |
| *** pcaruana has joined #openstack-sahara | 07:48 | |
| *** pcaruana has quit IRC | 07:55 | |
| *** pcaruana has joined #openstack-sahara | 07:55 | |
| gokhan_ | hi tellesnobrega, we can not create cluster template on sahara dashboard because of DATA_UPLOAD_MAX_NUMBER_FIELDS. | 08:10 |
| *** hoonetorg has quit IRC | 08:10 | |
| gokhan_ | error is : [Wed Jan 17 08:03:43.324827 2018] [wsgi:error] [pid 18712:tid 139951544997632] DeprecationWarning: Passing authentication parameters to saharaclient is deprecated. Please construct and pass an authenticated session object directly.The number of GET/POST parameters exceeded settings.DATA_UPLOAD_MAX_NUMBER_FIELDS. | 08:10 |
| gokhan_ | tellesnobrega, ^^ | 08:10 |
| *** hoonetorg has joined #openstack-sahara | 08:26 | |
| ltosky[m] | gokhan_: this was fixed with https://review.openstack.org/500193/ | 08:27 |
| ltosky[m] | which version of openstack are you using? | 08:27 |
| *** caowei has joined #openstack-sahara | 08:28 | |
| *** tesseract has joined #openstack-sahara | 08:29 | |
| *** caowei has quit IRC | 08:39 | |
| *** rcernin has quit IRC | 08:45 | |
| *** caowei has joined #openstack-sahara | 08:50 | |
| *** shuying__ is now known as shuyingya | 09:09 | |
| *** alinanesterova has quit IRC | 09:10 | |
| openstackgerrit | Merged openstack/sahara master: Rename 'SAHARA_AUTO_IP_ALLOCATION_ENABLED' config parameter https://review.openstack.org/531069 | 09:14 |
| *** alinanesterova has joined #openstack-sahara | 09:17 | |
| *** caowei has quit IRC | 09:20 | |
| *** caowei has joined #openstack-sahara | 09:21 | |
| *** alinanesterova has quit IRC | 09:21 | |
| *** rcernin has joined #openstack-sahara | 09:40 | |
| *** caowei has quit IRC | 10:01 | |
| *** tosky has joined #openstack-sahara | 10:23 | |
| gokhan_ | ltosky[m], yes I see this commit but again get same error.sahara dashboard version is 7.0.1.dev | 10:32 |
| gokhan_ | my openstack version is pike | 10:32 |
| tosky | gokhan_: so try to manually apply the patch and see if it fixes the problem | 10:39 |
| tosky | the patch was applied only to master, so queens | 10:39 |
| tosky | if it works for you, we can backport it | 10:40 |
| gokhan_ | tosky, it was alreday applied on pike because I controlled it on sahara-dashboard code and I see this commit. but it doesn't work unfortunately | 10:45 |
| tosky | gokhan_: right, it was installed; is the file _12_toggle_data_upload_max_number_fields.py really copied? | 10:47 |
| tosky | or does it work if you increase the value? | 10:48 |
| tosky | can you see the problem with any node group template, or specifically bigger ones? | 10:49 |
| gokhan_ | tosky, ı see this file under the sahara_dashboard/local_settings.d/_12_toggle_data_upload_max_number_fields.py | 10:49 |
| gokhan_ | tosky, on node group template it is working successfully | 10:50 |
| tosky | uh, cluster templates | 10:50 |
| gokhan_ | when I try to create cluster template, it is failed | 10:50 |
| tosky | gokhan_: ok, but do you see the file deployed on the system, not just in the source code? | 10:50 |
| tosky | if it is a devstack deployment, it should be in /opt/stack/new/horizon/openstack_dashboard/local/local_settings.d/ | 10:52 |
| gokhan_ | tosky, I have a HA environment which is created by openstack ansible | 10:53 |
| gokhan_ | I am looking now | 10:53 |
| tosky | oh, maybe the ansible installer didn't copy the file | 10:53 |
| gokhan_ | tosky, yes you are right it didn't copy this file to openstack_dashboard/local/local_setings.d | 10:56 |
| tosky | ah! | 10:56 |
| gokhan_ | tosky, how sahara dashboard copies this file to openstack_dashboard | 10:58 |
| gokhan_ | tosky, may be need a patch on openstack ansible-os_saha | 10:58 |
| gokhan_ | *openstack-ansible-os_sahara | 10:58 |
| tosky | gokhan_: but the code is there: https://review.openstack.org/#/c/516748/ | 10:59 |
| tosky | gokhan_: when did you deploy this openstack installation? Maybe openstack-ansible-os_horizon was not up-to-date | 10:59 |
| tosky | (or maybe it is broken :) | 10:59 |
| tosky | openstack-ansible-os_sahara does not install the dashboard; the horizon module installs all the dashboards (I'm not sure I agree with that, but I don't contribute to that project) | 11:00 |
| gokhan_ | tosky, ok I am controlling horizon role | 11:01 |
| tosky | and also please check if copying the file (and restarting apache) really fixes the issue :) | 11:01 |
| gokhan_ | tosky, yes I installed my environment 2 monts ago and see that it is previous version. thanks for help. | 11:03 |
| tosky | oh, good, I like when a problem is already solved! | 11:04 |
| gokhan_ | tosky, yes it is already solved :) also I entered a bug for ambari centos 7 images. hdp plugin can only work with centos6 images | 11:04 |
| gokhan_ | tosky, https://bugs.launchpad.net/sahara/+bug/1740133 | 11:05 |
| openstack | Launchpad bug 1740133 in Sahara "when try to create HDP cluster with centos7 image, get timeout error on registration of hosts event." [Undecided,New] | 11:05 |
| tosky | gokhan_: there was an bug reported on HDP 2.5 some time ago, but it was not reproduced and closed | 11:06 |
| tosky | but then I think that you hit the same | 11:06 |
| tosky | let me find the link | 11:06 |
| tosky | gokhan_: does it look the same? http://bugs.launchpad.net/sahara/+bug/1672218/ | 11:07 |
| openstack | Launchpad bug 1672218 in Sahara "Using ambari 2.4 with 2.4.0.1 image - inconsistent server/agent version" [Undecided,Invalid] | 11:07 |
| gokhan_ | tosky, no it is not same | 11:09 |
| gokhan_ | I created image | 11:10 |
| gokhan_ | and I am using ambari 2.4.2.0 | 11:10 |
| tosky | gokhan_: it may be the same | 11:10 |
| gokhan_ | it fails on registration of hosts | 11:10 |
| tosky | yes, like the other bug | 11:10 |
| tosky | "It fails to deploy at step "Wait registration of hosts" in Horizon." | 11:11 |
| tosky | the reporter of the other bug didn't report the content of sahara-engine.log, but he did report the content of ambari-server.log on the ambari host | 11:12 |
| tosky | I pretty much think that this is the same error | 11:12 |
| gokhan_ | tosky, I am not sure but I created sahara image with ambari 2.4.2.0 | 11:16 |
| gokhan_ | may be your new image generation tool can solve this situation | 11:16 |
| gokhan_ | I am following your commits about ambari images :) | 11:16 |
| tosky | gokhan_: right, but for ambari is not 100% ready/tested, and we should have something working in the meantime | 11:17 |
| tosky | so if you happen to retest that image and grab the logs from the ambari node, that would be great! | 11:17 |
| openstackgerrit | Shu Yingya proposed openstack/sahara-tests master: Fix APIExcetion if cinder is not installed https://review.openstack.org/534737 | 12:02 |
| tosky | APIException :) | 12:06 |
| *** alinanesterova has joined #openstack-sahara | 12:06 | |
| *** alinanesterova has quit IRC | 12:11 | |
| *** pgadiya has quit IRC | 12:12 | |
| *** rcernin has quit IRC | 12:14 | |
| *** pgadiya has joined #openstack-sahara | 12:15 | |
| *** pgadiya has quit IRC | 12:21 | |
| openstackgerrit | Shu Yingya proposed openstack/sahara-tests master: Fix APIException if cinder is not installed https://review.openstack.org/534737 | 12:21 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-extra master: Enable also building Oozie for 2.7.1 https://review.openstack.org/532690 | 12:27 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-extra master: WIP Add a native Zuul v3 build job https://review.openstack.org/532691 | 12:27 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/python-saharaclient master: Beginnings of APIv2 support in client https://review.openstack.org/533066 | 12:32 |
| *** pgadiya has joined #openstack-sahara | 12:33 | |
| *** dave-mccowan has joined #openstack-sahara | 12:45 | |
| tellesnobrega | jeremyfreudberg, I added a new patch to APIv2 stuff, can you take a look and see if it makes sense | 12:53 |
| tellesnobrega | tosky, please take a look too ^ | 12:54 |
| tellesnobrega | shuyingya, also ^ | 12:54 |
| shuyingya | tellesnobrega, going back home. will review it ASAP. | 13:01 |
| tellesnobrega | thanks shuyingya | 13:01 |
| shuyingya | https://github.com/openstack/sahara-tests/blob/master/sahara_tempest_plugin/tests/clients/test_job_executions.py#L313-L316 tellesnobrega, tosky. tempest run has an error "AttributeError: type object 'JobExecutionTest' has no attribute ‘client’". What do you think about removing it directly? or do you have any idea about fixing it? | 13:06 |
| tosky | shuyingya: always? Or only when there is an error? | 13:07 |
| shuyingya | always. | 13:07 |
| shuyingya | job execution test case is skipped. | 13:08 |
| shuyingya | I think this method has syntax error. | 13:08 |
| tosky | well, client is an attribute of the instance for sure, but it should work for the class too once initialized | 13:09 |
| shuyingya | yes, that is it. | 13:09 |
| tosky | anyway, there is a bug for that, and there were some (proposed but unsuccessful) patches | 13:09 |
| tosky | I'd rather had it fixed | 13:10 |
| tosky | just... priorities | 13:10 |
| shuyingya | thanks. | 13:10 |
| shuyingya | It's quite hard for me to fix it. happy to hear that you will fix it. | 13:11 |
| shuyingya | don't worry. It's not so urgent | 13:11 |
| shuyingya | bye. I need gooooo back home. :) and will review apiv2 client patch when I arrived | 13:12 |
| tosky | bye! | 13:12 |
| *** shuyingya has quit IRC | 13:13 | |
| tosky | tellesnobrega: one thing I didn't get from Jeremy's comment on devstack: if we don't deploy the new endpoint, how does it work? | 13:51 |
| openstackgerrit | Merged openstack/puppet-sahara master: Prepare for Queens Milestone 3 https://review.openstack.org/534494 | 13:52 |
| tellesnobrega | I think he meant that we can add the endpoint creation on devstack later, this patch should work if installed by pip, and we can always add the endpoint manually on devstack for testing | 13:54 |
| tosky | it works because we don't need new endpoints for now due to some magic? That's the part I didn't get well | 13:55 |
| gokhan_ | tosky, ok I will again retest | 13:56 |
| tosky | gokhan_: thanks! | 13:58 |
| *** pgadiya has quit IRC | 13:58 | |
| openstackgerrit | Merged openstack/puppet-sahara master: Remove pinned versions from test-requirements.txt https://review.openstack.org/534112 | 13:59 |
| tellesnobrega | tosky, what I understood, and I might be wrong, is that with the saharaclient version discovery it should just work | 14:00 |
| tellesnobrega | but we can ask jeremy for a better explanation | 14:00 |
| tosky | ack | 14:00 |
| tosky | but anyway, your patch failed :P | 14:00 |
| tellesnobrega | it did | 14:00 |
| tellesnobrega | it is considering saharaclient.api.v2 as third-party import | 14:04 |
| tellesnobrega | not sure why | 14:04 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/python-saharaclient master: Beginnings of APIv2 support in client https://review.openstack.org/533066 | 14:20 |
| tellesnobrega | tosky, fixed ^ | 14:20 |
| tosky | uh, missing module init | 14:21 |
| tosky | oki | 14:21 |
| tellesnobrega | exactly | 14:21 |
| *** jeremyfreudberg has joined #openstack-sahara | 14:25 | |
| jeremyfreudberg | guys, here's what i meant | 14:25 |
| jeremyfreudberg | a devstack might have client installed from pip or from git. the current version of client needs a v1.1 endpoint in catalog. so if i change the endpoint creation from versioned to unversioned it will break until minimum version of client is changed | 14:27 |
| jeremyfreudberg | after we merge the v2 client patch, and you request saharaclient to be from git master not from pip, then you can still do v2 stuff with a v1.1 endpoint in the catalog | 14:27 |
| jeremyfreudberg | ^ tellesnobrega tosky | 14:28 |
| tosky | but then, if I want to test v2, with the client patch (or without if I would use curl directly), do I simply need to create the unversioned endpoints manually? | 14:29 |
| jeremyfreudberg | saharaclient.client.Client('2', ...) or saharaclient.api.client.ClientV2(...) will mutate any endpoint it finds in the catalog into a proper v2 endpoint | 14:31 |
| tosky | ah | 14:31 |
| tosky | so basically the "form" of the endpoint does not matter really | 14:32 |
| tosky | too magic, I will need to study it :) | 14:32 |
| tellesnobrega | jeremyfreudberg, let me see if I fully understand. Once the client patch merges, we only need to force devstack to install saharaclient from master and we are using apiv2? | 14:32 |
| jeremyfreudberg | tosky: yes, exactly, the form of the endpoint does not matter. and my magic is only slightly magical, i'm still hoping monty will rewrite it with the real magic | 14:33 |
| jeremyfreudberg | tellesnobrega: yes, when client patch merges then sahara master + client master will give a functioning v2 deployment | 14:34 |
| tellesnobrega | jeremyfreudberg, awesome | 14:34 |
| tellesnobrega | jeremyfreudberg, also, take a look at the patch I sent, if it makes sense to you | 14:35 |
| jeremyfreudberg | tellesnobrega, yes, i'll review it soon | 14:35 |
| tellesnobrega | thanks | 14:35 |
| *** shuyingy_ has joined #openstack-sahara | 14:48 | |
| jeremyfreudberg | tellesnobrega, it's looking good, i have a few suggestions but i think it's faster if i just write the new patchset | 14:49 |
| tellesnobrega | jeremyfreudberg, please do | 14:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-sahara stable/newton: Prepare for Newton post release https://review.openstack.org/534801 | 15:03 |
| *** links has quit IRC | 15:11 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Beginnings of APIv2 support in client https://review.openstack.org/533066 | 15:55 |
| jeremyfreudberg | that's it for me for today, unfortunately | 16:02 |
| jeremyfreudberg | i'm sure the patch is still imperfect, i think once we have sahara-tests using the v2 client we'll be able to tell what to prioritize | 16:03 |
| *** jeremyfreudberg has quit IRC | 16:03 | |
| openstackgerrit | Merged openstack/sahara master: Changing expected value to job_template_id https://review.openstack.org/534422 | 16:42 |
| *** pcaruana has quit IRC | 16:46 | |
| *** shuyingy_ has quit IRC | 17:05 | |
| *** tesseract has quit IRC | 17:21 | |
| *** ukaynar has joined #openstack-sahara | 17:31 | |
| *** openstackstatus has quit IRC | 18:40 | |
| *** openstackstatus has joined #openstack-sahara | 18:42 | |
| *** ChanServ sets mode: +v openstackstatus | 18:42 | |
| *** ukaynar has quit IRC | 20:15 | |
| tellesnobrega | tosky, looks like swift worked | 20:18 |
| *** ukaynar has joined #openstack-sahara | 20:19 | |
| tosky | tellesnobrega: so it was a leftover | 20:23 |
| tosky | good | 20:23 |
| tellesnobrega | basically removed the v2.0 and it worked, not sure about the user_domain_proxy stuff | 20:24 |
| tellesnobrega | it currently returns v3/auth, but I believe that it will break | 20:24 |
| tellesnobrega | it should be always v3 | 20:24 |
| tosky | I'm not even sure if a v3 url should always end by v3 | 20:27 |
| tellesnobrega | hmm | 20:30 |
| tellesnobrega | https://github.com/openstack/sahara/blob/master/sahara/swift/utils.py#L39 | 20:36 |
| tellesnobrega | tosky, basically changed that to v3 and other place that requests from swiftclient to request api v3 | 20:37 |
| -openstackstatus- NOTICE: Zuul will be offline for a few minutes; existing changes will be re-enqueued; approvals during the downtime will need to be re-added. | 20:39 | |
| tosky | tellesnobrega: on the other side, I just discovered that, despite the unit tests, sahara-templates seems to be broken | 20:47 |
| tosky | apparently (from initial investigation) a change in validation | 20:47 |
| tellesnobrega | sahara-tempest? | 20:47 |
| tosky | sahara-templates | 20:48 |
| tosky | default templates | 20:48 |
| tellesnobrega | oh | 20:55 |
| tellesnobrega | I see, I will take a look into them | 20:55 |
| *** ukaynar has quit IRC | 20:55 | |
| tellesnobrega | tosky, about the file transfer bug, I was able to get a 115MB file transfer from 7 minutes to about 3 | 20:55 |
| tosky | which is already better, even if not optimal | 20:56 |
| tellesnobrega | yes, optimal would be getting to maximum 2 minutes which is the default timeout | 20:56 |
| tellesnobrega | but we can increase timeout to 5 minutes if necessary | 20:56 |
| tellesnobrega | so we can work with even bigger files | 20:56 |
| tosky | or at least in line with a normal transfer between the same hosts | 20:56 |
| tosky | but does if it scales linearly, 5 minutes will be ~250 MB at most | 20:57 |
| tellesnobrega | hmm, we have to work out the best option | 21:04 |
| tellesnobrega | let me see if this 3 minute is reproducible | 21:04 |
| tosky | how much does it take a normal scp between the nodes? | 21:06 |
| tellesnobrega | should be about 3 or 4 seconds IIRC | 21:06 |
| tellesnobrega | the problem is with paramiko sftp file transfer | 21:07 |
| *** alinanesterova has joined #openstack-sahara | 21:07 | |
| tellesnobrega | I'm removing buffering and setting pipelined to see if it improves | 21:07 |
| tellesnobrega | tosky, maybe the case is to see if there is anything better than paramiko for us to use | 21:07 |
| tosky | if I remember the discussion, removing the buffering and setting pipeline was one of the solutions, hopefully the less impacting | 21:08 |
| tellesnobrega | yes | 21:08 |
| tellesnobrega | this was the combination that took it down to 3 minutes | 21:08 |
| tellesnobrega | trying to reproduce it now | 21:08 |
| tellesnobrega | went down to less than 3 minutes | 21:11 |
| tellesnobrega | I believe this is the right way to go | 21:11 |
| tosky | I wonder if it scales linearly due to some initial cost (and it speeds up later) | 21:11 |
| tellesnobrega | I will try bigger files as well | 21:11 |
| tellesnobrega | and let you know | 21:11 |
| *** alinanesterova has quit IRC | 21:12 | |
| *** rcernin has joined #openstack-sahara | 22:02 | |
| *** ukaynar has joined #openstack-sahara | 22:38 | |
| *** ukaynar has quit IRC | 22:39 | |
| openstackgerrit | inspurericzhang proposed openstack/sahara-dashboard master: horizon misspelling in README.rst https://review.openstack.org/534976 | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!