| *** zhurong has quit IRC | 00:04 | |
| *** zhurong has joined #senlin | 01:10 | |
| *** elynn has joined #senlin | 01:19 | |
| *** elynn_ has joined #senlin | 01:26 | |
| *** elynn has quit IRC | 01:26 | |
| *** Drago2 has joined #senlin | 01:39 | |
| *** Drago has quit IRC | 01:39 | |
| *** shu-mutou-AFK is now known as shu-mutou | 01:46 | |
| *** yanyanhu has joined #senlin | 01:58 | |
| *** eldon has joined #senlin | 02:30 | |
| eldon | 2016-08-18 02:28:16.996 237 INFO rally.task.runner [-] Task c3c6 | ITER: 66 END: Error InternalServerError: Remote error: RemoteError Remote error: InvalidRequestError A transaction is already begun. Use subtransactions=True to allow subtransactions. | 02:32 |
|---|---|---|
| eldon | I used newest code to test, it also has such error. | 02:32 |
| *** yuanying_ has quit IRC | 02:46 | |
| yanyanhu | hi, eldon, I'm in a meeting, could you plz paste the log of senlin-engine and I will check it later, thanks | 02:49 |
| *** yuanying has joined #senlin | 02:52 | |
| openstackgerrit | Merged openstack/senlin: Fix node join behavior https://review.openstack.org/356301 | 02:53 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Fix nova_v2 fake driver https://review.openstack.org/356845 | 03:01 |
| *** yuanying has quit IRC | 03:18 | |
| *** yuanying has joined #senlin | 03:19 | |
| *** yuanying has quit IRC | 03:42 | |
| *** yuanying has joined #senlin | 03:42 | |
| *** yuanying has quit IRC | 03:43 | |
| *** yuanying has joined #senlin | 03:48 | |
| *** Drago2 has quit IRC | 04:11 | |
| xuhaiwei | hi Qiming | 04:57 |
| yanyanhu | hi, xuhaiwei, Qiming is not here today. | 05:22 |
| xuhaiwei | ok, Tacker's PTL talked to me today, tacker may use Senlin to manage vnf cluster, yanyanhu | 05:32 |
| elynn_ | That's a great news ! | 05:33 |
| xuhaiwei | elynn_, yes | 05:33 |
| xuhaiwei | Tacker may Senlin's heat resource to create vnf cluster, elynn_ | 05:34 |
| elynn_ | tacker give up heat? or use senlin as alternative? | 05:34 |
| elynn_ | I might take some time to examine the senlin resources in heat. | 05:34 |
| xuhaiwei | up to my understanding, tacker will still use Heat template to create VNFs, but in the heat template senlin resources will be created, elynn_ | 05:35 |
| xuhaiwei | I am very familiar with senlin resources, can you give me a sample template file? elynn_ | 05:37 |
| elynn_ | xuhaiwei https://github.com/openstack/heat-templates/blob/master/hot/senlin/cluster.yaml | 05:37 |
| elynn_ | will tacker still use autoscalegroup? | 05:37 |
| yanyanhu | xuhaiwei, that's great | 05:37 |
| xuhaiwei | That is a good question, I am not sure, either, but if peopleo find Senlin resources are nore feasible, maybe autoscalgroup will not be used | 05:38 |
| xuhaiwei | I think it depends on user's choice | 05:39 |
| yanyanhu | so they will keep using heat as orchestration engine for VNFs and use senlin to manage cluster type of resource collection? | 05:39 |
| xuhaiwei | currently, there is no conflict between these two features, tacker may support both of them I think | 05:39 |
| yanyanhu | xuhaiwei, that's good if tacker supports both as backend | 05:40 |
| xuhaiwei | yanyanhu, yes, I think, It maybe like Create some VNFs -> heat template -> senlin resource -> senlin profile -> instances creation | 05:40 |
| yanyanhu | yes, that's reasonable | 05:41 |
| xuhaiwei | yes | 05:41 |
| yanyanhu | since senlin itself is not an orchestration service | 05:41 |
| yanyanhu | heat is | 05:41 |
| yanyanhu | if user needs a combination of different type of resources, heat should be the choices | 05:41 |
| xuhaiwei | right, tacker won't drop heat I think | 05:42 |
| yanyanhu | just for homogeneous resource collection management, senlin might be a better option | 05:42 |
| yanyanhu | xuhaiwei, so you will make some contribution about cluster and scaling to taker in future? | 05:42 |
| xuhaiwei | yes, yanyanhu, another Chinese guy submitted a BP, I will join | 05:43 |
| yanyanhu | great | 05:43 |
| yanyanhu | hope there can be more collaboration between senlin and taker | 05:44 |
| xuhaiwei | Since tacker's ptl said tacker dont want to use senlin, I didn't propose senlin any more, but now it seems he changed his mind :) | 05:44 |
| yanyanhu | :) | 05:45 |
| yanyanhu | this will be a wise choice | 05:45 |
| yanyanhu | IMO :) | 05:45 |
| xuhaiwei | IMO few users want to use Senlin directly to controll the whole clusters currently, but if make senlin a heat resource, some users will accept | 05:46 |
| yanyanhu | xuhaiwei, understand | 05:46 |
| yanyanhu | since they may have built a complete solution around heat | 05:47 |
| yanyanhu | heat will be the only dependency for them | 05:47 |
| yanyanhu | that makes thing much simple | 05:47 |
| xuhaiwei | yes, people want to use heat as the first layer, but not senlin | 05:47 |
| yanyanhu | by contrast with talking to multiple services | 05:47 |
| yanyanhu | I see | 05:48 |
| yanyanhu | but for cluster management, senlin is actually a better choice for resourcegroup | 05:48 |
| yanyanhu | technically, it is really is :) | 05:48 |
| xuhaiwei | I am not clear if use Senlin as a Heat resource, Senlin will work feasibly? | 05:49 |
| yanyanhu | xuhaiwei, it will | 05:49 |
| xuhaiwei | That's great | 05:50 |
| yanyanhu | I think almost all functionalities have been supported by heat senlin resources | 05:50 |
| yanyanhu | if you meet any issues, you can ask us, especially elynn_ | 05:50 |
| yanyanhu | looking forward to the progress and feedback | 05:51 |
| xuhaiwei | In that case, user can still use senlin commands | 05:51 |
| yanyanhu | no, I think, user just need to update stack | 05:51 |
| xuhaiwei | ok, the job is not started, just under discussion | 05:51 |
| yanyanhu | if they want to make any operations on senlin cluster which stays at backend | 05:51 |
| yanyanhu | so from the tacker(the user here) perspective, it manipulate a heat resource | 05:52 |
| xuhaiwei | from 'senlin cluster-list' the clusters can be listed, right? | 05:52 |
| yanyanhu | which is actually senlin cluster | 05:52 |
| yanyanhu | yes, if you can access senlin api, that can be done | 05:53 |
| yanyanhu | just as you can list nova servers which are created inside heat stack | 05:53 |
| xuhaiwei | yes | 05:53 |
| eldon | http://paste.openstack.org/show/560718/ | 06:00 |
| eldon | Hi, yanyan, I also in a meeting this morning then. | 06:00 |
| yanyanhu | eldon, no problem, will check it | 06:05 |
| eldon | thanks~ | 06:06 |
| yanyanhu | my pleasure :) | 06:07 |
| yanyanhu | hi, eldon, I checked the code and didn't find clue... | 06:33 |
| yanyanhu | did this error always happen when performing operation like cluster-create,delete? | 06:34 |
| yanyanhu | or occasionally | 06:34 |
| xuhaiwei | yanyanhu, is the database changed recently? maybe you should rebuild the database, eldon | 06:35 |
| yanyanhu | xuhaiwei, yes, it changes | 06:36 |
| yanyanhu | it could be a reason | 06:36 |
| yanyanhu | but if eldon is using a fresh installed env, there could be other problems there | 06:37 |
| yanyanhu | eldon, you can try to rebuild the db by running "senlin-manage db_sync" | 06:38 |
| yanyanhu | hi, eldon, if possible, could you please paste the start_action method in scheduler.py, I noticed the code embedded in engine log is still the old one | 06:45 |
| eldon | ok | 06:51 |
| eldon | I uses rally to create and delete cluster in 40 parallels | 06:51 |
| eldon | http://paste.openstack.org/show/560732/ | 06:55 |
| yanyanhu | hi, eldon, this is still the old implementation of scheduler | 07:02 |
| eldon | how old? | 07:02 |
| yanyanhu | hmm, should be Mitaka release or earlier version | 07:03 |
| yanyanhu | https://review.openstack.org/#/c/312843/ | 07:03 |
| yanyanhu | this is the refactoring to scheduler which was done in May | 07:04 |
| yanyanhu | just after austin summit :) | 07:04 |
| yanyanhu | there is a significant change on senlin scheduler's behavior which is expected to improve the scalability | 07:05 |
| eldon | yes | 07:05 |
| yanyanhu | you can try with it | 07:05 |
| eldon | I remember I merged mitaka to our code | 07:05 |
| yanyanhu | hope can get some different result :) | 07:05 |
| eldon | Sorry, we have no time update code again. | 07:06 |
| yanyanhu | I see | 07:06 |
| eldon | Do you have any idea optimise this task in mitaka version? | 07:07 |
| yanyanhu | there could be concurrent issue in scheduler in Mitaka release | 07:07 |
| yanyanhu | let me see | 07:07 |
| yanyanhu | maybe we need a backport here | 07:07 |
| yanyanhu | let me check the code to see whether you can directly apply it into mitaka code base | 07:07 |
| eldon | that should be great. | 07:08 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Rework engine scheduler https://review.openstack.org/356975 | 07:09 |
| yanyanhu | hi, elynn_ , around? | 07:10 |
| yanyanhu | I tried to cherry pick a patch to mitaka release, do we need help from release team to backport the change? | 07:10 |
| yanyanhu | or it can be done at senlin side | 07:10 |
| yanyanhu | hi, eldon, let's wait for the check result of this cherry pick | 07:11 |
| yanyanhu | not sure it works | 07:11 |
| yanyanhu | eldon, anyway, I strongly suggest to apply this change since it does fix several critical issues of scheduler and will significantly improve the performance | 07:12 |
| elynn_ | Hi yanyanhu | 07:14 |
| yanyanhu | hi, elynn_ | 07:14 |
| elynn_ | I'm not sure how it works | 07:15 |
| eldon | I see | 07:15 |
| elynn_ | I saw someone will handle the stable release, better contact them before approve it yanyanhu | 07:15 |
| eldon | We also have our git. | 07:15 |
| yanyanhu | eldon, oh | 07:16 |
| eldon | So, if you can help me change the code | 07:16 |
| eldon | That will also work | 07:16 |
| yanyanhu | elynn_, I see. Maybe we just check whether it can work | 07:16 |
| yanyanhu | if so, eldon can try to manually apply it back to their stable/Mitaka repo | 07:16 |
| yanyanhu | I just triggered the functional test, lets whether it can pass | 07:17 |
| yanyanhu | if so, I think it could be safe to apply it since it doesn't touch any user visible interface | 07:18 |
| elynn_ | Ihar Hrachyshka , I think he would be the right person to contact | 07:18 |
| yanyanhu | elynn_, I see. Maybe we don't merge it back. | 07:19 |
| yanyanhu | let me mark that patch | 07:19 |
| elynn_ | Since this is not a new feature, back port it is a good idea. | 07:20 |
| elynn_ | Let's see if it can solve eldon's problem. | 07:20 |
| yanyanhu | hmm, it does enhance senlin scheduler | 07:20 |
| yanyanhu | yes | 07:20 |
| elynn_ | Maybe caused by db api.... | 07:21 |
| yanyanhu | elynn_, yes, that is possible as well. But a little bit looks like db concurrent issue... | 07:21 |
| yanyanhu | not sure... | 07:22 |
| eldon | a little confused:) | 07:22 |
| yanyanhu | :) | 07:24 |
| *** Qiming_ has joined #senlin | 07:27 | |
| *** Qiming_ has quit IRC | 07:28 | |
| yanyanhu | hi, eldon, I think you can apply this patch to stable/mitaka directly. It works according to the functional test I ran locally. | 07:55 |
| yanyanhu | the only problem is there is once DB upgrading after Mitaka release | 07:55 |
| yanyanhu | you may need to downgrade or recreate you senlin table to make it work | 07:55 |
| yanyanhu | another thing need to change is the api-paste.ini file | 07:57 |
| yanyanhu | plz use the one belongs to stable mitaka release | 07:57 |
| *** eldon_ has joined #senlin | 09:21 | |
| xuhaiwei | yanyanhu, are you there? | 09:21 |
| yanyanhu | xuhaiwei, yes | 09:22 |
| xuhaiwei | I reinstalled senlin, but it doesn't work at all now | 09:22 |
| xuhaiwei | no matter what commands I run, got an 500 error | 09:22 |
| yanyanhu | what error was raised | 09:23 |
| yanyanhu | so the request can reach senlin-api service? | 09:23 |
| xuhaiwei | http://paste.openstack.org/show/560760/ this is the log | 09:23 |
| yanyanhu | are you using latest senlin service code and client code | 09:23 |
| xuhaiwei | yes, this is the log of senlin-api | 09:23 |
| yanyanhu | ok, let me check it | 09:23 |
| xuhaiwei | I reinstalled senlin and senlinclient, everything is the latest I think | 09:24 |
| yanyanhu | need to sync db? | 09:24 |
| xuhaiwei | db is rebuilded too | 09:24 |
| *** eldon has quit IRC | 09:25 | |
| yanyanhu | ok, let me try in my env | 09:25 |
| *** eldon_ has quit IRC | 09:25 | |
| xuhaiwei | the error seems from sdk | 09:25 |
| *** eldon_ has joined #senlin | 09:25 | |
| elynn_ | update keystone? | 09:26 |
| yanyanhu | oh, are you using the latest sdk code | 09:26 |
| xuhaiwei | looks like a db error | 09:26 |
| xuhaiwei | using sdk 0.9.2 | 09:27 |
| yanyanhu | I think there are several big changes recently | 09:27 |
| yanyanhu | you can try to pull the latest sdk and reinstall it from local | 09:27 |
| xuhaiwei | you also got the same error? | 09:27 |
| yanyanhu | then try again | 09:27 |
| yanyanhu | no, it works well in my env | 09:28 |
| xuhaiwei | I have done that in fact | 09:28 |
| yanyanhu | hmm | 09:28 |
| yanyanhu | looks like error happened during senlin talked with keystone through sdk | 09:29 |
| xuhaiwei | what is the sdk version you are using | 09:29 |
| yanyanhu | let me recreate the db and try it again | 09:30 |
| yanyanhu | no error happened | 09:31 |
| xuhaiwei | when I uninstall sdk and reinstall sdk from local, sdk 0.9.3 is installed, but this time senlin-api can't be started | 09:32 |
| yanyanhu | what error was given | 09:32 |
| xuhaiwei | can you check your sdk version? | 09:32 |
| yanyanhu | openstacksdk-0.9.3.dev3 | 09:32 |
| yanyanhu | mine is this | 09:32 |
| xuhaiwei | http://paste.openstack.org/show/560768/ | 09:32 |
| yanyanhu | this is weird | 09:33 |
| yanyanhu | what is version of your keystoneauth1 | 09:34 |
| xuhaiwei | 2.11.1 | 09:34 |
| yanyanhu | me too | 09:34 |
| xuhaiwei | and your pbr? | 09:35 |
| xuhaiwei | my pbr is 1.10.0 | 09:35 |
| yanyanhu | let me check | 09:35 |
| yanyanhu | me too... | 09:35 |
| Qiming | xuhaiwei, it is a pbr mess ... | 09:37 |
| Qiming | I encountered that before | 09:38 |
| Qiming | cannot accurately describe how I fixed it | 09:38 |
| Qiming | things to try | 09:38 |
| xuhaiwei | but pbr version seems good | 09:38 |
| Qiming | remove all .pyc files under /usr/lib/python2.7/site-packages/ | 09:38 |
| xuhaiwei | ok, let me try | 09:39 |
| Qiming | some stall packages may cause pbr confusion | 09:39 |
| Qiming | check if you have keystoneauth installed, uninstall it if you have it | 09:39 |
| Qiming | what sdk needs is keystoneauth1, not keystoneauth | 09:39 |
| xuhaiwei | I have keystonemiddleware | 09:40 |
| xuhaiwei | should it be deleted? | 09:40 |
| Qiming | remove all subdirs with names like 'keystone*', under /usr/lib/python2.7/site-packages/ | 09:40 |
| Qiming | no keystonemiddleware is important | 09:40 |
| *** elynn_ has quit IRC | 09:40 | |
| Qiming | it is sometime frustrating | 09:40 |
| Qiming | but ... that is life, python style | 09:40 |
| xuhaiwei | removing .pyc doesn't work | 09:41 |
| Qiming | uninstall keystoneauth1 | 09:41 |
| Qiming | and reinstall openstacksdk? | 09:42 |
| xuhaiwei | let me try | 09:42 |
| Qiming | sdk will automatically reinstall keystoneauth1 | 09:42 |
| *** elynn_ has joined #senlin | 09:44 | |
| *** eldon_ has quit IRC | 09:56 | |
| *** elynn_ has quit IRC | 09:57 | |
| *** zhurong has quit IRC | 09:58 | |
| *** yanyanhu has quit IRC | 09:59 | |
| *** eldon_ has joined #senlin | 10:02 | |
| *** eldon_ has quit IRC | 10:29 | |
| *** zhurong has joined #senlin | 12:45 | |
| *** elynn_ has joined #senlin | 12:51 | |
| *** elynn_ has quit IRC | 12:59 | |
| *** Drago has joined #senlin | 14:31 | |
| *** Drago has quit IRC | 14:31 | |
| *** Drago has joined #senlin | 14:31 | |
| *** zhurong has quit IRC | 15:34 | |
| *** zhurong has joined #senlin | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!