Wednesday, 2016-01-20

*** pratikmallya has quit IRC00:10
openstackgerritQiming Teng proposed openstack/senlin: Fix db problem for node creation  https://review.openstack.org/26993100:17
Qimingxuhaiwei, ^00:18
xuhaiweimorning, Qiming00:18
Qimingxuhaiwei, another way to fix db problem:00:18
Qimingenter mysql00:18
Qiminguse senlin;00:18
Qimingalter table node drop foreign key node_ibfk_1;00:18
Qimingthe foreign key was creating problems00:19
xuhaiweidon't make cluster_id as a foreign key?00:20
Qimingyes00:20
Qimingso, here is the reason00:20
Qimingif we have it a foreign key00:21
xuhaiweiIf we make it a foreign key, it can't be ''?00:21
Qimingfor nodes that don't belong to any cluster00:21
Qimingthe cluster_id must be set to None00:21
Qimingthis was seemingly okay00:21
Qiminghowever, when we do node_get_all(cluster_id=None)00:21
Qimingthe semantics becomes ambiguous00:22
Qimingit can mean: get me all nodes that don't belong to any cluster, i.e. orphan nodes00:22
Qimingit can mean: get me all nodes, I don't care about cluster_id, so cluster_id default is None00:22
Qimingquestion?00:25
xuhaiweiso if we make the orphan nodes' cluster_id '', node_get_all(cluster_id=None) will only have the seconde meaning?00:25
Qimingexactly00:26
Qimingor, if you want to get orphan nodes, do node_get_all(cluster_id=''), i.e. explicitly tell me so00:26
Qimingdidn't realize that a '' value is not passing mysql constraint checking00:27
Qimingsince it is a bug fix00:28
QimingI don't think we should bump db version in this case00:28
xuhaiweiif we drop the foreign key cluster_id, there won't be any bad effect?00:28
QimingI don't think so00:28
xuhaiweiI am curious why unit test didn't fail00:29
xuhaiweiin db unit test, db_api.create_node can succeed00:30
Qimingforeign keys are nice to have, but not a must00:30
Qimingsigh ... that is because for unit tests, we are not using mysql00:30
Qimingwe are using sqlalchemy-lite00:30
Qimingsorry sqlite00:32
xuhaiweioh00:35
xuhaiweineed to study this00:35
Qiminghttp://git.openstack.org/cgit/openstack/senlin/tree/senlin/tests/unit/common/utils.py#n3200:36
Qimingalso pls help review https://review.openstack.org/26971500:39
Qimingjust finished it before going to bed last night00:39
xuhaiweiok, I am reading it00:41
* Qiming is dancing off to lab00:41
*** Qiming has quit IRC00:45
*** Qiming has joined #senlin01:42
*** Yanyanhu has joined #senlin02:02
openstackgerritYanyan Hu proposed openstack/senlin: Add fixed priority for built-in policies  https://review.openstack.org/26947802:03
*** lixinhui_ has joined #senlin02:09
*** elynn has joined #senlin02:09
elynnMorning02:09
openstackgerritYanyan Hu proposed openstack/senlin: Rework lb policy  https://review.openstack.org/26467002:09
Yanyanhumorning02:10
*** Liuqing has joined #senlin02:12
*** Liuqing has left #senlin02:13
*** Liuqing has joined #senlin02:13
*** elynn_ has joined #senlin02:16
openstackgerritCindia-blue proposed openstack/senlin: Add check and Recover into RPC API  https://review.openstack.org/26792202:17
openstackgerritCindia-blue proposed openstack/senlin: Add check and Recover into RPC API  https://review.openstack.org/26792202:18
*** elynn has quit IRC02:19
openstackgerritMerged openstack/senlin: Fix db problem for node creation  https://review.openstack.org/26993102:29
openstackgerritMerged openstack/senlin: Add Check and Recover into Node Action  https://review.openstack.org/26456803:02
openstackgerritMerged openstack/senlin: Add Check and Recover as a Cluster Action  https://review.openstack.org/26741903:12
*** yuanying_ has joined #senlin03:18
openstackgerritCindia-blue proposed openstack/senlin: Add check and Recover into RPC API  https://review.openstack.org/26792203:19
*** yuanying has quit IRC03:19
openstackgerritCindia-blue proposed openstack/senlin: Add check and Recover into RPC API  https://review.openstack.org/26792203:21
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-api a console script entry point  https://review.openstack.org/26737503:22
openstackgerritEthan Lynn proposed openstack/senlin: Fix devstack script to use console scripts  https://review.openstack.org/26948403:22
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-engine a console script entry point  https://review.openstack.org/26737603:22
openstackgerritMerged openstack/senlin: Add check and Recover into RPC API  https://review.openstack.org/26792203:33
elynn_Hi do you test latest codes ? I do a cluster-list and can't get any response03:34
Qimingelynn_, I cannot reproduce it03:37
elynn_ok, maybe I need to dig more.03:38
Qimingdo a 'senlin -d cluster-list'03:41
Qimingsee if anything abnormal from client side03:41
Qimingthen check api log and engine log03:41
elynn_Qiming, thanks, I'm checking03:43
AyushHi Qiming, Sorry for very late response, There is the stack trace of my command03:54
AyushDEBUG (extension) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token') DEBUG (extension) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken') DEBUG (extension) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode') DEBUG (extension) found extension EntryPoint.parse('v2pass03:54
AyushTraceback (most recent call last):   File "/usr/local/bin/senlin", line 11, in <module>     sys.exit(main())   File "/usr/local/lib/python2.7/dist-packages/senlinclient/shell.py", line 308, in main     SenlinShell().main(args)   File "/usr/local/lib/python2.7/dist-packages/senlinclient/shell.py", line 288, in main     sc = self._setup_senlin_client(api_ver, args)   File "/usr/local/lib/python2.7/dist-packages/senlinclient/shell.py", l03:57
AyushFile "/usr/local/lib/python2.7/dist-packages/senlinclient/client.py", line 24, in Client     return cls(*args, **kwargs)   File "/usr/local/lib/python2.7/dist-packages/senlinclient/v1/client.py", line 27, in __init__     conn = sdk.create_connection(preferences, user_agent, **kwargs)   File "/usr/local/lib/python2.7/dist-packages/senlinclient/common/sdk.py", line 130, in create_connection     **kwargs)03:58
Ayush File "/usr/local/lib/python2.7/dist-packages/openstack/connection.py", line 199, in __init__     **auth_args)   File "/usr/local/lib/python2.7/dist-packages/openstack/connection.py", line 211, in _create_authenticator     loader = ksa_loader.get_plugin_loader(auth_plugin)   File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/base.py", line 74, in get_plugin_loader     raise exceptions.NoMatchingPlugin(name) keystoneaut03:58
QimingAyush, looks like openstacksdk and/or keystoneauth1 has problems04:00
*** yuanying_ has quit IRC04:05
openstackgerritQiming Teng proposed openstack/senlin: Rework region placement policy  https://review.openstack.org/26971504:06
*** yuanying_ has joined #senlin04:07
openstackgerritMerged openstack/senlin: Add fixed priority for built-in policies  https://review.openstack.org/26947804:09
*** yuanying has joined #senlin04:20
*** yuanying_ has quit IRC04:22
*** elynn_ has quit IRC04:23
openstackgerritLiuqing Jing proposed openstack/senlin-dashboard: Add event table in node detail  https://review.openstack.org/27000204:34
*** yuanying has quit IRC04:37
*** yuanying has joined #senlin04:39
*** yuanying has quit IRC04:40
*** yuanying has joined #senlin04:44
*** yuanying_ has joined #senlin04:49
*** yuanying has quit IRC04:50
xuhaiweielynn_, maybe you need to rebuild DB04:57
*** elynn_ has joined #senlin04:59
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-api a console script entry point  https://review.openstack.org/26737505:04
openstackgerritEthan Lynn proposed openstack/senlin: Fix devstack script to use console scripts  https://review.openstack.org/26948405:04
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-engine a console script entry point  https://review.openstack.org/26737605:04
*** elynn_ has quit IRC05:04
*** elynn_ has joined #senlin05:04
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-api a console script entry point  https://review.openstack.org/26737505:08
openstackgerritEthan Lynn proposed openstack/senlin: Fix devstack script to use console scripts  https://review.openstack.org/26948405:08
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-engine a console script entry point  https://review.openstack.org/26737605:08
*** dixiaoli has joined #senlin05:18
elynn_Hi xuhaiwei , I encounter the same error when create new nodes and I report a bug for it . https://bugs.launchpad.net/senlin/+bug/153600005:29
openstackLaunchpad bug 1536000 in senlin "Can not create node without attaching to a cluster" [Undecided,New]05:29
xuhaiweielynn_, it is fixed by this patch05:30
xuhaiweiyou need to rebuild DB05:30
elynn_Oh....05:31
xuhaiweihttps://review.openstack.org/26993105:32
xuhaiweisorry, this patch05:32
elynn_cool, that works.05:40
elynn_but I still can't delete node, the status will remain in deleting. Don't know why05:41
xuhaiweilet me have a try05:44
xuhaiweiit works for me05:45
elynn_Are you using heat profile?05:46
xuhaiweino05:46
xuhaiweiI created a nova type profile05:46
Yanyanhuhi, xuhaiwei, are you around?05:54
xuhaiweiyes05:54
YanyanhuI'm now trying to move select_candidates function out of deletion_policy05:54
Yanyanhuto define it as an util method05:54
Yanyanhudoes this work conflict to your current work?05:55
xuhaiweicurrently no05:55
Yanyanhuok05:55
Yanyanhuif so, I will keep working on it. Plan to finish it this afternoon05:55
xuhaiweiok05:55
Yanyanhuno other change in DeletionPolicy05:55
xuhaiweiit's better to make some method as a common method so that it can be used by many policies05:57
Yanyanhuyes, that is the purpose for this change :)05:59
openstackgerritQiming Teng proposed openstack/senlin: Rework region placement policy  https://review.openstack.org/26971506:08
openstackgerritCindia-blue proposed openstack/senlin: Add Doc for Check and Recover Actions  https://review.openstack.org/27002006:24
lixinhuiQiming, Yanyanhu06:34
Yanyanhuhi06:34
lixinhuiwhat priority you think should give to health_policy06:34
openstackgerritMerged openstack/senlin: Rework region placement policy  https://review.openstack.org/26971506:35
openstackgerritQiming Teng proposed openstack/senlin: Enable nova driver to validate azs  https://review.openstack.org/27002206:35
Qiminglixinhui, I think we can pause at enabling check/rover RPC APIs06:36
Yanyanhuah, this is a good question :) that depends on the timing we want it to take effect06:36
Qimingnext thing would be about the usage scenario discussion06:36
Yanyanhuagree06:36
Qiminghow would we do polling (i.e. call do_check()), and how would we do recovery (i.e. call do_recover())06:36
QimingI believe we do have a rough idea06:37
lixinhuiokay, Yanyanhu and Qiming06:37
Qimingwe can make it happen step by step06:37
Qimingthese days we are too much trapped into fixing things to make the engine working again06:39
QimingYanyanhu, zone placement PRIORITY should be?06:42
Qiming300?06:43
*** lixinhui has quit IRC06:43
Yanyanhuyes, it is 300 now06:43
Qimingokay06:43
*** lixinhui has joined #senlin06:43
lixinhuiI see. Qiming. xinku.06:44
openstackgerritYanyan Hu proposed openstack/senlin: Minor tweak DeletionPolicy  https://review.openstack.org/27002706:47
openstackgerritMerged openstack/senlin: Make senlin-api a console script entry point  https://review.openstack.org/26737506:48
Yanyanhuhi, lixinhui, I think06:50
Yanyanhuwe can make a discussion about this topic later to decide possible scenarios that health check action is triggered06:50
Yanyanhuthen we will know how to decide health policy's priority :)06:50
lixinhuiI see. Yes, Yanyanhu :)06:51
openstackgerritYanyan Hu proposed openstack/senlin: Make select_candidates an util function  https://review.openstack.org/27002706:53
xuhaiweiYanyanhu07:09
Yanyanhuhi07:09
xuhaiweilb policy has similar cretirias like 'oldest_first', 'youngest_first'?07:10
xuhaiweiwhen deleting nodes07:10
Yanyanhuno, just if there is no deletion_policy attached to cluster, it may need to decide 'candidates' by itself07:10
xuhaiweiso when deciding candidates by lb policy itself, is there any creteria for it?07:11
Yanyanhuso the basic idea is: if a policy finds its expected input doesn't exist in action data, it will try to generate it based on action input07:11
Yanyanhurandom will be the reteria07:11
Yanyanhusorry, creteria07:12
xuhaiweiin this case, I am afraid the _select_candidates() method of deletion policy is not suitable to become a common method totally07:13
xuhaiweiwhat about making a new method select_candidates_random()07:13
Yanyanhuhmm, there will be duplication if so07:13
xuhaiweiwhich will be used by both deletion policy and lb policy07:13
Yanyanhuhmm, let me see07:16
xuhaiweiwhat will be duplicated07:16
Yanyanhuthose code for getting node list and err nodes filtering I think07:17
xuhaiweithat can be put in the method07:18
Yanyanhusince the new select_candidates_random() method actually has to implement the entire workflow of node list getting, node status check and node selecting07:19
xuhaiweiyes07:19
Yanyanhuthe only difference from existing _select_candidates() is two selecting algorithm07:20
xuhaiweiok, let me think about it more07:21
Yanyanhuhmm, I will think about it as well. Just feel select_candidates() is an useful method07:21
Yanyanhuin cases we want to select some candidates from a given cluster with given rule07:22
Yanyanhuhmm, maybe we can further split algorithm from workflow07:24
Yanyanhuas qiming suggested07:24
YanyanhuJust try to address policy check issue before Friday. So will think about it for further optimization later :)07:26
xuhaiweiwhat do you mean by 'split algorithm from workflow'?07:27
*** lixinhui_ has quit IRC07:27
YanyanhuI mean separating the progress of selecting candidates from a list and how to getting the list07:30
Yanyanhuhow to get the list of nodes07:30
Yanyanhue.g. there is no context and cluster_id input parameters for select_candidates()07:31
Yanyanhusince we don't need them for candidates selecting07:32
Yanyanhuwe just need them to get nodes and profiles list07:32
Yanyanhuof course, I'm not sure this kind of change is necessary07:33
Yanyanhuin current stage07:33
openstackgerritYanyan Hu proposed openstack/senlin: Make select_candidates an util function  https://review.openstack.org/27002707:36
openstackgerritQiming Teng proposed openstack/senlin: Enable nova driver to validate azs  https://review.openstack.org/27002208:10
openstackgerritQiming Teng proposed openstack/senlin: Rework zone placement policy  https://review.openstack.org/27004708:10
openstackgerritYanyan Hu proposed openstack/senlin: Make select_candidates an util function  https://review.openstack.org/27002708:22
openstackgerritQiming Teng proposed openstack/senlin: Remove useless test module  https://review.openstack.org/27004908:29
Qimingelynn_, about the revision to senlin-engine, we need to test if it works08:34
Qimingto test that, we need a fresh install from start, creating conf file and test if the oslo_config and oslo_log still works08:35
elynn_Hi Qiming , I will do more tests, but for safe, I can move it back.08:36
elynn_I clean my env yesterday, so my env is base on my patches.08:36
Qimingyes, because it is not a critical part of the change08:36
Qimingplease revert it back08:37
Qimingbefore we get oslo_log and oslo_config, oslo_messaging properly configured, import engine service at the very begining would somehow break the whole initialization process08:38
QimingI learned that lesson in a hard way when migrating to oslo_config I guess08:39
elynn_oh, jiang hai shi lao de la08:39
* Qiming sweating ...08:40
elynn_:)08:41
openstackgerritEthan Lynn proposed openstack/senlin: Fix devstack script to use console scripts  https://review.openstack.org/26948408:42
openstackgerritEthan Lynn proposed openstack/senlin: Make senlin-engine a console script entry point  https://review.openstack.org/26737608:42
elynn_Seems wait_for_delete in openstacksdk still doesn't work, I need to find out why...08:46
Qimingstill?08:48
elynn_Qiming, yes, my nodes are always in deleting status, and logs doesn't output any error.08:48
Qimingso ... NotFound exception is not caught08:50
elynn_Qiming, I think maybe not found doesn't raise08:51
Qiminghttp://git.openstack.org/cgit/openstack/python-openstacksdk/commit/openstack/resource.py?id=e0ad907a361a71ce22e89e28bd319b8c8dc1843208:52
Qimingthis could be the reason!08:52
elynn_might be...08:53
elynn_Seems not, revert this patch doesn't work for me.08:57
Qimingit is not simply about that patch I guess08:57
elynn_I might pus some time on it.08:58
Qimingyou mean the whole patch? http://git.openstack.org/cgit/openstack/python-openstacksdk/commit/openstack?id=e0ad907a361a71ce22e89e28bd319b8c8dc1843208:58
Qimingthat patch is not just changing resource module08:59
elynn_Yes.08:59
openstackgerritMerged openstack/senlin: Enable nova driver to validate azs  https://review.openstack.org/27002209:03
elynn_Ok, I know why, heat will always return deleted stack at https://github.com/openstack/python-openstacksdk/blob/master/openstack/resource.py#L102409:07
elynn_Maybe we can use wait_for_status in senlin.09:08
*** lixinhui_ has joined #senlin09:09
*** lixinhui_ has quit IRC09:10
Qimingoh09:12
Qimingsounds reasonable09:12
xuhaiweiQiming, after all, you still determined to do deletion in placement policies?09:14
Qimingno09:14
Qimingwe have discussed this a hundred times09:15
Qimingit is not about doing deletion09:15
Qimingit is about maintaining zone/region distribution09:15
Qimingif we only handle node creation, that policy is useless09:16
Qimingit was a design philosophy called 'separation of concerns'09:16
Qiminglet each policy do one thing, and make it do it correctly09:16
Qimingall policies should work without dependency on any other policies09:16
Qimingif you have a better idea, we can discuss it09:17
xuhaiweibut from the region_placement implement, when CLUSTER_SCALE_IN is triggered, the policy will handle it right?09:18
QimingYanyan and I had quite some discussions on whether/how to switch from a policy-centric design to an action-centric design09:18
Qimingthat switching is simply not possible and it will introduce a lot of duplicated code09:18
xuhaiweiwe don't need to do anything09:19
Qimingyes, CLUSTER_SCALE_IN would be handled09:19
xuhaiweisorry miss type09:19
Qimingwhat do you suggest?09:19
xuhaiweiso there would be no deletion policy at this time09:19
QimingI'm open to any alternatives09:19
Qimingwhat do you mean no deletion policy?09:21
xuhaiweiI mean deletion policy will not be triggered09:21
Qimingthat is orthogonal09:21
xuhaiweiin other words, deletion policy can not be attached to a cluster when placement policy is attached09:22
Qimingdeletion policy has to be improved to handle inputs like {'deletion': {'zone1': 3, 'zone2': 2}}09:22
Qimingwe need hands to work on that09:22
openstackgerritQiming Teng proposed openstack/senlin: Remove a dead function in test case  https://review.openstack.org/27007609:26
xuhaiweiI can't think out a case to use deletion policy to handle {'deletion': {'zone1': 3, 'zone2':2}}09:27
openstackgerritMerged openstack/senlin: Added release notes about recent changes  https://review.openstack.org/26841809:28
openstackgerritMerged openstack/senlin: Update text strings  https://review.openstack.org/26885309:29
openstackgerritMerged openstack/senlin: Remove purge_deleted from senlin-manage  https://review.openstack.org/26891509:29
QimingYanyanhu, about the refactoring of deletion policy09:32
QimingI'm really concerned moving all deletion decisions into scaleutils09:33
Yanyanhuyes, understand09:35
YanyanhuI'm now trying to put it into Cluster module09:35
Yanyanhuafter thinking, I believe you and haiwei are right09:36
Yanyanhuwe actually don't need that function to be such common09:37
Qimingyip09:39
Yanyanhuhope can finish it today09:40
Qimingon the contrary, we do need the random selection method to be a util09:40
Qimingplease help work on that09:40
Yanyanhuyes, I'm working on it09:40
QimingI'll start fix deletion policy to handle cross-region case09:40
Yanyanhugot it09:40
*** elynn_ has quit IRC09:54
*** dixiaoli has quit IRC10:05
*** dixiaoli has joined #senlin10:05
*** Qiming has quit IRC10:12
openstackgerritYanyan Hu proposed openstack/senlin: Add select_random_nodes function  https://review.openstack.org/27010710:31
*** Yanyanhu has quit IRC10:40
*** dixiaoli has quit IRC10:49
*** Qiming has joined #senlin11:08
openstackgerritMerged openstack/senlin: Add select_random_nodes function  https://review.openstack.org/27010711:22
*** openstackgerrit has quit IRC11:43
-openstackstatus- NOTICE: review.openstack.org is being restarted to apply patches11:44
*** ChanServ changes topic to "review.openstack.org is being restarted to apply patches"11:44
*** openstackgerrit has joined #senlin11:45
*** ChanServ changes topic to "IRCLog: http://eavesdrop.openstack.org/irclogs/%23senlin/ | Bugs: bugs.launchpad.net/senlin | Review: https://review.openstack.org/#/q/project:openstack/senlin,n,z"11:55
-openstackstatus- NOTICE: Restart done, review.openstack.org is available11:56
openstackgerritQiming Teng proposed openstack/senlin: Add node query by region and zone to cluster  https://review.openstack.org/27016312:19
openstackgerritQiming Teng proposed openstack/senlin: Rework zone placement policy  https://review.openstack.org/27004712:20
openstackgerritMerged openstack/senlin: Rework zone placement policy  https://review.openstack.org/27004712:42
*** Ayush has quit IRC13:14
openstackgerritMerged openstack/senlin: Make senlin-engine a console script entry point  https://review.openstack.org/26737613:44
*** Liuqing has quit IRC13:53
*** pratikmallya has joined #senlin15:09
*** pratikma_ has joined #senlin15:10
*** pratikmallya has quit IRC15:13
*** bdrich has joined #senlin15:35
*** Liuqing has joined #senlin15:44
openstackgerritQiming Teng proposed openstack/senlin: Add node selection utils to scale utils  https://review.openstack.org/27032117:07
openstackgerritMerged openstack/senlin: Add node query by region and zone to cluster  https://review.openstack.org/27016317:12
openstackgerritQiming Teng proposed openstack/senlin: Add node selection utils to scale utils  https://review.openstack.org/27032117:17
openstackgerritQiming Teng proposed openstack/senlin: Add node selection utils to scale utils  https://review.openstack.org/27032117:20
openstackgerritQiming Teng proposed openstack/senlin: Add node selection utils to scale utils  https://review.openstack.org/27032117:31
openstackgerritQiming Teng proposed openstack/senlin: Add node selection utils to scale utils  https://review.openstack.org/27032117:31
*** Liuqing has quit IRC17:35
*** pratikma_ has quit IRC17:46
openstackgerritMerged openstack/senlin: Add node selection utils to scale utils  https://review.openstack.org/27032117:46
openstackgerritQiming Teng proposed openstack/senlin: Reworked deletion policy  https://review.openstack.org/27034818:07
*** lixinhui has quit IRC18:22
openstackgerritAndreas Jaeger proposed openstack/python-senlinclient: Remove argparse from requirements  https://review.openstack.org/27039118:22
*** lixinhui has joined #senlin18:22
*** Qiming has quit IRC18:34
*** pratikmallya has joined #senlin18:57
*** pratikma_ has joined #senlin18:58
*** pratikma_ has quit IRC19:00
*** pratikma_ has joined #senlin19:00
*** pratikmallya has quit IRC19:02
*** bdrich has quit IRC19:32
*** bdrich has joined #senlin20:33
*** pratikma_ has quit IRC20:48
*** pratikmallya has joined #senlin21:45
*** bdrich has quit IRC22:55
*** pratikmallya has quit IRC23:16

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!