13:00:38 <Qiming> #startmeeting senlin
13:00:39 <openstack> Meeting started Tue Apr 11 13:00:38 2017 UTC and is due to finish in 60 minutes.  The chair is Qiming. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:40 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:42 <openstack> The meeting name has been set to 'senlin'
13:01:11 <XueFengLiu> hi,QiMing
13:01:23 <ruijie> evening :)
13:01:27 <yanyanhu> hi, evening
13:01:38 <Qiming> evening, guys
13:01:39 <XueFengLiu> evening:)
13:02:17 <Qiming> pls add items to the agenda if you have things to talk about
13:02:19 <Qiming> #link https://wiki.openstack.org/wiki/Meetings/SenlinAgenda#Weekly_Senlin_.28Clustering.29_meeting
13:02:55 <Qiming> #topic pike work items
13:03:04 <Qiming> #link https://etherpad.openstack.org/p/senlin-pike-workitems
13:03:58 <Qiming> API errors check, no seeing any patches
13:05:13 <Qiming> I will do that
13:05:35 <Qiming> VDU profile, not seeing progress from ethan
13:06:00 <Qiming> CLUSTER_CHECK improvement, still there for review
13:07:12 <Qiming> I was hoping a revision that makes some actions self destroyable
13:07:28 <XueFengLiu> Qiming
13:07:47 <XueFengLiu> self destroyable in db layer or in senlin ?
13:08:27 <Qiming> in db layer
13:08:38 <Qiming> or else the logic propagates to many places
13:08:50 <XueFengLiu> ok
13:09:07 <Qiming> a NODE_CHECK can be maked transient
13:09:27 <Qiming> and it will remove itself from the dependency table after sucessfully completed
13:09:54 <Qiming> Node adoption: two patches for review
13:10:06 <Qiming> still stuck at API endpoint
13:10:28 <Qiming> basically, node_adopt_preview and node_adopt are two different operations
13:10:53 <Qiming> because the first one should return the (complete) spec of the profile to be created, while
13:11:07 <Qiming> the second will return the node properties after a successful adoption
13:11:53 <Qiming> so ... the first api may eventually land at /v1/profiles/adopt, and the second may eventually land at /v1/nodes/adopt
13:12:07 <Qiming> neither one looks decent
13:12:15 <Qiming> would like to know your opinions
13:12:48 <XueFengLiu> ok, spend time to review
13:13:09 <Qiming> the question I raised is not yet there for review
13:13:22 <Qiming> the two patches are some basic mechanisms for node adoption
13:13:40 <Qiming> next thing is about scaling improvement
13:13:49 <Qiming> rui jie is onto it
13:14:30 <Ruijie_> yes, its in good progress now :)
13:14:32 <Qiming> the plan is to do an optional health check (by invoking node.do_check() directly) before a scale-in operation
13:15:14 <Qiming> it makes good senses to me because this is solving a practical problem without introducing a lot of disruptive changes
13:16:04 <Qiming> next thing to work on is to do an optional converge to the (last) desired_capacity
13:16:16 <Qiming> it is also a wanted feature
13:16:35 <Qiming> about RDO prep, XueFengLiu any update?
13:16:58 <XueFengLiu> In process
13:17:24 <Qiming> alright
13:17:37 <Qiming> pls let team know when there are new progress
13:17:47 <Qiming> health management
13:17:50 <XueFengLiu> Ok, I will
13:18:06 <Qiming> mistral workflow work is stagnated, we need hands on it
13:18:27 <XueFengLiu> yes
13:18:29 <Qiming> I don't think the neutron LB problem has a quick fix
13:18:44 <Qiming> maybe we should put fencing support back to FEATURES.rst
13:20:05 <Qiming> fencing physical nodes are desirable, but we may need some IPMI logics to get that done
13:20:05 <Qiming> fencing virtual machine is already there, i.e. a forced delete
13:20:05 <Qiming> Tempest tests
13:20:05 <Qiming> XueFengLiu is on it
13:20:26 <Qiming> however, my perceiption is that there were some misunderstandings about cloud_backend
13:20:33 <XueFengLiu> yes, start recently
13:20:37 <yanyanhu> yes, seems so :)
13:20:41 <XueFengLiu> en
13:20:48 <Qiming> but the logics proposed are still very valuable for sovling integration test problems
13:20:49 <yanyanhu> I guess we don't need this https://review.openstack.org/#/c/455331/
13:20:53 <XueFengLiu> I read the readme.rst
13:21:20 <Qiming> feel free to propose clarifications or documentation anywhere
13:21:20 <yanyanhu> creating of new key and network is only required for integration test I feel
13:21:44 <XueFengLiu> It did not mention cloud_backend
13:21:56 <Qiming> don't hesitate don't feel shy about english, team can help you polish it
13:22:30 <Qiming> or you can report a bug and let me know, I can help fix the doc
13:22:35 <yanyanhu> XueFengLiu, that is our fault to forget to mention it :)
13:22:54 <XueFengLiu> :)
13:23:21 <XueFengLiu> Some where we have add the cloud_backend conf
13:23:28 <Qiming> I believe it has been documented: https://docs.openstack.org/developer/senlin/developer/testing.html
13:23:33 <XueFengLiu> But not in tempest readme.rst
13:23:44 <XueFengLiu> yes
13:24:28 <Qiming> feel free to add a pointer then
13:24:46 <XueFengLiu> ok. will add
13:25:11 <Qiming> about CLUSTER_RECOVER and LB policy, Ruijie_ still have proposals?
13:25:30 <Ruijie_> np Qiming, will revise it when doing current bp
13:25:47 <Qiming> okay, so will leave the item there
13:25:52 <Ruijie_> I thought it too simple before
13:26:30 <Qiming> :) nothing is simple in real life
13:26:58 <Ruijie_> the node.status=WARNING need to be handled in several places..
13:27:08 <Qiming> yep
13:27:26 <Qiming> better think it through before introducing piecemeal changes
13:27:42 <Qiming> as for NFV work, haiwei has been working on it
13:28:22 <Qiming> here is the link: https://review.openstack.org/#/c/449471/
13:28:58 <Qiming> when he worked on this he also found a concurrency bug
13:29:03 <Qiming> I believe he is solving it
13:29:47 <Qiming> that's all I have on the etherpad items
13:29:53 <Qiming> anything to add?
13:30:19 <XueFengLiu> no from me
13:30:41 <Qiming> okay, moving on
13:30:48 <Qiming> #topic pike-1 release
13:30:48 <XueFengLiu> ok
13:31:00 <Qiming> I'm planning to cut a p-1 release tomorrow
13:31:07 <Qiming> however, bad things happened
13:31:12 <Qiming> our gate is broken
13:31:24 <Qiming> we have just stood up py35 gate jobs
13:31:40 <XueFengLiu> yes, seems problem with py35
13:31:50 <Qiming> the gate is failing us, which means no patch can be validated for merge
13:32:04 <Qiming> I spent some time just now looking into the logs
13:32:20 <Qiming> it seems that we have glance-api failing during devestack installation
13:32:22 <yanyanhu> the change in gate side has been merged?
13:32:28 <Qiming> yes
13:32:32 <Qiming> yanyanhu,
13:32:32 <yanyanhu> the patch for enabling py35 test
13:32:55 <yanyanhu> could be devstack's problem?
13:33:07 <Qiming> the glance store by default is using swift as its backend
13:33:16 <Qiming> and ... swift is not yet py35 ready
13:33:22 <yanyanhu> sigh...
13:33:45 <Qiming> so ... need some fix to disable glance install
13:33:57 <Qiming> for api and functional tests, we need nothing
13:34:05 <XueFengLiu> can we remove py35 in gate side?
13:34:08 <Qiming> there must be a way to disable all them
13:34:16 <Qiming> we can set it to non-voting
13:35:02 <Qiming> if you are interested, you can check the log here: http://logs.openstack.org/75/455575/1/gate/gate-senlin-dsvm-tempest-py35-api/66595c2/logs/screen-g-api.txt.gz
13:35:44 <Qiming> glance is trying to invoke swift ...
13:35:53 <XueFengLiu> yes
13:36:01 <XueFengLiu> I saw it
13:36:11 <yanyanhu> one possible way is to define which services we want to install specificly in job template, rather then using default list. But this is not recommened
13:36:15 <XueFengLiu> So why we support py35 recently?
13:36:24 <XueFengLiu> Can we do this work later?
13:36:38 <Qiming> it is a pike cycle goal for the whole community
13:36:51 <yanyanhu> you need to carefully add every depedencies one by one
13:37:11 <yanyanhu> so it could be easy to make mistake and then fail the devstack installation...
13:37:12 <Qiming> another cross-community goal is to have senlin-api deployed as a wsgi module
13:37:31 <Qiming> I have checked our gate job script
13:37:49 <Qiming> we didn't do it very "carefully"
13:38:26 <yanyanhu> currently, we just use default list plus extra services we need
13:38:43 <Qiming> e.g. http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/senlin.yaml#n37
13:38:59 <Qiming> this line may have glance installed, and possibly swift
13:39:21 <Qiming> or swift wasn't installed ... so glance is getting connection refused error
13:39:42 <yanyanhu> but without it, heat installation will fail
13:39:51 <yanyanhu> since it is not in default project_list
13:40:01 <Qiming> and we were installing nova components using this line, right?
13:40:25 <yanyanhu> so git_clone won't work...
13:40:29 <yanyanhu> no
13:40:38 <yanyanhu> this is only for heat
13:40:49 <XueFengLiu> Yes, this need a  repeated attempts to conf local.conf
13:40:52 <yanyanhu> since heat is installed as a plugin
13:41:17 <Qiming> then how nova gets installed?
13:41:18 <yanyanhu> but nova/neutron/glance/cinder/keystone are not
13:41:28 <yanyanhu> those services are in default list
13:41:36 <yanyanhu> they will get installed by default
13:41:44 <yanyanhu> in default project_list
13:41:46 <XueFengLiu> yes
13:41:51 <yanyanhu> so git_clone works with them
13:42:00 <XueFengLiu> nova/cinder/glance/neutron/keystone
13:42:02 <yanyanhu> even you don't add them to project_list manually
13:42:15 <yanyanhu> but heat is not...
13:42:22 <Qiming> right, project list is the problem I was suspecting
13:42:24 <XueFengLiu> en
13:42:27 <yanyanhu> that's why we enable it explicitly and add it to project_list
13:42:29 <Qiming> if you compare this to heat.yaml
13:42:46 <Qiming> http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/heat.yaml#n50
13:43:04 <Qiming> they enable each and every service explicitly
13:43:08 <yanyanhu> Qiming, heat's job template is special :) they define all required services specificly using OVERRIDE_ENABLED_SERVICES
13:43:41 <Qiming> okay, that is a good trick
13:43:43 <yanyanhu> we tried this way, but per Andreas's suggestion, we'd better using default list + extra ones required
13:43:53 <yanyanhu> so we changed it back :)
13:44:10 <Qiming> I was really curious why glance store was set to swift
13:44:12 <Qiming> and where
13:44:47 <yanyanhu> could be in glance devstack plugin?
13:44:59 <yanyanhu> not sure... but that sounds weird...
13:45:13 <Qiming> yes, but I am not seeing any projects gate job explicitly overwriting it
13:45:47 <Qiming> only when s-api is enabled will devstack set the backend to swift
13:45:58 <Qiming> that brings me back to the services set up
13:46:19 <Qiming> because swift is known to be bad wrt py35
13:46:25 <Qiming> we need to exclude it
13:46:51 <Qiming> http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/heat.yaml#n64
13:47:11 <yanyanhu> honestly, I do expect a service_exlcude support in devstack gate...
13:47:43 <yanyanhu> just found there isn't such feature after searching...
13:47:48 <Qiming> that is beyond our control and I don't think we have extra bandwidth on fixing it that way
13:47:59 <Qiming> any way
13:48:06 <yanyanhu> they only support "services+=", but no "services-="...
13:48:21 <yanyanhu> so the only way is what heat is using now...
13:48:31 <yanyanhu> using OVERRIDE_ENABLED_SERVICES
13:49:01 <Qiming> right, we have returned to the same problem now
13:49:12 <yanyanhu> yes...
13:49:18 <Qiming> will try set the py35 gate nv tomorrow
13:49:26 <Qiming> then cut p-1 release
13:49:32 <yanyanhu> ok
13:49:43 <Qiming> are you aware of any high priority bugs to fix?
13:49:49 <Qiming> except for this: https://bugs.launchpad.net/senlin/+bug/1681620
13:49:51 <openstack> Launchpad bug 1681620 in senlin "ACTION failed when can't get the cluster lock" [Critical,New]
13:50:01 <yanyanhu> nope from me...
13:51:02 <XueFengLiu> Let me see the bug list
13:51:06 <Qiming> okay
13:51:21 <Qiming> #link https://bugs.launchpad.net/senlin/
13:51:30 <Qiming> #topic open discussions
13:51:43 <Qiming> since neither xinhui or elynn is in
13:52:02 <Qiming> I'm not sure anyone else have summit proposals to discuss
13:52:50 <XueFengLiu> I and haiwei in prepare for the proposal
13:52:52 <yanyanhu> ok, maybe next week. Only 3 weeks left?
13:53:11 <Qiming> yep
13:53:32 <Qiming> if you need help on polish it, pls share with the team
13:53:48 <XueFengLiu> sure, we will
13:54:04 <Qiming> cool
13:54:08 <Qiming> anything else?
13:54:38 <yanyanhu> nope
13:54:39 <XueFengLiu> Depends on adoption patch in senlin and the patch in tacker
13:54:58 <Qiming> okay
13:55:30 <Qiming> your work has a hard dependency on node adoption?
13:55:49 <XueFengLiu> not hard dependency
13:56:12 <XueFengLiu> :)
13:56:29 <XueFengLiu> If we have the feature, we can show it better
13:56:51 * Qiming feeling a little bit relieved
13:57:14 <Qiming> you will have it before the summit
13:57:15 <XueFengLiu> :)
13:57:28 <XueFengLiu> ok, greate
13:57:36 <Qiming> so, please review, :D
13:57:50 <XueFengLiu> sure, I will
13:58:02 <XueFengLiu> Thanks Qiming for the hard work
13:58:04 <Qiming> guess that's all for today?
13:58:12 <yanyanhu> seems so
13:58:14 <yanyanhu> :)
13:58:22 <XueFengLiu> no more from me
13:58:30 <Qiming> thank you all for joining, take back your two minutes
13:58:32 <Qiming> good night
13:58:34 <Qiming> #endmeeting