13:00:07 <Qiming> #startmeeting senlin
13:00:08 <openstack> Meeting started Tue Jul 12 13:00:07 2016 UTC and is due to finish in 60 minutes.  The chair is Qiming. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:11 <openstack> The meeting name has been set to 'senlin'
13:00:52 <Qiming> evening
13:00:58 <zzxwill_> Good evening.
13:01:01 <cschulz> Hi
13:01:10 <xuhaiwei_> hi
13:01:25 <yanyanhu> hi
13:01:25 <yanyanhu> Qiming, I thought you are offline :) my network is really poor, sigh
13:01:27 <elynn> Hi, already started?
13:01:38 <Qiming> yanyanhu, ok
13:01:49 <Qiming> let's get started
13:02:00 <Qiming> #link https://etherpad.openstack.org/p/senlin-newton-workitems
13:02:11 <Qiming> newton work items
13:02:56 <Qiming> I don't have additional agenda items to propose, if you have some, please revise the agenda and let me know
13:02:59 <Qiming> #link https://wiki.openstack.org/wiki/Meetings/SenlinAgenda
13:03:06 <lixinhui_> hi
13:03:12 <Qiming> back to etherpad
13:03:24 <Qiming> tempest testing - done?
13:03:35 <yanyanhu> hi, looks like my network disconnected for a little while
13:04:00 <yanyanhu> Qiming, yes
13:04:04 <yanyanhu> tempest test is done
13:04:07 <Qiming> great, thx
13:04:13 <Qiming> removing line 4-6
13:04:36 <Qiming> do we want to add some integration tests?
13:05:09 <Qiming> doesn't look like a challenging task, but still need some design
13:05:11 <yanyanhu> no problem :)
13:05:35 <Qiming> ok
13:05:46 <yanyanhu> Qiming, yes, that's a good idea
13:05:48 <Qiming> next is performance evaluation
13:06:01 <yanyanhu> we can start from a simple one
13:06:06 <Qiming> any updates? lixinhui_ ?
13:06:18 <yanyanhu> just to verify the interaction between senlin and nova
13:06:42 <lixinhui_> around the end of July
13:06:49 <lixinhui_> I can get performance test bed again
13:06:51 <Qiming> yanyanhu, that's fine
13:07:07 <Qiming> thanks, lixinhui_
13:07:11 <lixinhui_> then we can try the nova
13:07:12 <lixinhui_> settings
13:07:20 <Qiming> okay
13:07:49 <Qiming> what's the status of rally stress testing now? yanyanhu
13:08:21 <yanyanhu> that patch for cluster/profile plugins is still pending there
13:08:31 <yanyanhu> will contact with rally team to merge it
13:08:37 <yanyanhu> I think it has been ready
13:08:46 <Qiming> sounds good
13:09:06 <yanyanhu> in my local repo. I also finished test job for scaling and resizing
13:09:08 <Qiming> moving on ...
13:09:12 <Qiming> health management
13:09:21 <yanyanhu> but will wait for the basic cluster/profile support
13:10:03 <lixinhui_> I am investigating the nova event
13:10:09 <lixinhui_> about host failure
13:10:30 <Qiming> are they sending out any notifications when a compute node is down?
13:10:37 <lixinhui_> and manually try live miggration
13:10:56 <lixinhui_> Qiming, still need check
13:11:15 <lixinhui_> will give result this week
13:11:21 <Qiming> great!
13:11:29 <lixinhui_> seems live migration
13:11:44 <lixinhui_> behave not steady
13:11:54 <lixinhui_> and need quiet few of settings
13:12:08 <lixinhui_> like libvert version
13:12:31 <lixinhui_> and ssh connection between the source and target host
13:12:50 <Qiming> live migration is not easy
13:13:15 <lixinhui_> if so, we may need to involve system level operation
13:13:46 <Qiming> not sure how many people are doing live migration in production environments
13:13:48 <lixinhui_> since we can not think all the nodes ready for live migration
13:14:00 <Qiming> but we will need to know host failures anyway
13:14:22 <lixinhui_> okay
13:15:10 <lixinhui_> anyway, KVM live migration is less efficient than vsphere
13:15:37 <Qiming> the lbaas bug has been there for quite a while, just checked the bug comment history
13:16:06 <lixinhui_> the bug is tagged as "low handing fruit"
13:16:13 <Qiming> it is labeled 'low-hanging-fruit' ... but no one is fixing it, it is assigned and reassigned and reassigned ...
13:16:27 <lixinhui_> ...
13:16:36 <lixinhui_> I already assigned it to myself
13:16:45 <lixinhui_> but need people to review
13:16:50 <lixinhui_> if saw the history
13:17:00 <Qiming> yes, it was reported on Feb 23
13:17:03 <lixinhui_> I already sticked the fix link there
13:18:04 <Qiming> okay, that two patches are all that needed to fix the bug?
13:18:23 <lixinhui_> yes
13:18:55 <lixinhui_> one is framework, the other is implements for driver of octvia
13:19:13 <Qiming> then you are supposed to add the 'bug' tag like this in both patches: partial-bug: #1548774
13:19:13 <openstack> bug 1548774 in neutron "LBaas V2: operating_status of 'dead' member is always online with Healthmonitor" [Undecided,Incomplete] https://launchpad.net/bugs/1548774
13:19:25 <Qiming> because neither patch is closing the bug
13:21:08 <lixinhui_> okay
13:21:08 <lixinhui_> I will try
13:21:20 <Qiming> if we need to push octvia team, we should do it
13:22:04 <lixinhui_> need let the framework one pass firstly
13:22:23 <lixinhui_> then octavia and nsx will follow quickly
13:22:46 <Qiming> so there is a dependency between those patches
13:22:56 <Qiming> need to remind the reviewers as well
13:22:59 <lixinhui_> yes
13:23:16 <Qiming> ok
13:23:19 <Qiming> threshold for cluster health status
13:23:22 <lixinhui_> already add the dependency in the commit message
13:23:32 <lixinhui_> will add comments if help
13:23:56 <Qiming> lixinhui_, contact the reviewers if necessary
13:24:17 <Qiming> not all reviewers have subscribed to bug reports I think
13:24:36 <lixinhui_> oh, okay
13:24:41 <Qiming> comments there may get unnoticed/ignored easily
13:25:17 <Qiming> threshold for cluster health status, do we agree to remove that item from the etherpad?
13:26:00 <Qiming> we had that discussion at least twice in the past, it seems to me we don't have an urgent need to add an additional watermark at the moment
13:26:55 <Qiming> ?
13:27:35 <lixinhui_> I agree
13:27:40 <yanyanhu> hi, just dropped
13:27:43 <xuhaiwei_> yes, we can remove I think
13:28:01 <Qiming> okay, removing it
13:28:50 <Qiming> about the HA story line, I'm not aware of any comments/revisions at this moment
13:29:24 <Qiming> please don't hesitate to input your thoughts if you have got some
13:29:37 <Qiming> fencing support
13:30:02 <Qiming> any design/code for review?
13:30:39 <lixinhui_> I can share manual steps out
13:30:52 <lixinhui_> but got no much time thinking design yet
13:31:15 <Qiming> okay, if needed, we can create a BP for this
13:31:24 <Qiming> or a spec
13:31:28 <lixinhui_> okay
13:31:43 <Qiming> need a "whiteboard" to collect ideas
13:32:08 <Qiming> moving on
13:32:20 <lixinhui_> need cluster-do firstly to install fencing lib
13:32:28 <lixinhui_> maybe ...
13:32:40 <lixinhui_> we discuss more in next few days
13:32:57 <Qiming> no update on documentation from my side, deeply trapped by troubles/happiness at home
13:33:27 <Qiming> okay, we can discuss that for sure
13:33:27 <zzxwill_> 'some use cases' for what function?
13:33:48 <Qiming> I'm not sure if we need to install something
13:34:27 <Qiming> zzxwill_, it is about autoscaling, load-balancing, rolling-upgrade, high-availability ...
13:34:44 <zzxwill_> Noted.
13:34:57 <Qiming> the idea is to write something so that users will get a better idea what they can do with senlin
13:35:16 <Qiming> moving on ...
13:35:25 <Qiming> container support, haiwei?
13:35:51 <xuhaiwei_> I want to have some discussion with you guys
13:36:30 <Qiming> sure, feel free to call for a online discussion or a phone call, ... whatever
13:36:36 <xuhaiwei_> first one is shall we use floating_ip in docker client's url
13:36:42 <Qiming> last review comment was from May 26
13:36:53 <Qiming> (referring to the spec)
13:37:16 <xuhaiwei_> the spec may be not a good place to discuss
13:37:25 <zzxwill_> Qiming, for example, we need to complete 'Making Your Cluster Elastic
13:37:25 <zzxwill_> ' http://docs.openstack.org/developer/senlin/tutorial/autoscaling.html?
13:37:44 <xuhaiwei_> I will create an ether pad to list the questions I want to discuss with you
13:37:53 <Qiming> okay, got it, we have quite some tech details for discussion
13:38:08 <Qiming> okay, pls share with the team
13:38:15 <xuhaiwei_> ok
13:38:37 <Qiming> zzxwill_, correct, that is the tutorial documentation we have yet to work on
13:39:34 <zzxwill_> Got it, thanks.
13:39:37 <Qiming> engine side, no update from me
13:39:51 <Qiming> zaqar receiver
13:40:13 <Qiming> yanyan has proposed patches to openstacksdk, not elegant solution, but workable
13:40:26 <yanyanhu> yes, spent some time working on zaqar driver last two weeks
13:40:38 <Qiming> hopefully we can convince that team that a 'headers' param is needed for sure
13:41:09 <yanyanhu> I think biran has agree we can use current implementation as solution for zaqar api resources
13:41:12 <Qiming> we need that param to send 'openstack-api-version' header so that 'cluster-collect' can work
13:41:22 <yanyanhu> but not sure whether there will be a more generic solution come in future
13:41:29 <Qiming> I think that is a wrong decision
13:41:52 <Qiming> the correct fix should be revise the resource2.Resource code, so that a 'headers' param can be passed
13:42:11 <Qiming> then you don't have to duplicate the code in your queue implementation
13:42:33 <yanyanhu> Qiming, yes
13:42:33 <Qiming> anyway, a small step is still a step forward
13:42:53 <yanyanhu> btw, a more difficult issue we will encounter is zaqar queue update API needs special definition for 'content-type' header
13:43:11 <Qiming> event/notification --- I don't have progress to update, sorry
13:43:42 <Qiming> we can hide that behind the zaqar service api in SDK
13:43:45 <yanyanhu> but in keystoneauth, request method will always set 'content-type' to 'application/json' when sending out patch request
13:44:12 <yanyanhu> so still didn't find a workaround for it
13:44:14 <Qiming> the internal resource2.Resource class should open to all headers
13:44:43 <Qiming> .... I'm speechless on this
13:44:54 <Qiming> have you mentioned that to feilong?
13:45:13 <yanyanhu> unless we can make some changes on the implementation of keystoneauth1.session
13:45:13 <yanyanhu> although we currently don't need queue update API interface I think
13:45:20 <Qiming> it sounds like their problem to fix
13:45:25 <yanyanhu> Qiming, yes
13:45:28 <yanyanhu> I feel so as well
13:45:33 <yanyanhu> will talk with feilong about it
13:45:38 <yanyanhu> looks like a big problem
13:45:39 <Qiming> okay
13:46:04 <Qiming> let's move on to the next topic
13:46:15 <Qiming> #topic barcelona summit
13:46:46 <Qiming> I have spent some time yesterday and today on reviewing all the proposals I know of
13:46:59 <Qiming> 15208: Get Your Resource Pool Managed on OpenStack
13:46:59 <Qiming> 15209: Managing clusters of thousands of VMs using Senlin
13:46:59 <Qiming> 15037: On Building an Auto-healing Resource Cluster using Senlin
13:47:43 <lixinhui_> And
13:47:45 <lixinhui_> 15251
13:47:58 <lixinhui_> Extensible, Flexible, Fine-granularity Cluster Management and Controlling Tool - Senlin
13:48:07 <Qiming> when reviewing these proposals, I really like 15037 - the HA story and 15209 - the scalability analysis and experience sharing
13:48:46 <Qiming> still strugging with 15208 - the comparison between heat resource types and senlin service
13:48:51 <yanyanhu> for 15208, maybe change the name to "Get Your Resource Pool Better Managed on OpenStack"?
13:49:20 <Qiming> when revising that one, I have got some feeling that the comparison is unfair
13:49:32 <Qiming> we are comparing apple to orange
13:49:45 <Qiming> we cannot compare a resource type to a service
13:50:31 <xuhaiwei_> I think we can say by using Senlin, user can use Heat better
13:50:37 <Qiming> what's more, we have already delivered a strong message during Austin summit -- what senlin is, why we started it, it's status, its plan
13:50:50 <elynn> I just want users to be aware of senlin cluster and maybe migrate from heat ASG to senlin cluster after listening this presentation.
13:51:13 <Qiming> elynn, users have their own concerns when doing this migration
13:51:32 <Qiming> if that is the point we want to deliver, we'd better provide some tool/guide for them
13:51:53 <yanyanhu> so maybe we focus on a specific scenario, autoscaling to deliver the idea
13:52:09 <Qiming> it doesn't sound that sexy a topic for the barcelona summit
13:52:20 <yanyanhu> we compare the cons and pros and why there are those difference
13:52:40 <yanyanhu> then give user the knowledge about both of them
13:52:42 <elynn> By comparing the details of heat ASG and senlin, they might know of what heat ASG can do and what Senlin cluster can do, so they won't keep asking heat to add more features to ASG
13:52:48 <Qiming> why do we need to do that again and again?
13:53:22 <Qiming> elynn, are you aware of any such requests lately?
13:53:41 <yanyanhu> this can be a pure technical comparison I feel
13:53:51 <Qiming> if there are such request, can we redirect them to senlin ?
13:54:06 <elynn> Not from community
13:54:19 <yanyanhu> I guess some users will ask this question :P
13:54:30 <Qiming> back in austin, we have three talks, and in each one of them, we mentioned that difference
13:54:46 <yanyanhu> since they don't have time to read the code and they want to understand the tech differentiation between these two designs :)
13:54:55 <Qiming> it is not interesting to me
13:55:09 <Qiming> I'm more interested in topic 4
13:55:17 <elynn> I think we might need to address those questions in a public place, like summit.
13:55:17 <Qiming> the cluster-do feature
13:55:39 <Qiming> which is totally new, and designed to solve real life problems for operators
13:56:14 <Qiming> this time, we have got some restrictions on proposals
13:56:40 <Qiming> each presenter can have at most 3 talk proposals, and each proposal may have at most 3 speakers
13:57:12 <Qiming> so ... I'd like moving my name from the 3rd topic to the 4th one
13:57:42 <yanyanhu> that is ok I think. We can reorg our presentation later if they are accepted I think ?
13:58:17 <yanyanhu> to better present to the audience
13:58:19 <lixinhui_> sounds good
13:58:30 <lixinhui_> the limits are hash
13:58:37 <lixinhui_> s/hash/harsh
13:58:39 <yanyanhu> so the deadline is tomorrow night beijing time?
13:59:00 <Qiming> yes
13:59:06 <Qiming> we don't have much time on this
13:59:13 <yanyanhu> ok, so we have one day to revise the proposal
13:59:19 <Qiming> I have revised topic 3
13:59:26 <yanyanhu> need to finish it tomorrow morning
13:59:42 <zzxwill_> 'The content on this page is obsolete. The autoscaling solution is offloaded from Heat to Senlin since Mitaka." I learn this official message recently. Sounds cool!
13:59:42 <Qiming> I mean the text, I'd propose replace my name by someone else
13:59:44 <elynn> yes, not much time to do it
13:59:52 <Qiming> ... oh ... time's up
14:00:02 <Qiming> pls go back to #senlin channel for this
14:00:05 <yanyanhu> lets move back to senlin channel
14:00:10 <Qiming> we'd release this channel
14:00:12 <lixinhui_> ok
14:00:14 <zzxwill_> Okay.
14:00:15 <Qiming> thanks for joining everyone
14:00:17 <Qiming> #endmeeting