13:00:11 <yanyanhu> #startmeeting senlin
13:00:12 <openstack> Meeting started Tue Nov 22 13:00:11 2016 UTC and is due to finish in 60 minutes.  The chair is yanyanhu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:16 <openstack> The meeting name has been set to 'senlin'
13:00:28 <yanyanhu> hello
13:00:32 <lixinhui> hi
13:00:37 <XueFengLiu> hi
13:00:39 <haiwei_> hi
13:00:42 <yanyanhu> evening
13:00:56 <yanyanhu> hi, xinhui, xuefeng, haiwei
13:01:08 <XueFengLiu> hi,all
13:01:17 <Qiming> hello
13:01:18 <yanyanhu> lets wait for a while for other attenders
13:01:20 <yanyanhu> hi, Qiming
13:02:03 <yanyanhu> ok, lets start
13:02:05 <yanyanhu> https://wiki.openstack.org/wiki/Meetings/SenlinAgenda#Agenda_.282016-11-22_1300_UTC.29
13:02:10 <yanyanhu> here is the agenda
13:02:16 <yanyanhu> please feel free to add items
13:02:24 <yanyanhu> #topic ocata work items
13:02:33 <yanyanhu> ocata workitems
13:02:53 <yanyanhu> ocata-1 has been released in last week
13:03:08 <yanyanhu> so now we are working for ocata-2
13:03:23 <yanyanhu> test
13:03:28 <yanyanhu> performance test, no progress
13:04:01 <yanyanhu> improve tempest API test. since the versioned request support is almost done, will consider to resume this work
13:04:35 <yanyanhu> the basic idea is adding verifcation of exception message
13:05:10 <yanyanhu> next one
13:05:13 <yanyanhu> HA support
13:05:20 <yanyanhu> hi, Qiming, xinhui, your turn
13:05:44 <Qiming> no update from me
13:05:54 <yanyanhu> ok
13:06:00 <yanyanhu> lixinhui, and you?
13:06:10 <yanyanhu> I noticed this bug is marked as won't fix
13:06:12 <yanyanhu> https://bugs.launchpad.net/neutron/+bug/1548774
13:06:12 <openstack> Launchpad bug 1548774 in senlin "LBaas V2: operating_status of 'dead' member is always online with Healthmonitor" [Undecided,New]
13:06:17 <yanyanhu> the bug report about lbaas
13:06:22 <lixinhui> I will propose a BP this week
13:06:24 <Qiming> yes, I challenged that decision
13:06:33 <yanyanhu> Qiming, I think you have asked armando
13:06:38 <yanyanhu> yes
13:07:02 <yanyanhu> lixinhui, great
13:07:20 <Qiming> lixinhui, bp about?
13:07:35 <lixinhui> to ocativia
13:07:41 <Qiming> okay
13:07:46 <lixinhui> no matter what is the result
13:07:51 <lixinhui> we should try
13:07:56 <Qiming> yep
13:08:07 <yanyanhu> btw, the following patch is abandon for no update in last 4 weeks
13:08:09 <yanyanhu> https://review.openstack.org/#/c/325624/
13:08:19 <yanyanhu> lixinhui, you may want to restore it if needed :)
13:08:20 <lixinhui> yes
13:08:24 <lixinhui> I know this
13:08:40 <lixinhui> but totally different resolution
13:08:44 <yanyanhu> ok
13:08:58 <Qiming> cool
13:09:05 <yanyanhu> so you will reuse this patch or propose a new one?
13:09:15 <lixinhui> Propose a new one
13:09:25 <Qiming> while we are keeping an eye on the LB service
13:09:41 <Qiming> I'm wondering if we should look for some alternatives
13:10:20 <yanyanhu> Qiming, you mean some loadbalancers outside openstack?
13:10:29 <yanyanhu> e.g. hardware based ones
13:10:29 <Qiming> the reason we are investigating and even try making things right with LB is for health check
13:11:11 <Qiming> users will decide whether they will use neutron-lbaas or not
13:11:36 <Qiming> by alternative, I mean some 'ping', 'http get' requests sent to the cluster nodes
13:11:49 <yanyanhu> I see
13:12:00 <yanyanhu> as poller for health check
13:12:25 <Qiming> it is a little bit different from what LB is doing, because what we will be doing is from the management network
13:12:43 <Qiming> LB can do it from guest network
13:13:42 <yanyanhu> guest network, you mean?
13:13:49 <Qiming> tenant network
13:14:10 <yanyanhu> data plane?
13:14:14 <yanyanhu> I see
13:14:16 <Qiming> yes
13:14:47 <Qiming> if we decide to do this, I'd suggest we start a new service process for this
13:14:56 <yanyanhu> if so, we may need to find a way to make senlin hm reach to tenant network
13:14:59 <Qiming> poller will consume a lot of cpu cycles I believe
13:15:11 <yanyanhu> Qiming, yes, that makes sense
13:15:33 <Qiming> having health manager configured into the tenant network is an option though
13:15:35 <yanyanhu> currently, all senlin components are running in management network
13:15:41 <Qiming> yes
13:15:55 <Qiming> all VMs should be reachable from management network
13:16:02 <Qiming> that is a safe assumption I think
13:16:13 <Qiming> or else, ceilometer cannot work, heat cannot work ...
13:16:18 <yanyanhu> Qiming, yes, just we need a graceful and scalable way to support it
13:16:27 <Qiming> I never worry about management network reachability
13:16:51 <yanyanhu> for if there are lots of vms in a huge cluster, ping them one by one could be a big problem
13:17:36 <Qiming> if you have a larget scale cluster, you will highly likely prolong the interval between two polling operations
13:17:48 <yanyanhu> yes
13:18:12 <Qiming> because the base is large, the crashes of one or two VM should be solved, but it won't be that urgent
13:18:30 <yanyanhu> yes
13:18:56 <yanyanhu> anyway, we hope lbaas can provide us some help here
13:19:05 <Qiming> there won't be a decent solution for this, even for listeners, you still have to process a lot of events
13:19:14 <Qiming> right
13:19:21 <yanyanhu> if not, we may consider plan B to support status check by ourselves
13:19:31 <Qiming> +2
13:19:48 <haiwei_> tacker is using ping
13:19:50 <yanyanhu> hope xinhui's work can help to solve this issue :)
13:20:12 <yanyanhu> haiwei_, any detail about how they support it?
13:20:24 <Qiming> ping can be used, but ping won't prove you that the service is still working
13:20:26 <lixinhui> could you point us to the code, haiwei_
13:20:35 <lixinhui> thx
13:20:40 <haiwei_> just ping the vm, not anything special
13:20:54 <lixinhui> ok
13:21:28 <yanyanhu> haiwei_, do they perform the ping operation periodically?
13:21:35 <Qiming> http://git.openstack.org/cgit/openstack/tacker/tree/tacker/vnfm/monitor_drivers/http_ping/http_ping.py
13:21:36 <yanyanhu> or it is triggered by some events
13:21:54 <haiwei_> https://github.com/openstack/tacker/blob/master/tacker/vnfm/monitor_drivers/ping/ping.py
13:21:59 <haiwei_> you are quick
13:22:01 <haiwei_> that is
13:22:07 <yanyanhu> :)
13:22:24 <Qiming> and a native ping: http://git.openstack.org/cgit/openstack/tacker/tree/tacker/vnfm/monitor_drivers/ping/ping.py
13:22:55 <lixinhui> thanks, Qiming
13:23:05 <yanyanhu> great. we can refer to it
13:23:07 <haiwei_> seems not periodically, yanyanhu
13:23:15 <yanyanhu> haiwei_, ok
13:23:29 <Qiming> looks like this is how it is invoked: http://git.openstack.org/cgit/openstack/tacker/tree/tacker/vnfm/monitor.py#n96
13:23:37 <yanyanhu> periodical ping could cause huge overhead as Qiming said
13:23:50 <Qiming> it is doing periodical pings
13:23:57 <haiwei_> yes
13:24:07 <yanyanhu> oops, I saw check_intvl
13:24:20 <Qiming> default 10 secons
13:24:32 <yanyanhu> the logic is not complicated
13:24:52 <yanyanhu> so may need some evaluations here
13:25:12 <yanyanhu> once we decide to apply similar design
13:25:16 <Qiming> code is easy, design is the difficult part
13:25:17 <haiwei_> not test this yet, don't know if it works or not
13:25:26 <yanyanhu> I see
13:25:30 <yanyanhu> Qiming, +1
13:26:50 <yanyanhu> ok, lets wait for xinhui's work in lbaas before going along this way
13:27:13 <Qiming> the other side of the HA solution is about usage scenarios I think
13:27:29 <Qiming> need some tutorials from xinhui some day on mistral
13:27:35 <Qiming> NOT today
13:27:45 <yanyanhu> Qiming, sure, also looking forward to it
13:28:04 <yanyanhu> maybe we can get a lecture from xinhui someday :)
13:28:06 <lixinhui> w:)
13:28:07 <Qiming> I can pay for the lunchbox
13:28:19 <yanyanhu> haha, I can pay for beverage
13:28:22 <lixinhui> hope so:)
13:28:49 <yanyanhu> ok, lets decide it offline after the meeting :)
13:28:49 <haiwei_> a small senlin meetup
13:28:57 <yanyanhu> haiwei_, yep :)
13:29:02 <lixinhui> :)
13:29:04 <haiwei_> ptg?
13:29:14 <Qiming> haiwei_, can call in and provide some music as background
13:29:16 <yanyanhu> oh, right, for we won't appear in ptg
13:29:33 <elynn> : D
13:29:37 <yanyanhu> a meetingup may be needed for us
13:29:41 <haiwei_> my boss asked me today about senlin ptg
13:30:07 <yanyanhu> haiwei_, we don't have plan to join it :( and also for we are marked as single-vendor now
13:30:24 <yanyanhu> so one mission in this cycle is adding our diversity :)
13:30:30 <haiwei_> I know it, yanyanhu, so I just smiled to him
13:30:38 <Qiming> the reason we were "awarded" that label is review statistics
13:30:52 <yanyanhu> hope more contribution especially code review from you guys who are not ibmer :P
13:30:53 <Qiming> 90% reviews are from IBM
13:30:57 <yanyanhu> yep
13:31:11 <haiwei_> ok
13:31:35 <Qiming> we have been improving quickly recently
13:32:00 <yanyanhu> yep, especially thanks XueFengLiu lvdongbin and also Ruijie_ :)
13:32:39 <yanyanhu> lets spend more effort here :)
13:32:53 <yanyanhu> ok, we can talk about meetup later
13:32:54 <Ruijie_> no problem :)
13:32:56 <lixinhui> :)
13:33:02 <yanyanhu> so lets move on?
13:33:06 <XueFengLiu> :)
13:33:13 <Qiming> ibm reviews: 68% now
13:33:27 <Qiming> ibm commits: 41%
13:33:51 <yanyanhu> great
13:34:06 <yanyanhu> ok, next item, document
13:34:10 <Qiming> very happy to see fresh blood in the team
13:34:10 <yanyanhu> no progress I think
13:34:19 <yanyanhu> Qiming, definitely :)
13:34:20 <Qiming> yes, skip that please, :(
13:34:30 <yanyanhu> ok
13:34:34 <yanyanhu> versioned request support
13:34:35 <yanyanhu> almost done
13:34:44 <yanyanhu> with the effort from all the team
13:34:55 <Qiming> em, one problem though
13:34:57 <Qiming> https://blueprints.launchpad.net/senlin/+spec/objectify-service-requests
13:35:01 <yanyanhu> hopefully it can be finished in one or two weeks
13:35:06 <Qiming> no patch has been referencing this BP
13:35:18 <yanyanhu> ...that's true
13:35:23 <Qiming> although we may have a few dozens of patches working on this
13:35:38 <yanyanhu> sigh...
13:35:51 <yanyanhu> didn't notice this issue before
13:35:56 <Qiming> I'll mark it ... "Good Progress", if no objections
13:36:05 <yanyanhu> agree...
13:36:10 <Qiming> it was my fault, I didn't do this, so ...
13:36:23 <yanyanhu> you built the basement :)
13:36:45 <Qiming> and the habit of not referencing the bp
13:36:59 <yanyanhu> :P
13:37:11 <yanyanhu> ok, next one
13:37:17 <yanyanhu> container profile
13:37:31 <yanyanhu> haiwei_, any new progress?
13:37:35 <haiwei_> no much progress , yanyanhu
13:37:45 <yanyanhu> ok
13:37:51 <yanyanhu> so, next one
13:37:54 <yanyanhu> event/notification
13:38:01 <Qiming> I think the dependency handling work is good
13:38:03 <yanyanhu> I believe Qiming did lots of work here
13:38:18 <Qiming> yes, quite some code written and deleted locally
13:38:43 <Qiming> I believe the design (not written out anywhere) is 70%
13:38:48 <yanyanhu> great
13:39:11 <Qiming> the last mile is about generalization about cluster/node actions
13:39:44 <yanyanhu> generalization, you mean?
13:39:57 <yanyanhu> the format? or the timing that generate event
13:40:04 <Qiming> extract generic parameters/properties while ensuring important data can be presented to users, that is ... a big puzzle for me
13:40:32 <yanyanhu> I see
13:40:54 <Qiming> the format, do we want a ClusterScaleOutActionPayload and a ClusterResizeActionPayload, or we stop at ClusterActionPayload
13:41:12 <yanyanhu> myself prefer the last one :)
13:41:13 <Qiming> have been playing with different options recently
13:41:21 <Qiming> yes, me too
13:41:49 <Qiming> then that payload have to be very expressive, capable of delivering the event details for all actions
13:41:56 <yanyanhu> yes
13:42:05 <yanyanhu> so the generalization is very important
13:42:28 <Qiming> notification object, when serialized ...
13:42:58 <Qiming> will be something like this: http://paste.openstack.org/show/590069/
13:43:29 <yanyanhu> wow, looks great
13:43:44 <Qiming> line 33 in that paste will only appear when something wrong happens
13:44:06 <yanyanhu> I see
13:44:33 <yanyanhu> so it will be empty when everything is good
13:44:36 <Qiming> still working on the line 44 part, trying to get the most interesting properties from all action types, while keeping the structure concise
13:44:58 <Qiming> you know ... exception is an ObjectField with nullable=True
13:45:07 <yanyanhu> ah, right
13:45:38 <Qiming> the lin 19 part will be replace by a 'node' dict when dealing with nodes
13:45:39 <yanyanhu> great
13:46:36 <yanyanhu> the structure looks good
13:46:57 <yanyanhu> thanks for this great work, Qiming :)
13:46:59 <Qiming> other things to be settled is about event_type, which isn't a big issue
13:47:34 <haiwei_> what about the event log file?
13:47:45 <haiwei_> there will be a output file, right?
13:48:07 <Qiming> currently we focus on database and message backend (aka. driver)
13:48:18 <haiwei_> ok
13:48:34 <yanyanhu> haiwei_, file will be an option
13:48:36 <Qiming> if there are requirements, we can quickly add a new driver, dumping  these into a JSON file, for example
13:49:02 <haiwei_> got it
13:49:09 <yanyanhu> Qiming, the target milestone of basic support is ocata-2?
13:49:30 <Qiming> just leave it there and see if I can work my ass off
13:49:51 <yanyanhu> great, just take your time :)
13:49:59 <Qiming> will propose a postpone if I figured I cannot finish it by o-2
13:50:06 <yanyanhu> ok
13:50:35 <yanyanhu> great, this will be a very important feature we will include in o release
13:50:55 <yanyanhu> so those are all items in the list
13:51:17 <yanyanhu> any more you guys are working on?
13:51:35 <yanyanhu> #topic open discussion
13:51:44 <yanyanhu> ok, open discussion now
13:52:08 <yanyanhu> any topic you want to discuss?
13:52:16 <Qiming> if there are still some bandwidth from team, I'd propose we work on improving container cluster support
13:52:34 <Qiming> haiwei has done a great job, but it is still a starting point
13:52:44 <yanyanhu> yes
13:53:04 <XueFengLiu> good idea
13:53:10 <yanyanhu> hope we can have at least one case that can be successfully run based on senlin container cluster
13:53:40 <Qiming> my dreamed goal is: start a container cluster using a single senlin command, which will be load-balanced, auto-scalable and health managed
13:54:08 <yanyanhu> :)
13:54:14 <elynn> Wow
13:54:31 <Qiming> too ambitious ?
13:54:43 <yanyanhu> honestly, a little bit :P
13:54:56 <yanyanhu> at least for this cycle, it is
13:55:04 <elynn> a little bit, but we should be there
13:55:17 <Qiming> I didn't say the goal is for this cycle, :D
13:55:29 <Qiming> but we have to work on it
13:55:33 <yanyanhu> haha, understand
13:55:34 <XueFengLiu> :)
13:55:35 <yanyanhu> sure
13:56:10 <Qiming> that will be a great show of senlin's capability
13:56:20 <yanyanhu> yes, definitely
13:56:53 <Qiming> been looking at spark architecture recently, as a beginner
13:57:04 <yanyanhu> Qiming, :)
13:57:14 <Qiming> when comparing its low-level architecture, we only miss the scheduling part
13:57:48 <yanyanhu> since we are not a scheduler actually
13:57:53 <yanyanhu> senlin is more like an engine
13:57:55 <Qiming> we don't have one
13:58:00 <Qiming> even a simple one
13:58:17 <yanyanhu> maybe placement policy can help to address this issue
13:58:22 <Qiming> that has been a blocking factor for haiwei I believe
13:58:43 <Qiming> yes, for most short-lived containers, it is just a placement decision
13:58:52 <Qiming> you won't migrate them around
13:59:02 <yanyanhu> yes
13:59:24 <yanyanhu> ok, time is almost over. lets move back to senlin channel
13:59:28 <Qiming> aren't containers designed to be short-lived?
13:59:34 <Qiming> :D
13:59:37 <Qiming> a big question
13:59:38 <yanyanhu> Qiming, yes, it is :)
13:59:41 <yanyanhu> in most case
13:59:57 <yanyanhu> ok, we can have further discussion later
14:00:02 <yanyanhu> thanks all you guys for joinging
14:00:05 <yanyanhu> have a good night
14:00:12 <yanyanhu> #endmeeting