13:00:26 <Qiming> #startmeeting senlin
13:00:27 <openstack> Meeting started Tue Jul 19 13:00:26 2016 UTC and is due to finish in 60 minutes.  The chair is Qiming. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:31 <openstack> The meeting name has been set to 'senlin'
13:01:00 <Qiming> hello
13:01:09 <zzxwill> Good evening.
13:01:17 <lixinhui_> good evening
13:01:32 <cschulz> good evening
13:01:51 <Qiming> evening everyone
13:02:04 <Qiming> #topic agenda items
13:02:29 <Qiming> if you have any topics for discussion, please add them here: https://wiki.openstack.org/wiki/Meetings/SenlinAgenda
13:03:23 <Qiming> let's start with the progress etherpad
13:03:29 <Qiming> #topic newton work items
13:03:33 <Qiming> #link https://etherpad.openstack.org/p/senlin-newton-workitems
13:04:00 <Qiming> seems yan yan is not online
13:04:23 <Qiming> the rally tests are pending review
13:04:49 <Qiming> hopefully, with some minor tweaks, they will get merged
13:05:05 <Qiming> lixinhui_, any update from scaling tests?
13:05:49 <lixinhui_> Intel will loan vmware a super large test bed
13:06:06 <lixinhui_> if time permits, we can try to run senlin on it
13:06:15 <Qiming> wow, what do you mean by "super large"?
13:06:37 <lixinhui_> around thousand machines
13:06:46 <lixinhui_> physical hosts
13:07:01 <lixinhui_> now it is used to running vio long run tests
13:07:02 <Qiming> it would be super great if we can get some perf data on a large scale deployment
13:07:17 <lixinhui_> after that if the time permits, we can try to run senlin
13:07:24 <Qiming> great
13:07:50 <lixinhui_> that belongs to OpenStack foundation
13:07:58 <Qiming> please take notes on all aspects of senlin behavior
13:08:11 <lixinhui_> all the machines come from different aspects
13:08:13 <Qiming> not just performance, but also concurrency, consistency etc
13:08:21 <lixinhui_> hope so
13:08:33 <lixinhui_> please share me details of china mobile deployment
13:08:46 <lixinhui_> hope we can remove basic problems
13:08:54 <lixinhui_> by these experience
13:09:05 <Qiming> I don't have lot to share other than their public presentation
13:09:15 <lixinhui_> ok
13:09:36 <lixinhui_> little bit pity if most of time spent on small problems
13:09:43 <Qiming> I believe you have got some doc from liu junwei, correct?
13:10:08 <lixinhui_> not really
13:10:08 <lixinhui_> I just got his chart
13:10:12 <lixinhui_> but no detailed configurations on nova and so on
13:10:12 <Qiming> about message queue, about nova api etc
13:10:32 <lixinhui_> do not have these deyails
13:10:34 <Qiming> if you do a search
13:10:39 <lixinhui_> if possible
13:10:45 <Qiming> I think he has some presentation on last summit
13:10:45 <lixinhui_> I want to get these info
13:10:56 <Qiming> about how they solved the bottleneck problem
13:11:13 <lixinhui_> I once learned that
13:11:17 <Qiming> it is something you can find online
13:11:21 <lixinhui_> but still need details
13:11:48 <lixinhui_> the talk only mention the problem side but need details to reproduce
13:11:53 <lixinhui_> you know
13:12:01 <lixinhui_> just like paper and deployment document
13:12:29 <Qiming> then you have to contact liu junwei directly
13:12:40 <Qiming> see if they have details that meet your request
13:14:11 <Qiming> moving on
13:14:15 <Qiming> health management
13:14:21 <lixinhui_> performance/salability really need more time
13:14:52 <Qiming> yes, I know that, I have been working on performance for at least 3 years in a row, you know
13:15:13 <lixinhui_> yes, I see
13:15:29 <Qiming> any progress would be good to know for the team
13:15:38 <Qiming> not just the final result
13:16:09 <Qiming> the LB side, what's the status of the bug?
13:16:12 <lixinhui_> will yanyan or eldon can get the deployment document from china mobile?
13:16:28 <Qiming> eldon is from china mobile
13:16:52 <Qiming> you can just ask for this doc for sharing, if there is such a doc
13:16:57 <lixinhui_> you guys have a talk on this
13:16:57 <Qiming> it is not secret
13:17:00 <lixinhui_> for summit
13:17:08 <Qiming> we didn't
13:17:28 <Qiming> we don't have any prep on that yet
13:18:02 <lixinhui_> then can not put this into schedule
13:18:07 <lixinhui_> will need more time to prepare
13:18:16 <Qiming> just like the cluster do, we don't have implementation there
13:18:16 <lixinhui_> if I got this test bed, only one week maybe
13:18:18 <lixinhui_> for us
13:18:20 <Qiming> correct?
13:18:42 <lixinhui_> deadline for the test bed is end of July
13:18:58 <Qiming> okay ...
13:19:24 <lixinhui_> maybe eldon is the proper name for the scalability tests
13:19:48 <lixinhui_> at least, he can summarize all the configs of china mobile
13:19:55 <lixinhui_> as the first step
13:20:00 <Qiming> can we do some perf test if we cannot get physical machines?
13:20:02 <lixinhui_> do you agree
13:20:06 <lixinhui_> ?
13:20:07 <Qiming> yes sure
13:20:16 <Qiming> could have changed the name to someone else
13:20:43 <Qiming> the item has been there for quite some time
13:20:50 <lixinhui_> okay
13:20:54 <lixinhui_> if we have the first doc
13:20:55 <Qiming> if priority is not high, we can move it back to TODO.rst
13:21:12 <lixinhui_> then we can run further tests to reproduce or optimize
13:21:26 <Qiming> I see now
13:21:28 <lixinhui_> seems 600 or 800 nodes
13:21:41 <lixinhui_> in china mobile's cloud
13:21:44 <lixinhui_> right?
13:21:51 <Qiming> I said I don't know
13:21:56 <lixinhui_> ok
13:21:59 <lixinhui_> let it go
13:22:11 <lixinhui_> just trying to know more information about it
13:22:14 <Qiming> you can either contact eldon
13:22:25 <Qiming> or you can ask yanyan to do the communication
13:22:27 <Qiming> whatever
13:22:40 <lixinhui_> let it go
13:22:49 <Qiming> moving on
13:23:00 <Qiming> the health management topic
13:23:15 <Qiming> no comment on the etherpad during the past weeks
13:23:19 <Qiming> fine
13:23:36 <lixinhui_> the bug is reviewed
13:23:36 <Qiming> LBaaS bug
13:23:45 <Qiming> marked as incomplete
13:24:10 <lixinhui_> https://review.openstack.org/#/c/325624/
13:24:53 <lixinhui_> Stephen hope to add tests
13:25:12 <lixinhui_> but feel no tests need to add if we follow the original flavor
13:25:13 <Qiming> finally someone is reviewing it
13:25:20 <lixinhui_> enen
13:25:32 <Qiming> then argue with the reviewer
13:25:42 <lixinhui_> but if he reviewed the code really, he will know now test should be added
13:25:49 <Qiming> try convince him/her that a test is not necessary
13:27:02 <Qiming> still I agree to the reviewer that some tests would be good
13:27:27 <Qiming> e.g. make sure BasePoolManager is a subclass of driver_mixins.BaseStatusMixin
13:27:44 <lixinhui_> sounds reasonable
13:27:51 <Qiming> if, for some reasons, this subclassing is changed in future, your tests there will detect it
13:28:29 <Qiming> pls continue work with those guys to get the bug fixed
13:28:31 <lixinhui_> there is no test for original BasePoolManager even
13:28:43 * Qiming is still wondering why that bug is marked incomplete
13:28:52 <Qiming> then you can add one
13:29:01 <Qiming> it is just a few lines code
13:29:31 <lixinhui_> then someone review to suggest add more tests
13:29:32 <lixinhui_> ...
13:29:39 <lixinhui_> for the whole functions
13:29:44 <lixinhui_> anyway
13:29:48 <Qiming> some times, people get picky when reviewing code ...
13:29:59 <lixinhui_> we can try if anything helpful
13:30:57 <Qiming> if it is not relevant to your patch, you can tell the reviewers why a separate patch is needed to cover all those functions
13:31:31 <Qiming> moving on
13:31:33 <lixinhui_> :)
13:31:37 <Qiming> fencing code
13:31:50 <Qiming> we don't have anything to commit into our code base?
13:32:30 <lixinhui_> let us do thing one by one
13:32:52 <lixinhui_> no much progress on this
13:32:58 <Qiming> okay, moving on
13:33:05 <Qiming> no update from me on documentation
13:33:22 <Qiming> haiwei is online?
13:33:28 <haiwei_> yes
13:33:38 <Qiming> hi
13:33:47 <Qiming> posted some comments to your etherpad
13:33:57 <haiwei_> hi, saw your comment on the ether pad
13:34:02 <haiwei_> https://etherpad.openstack.org/p/senlin-container-cluster-issues
13:34:16 <Qiming> any more questions?
13:34:37 <Qiming> okay
13:34:42 <haiwei_> I commented it below
13:34:48 <Qiming> if you have many nova servers created in a single heat stack
13:35:02 <Qiming> that heat stack is not designed for this purpose
13:35:14 <haiwei_> ok
13:35:26 <Qiming> we cannot force people to write a heat template in a particular way
13:35:47 <Qiming> but we can say that we only accept a template that has a single ip_address output
13:36:08 <haiwei_> make sense
13:36:22 <Qiming> if your template looks that way, senlin can somehow treat it as a "virtual" nova server
13:36:35 <Qiming> "virtual" <=> "conceptual"
13:36:58 <Qiming> I don't know much about ip namespace
13:37:15 <Qiming> in my simple deployment, I am not using namespace
13:37:32 <Qiming> I can ssh to each and every nova servers I created
13:37:55 <Qiming> it really depends on the network configuration you are using
13:38:04 <Qiming> so ... I think it is not a senlin-specific problem
13:38:19 <Qiming> we don't have to worry about it too much
13:38:27 <haiwei_> for example, when you want to ssh to a nova server by private ip address, you need to specify namespace to do it, right?
13:38:36 <Qiming> no
13:38:38 <Qiming> I don't have to
13:38:59 <Qiming> just 'ssh cirros@10.0.0.2', for example
13:39:14 <haiwei_> that will work?
13:39:17 <Qiming> yes
13:39:41 <Qiming> if you have complicated network setting, things may become not so straightforward
13:39:51 <Qiming> but still, you will have a way to do that
13:40:05 <Qiming> let's assume the basics
13:40:14 <Qiming> I can find a email thread on this for you
13:40:41 <Qiming> it is not a problem specific to senlin
13:40:46 <haiwei_> ok, I will investigate it to see in which case we need nameapace
13:40:56 <Qiming> ...
13:41:15 <Qiming> I'd assume we don't need a namespace and continue innovate our solution, ;)
13:41:31 <Qiming> but anyway, it is up to you
13:41:40 <haiwei_> I will test in your way
13:41:53 <Qiming> okay
13:42:01 <Qiming> ping me if helps needed
13:42:04 <Qiming> moving on
13:42:16 <Qiming> engine rework for NODE_CREATE/NODE_DELETE
13:42:19 <Qiming> this item was wrong
13:42:38 <Qiming> when I started walking through the code of those actions
13:42:52 <Qiming> I noticed that policy checking are also on those paths
13:43:23 <Qiming> so it is a matter of checking which policy should be impoved to handle NODE_CREATE/NODE_DELETE
13:43:30 <Qiming> rather than rework the action logic
13:43:54 <Qiming> I'm revising the work item and I'll start check each policy and see how they impact these two actions
13:44:54 <Qiming> moving on
13:45:02 <Qiming> the zaqar receiver work
13:45:12 <Qiming> yanyan has been pushing hard at the sdk side
13:45:14 <qwebirc33749> Hi
13:45:22 <Qiming> hi, qwebirc33749
13:45:40 <qwebirc33749> Not able to login for https://review.openstack.org
13:46:00 <Qiming> their api doc is a little bit unstable
13:46:09 <Qiming> hopefully things can be improved soon
13:46:22 <Qiming> we have a zaqar basic driver merged now
13:46:43 <Qiming> event/notifications generalization
13:46:55 <Qiming> em ... no progress from me on this either
13:47:14 <lixinhui_> Qiming
13:47:21 <Qiming> I'm even wondering if we can finish it by newton-3
13:47:26 <lixinhui_> I have one question
13:47:27 <Qiming> yes?
13:47:35 <lixinhui_> about listener
13:47:43 <lixinhui_> of vm-lifecyle-event
13:48:05 <lixinhui_> today I configured nova but got nothing from health_manager
13:48:05 <Qiming> moving on to the last few lines
13:48:14 <Qiming> proposal are all submitted
13:48:20 <Qiming> deleting those lines
13:48:52 <lixinhui_> I should propose this question on open disucssion
13:49:01 <Qiming> what do you mean you 'configured nova'?
13:49:16 <lixinhui_> enable nova notification
13:49:42 <Qiming> have you cross-verified with ceilometer?
13:49:44 <lixinhui_> then attach health policy with the cluster
13:50:11 <lixinhui_> I saw nova print the notifications out in n-cpu.log
13:50:33 <Qiming> can you try a script that monitors the message queue and see if you can get any notification there?
13:51:50 <lixinhui_> i just tried health manager
13:52:12 <lixinhui_> even I put the filter to quiet common rule, still nothing to get
13:52:28 <Qiming> so?
13:52:56 <lixinhui_> so I wanna know if the filter is not right for new oslo.messging versio
13:52:58 <Qiming> what's your filter look like?
13:53:34 <Qiming> do you have a simple script watching the message queue?
13:53:49 <Qiming> to isolate problems between components
13:54:07 <lixinhui_> never try that before
13:54:17 <lixinhui_> it will be nice if you can share me one
13:54:23 <Qiming> it worth trying
13:54:28 <lixinhui_> or I will try to work out one tomorrow
13:54:35 <Qiming> okay
13:54:44 <Qiming> still one thing on agenda
13:54:54 <Qiming> but ... I don't think we have time for it today
13:55:03 <Qiming> it is about senlin delieverables for newton release
13:55:29 <Qiming> we still have one milestone (newton-3) to cut the release
13:56:01 <Qiming> team please push your patches if you want them to be part of this 2.0 version
13:56:59 <Qiming> for example, we planned to do policy-validate, profile-validate, but we haven't touched that yet
13:57:20 <Qiming> we may have to postpone that to next release I guess
13:58:34 <Qiming> the cluster-collect and cluster-do apis are not yet completed, due to lazy reviews in openstacksdk
13:58:42 <Qiming> sigh .
13:59:02 <Qiming> we are introducing micro-versioning only to block our own progress
13:59:39 <Qiming> anyway
13:59:46 <Qiming> thank you all for joining today
14:00:01 <Qiming> we are running out of time
14:00:06 <Qiming> #endmeeting