13:00:52 <Qiming> #startmeeting senlin
13:00:53 <openstack> Meeting started Tue Jul 18 13:00:52 2017 UTC and is due to finish in 60 minutes.  The chair is Qiming. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:55 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:57 <openstack> The meeting name has been set to 'senlin'
13:02:13 <Qiming> hello?
13:02:21 <Ruijie_> hi Qiming
13:03:23 <Qiming> hi, Ruijie_
13:04:18 <Qiming> let's see if others are joining
13:05:16 <Qiming> Ruijie_, next week would be p-3 milestone, are you aware of any high priority bugs to fix?
13:05:49 <Ruijie_> no Qiming, the support for recovery action in lb policy had been done
13:06:03 <Qiming> great
13:07:08 <Ruijie_> Oh, another thing is about the deletion process of cluster recover action
13:07:14 <Ruijie_> that still need to be improved
13:07:36 <Qiming> okay
13:07:49 <Qiming> can we get that landed in this week?
13:08:09 <Ruijie_> I am afraid not Qiming ..:(
13:08:16 <Qiming> I don't have high priority work items before p-3 release
13:08:59 <Qiming> okay
13:09:16 <Qiming> have you submitted a talk proposal for the sydney summit?
13:09:26 <Ruijie_> I am with you Qiming :)
13:09:43 <Ruijie_> about managing clearwater clusters
13:10:14 <Qiming> I know xinhui and xuefeng have each submitted topics
13:10:30 <Qiming> elynn submitted one also
13:10:58 <Qiming> my top priority in the near term is about federated kubernetes clusters
13:11:09 <Ruijie_> I am not sure, xinhui, me and you are in the NFV: clearwater .. proposal
13:11:20 <Qiming> okay
13:11:44 <Ruijie_> I can do the poc job for that since elynn already submitted the VDU profile
13:12:01 <Qiming> I see
13:12:21 <Qiming> I'll be doing k8s + senlin integration
13:12:37 <Qiming> I know many of you have expressed interests in this as well
13:12:42 <Ruijie_> that's great
13:12:52 <Qiming> just not sure how to frame this work
13:13:33 <Qiming> I think elynn might have started looking into standing up vms with scripts for k8s install
13:14:13 <Ruijie_> hope you can sync that with us, that topic is interesting
13:14:16 <Qiming> I'm looking from the other side, getting it up and summarize what we can do to make things easy to reproduce
13:14:34 <Qiming> it is not a talk for the summit
13:14:49 <Qiming> I'm doing that for real-life workload
13:15:18 <Qiming> seems elynn has 15 minutes jet-lag from us, :)
13:15:20 <elynn> Hi, sorry about late, just trapped in another meeting.
13:15:36 <Qiming> hello sir
13:15:47 <Qiming> we were talking about k8s on senlin
13:16:12 <elynn> ooo, anything important I missed?
13:16:26 <Qiming> <Qiming> I think elynn might have started looking into standing up vms with scripts for k8s install
13:16:35 <Qiming> <Qiming> I'm looking from the other side, getting it up and summarize what we can do to make things easy to reproduce
13:17:20 <Qiming> I think XueFeng and friends are interested in this as well
13:17:57 <elynn> Yes, XueFeng discuss this with me last week, and He said he could find some colleagues to help if available.
13:18:22 <elynn> I'm still thinking about how to setup a k8s
13:18:50 <elynn> Basically we will have two clusters, one for k8s masters and one for k8s nodes, right?
13:19:25 <elynn> Am I lost connection again?
13:19:27 <Qiming> master node doesn't have to be a separate cluster, a cluster is needed only for HA setup
13:19:54 <Ruijie_> all the nodes belong to one cluster?
13:20:04 <Qiming> on master nodes, you install apiserver, controller-manager, scheduler, and dns
13:20:20 <Qiming> on all worker nodes, you install kubelet and kube-proxy
13:20:33 <Qiming> these are the basics
13:20:55 <elynn> how do we distinguish the master and worker node if all in one cluster?
13:20:57 <Qiming> Ruijie_, I'm afraid not
13:21:10 <elynn> Like scale up and down operations..
13:21:12 <Qiming> if you do, we distinguish them by 'role'
13:21:43 <elynn> Policies doesn't respect 'role' for now, I guess?
13:22:10 <Qiming> u r rt, elynn
13:22:47 <Qiming> and ... I don't see clear benefits having master node and workers live in the same senlin cluster
13:23:22 <elynn> So your proposal is ... one master node and a cluster of worker nodes?
13:23:52 <Qiming> however, if we have a master 'cluster' and a node 'cluster', we will need to synchronize quite some things between the two
13:24:08 <Qiming> that is a dilemma
13:25:16 <elynn> I'm still confused here...
13:27:05 <Qiming> basically, some network settings, authentication data should be shared from the apiserver to all kubelets
13:27:15 <Qiming> kubelets don't have to know each other
13:28:24 <elynn> Okay, I got your point, from your perspective , make this as simple as possible
13:28:32 <Qiming> yes
13:28:58 <elynn> Deploy a cluster, then user will get a k8s cluster later.
13:29:07 <Qiming> we can postpone advanced features later, such as tls bootstrap ...
13:29:21 <elynn> But many details need to figure out.
13:29:26 <Qiming> yes
13:29:29 <elynn> I think kubeadm is a good start?
13:29:40 <Qiming> kubeadm is very limited
13:29:58 <Qiming> it encapsulates too many things
13:30:09 <Qiming> but ... you are right, it can be a starting point
13:30:46 <Qiming> maybe just run 'kubeadm init' on master with a predefined token
13:30:50 <elynn> That's only for tools part, we still need to solve how to share configuration info between nodes in a cluster
13:31:04 <Qiming> at the same time, run 'kubeadm join' on all other nodes with the same token
13:31:22 <elynn> And how to pass it to user-data, etc...
13:31:50 <Qiming> em, right
13:31:57 <elynn> Since only vdu profile in contrib folder support passing parameters to user-data...
13:32:12 <Qiming> I don't want to use any vendor specific tools
13:32:21 <elynn> And also how to avoid deleting master node in cluster..
13:32:49 <Qiming> cluster should be operatable if masters are gone ..
13:33:02 <Qiming> although no new resources can be created, :)
13:33:24 <Qiming> so ... ideally, master nodes should be a separate cluster
13:33:49 <Qiming> in the future we can even move etcd out to a third cluster
13:34:26 <elynn> I know your concern, eventually we will have some scripts to setup k8s if that is ... I have enough time to implement such a script..
13:34:30 <Qiming> then ... we will need the inter-cluster dependencies properly
13:34:46 <Qiming> oh really
13:35:22 <elynn> Do you suggest we start from HA mode or just a single master node?
13:35:33 <Qiming> just a single could be fine
13:35:49 <Qiming> I was thinking about a kube-master profile type and a kube-worker profile
13:36:34 <elynn> O? New profile?
13:36:44 <elynn> Sounds good actually.
13:37:02 <Qiming> these kinds of nodes have a lot of things to do upon bare VM
13:37:17 <Qiming> implementation wise we can subclass Server
13:37:29 <elynn> Yes, agree
13:37:52 <Qiming> this way we can quickly stand up something
13:38:17 <Qiming> and that "something" won't become garbage soon
13:38:30 <elynn> And we can put them in contrib folder firstly?
13:38:46 <Qiming> we can gradually expose properties when there is a need
13:38:49 <Qiming> sure
13:39:28 <Qiming> ideally, here would be what you can deploy a k8s with senlin
13:39:29 <elynn> Do you think what kind of properties we can expose for now?
13:40:33 <Qiming> openstack cluster create --profile master.k8s --count 3 my_master
13:40:36 <elynn> Hi XueFeng
13:40:45 <XueFeng> hi
13:40:50 <XueFeng> elynn
13:40:54 <XueFeng> hi, all
13:41:10 <elynn> We are talking about k8s now.
13:41:16 <Qiming> openstack cluster create --profile worker.k8s --count 5 --metadata "master=my_master" my_workers
13:41:24 <XueFeng> ok
13:41:41 <elynn> Sounds doable.
13:41:47 <Ruijie_> clusters need to talk to each other?
13:42:16 <Qiming> workers need to know master's api
13:42:46 <Qiming> if you do TLS, you will need their key-pairs etc, you will need to know their cidr configurations etc
13:42:57 <Qiming> it is a one way depenency
13:44:11 <Ruijie_> em.. okay Qiming, that makes sense. How about the monitor for the workers?
13:44:20 <elynn> I would suggest we expose those kind of configurations to properties, what do you think?
13:44:35 <XueFeng> Just come back, sorry for late. Let me see the log http://eavesdrop.openstack.org/meetings/senlin/2017/senlin.2017-07-18-13.00.log.txt
13:45:13 <Qiming> monitors, loggings, kernel optimization ... etc can be added to the worker.k8s profile later
13:46:51 <Qiming> elynn, we can add properties one by one
13:47:32 <Qiming> if we expose a hundred properties at the very beginning, ... the framework will fall apart pretty soon
13:48:03 <elynn> yes, yes
13:48:28 <Qiming> the challenge, though, is about the details
13:48:45 <elynn> Speaking about monitors, that reminds me that we might need to control the sequence of scripts between workers and masters...
13:48:54 <Qiming> things work differently when you are using ubuntu or fedora ...
13:49:06 <Qiming> elynn, there is no such need
13:49:25 <Qiming> they will be able to "find" each other
13:49:25 <elynn> Deploy master -> deploy workers -> deploy monitor yaml on master ?
13:49:55 <Qiming> oh, no
13:50:21 <Qiming> don't introduce circular dependency
13:50:32 <Qiming> we have to do this in one pass
13:51:05 <Qiming> if you want monitoring, deploy it with your master components on master nodes
13:51:25 <Qiming> or you can add them later by upgrading the master node cluster
13:52:20 <elynn> The devil is in the detail..
13:52:25 <Qiming> yes
13:52:43 <elynn> I can record it  in my notes and think about it later.
13:53:01 <XueFeng> So we will add  kube-master profile type and a kube-worker profile?
13:53:11 <elynn> For image, I would suggest we start from ubuntu?
13:53:55 <elynn> Details again, how can we limit the image they used...like os version, 14.04 or 16.04...
13:53:56 <Qiming> agree
13:54:10 <Qiming> don't want to reinstall my local machines, :)
13:54:50 <Qiming> the most annoying thing is about os distros
13:55:16 <Qiming> I believe we can start from one of them say ubuntu16.04
13:55:29 <Qiming> then extend to other distros later
13:55:39 <XueFeng> I think the docker deamon version is limited
13:56:02 <Qiming> most version above 1.12 should work
13:57:28 <XueFeng> For vm 1404 is not suggest to support
13:57:37 <XueFeng> ub14.04
13:57:49 <Qiming> yes, it is ... history now
13:57:59 <XueFeng> So first support in 16.04 is ok
13:58:41 <Ruijie_> .....just change to 16.04 these days ..
13:58:52 <Qiming> there are more to design/implement regarding network and storage
13:58:57 <elynn> Silently shutdown the VM in script if we don't support it ;)
13:59:10 <Ruijie_> hahha
13:59:36 <Ruijie_> I can do some work if needed
13:59:53 <Qiming> alright
13:59:56 <elynn> About network and storage, what do you suggest? Qiming
13:59:58 <Qiming> time's up guys
14:00:00 <Qiming> thanks for joining
14:00:05 <elynn> Okay...
14:00:06 <Qiming> pls switch to #senlin
14:00:10 <Qiming> #endmeeting