21:03:29 <strigazi> #startmeeting containers
21:03:30 <openstack> Meeting started Tue Nov 20 21:03:29 2018 UTC and is due to finish in 60 minutes.  The chair is strigazi. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:03:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:03:33 <openstack> The meeting name has been set to 'containers'
21:03:44 <strigazi> #topi Roll Call
21:03:46 <strigazi> o/
21:03:51 <strigazi> #topic Roll Call
21:04:02 <strigazi> o/
21:04:02 <cbrumm_> o/
21:04:30 <imdigitaljim> o/
21:04:32 <flwang> o/
21:05:18 <strigazi> #topic Stories/Tasks
21:05:45 <strigazi> tasks I added in the agenda:
21:05:48 <strigazi> #link https://wiki.openstack.org/wiki/Meetings/Containers#Agenda_for_2018-11-20_2100_UTC
21:06:06 <strigazi> 1. Helm in k8s clusters
21:06:34 <strigazi> After some discussion in gerrit, we decided not to have the binary in the heat-agent
21:07:09 <strigazi> since the agent has specific purpose related to heat and versioning is complicated.
21:07:41 <strigazi> I implemented an alternative where a k8s job is deployed by the agent
21:08:15 <strigazi> The job runs in a minimal container which includes the helm bin
21:08:32 <strigazi> and this container is versioned with the helm verion
21:08:44 <strigazi> Does this make sense?
21:10:29 <flwang> you mean running Helm as a pod on top of k8s cluster
21:10:30 <flwang> ?
21:10:30 <cbrumm_> I'd like to see more, but yes it makes sense
21:10:59 <strigazi> one moment to dig the code.
21:13:15 <strigazi> Can't http://paste.openstack.org/raw/735868/
21:13:18 <openstackgerrit> Jim Bach proposed openstack/magnum master: Make providing a keypair optional  https://review.openstack.org/590443
21:14:54 <strigazi> Since we create the role, we can leverage it deploy tiller.
21:15:15 <strigazi> In the same way we can deploy charts.
21:15:54 <cbrumm_> looks good, pretty straight forward .
21:16:01 <strigazi> What I need your input on though is the following.
21:16:28 <strigazi> The best practice of helm, is secure tiller with tls and one tiller per ns
21:17:39 <strigazi> I think it would make sense to have tiller configured in kube-system for us to deploy additional components, eg prometheus, sssd, node problem detector, k8s-keystone-auth and so on
21:19:16 <strigazi> And for users can deploy tiller in other namespaces, they shouldn't use the global tiller anyway and tiller in kube-system should have powers only there.
21:19:23 <strigazi> makes some sense?
21:19:41 <imdigitaljim> yup
21:19:47 <strigazi> the default tiller sounds like a no-go to me
21:19:48 <cbrumm_> yes, split tillers are better I feel
21:20:01 <strigazi> no tls and admin access is pretty bad
21:21:04 <strigazi> on the other hand, if we use the job model I posted, we can deploy components with helm template && k apply -f
21:22:51 <strigazi> Enough with helm, we can continue on gerrit. let's mode on to upgrades.
21:23:56 <strigazi> 2. We discusses briefly with flwang in Berlin about it, the work I've done I'll break it in four patches and move things:
21:24:30 <strigazi> a. patch for the API: https://review.openstack.org/#/c/514959/
21:25:04 <strigazi> b. add the heat agent in all nodes: https://review.openstack.org/#/c/561858/
21:25:57 <strigazi> c. part from https://review.openstack.org/#/c/561858/  to move most software configs in software deployments
21:26:24 <strigazi> d. final software deployment to upgrade the nodes: https://review.openstack.org/#/c/514960/
21:26:41 <flwang> looks good for me
21:26:54 <strigazi> This way it will be more review friendly and modular.
21:27:05 <strigazi> well "modular"
21:27:14 <cbrumm_> will check these out
21:27:56 <strigazi> and one last item from me
21:28:08 <strigazi> For k8s-keystone-auth
21:28:54 <strigazi> I discussed with Ricardo chaning the magnum client to produce a k8s-keystone-auth friendly kubeconfig, would that make sense?
21:29:08 <flwang> strigazi: that would be great
21:29:11 <cbrumm_> I think so
21:29:14 <strigazi> openstack coe cluster config --keystone-auth
21:29:19 <strigazi> or similar
21:29:21 <flwang> i will pick up that patch asap
21:29:28 <strigazi> --k8s-keystone-auth
21:29:38 <flwang> strigazi: i love it
21:29:43 <strigazi> The question is, what would that be
21:29:51 <strigazi> the simplest way is:
21:30:25 <strigazi> http://paste.openstack.org/raw/735869/
21:30:48 <strigazi> with the token encoded in  kubeconfig.
21:31:02 <strigazi> it is very similar the the certs options
21:31:21 <strigazi> in that case you have everything in one file in the filesystem
21:31:37 <cbrumm_> would the user be adding their token to it or would the cli be filling that part in?
21:31:46 <strigazi> the clo
21:31:51 <strigazi> the cli
21:32:28 <strigazi> IMO the user should always do a single cmd.
21:32:48 <imdigitaljim> so they have to get a new config when their token expires or edit it every time?
21:33:09 <strigazi> there is the option with the client in the cpo repo but at cern is a no-go due to the lack of kerberos.
21:33:36 <strigazi> imdigitaljim: every 24h at least for us. how long is for you?
21:34:11 <imdigitaljim> i think the same
21:34:21 <strigazi> other options include getting a token per call. but user will hate it
21:34:26 <strigazi> I would
21:34:49 <strigazi> with certs in our cloud k8s replies in 70ms
21:35:05 <strigazi> with the token in kubeconfig in ~120ms
21:35:47 <strigazi> with the small script in exec to do a single openstack token issue it takes 1s
21:36:04 <strigazi> pythonclient speed levels...
21:36:34 <imdigitaljim> http://paste.openstack.org/show/735870/
21:36:37 <imdigitaljim> this is what we do
21:36:58 <imdigitaljim> (ive shared this in irc before)
21:37:06 <imdigitaljim> and i would propose this as well
21:37:16 <cbrumm_> it works until the token expires, then users lose their minds and don't know what to do
21:37:59 <strigazi> that works too, I couldn'g find it :)
21:38:00 <imdigitaljim> ^ not referring to what was pasted
21:38:13 <colin-> hi, sorry i'm late
21:39:00 <flwang> imdigitaljim: btw, mind me picking up https://review.openstack.org/#/c/577477/ ?
21:39:21 <imdigitaljim> oh for sure
21:39:22 <imdigitaljim> go for it
21:39:29 <imdigitaljim> we've switched to a centos driver
21:39:35 <colin-> flwang: thought of you when we came across this the other day https://github.com/kubernetes/kubernetes/pull/70398 in case it affects your implementation of ipvs
21:39:43 <colin-> (it does ours)
21:39:44 <imdigitaljim> that id like to shoot for putting it up by stein
21:39:58 <imdigitaljim> ill try to have blueprint documents for everyone to review in the meantime
21:40:32 <strigazi> imdigitaljim: mainly to use a custom image?
21:40:33 <imdigitaljim> its isolated from existing except https://review.openstack.org/#/c/615592/ https://review.openstack.org/#/c/615591/ https://review.openstack.org/#/c/590443/
21:40:55 <imdigitaljim> it can use upstream centos image
21:41:02 <imdigitaljim> but yes we additionally customize it
21:41:08 <flwang> colin-: thanks
21:41:36 <strigazi> how do you install k8s?
21:41:45 <imdigitaljim> ill have it in the blueprints :D
21:42:21 <flwang> imdigitaljim: for your adding clients patches, are you aware of the patch lingxian proposed?
21:42:24 <strigazi> in stories
21:42:33 <flwang> to add a hook for deleting resources
21:42:57 <imdigitaljim> which one particularly
21:44:07 <imdigitaljim> https://review.openstack.org/#/q/owner:anlin.kong%2540gmail.com+status:merged
21:44:18 <imdigitaljim> * https://review.openstack.org/#/q/owner:anlin.kong%2540gmail.com
21:45:01 <flwang> imdigitaljim: https://review.openstack.org/497144
21:46:34 <imdigitaljim> oh ok thats an okay PR but unnecessary
21:46:55 <imdigitaljim> you can just make function calls in the delete section
21:46:58 <imdigitaljim> ./shrug
21:47:14 <flwang> imdigitaljim: we do know that
21:47:18 <imdigitaljim> but yeah our driver is isolated from that
21:47:36 <flwang> imdigitaljim: you better understand the whole picture
21:47:46 <strigazi> imdigitaljim: are you going to maintain the centos driver upstream?
21:47:51 <flwang> there are some case we'd like to handle with a plugin approach
21:48:15 <imdigitaljim> yeah id imagine so
21:48:30 <imdigitaljim> especially if any of you were considering switching once weighing the pros/cons
21:48:37 <flwang> imdigitaljim: is there any big difference between your driver and the upstream version?
21:48:41 <imdigitaljim> much
21:48:57 <imdigitaljim> its faster, easier to read, easier to maintain, less effort to operate
21:49:00 <flwang> much is not a sound answer ;D
21:49:21 <imdigitaljim> allows easy customization based on your needs
21:49:25 <flwang> but anyway, just propose it as a v2 or something like that, so that we can review
21:49:37 <imdigitaljim> yeah itll be just a standalone in the drivers folder
21:49:48 <flwang> imdigitaljim: cool
21:49:52 <imdigitaljim> ill throw up a blue print of design and explain things
21:50:09 <flwang> imdigitaljim: that would be nice
21:50:26 <flwang> or you can even do both in parallel
21:50:28 <strigazi> imdigitaljim: how fase?
21:50:31 <strigazi> imdigitaljim: how fast?
21:50:38 <strigazi> less than 3mins?
21:50:40 <imdigitaljim> bootstraps in like 3 minutes
21:50:42 <imdigitaljim> yeah
21:51:01 <imdigitaljim> unless its spinning disk
21:51:02 <flwang> i can't wait to see the code
21:51:07 <imdigitaljim> which is like 6-8
21:51:27 <imdigitaljim> we were talking about sharing it offline
21:51:29 <imdigitaljim> sending a zip
21:51:35 <imdigitaljim> to you both
21:51:45 <imdigitaljim> if you're interested in an unofficial preview at some point
21:51:53 <flwang> imdigitaljim: that works as well
21:52:27 <strigazi> the sooner the better we diverge.
21:52:34 <strigazi> the sooner the better we converge . :)
21:53:18 <imdigitaljim> https://imgur.com/a/gTazcKl
21:53:30 <imdigitaljim> kind of a layout
21:54:32 <flwang> looks good, code please ;)
21:54:37 <strigazi> sounds good
21:55:40 <imdigitaljim> also a side note as well, my wife is having a kid and ill be out about a month
21:55:55 <imdigitaljim> but my colleagues will continue to meet
21:56:22 <flwang> congrats first, then i would suggest sharing your code for review before your leaving
21:56:37 <strigazi> imdigitaljim: congratulations :)
21:58:46 <strigazi> We are reaching an hour, if there is anything else to discuss we can continue in the channel or tmr (tmr for me)
21:58:46 <imdigitaljim> thanks@!
21:59:18 <flwang> strigazi: i'm good
21:59:54 <strigazi> cool, thanks for joining the meeting guys
22:00:09 <strigazi> See you next week
22:00:24 <flwang> see you all
22:00:43 <strigazi> imdigitaljim: congrats again
22:00:49 <strigazi> #endmeeting