16:00:39 <inc0> #startmeeting kolla
16:00:41 <flaper87> o/
16:00:43 <openstack> Meeting started Wed Jul 19 16:00:39 2017 UTC and is due to finish in 60 minutes.  The chair is inc0. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:45 <Daidv> o/
16:00:47 <openstack> The meeting name has been set to 'kolla'
16:00:48 <inc0> #topic rollcall
16:00:51 <spsurya_> woot 0/
16:00:51 <flaper87> o/
16:00:54 <Duonghq> o/
16:00:54 <kfox1111> o/
16:01:03 <sbezverk> o/
16:01:07 <Jeffrey4l> o/
16:01:12 <egonzalez> woot o/
16:01:17 <rwellum> o/
16:01:27 <vhosakot> woot \o/
16:01:37 <EmilienM> hello
16:01:40 <jascott1> woot
16:02:12 <inc0> let's get to it right away
16:02:17 <inc0> #topic announcement
16:02:33 <shardy> Hi all!
16:03:03 <mwhahaha> o/
16:03:06 <inc0> We're closing to P-3 which will be feature freeze for us, I'm not aware of any features that will be hit by this, but just so you know. Regular services has it next week, which gives us 2 weeks more
16:03:10 <kfox1111> special welcome to all the tripleo folks. :)
16:03:27 <inc0> we need to make sure upgrades works after services tags
16:03:37 <inc0> yup and that, welcome :)
16:03:48 <inc0> let's get to meaty stuff then
16:03:55 <sdake> o/
16:03:57 <inc0> #topic TripleO+Kolla-k8s discussion (inc0)
16:04:02 <flaper87> o/
16:04:16 <flaper87> if you don't mind, I've some stuff I can copy/pasta to introduce the topic
16:04:17 <krtaylor> o/
16:04:23 <inc0> #link https://etherpad.openstack.org/p/tripleo-ptg-queens-kubernetes this is etherpad flaper87 linked
16:04:32 <inc0> flaper87: go ahead
16:04:34 <kfox1111> flaper87: sounds good.
16:04:37 <flaper87> sorry in advance ;)
16:04:40 <flaper87> hey folks, thanks for hosting us. Going to cut to the chase. TripleO is looking for a tool to manage an OpenStack deployment on Kubernetes.
16:04:43 <flaper87> Preferably, this tool should be based on Ansible, given the current goal of the project to migrate a significant part of the code base to it.
16:04:45 <flaper87> I've prepared an etherpad that will hopefully help framing the meeting as I do want to take the most out of it. https://etherpad.openstack.org/p/tripleo-ptg-queens-kubernetes
16:04:47 <flaper87> By all means, you should not let the etherpad dictate what should be talked about but I do want us to stay away from discussions about competition, wheel re-invention or any other thing that is not going to help with what I'm (we are) actually interested in: What can we collaborate on?
16:04:49 <flaper87> I know there are *many* things we have to say and want to talk about but this is IRC and the time is limited. Let's try to allow ppl to speak and participate and avoid doing brain dumps on IRC. I'm not good with 10 parallel IRC conversations.
16:04:51 <flaper87> I've listed some of the motivations why I believe a helm based solution would not be the best solution for TripleO. It has more to do with TripleO current direction than Helm itself, although there are some aspects directly related to Helm that I'm not entirely happy with.
16:04:53 <flaper87> This is of course my opinion based on my own research. Some members of the TripleO team seem to agree with this assestment while others seem to disagree. This is perfect!
16:04:55 <flaper87> I think we know what we would have to do if TripleO were to adopt kolla-kubernetes (k-k) and the different benefits of this. We would all contribute to k-k, we would have ansible+helm, etc.
16:04:57 <flaper87> Now, what would it mean for OpenStack and TripleO if the team decides to not use k-k but to wok on a solution like the one proposed in the email (ansible --(direct)-> k8s )? Can we still use k-k to some extent? Are there things we can collaborate on? What would we be missing (besides the opportunity to collaborate on k-k, which is important)?
16:04:59 <vhosakot> wow, that's fast typing
16:04:59 * flaper87 is trying to save time by typing ahead of the meeting
16:05:08 <shardy> lol :)
16:05:43 * flaper87 stfu and lets ppl talk
16:05:49 <flaper87> hope that's a fair intro
16:05:56 <inc0> ok, so I did some preparation myself and I have few comments;)
16:06:12 <flaper87> awesome
16:06:18 <shardy> Yeah, I think the tl;dr is what would collaboration look like if it was just ansible+k8s vs helm+k8s
16:06:25 <inc0> there are few reasons why helm might be good idea after all and at least won't be that impactful
16:06:37 <sdake> shardy that is how i parsed it as well
16:06:41 <shardy> e.g I'm intersted in how the archtiecture might look if we layered things to enable e.g common ansible roles
16:06:55 <flaper87> shardy: sdake yup
16:07:01 <coolsvap> o/
16:07:08 <inc0> 1. in our arch helm is just a templating language, you don't even really need tiller (service run in k8s) for kolla-k8s
16:07:42 <sdake> inc0 - i think the thing that interests flaper87 and shardy is what would collaboratin look like in a helm-free world
16:07:46 <inc0> so clue of discussion here is how do we get from templates to k8s yamls
16:07:59 <kfox1111> so, part of the issue there I think is language.
16:08:09 <trinaths> o/
16:08:11 <kfox1111> we did origionally use jinja2 for the templates.
16:08:12 <inc0> because I assume we should strive to have yamls similar if not identical
16:08:26 <kfox1111> then spent months switching to gotl. (go templating language).
16:09:06 <inc0> and now helm accepts different templating languages so technically (maybe!) we could switch back to jinja and be consumable by both ansible and helm?;)
16:09:13 <kfox1111> we could either come up with a gotl -> jinja2 converter, or a meta yaml template that can generate the other two types of templates, but that gets back to that xkcd comic. :/
16:09:22 <sdake> shardy to answer your question in a tldr format - helm is based upon gotpl and ansible is fundamentally based upon jinja2
16:09:31 <kfox1111> inc0: possible, but its only theoreticle.
16:09:35 <sdake> these two DSLs can be translated
16:09:37 <kfox1111> they don't have any non gotl template engine currently.
16:09:49 <inc0> jascott1: do you knwo anything about helm support for jinja2?
16:09:58 <jascott1> no
16:10:05 <jascott1> can look into it
16:10:09 <inc0> please do
16:10:11 <kfox1111> inc0: helm itself has no interest in implementing it. but wont block it.
16:10:16 <jascott1> they  are all out this week
16:10:19 <kfox1111> its a go vs python issue.
16:10:28 <flaper87> so, assuming the templating thing is not a problem, what would we be able to collaborate on? Can we use kolla-kubernetes roles anyway ?
16:10:35 <flaper87> what parts of these roles can we use?
16:10:46 <sdake> the fundamental thing i think possibly shardy or flaper87 don't understand is the amount to produce the "fundamental building blocks of openstack on kubernetes"
16:10:52 <flaper87> we can come up with ansible modules that would make the templating part "pluggable/flexible" I think
16:10:56 <inc0> flaper87: as of today k8s is collection of k8s resources with little orchestration
16:11:00 <kfox1111> flaper87: at the moment we dont' have roles.
16:11:02 <sdake> rather amount of work
16:11:09 <inc0> so it's very unopinionated today
16:11:19 <kfox1111> lets back up and let me try and explain where I think we're at.
16:11:27 <shardy> sdake: ack, I think it's more than just the templating language, as e.g the OpenShift community have interest in using "playbook bundles" to define the apps deployed on k8s and their lifecycle
16:11:28 * flaper87 reads carefully
16:11:31 <inc0> that being said we have things like this
16:11:34 <shardy> https://github.com/openshift/ansible-service-broker
16:11:35 <inc0> #link https://review.openstack.org/#/c/473588/
16:11:40 <kfox1111> tripleo's got a pretty solid installer and is looking at changing the orchestration engine.
16:11:56 <shardy> so, while we don't have to do exactly that, there's interest in how the different approaches may align and/or be integrated
16:12:01 <inc0> shardy: what I'm saying is kolla-k8s is really written in a way that it's nothing more than templating
16:12:20 <sdake> shardy - kolla-kubernetes really has no orchestration that is feasible at this time - only the fundamental buliding blocks of building openstack on kubernetes
16:12:24 <inc0> and another alternative would be to write gotpl parser in python, make ansible module out of it and just use kolla-k8s resources
16:12:38 <flaper87> shardy: your last message is important. This conclusion has little to do with what OpenShift is doing and more with what is useful for TripleO that looks like a good common ground
16:12:43 <inc0> we don't use helm lifecycle features
16:12:45 <kfox1111> kolla kubernetes has gotten to the point where it is the equivelenet of kolla-containers for k8s. a db of prebaked openstack building blocks.
16:12:58 <kfox1111> the middle piece, the orchestration piece is really rather undefined in both projects at the moment.
16:13:12 <inc0> yes, and helm in our way of using it won't stand in a way
16:13:24 <inc0> it's again, just client-only template parser
16:13:26 <flaper87> inc0: sdake kfox1111 let's try to just have 1 conversation at a time. I think what kfox1111 is saying is important background. Let's build on top of that
16:13:47 <inc0> right
16:13:58 <inc0> issue is, our building blocks are written in gotph
16:14:00 <shardy> flaper87: yes, I'm just trying to provide a little more context around your ML approach/summary, and why the prototyping has gone in that direction so far
16:14:01 <inc0> gotpl
16:14:09 <sdake> flaper87 i am expanding on kfox1111 's statements above not 2 conversations :)
16:14:23 <flaper87> sdake: sorry, got confused :P Thanks!
16:14:29 <inc0> really what we can say is kolla-k8s doesn't use helm, it's written in gotpl and that's it
16:14:46 <inc0> and have helm-based dir structure
16:15:04 <kfox1111> yeah, its more gotl then helm, but can be consumed direcly as a helm chart.
16:15:23 <flaper87> inc0: so, how do you guys run services on k8s? Is that by calling kubectl directly ?
16:15:39 <inc0> no, we call helm, but helm underneath calls kubectl
16:15:41 <sdake> flaper87 parsing gotpl is moost optimal atm using helm
16:15:50 <kfox1111> we currently launch via helm, but you could also:
16:16:03 <inc0> we could do sth like "gotemplatethis our-dir && kubectl create -f our-dir"
16:16:05 <kfox1111> helm template kolla/nova-api-deployment | kubectl create -f -
16:16:17 <kfox1111> or whatever golang thing to do the generation.
16:16:22 <inc0> where helm template is just generation thingy;)
16:16:37 <kfox1111> helm template is a command that runs gotl templtaing without tiller.
16:16:37 <sdake> fundamentally helm is not an orchestration system
16:16:48 <inc0> so I understand issue of new templating language and I, for one, am not huge fan of go template
16:17:26 <flaper87> mind if I ask whether ansible-kubernetes module was ever an option?
16:17:32 <inc0> however, I'd like to say that in my most honest of opinions, it's harder to learn how to run services on top of k8s than templating language
16:17:34 <flaper87> or wasn't it around at that time?
16:17:41 <kfox1111> yeah. I have my qualms about it. but it is an established k8s standard too.
16:17:42 <inc0> flaper87: https://review.openstack.org/#/c/473588/
16:17:43 <EmilienM> sdake: yeah, looks like a wrapper
16:17:47 <sdake> the challenge you will have flaper87 and shardy with kfox1111 's approach is that understanding the *building blocks* is complex - took me 3 months of constantly banging on it to understand them
16:18:06 <inc0> history behind our helm is an intereting one
16:18:17 <sdake> inc0 lets just not get into the histroy :)
16:18:20 <inc0> I'd be happy to tell you all about it over drinks;)
16:18:20 <kfox1111> sdake: it takes longer to write stable k8s objects for openstack.
16:18:33 <flaper87> inc0: I like that! Let's put the full history on hold
16:18:38 <sdake> kfox1111 agreed
16:18:52 <flaper87> I'm more interested in: "We looked into it but it's not good"
16:18:55 <inc0> thing is, ansible was our first run and we had some progress over time
16:18:57 <flaper87> or "It wasn't around"
16:19:00 <inc0> but we jumped on helm
16:19:00 <flaper87> etc
16:19:05 <flaper87> ok
16:19:09 <inc0> and after that our learning and ideas grew as well
16:19:16 <flaper87> I'll take that and I'll pay for the full story
16:19:26 <sdake> flaper87 you are asking for history then - our history in summation was that we wrote the templates in jinja2- then converted them to gotpl
16:19:37 <sdake> same building blocks
16:19:44 <flaper87> I'll take that and I'll pay for the full story :D
16:19:49 <inc0> and I think ansible is great tool for the job today, part of me wishes that we'd stick to it all along, but ansible is a bit loaded with religion among ops
16:19:51 <flaper87> (with beers)
16:19:53 <slagle> could someone possible point me to the templates in kolla-k8s that might parallel with what is done in e.g., https://github.com/fusor/apb-examples/blob/master/rhscl-mariadb-apb/roles/provision-rhscl-mariadb-apb/tasks/main.yml
16:20:29 <sdake> slagle we would hvae to break eqaqch of th ose into distiinct objects - because that is what kk does
16:20:31 <flaper87> so, if tripleo decideds to go with the ansible-kubernetes modules. What are the possible areas where we could collaborate?
16:20:32 <slagle> i was thinking that might be an area to collaborate, e.g., I assume helm has some parallel to the ansible k8s_v1_service ansible module
16:20:42 <kfox1111> slagle: https://github.com/openstack/kolla-kubernetes/tree/master/helm/microservice/mariadb*
16:20:50 <inc0> flaper87: modules themselves ofc
16:21:07 <kfox1111> note, those are for deployment, not orchestration. so you would still use an ansible role to do the deployment step by step.
16:21:08 <inc0> and frankly? I'd love to have these playbooks in kolla-k8s or at least pluggable to kolla-k8s
16:21:20 <inc0> we don't have orch figured out, you want to figure out orch, we do to
16:21:29 <inc0> we can figure out orch together and put it in whatever repo
16:21:41 <slagle> kfox1111: thanks
16:21:44 <flaper87> inc0: I like that, I like the sound of that
16:21:48 <inc0> but there is no need to have tripleo orch and kolla orch as far as I'm concerned
16:21:51 <flaper87> or the look of that
16:22:03 <flaper87> inc0: yes, I think we agree on that
16:22:11 <inc0> we can put it under tripleo, we can put it under kolla or have new one completely
16:22:19 <flaper87> (btw, I'll be incorporating some summaries of this meeting into the etherpad and later the email thread)
16:22:20 <inc0> logistics can be solved
16:22:27 <kfox1111> I think likely there may be some of both for a while.
16:22:39 <sdake> kfox1111 tend to agree
16:22:39 <kfox1111> since tripleo must have migration from baremetal to containers,
16:22:44 <flaper87> kfox1111: as long as the goal is to converge those, I think it's fine
16:22:47 <kfox1111> some of the roles may be pretty tripleo historic.
16:22:48 <sdake> the problem with the tripelo approach is it is openshift only
16:22:51 <inc0> today biggest barrier imho would be this go tpl as it doesn't make much sense in ansible world, we can discuss which approach would be best
16:23:00 <kfox1111> once the migration has been made though, maybe they can all be merged?
16:23:16 <flaper87> we can still share some of the playbooks and stuff
16:23:16 <rwellum> If we can serve kolla-k8s with helm, as it works already, I don't understand the aversion to helm?
16:23:27 <shardy> kfox1111: yeah, I think we can probably leave the upgrade problem aside for now, but it's something to consider for sure ;)
16:23:37 <inc0> and frankly? (this is my personal opinion, not official PTLish), after we release 1.0, I wouldn't be opposed to dropping gotpl and moving back to jinja (if we have manpower to do it)
16:23:49 * sdake smacks inc0 around
16:23:49 <dprince> sdake: some things optional things may require openshift but I think the underlying building blocks could be shared right?
16:23:50 <kfox1111> shardy: my number one concern is usually upgrades. if you lick that problem, usually all the rest is easy. :)
16:23:50 <inc0> since again, it's just templating yamls, we can provice smooth upgrade experience
16:24:06 <sdake> dprince right - i was speaking only of orchestration
16:24:06 <kfox1111> shardy: thats another reason why kolla-kubernetes looks the way it does. to facilitate upgrades. :)
16:24:34 <kfox1111> so, maybe one question is how tripleo wants to do roles.
16:24:35 <sbezverk> inc0: why?? in 6 month there will be another group of dudes who asked to re-write everything to XYZ thing, so we do it agaoin?
16:24:35 <sdake> dprince ansible-service-broker is openshift only - more specifically
16:24:42 <shardy> kfox1111: Yeah, I just mean the details of upgrading from TripleO as it exists today may not be a common problem?  But maybe it's the same as the brownfield problems that were discussed in atlanta
16:24:50 <sbezverk> maybe at one point we should stop bending for everybody
16:24:51 <flaper87> sdake: it's not openshift only. It uses openshift for their examples
16:24:52 <kfox1111> shardy: exactly.
16:24:59 <flaper87> let's not bring openshift into the conversation, please
16:25:01 <inc0> sbezverk: I'm just throwing this idea around, not planning anything:(
16:25:13 <inc0> keep an open mind folks
16:25:14 <flaper87> I'm interested in kubernetes and how we can support the upstream community and enable collaboration
16:25:21 <kfox1111> would tripleo be wanting to do an ansible role per openstack service?
16:25:21 <shardy> kfox1111: fwiw it's likely to be container->container upgrades, as we're close to switching to kolla containers running on baremetal, just not orchestrated by k8s yet
16:25:31 <inc0> but this is pure priority discussion
16:25:40 <kfox1111> shardy: ah. ok. that would probably make things easier.
16:26:04 <flaper87> TBH, I like the idea of collaborating on the orch pieces a lot
16:26:08 <dprince> shardy: true, but we'd likely still have remnants of old baremetal stuff to cleanup by then
16:26:09 <sdake> back on target - the common area for collaboration is in the templating of the bulding blocks - which at this point and probably forever will be gotpl bsaed
16:26:11 <dprince> shardy: maybe :)
16:26:30 <sdake> this is fundmanetally how we collobrate on docker images as well
16:26:34 <sdake> (at the building block level)
16:26:37 <sbezverk> I think if somebody does not want to use helm just for some political reason, is not good enough to start discussion to switch from it..
16:26:39 <inc0> flaper87: building blocks are pretty much ready, but there are always missing pieces
16:26:46 <flaper87> inc0: roger
16:26:51 <inc0> so if we use existing building blocks and together write orch from scrach
16:26:59 <inc0> we can get there pretty soon
16:27:06 <flaper87> also, note tripleo is currently using kolla-images and I don't think (keep me honest here) there's any plan to change that
16:27:13 <sdake> flaper87 i've depoyed kk in 3 node with ha - works like a champ with the buliding blocks as they are
16:27:19 <inc0> especially that we (kolla-k8s) were playing with different orch mechanisms for a year now
16:27:37 <inc0> and have lot of knowledge to bring to the table
16:27:38 <EmilienM> I see some collaboration on ansible role to generate configuration files for OpenStack services
16:27:50 <kfox1111> plus we have been gating almost everything for a long time now too. a lot of tests have gone into the objects.
16:27:51 <flaper87> EmilienM: was going to get to that now! :)
16:27:53 <EmilienM> flaper87 & dhellmann already kicked off serious amount of work, that is demoable today
16:27:58 <inc0> as of today config files are generated by ansible
16:28:06 <inc0> ripped out of kolla-ansible really, but that works
16:28:17 <EmilienM> inc0: we have something without doing templating
16:28:28 <EmilienM> and we use pure oslo.config tooling
16:28:44 <flaper87> inc0: could you point us to the pieces of kolla-ansible that do this?
16:28:46 <kfox1111> EmilienM: yeah, been following that tangentially a bit. what I've seen looks really slick. :)
16:29:08 <kfox1111> flaper87: the fork of kolla-ansible genconfig in kolla-kubernetes was done just for development purpuses.
16:29:13 <kfox1111> it wasn't intended to be perminant.
16:29:16 <inc0> yeah, but that injects dependency on oslo.config to deploy node. something to discuss. If we run ansible play in container (my personal favorite today) it's not an issue
16:29:17 <EmilienM> kfox1111: we gave up (for now) the etcd idea
16:29:22 <shardy> Yeah leaving the comments re helm aside we definitely have clear scope for collaboration on orchestration and config management here IMO
16:29:27 <sdake> fundamentally the kk buliding blocks don't require collobration on configuration methods - so there are two areas to collaborate (indepdently)
16:29:31 <flaper87> kfox1111: glad you like it so far. I've made some extra progress
16:29:31 <kfox1111> EmilienM: yeah. I think the etcd one has a lot of ugly corner cases.
16:29:32 <inc0> EmilienM: https://github.com/openstack/kolla-kubernetes/tree/master/ansible
16:29:39 <inc0> flaper87: sorry
16:29:40 <flaper87> kfox1111: roger
16:29:41 <inc0> ^
16:29:42 <flaper87> inc0: thanks
16:30:09 <kfox1111> for kolla-kubernetes, configgeneration is kind of like orchestration.
16:30:29 <kfox1111> a nice reference implementation isn't done yet, and you can always use your own.
16:30:46 <flaper87> kfox1111: ideally, we would all use the same tool/roles here
16:30:51 <inc0> yeah
16:30:55 <sdake> flaper87 - i'd suggest trying the ansible orchestraiton playb0ook we do hve to do deploymnet today to really get a feel for what building blocks are in the gotpl format
16:30:55 <kfox1111> kubectl create configmap <service>.... is all it needs.
16:31:09 <inc0> no need for duplication and none of us is emotionally bound to this ansible
16:31:09 <mwhahaha> config generation should be a step in the orchestration but should be properly separated so it can be reused
16:31:14 <kfox1111> flaper87: +1. just saying, if you have someting great already, it might be something we all can use. :)
16:31:19 <flaper87> the current PoC is pure ansible role but I think it'd be better to have an ansible module, which I'm playing with already. Then everyone can upload it to k8s the way they want
16:31:20 <inc0> we put it there because we didn't want to write it;)
16:31:33 <EmilienM> mwhahaha: yes, like in a role probably
16:31:33 <flaper87> kfox1111: will send emails soon (in between travels)
16:31:42 <kfox1111> flaper87: sounds good. :)
16:31:49 <inc0> flaper87: how about this helm module you wrote other day?;)
16:32:05 <inc0> again, if we write ansible module that could do helm template, we're good
16:32:17 <flaper87> inc0: the ansible one? That was part of my experimentation and research to be able to call helm from ansible
16:32:19 <kfox1111> inc0: yeah. I think thats all it would take.
16:32:30 <flaper87> the module is in ansible core now and it can be consumed after the next release
16:32:36 <inc0> that and learning gotpl which isn't that bad
16:32:42 <flaper87> seems to be working decently but it's of course missing some features
16:32:55 <flaper87> anyway, that removes the need of calling helm from the CLI
16:32:56 <inc0> we just need helm template thing really
16:33:00 <kfox1111> sweet. :)
16:33:19 <flaper87> inc0: that can be added
16:33:21 <kfox1111> I've been talking with technosophos about merging the template plugin directly into helm.
16:33:38 <kfox1111> he was neutral to mildly supportive of the idea.
16:33:46 <kfox1111> I think if more folks ask, it will tip the scales.
16:33:59 <inc0> especially if we commit to writing the thing
16:34:16 <kfox1111> yeah. he already basically wrote the thing for us. :)
16:34:18 <inc0> I mean making it mergable because it's written
16:34:46 <inc0> so, my personal favorite direction now is:
16:34:48 <kfox1111> but it requires 2 binaries instead of 1. this would just merge it into one.
16:34:54 <inc0> 1. use kolla-k8s resources
16:34:56 <EmilienM> flaper87: do you mind sharing the link of the help module that you wrote?
16:35:01 <sdake> slagle back on the tripleo playbook - if you reach the logical conclusion of compute kit - you end up with approximately 170 plays
16:35:01 <inc0> 2. use ansible-helm module to render templates
16:35:03 <EmilienM> flaper87: helm*
16:35:16 <sdake> slagle each of these steps is represented by a separate gotpl object in kolla-kubernetes
16:35:17 <flaper87> EmilienM: inc0 kfox1111 https://github.com/ansible/ansible/blob/2a7e586801712c949263315dd79580ae67f7443e/lib/ansible/modules/cloud/misc/helm.py
16:35:19 <inc0> 3. write full ansible orch for deployment
16:35:25 <flaper87> that's the ansible helm module
16:35:33 <EmilienM> flaper87: thanks
16:35:47 <kfox1111> flaper87: very nice. thanks. :)
16:35:54 <inc0> 4*. put it all into a container and run ansible from inside pod;)
16:36:11 <sdake> slagle naturally you could use each of these building blocks in the playbook individually as a result - that is how kolla-kubernetes is structured
16:37:10 <kfox1111> for reference, here's the helm template plugin: https://github.com/technosophos/helm-template
16:37:53 <slagle> sdake: when you say "building blocks", is it all of these? https://github.com/openstack/kolla-kubernetes/tree/master/helm/microservice
16:38:00 <sdake> those are them slagle
16:38:13 <slagle> thanks, just wanted to make sure i was looking in the right place
16:38:17 <flaper87> inc0: so I think 3 we can definitely collaborate on. Under what number do you put config management?
16:38:19 <kfox1111> slagle: yeah.
16:38:43 <inc0> conf mgmt is part of 3 as far as I'm concerned
16:38:51 <kfox1111> flaper87: probably very similar to orchestration.
16:38:52 <kfox1111> yeah.
16:39:01 <flaper87> ok
16:39:03 <inc0> part of orchestration
16:39:04 <flaper87> just wanted to make sure
16:39:14 <flaper87> so we can basically break #3 down into different areas
16:39:15 <sdake> slagle we also played with composition of these building blocks using helm (the helm/services) directory - I feel this has been sort of a failure of our understanding of hem initially - in that helm is not an orchestration system
16:39:17 <flaper87> but yeah
16:39:19 <flaper87> I agree
16:39:20 <kfox1111> it could be split out if we think it has some different qualities.
16:39:22 <EmilienM> flaper87: yes, breakdown
16:39:22 <inc0> thing is, I'm not sure if we can write orchestration for 2 different resource definitions
16:39:38 <inc0> and it won't be ideal
16:39:48 <EmilienM> I wish we could write some ansible roles for config management that would be re-usable outside kolla/tripleo too
16:39:55 <inc0> so I'd say we should try to use common resource definitions
16:40:15 <EmilienM> if we just write a stupid role that generate a file from a list of parameters
16:40:20 <kfox1111> EmilienM: big +1.
16:40:32 <inc0> ansible module would be nice
16:40:34 <flaper87> inc0: when you say common resource edefinition, you're referring to using gotl
16:40:41 <EmilienM> nothing more, so everyone can use it and do what they like before / after, as long as they use this role
16:40:43 <kfox1111> kolla-kubernetes has been writen with config generation completely seperate so that whatever the right tool for config gen was, it can be used.
16:40:49 <inc0> I'm referring to yamls that would be rendered by gotpl
16:40:55 <kfox1111> having a generic ansible genconfig woudl be sweet. :)
16:41:08 <inc0> templates in whatever language are just a means to get these yamls
16:41:09 <sdake> jinja2 / gotpl are analogs
16:41:18 <EmilienM> I'm pretty sure openstack-ansible would also be interested by using this role sometimes
16:41:26 <inc0> yamls should be identical for orch to work properly
16:41:38 <inc0> and if we have 2 codebases for them that will be hard to maintain
16:41:44 <kfox1111> EmilienM: yeah. could share all that between all the projects. :)
16:41:47 <inc0> EmilienM: ofc
16:42:01 <sdake> the cost to tripleo to using gotpl is in some way converting gotpl into a format suitable for consumption by whatever tooling you want to use (which sounds like jinja2)
16:42:39 <kfox1111> or maintaining 2 languages.
16:42:46 <sdake> the alternative is to write all those buliding blocks from scratch - which can be done - which is painful
16:42:53 <kfox1111> jinja2 for config, gotl for k8s objects.
16:43:04 <kfox1111> sdake: yeah.
16:43:07 <flaper87> technically, we could pass yaml files to ansible-kubernetes modules. That said, the resources topic is one that we should def figure out
16:43:29 <inc0> yeah, once we have yamls, we're good
16:43:49 <inc0> really orch is about having yamls and running them in correct order with wait conditions
16:43:52 <inc0> + config mgmt
16:43:57 <sdake> inc0+
16:44:06 <inc0> however getting these yamls correctly is not a trivial task
16:44:21 <sdake> sorting out the order for the building blocks is no easy task - although it has already been prototyped
16:44:39 <EmilienM> there is one important aspect we have to think about is tripleo is interested to simplify operator's life and reduce the number of technologies (in some sort) used to deploy OpenStack - I don't think we want to add too much langages in the stack, at least not more than we have now
16:44:41 <sdake> sorting out the kubernetes object is an order of magnitude more difficult
16:44:44 <flaper87> which is why I like the idea of just using ansible-kubernetes module
16:45:02 <sdake> object/objects
16:45:08 <inc0> yeah I think so too flaper87
16:45:19 <EmilienM> jinja2 is already part of tripleo and gotl is somehow required for k8s world, so I guess that would be fine - plus ansible which is also something we want
16:45:35 <flaper87> EmilienM: gotl is not really required for k8s
16:45:38 <inc0> well, I still have some issues with ansible k8s module, or rather that ansible is not great in templating to variable (can do it but there are caveats)
16:45:44 <flaper87> EmilienM: it's required by helm
16:45:48 <slagle> flaper87: do you know if it's possible to get the k8s yaml as output from the ansible-kubernetes modules?
16:46:00 <flaper87> slagle: I was literally looking into that
16:46:02 <flaper87> :P
16:46:04 <EmilienM> flaper87: oh ok. So yeah, we would have one new layer for our operators
16:46:07 <flaper87> slagle: if it isn't, I'm sure it can be added
16:46:08 <inc0> slagle: ansible-kubernetes takes yamls as input
16:46:12 <kfox1111> kubectl get pod <thingy> -o yaml
16:46:18 <flaper87> which was going to be my next comment
16:46:22 <inc0> output is kubectl create | echo templated-yaml
16:46:30 <inc0> or other way around rather
16:46:39 <flaper87> if we get ansible-kubernetes to output yaml files, it means that we can focus on orch pieces
16:46:53 <kfox1111> output yamls from where?
16:47:00 <sdake> what is ansible-kubernetes?
16:47:11 <inc0> if we get ansible-helm to output yamls then we can reuse whole of kolla-k8s;)
16:47:12 <slagle> inc0: yea, i wasn't sure if the yaml input was the same format expected by kubectl
16:47:21 <flaper87> sdake: ansible's module to talk to kubernetes.
16:47:22 <flaper87> sdake: https://github.com/ansible/ansible-kubernetes-modules
16:47:28 <sdake> oh right thanks
16:47:44 <flaper87> inc0: slagle the output is the same. It's a k8s yaml file
16:47:53 <kfox1111> ansible-kubernetes as I understand it is more of a, here's a yaml, load it into k8s?
16:48:02 <inc0> yeah, source of where it comes from is the question here I think
16:48:04 <kfox1111> something needs to generate the yaml though to feed in.
16:48:13 <flaper87> kfox1111: it's more like: Use ansible tasks to create k8s resources
16:48:19 <flaper87> but it can also take yamls as input
16:48:23 <kfox1111> thats what kolla-kubernetes is about.
16:48:27 <flaper87> That said, we could have the task output the yaml for us
16:48:53 <kfox1111> so, something like helm template kolla/nova-api-deployment > yamlfile.
16:48:55 <flaper87> register that yaml into a variable or something
16:48:57 <inc0> so again, one thing I'd not like to see personally is 2 different sets of templates with same orch layer
16:49:00 <flaper87> kfox1111: exactly
16:49:01 <kfox1111> ansible-kubernetes load yamlfile ?
16:49:19 <inc0> since 2 different sets of templates is *a lot of* duplicated work which has to be kept in sync because of common orch layer
16:49:24 <kfox1111> yeah. I thin kthat would work.
16:49:31 <mwhahaha> that's kinda the problem with templates is that trying to capture all use cases in templates is really hard
16:49:52 <sdake> mwhahaha not if decomposed properly
16:49:52 <mwhahaha> it'd be better to define the datastructures and reuse generic ones rather than templates
16:49:55 <inc0> but our use cases are largely the same
16:50:05 <kfox1111> mwhahaha: we do some of that today too.
16:50:17 <kfox1111> mwhahaha: see kolla-kubernetes/helm/kolla-common/templates/*
16:50:17 <mwhahaha> i'm not saying it can't be done, i'm just saying that it needs to be kept in mind
16:50:32 <flaper87> ok, we have like 10mins left
16:50:33 <inc0> at the end of the day both of us wants openstack running on top of k8s
16:50:33 <kfox1111> we share a lot of code between the microservices that way.
16:50:37 <mwhahaha> and templates generally is where you're problem is because there's always a bunch of edge cases that only gets worse the more template syou add
16:50:46 <inc0> yeah we won't make any decisions today
16:50:50 <flaper87> inc0: I would like to dive a bit more into what scenarios could happen if we end up with 2 set of yamls
16:51:09 <sdake> flaper87 an analog would be two sets of  dockerfile descriptors
16:51:13 <flaper87> inc0: wasn't planning to make any decision today :P but rather getting the conversation framed and started
16:51:17 <sdake> flaper87 except fastly more complex
16:51:21 <flaper87> sdake: gotcha
16:51:22 <kfox1111> flaper87: same senerio with any split code base? bugs get fixed in one and not in the other, etc.
16:51:32 <inc0> sure, and I'd like to make sure that we examine all possibilities to have single set of templates
16:51:43 <sdake> fastly/vastly
16:51:54 <EmilienM> we have 10 mins like flaper87 said - let's take some actions from now
16:51:58 <inc0> again, trust me in that, getting these resource devinitions right is not trivial
16:52:04 <kfox1111> k.
16:52:06 <inc0> took us months to get it right
16:52:15 <sdake> more like 14 months inc0 :)
16:52:23 <inc0> right
16:52:30 <EmilienM> what can we do "tomorrow" to start this effort?
16:52:38 <flaper87> Besides summarizing the meeting and trying to move the conversation forward on the mailing list, I would like to explore the idea of collaborating on the orchestration aspects of the deployment.
16:53:00 <sdake> EmilienM personallly I think a good first step is for the tripeo cats to leran a little bit about what orchestration looks like to them with the templates in kk
16:53:06 <flaper87> I would also like to explore a bit more on what it would mean to have a common set of resources from a tripleo perspective
16:53:08 <inc0> EmilienM: I'd like to have proper PTG meeting for technical planning, so we need to have direction agreed upon by then
16:53:14 <kfox1111> EmilienM: maybe a prototype of the helm template plugin -> ansible-kubernetes would be good to see if the idea has wings?
16:53:24 <flaper87> sdake: beat me by 10s
16:53:58 <flaper87> inc0: I don't want to wait until the PTG, fwiw. I'd like to get to the PTG with a more formal plan
16:54:00 <kfox1111> inc0: +1 to ptg meeting. though thats not 'tomorrow'
16:54:01 <inc0> jascott1 and me can handle prototype (he knows helm, I know ansible;))
16:54:07 <flaper87> unless that's what you meant and I just misunderstood you
16:54:10 <flaper87> in which case, I'm sorry
16:54:19 <EmilienM> yeah we have ~1 month and half - in the meantime we can do some work
16:54:29 <dprince> inc0: EmilienM can we try to make sure the PTG sessions for Kolla and TripleO don't overlap?
16:54:34 <inc0> flaper87: yeah what I was thingking about is PTG = actual implementation plans
16:54:44 <EmilienM> dprince: yes, I'll make sure of than, with inc0
16:54:53 <EmilienM> s/than/that
16:55:09 <flaper87> crazy (not?) question: Would it be possible to generate k-k templates without helm? As in, just gotl
16:55:17 <inc0> we have 3 days (incl Friday)
16:55:18 <sdake> i'd encourage folks to have a look at https://review.openstack.org/#/c/473588/
16:55:19 * flaper87 is just trying to understand kk boundaries better
16:55:28 <inc0> flaper87: yeah, if we write tooling
16:55:35 <sdake> this represents one model of orchestration (using ansible) that explores the consumption of helm directly
16:55:35 <kfox1111> flaper87: the templates are just gotl. just something needs to render it.
16:55:36 <inc0> that's what I was trying to say;)
16:55:44 <inc0> it's *just* templating renderer;)
16:55:47 <flaper87> kfox1111: right, some gotl cli I was thinking
16:55:48 <sdake> simple to replace all the helm calls with the helm template calls
16:55:52 <kfox1111> :)
16:56:00 <flaper87> asking because of the possiblity to collaborate on the orc parts
16:56:01 <inc0> flaper87: thing is, helm template is exactly cli you want
16:56:04 <kfox1111> flaper87: yeah. which I think thats all the helm template plugin does.
16:56:20 <kfox1111> take in gotl stuff, spit out yaml. :)
16:56:21 <inc0> it's helm only by name
16:56:47 <flaper87> gotcha
16:56:53 <kfox1111> I think you can even run it without helm.
16:57:00 <flaper87> kfox1111: that was my question :P
16:57:02 <kfox1111> all helm plugins do are chain the cli call.
16:57:05 <inc0> gotpl as language is the barrier here
16:57:10 <EmilienM> yeah, you use helm as a wrapper right?
16:57:20 <sdake> EmilienM right - helm is a wrapper around kubectl
16:57:20 <kfox1111> so helm <x> just execs ~/.helm/plugins/bin/<x>
16:57:33 <flaper87> ok
16:57:44 <kfox1111> helm is a couple of thigns.
16:57:46 <flaper87> I'll work on my action items as soon as possible and get back to y'all
16:57:48 <EmilienM> so if we wanted to re-use your templates we would need a convertor like sdake proposes to investigate
16:57:56 <kfox1111> 1. a gotl parser.
16:58:09 <flaper87> I'll probably start new threads (or just follow up on that one) to discuss further some of these topics
16:58:09 <inc0> or just use gotpl, it's not *that* bad
16:58:13 <kfox1111> 2. a package fetcher/searcher. (yum search, yum list)
16:58:41 <inc0> ok we're at the end of times
16:58:55 <sdake> check out that review - learn how it works :)
16:58:55 <inc0> or at least this meeting time;)
16:58:57 <flaper87> thank y'all for allowing us to "steal" your meeting
16:59:01 <flaper87> I really appreciate your time
16:59:02 <shardy> thanks for the dicussion everyone!
16:59:06 <EmilienM> thanks yeah
16:59:08 <flaper87> your patience and for walking us through the project
16:59:17 <flaper87> looking forward to collaborate more and as much as possible
16:59:19 <EmilienM> #action inc0 & EmilienM to work together on ptg scheduling
16:59:20 <inc0> thank you all, let's kick off ML thread and get some cool prototypes going
16:59:26 <inc0> right
16:59:32 <inc0> thank you all!
16:59:34 <flaper87> inc0: that's on my list
16:59:39 <inc0> #endmeeting kolla