15:00:17 <portdirect> #startmeeting openstack-helm
15:00:18 <openstack> Meeting started Tue May 21 15:00:17 2019 UTC and is due to finish in 60 minutes.  The chair is portdirect. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:21 <openstack> The meeting name has been set to 'openstack_helm'
15:00:25 <evrardjp> o/
15:00:26 <howell> o/
15:00:38 <megheisler> \o
15:00:38 <portdirect> hey all - the agenda is here: https://etherpad.openstack.org/p/openstack-helm-meeting-2019-05-21
15:00:42 <itxaka> o/
15:00:44 <portdirect> very light this week
15:00:50 <mfuller_> o/
15:00:57 <portdirect> lets give it 5 mins for people to roll up
15:01:04 <gagehugo> o/
15:01:14 <portdirect> though i expect this one may be quite light as many people are in bcn for kubecon
15:01:15 <georgk> hi
15:01:24 <srwilkers> o/
15:01:56 <mattmceuen> o/
15:03:40 <alanmeadows> o/
15:03:46 <georgk> quick question: who of you is at KubeCon - for a short chat
15:04:50 <evrardjp> I am not
15:04:56 <georgk> everybody is hiding... :-)
15:05:18 <mattmceuen> I am back at work as well :)
15:05:55 <itxaka> me neither :(
15:06:31 <portdirect> ok lets go
15:06:35 <portdirect> #topic tempest prerequisites from scripts to init containers
15:07:01 <itxaka> today its all me :)
15:08:03 <itxaka> so this was brougth up with our airship team, they find it quite a pain to do an automated airhship deployment that includes tempest because we are doing automated things in our deployment scripts to deploy a working tempest chart
15:08:41 <itxaka> and the idea was brougth up to explore having those prerequisites be a part of the init containers of tempest to provide a full unmaned deployment of the tempest chart by passing just values
15:09:17 <portdirect> what would the container do?
15:09:43 <evrardjp> and how do you test the idempotency impact on said change on the chart?
15:10:07 <itxaka> create network+subnetwork basically
15:10:27 <itxaka> and obtain the ids for the other needed components (only the image id I believe)
15:10:46 <itxaka> *correction: and flavor id
15:10:52 <portdirect> for the 2nd point ++
15:11:05 <evrardjp> what is the second point?
15:11:19 <portdirect> for the 1st, kinda ok with it - though with this not be better suited to being either in the boostrap job for the neutron chart
15:11:20 <evrardjp> having those at init containers?
15:11:45 <portdirect> or a generic heat driven 'openstack objects chart'
15:11:59 <itxaka> basically whats in here: https://github.com/openstack/openstack-helm/blob/f9231d720c2299c5d85f614c2e1787599b069199/tools/deployment/multinode/900-tempest.sh
15:12:12 <portdirect> evrardjp: point 1: `create network+subnetwork basically` point 2: `obtain the ids for...`
15:12:16 <itxaka> so we can have a tempest deployment like the other charts, just passing values to the chart
15:12:17 <evrardjp> ok
15:12:47 <evrardjp> point 1 can be a part of the bootstrap neutron or tempest itself?
15:12:48 <itxaka> mmmh using neutron to bootstrap any nets is also a good idea<
15:12:49 <portdirect> i wonder if this: https://github.com/openstack/openstack-helm/blob/f9231d720c2299c5d85f614c2e1787599b069199/tools/deployment/multinode/900-tempest.sh#L22-L43
15:12:53 <portdirect> should become its own chart
15:13:06 <evrardjp> that's what I suggested
15:13:43 <evrardjp> but then you have to pass values around for next chart
15:13:57 <itxaka> not really, you can store them in a shared configmap no?
15:14:03 <evrardjp> which means introspection of the first chart values or things like that, which sounds not really clean
15:14:04 <portdirect> id avoid that
15:14:12 <evrardjp> itxaka: exactly
15:14:16 <itxaka> and tempest can try to use those value if exists?
15:14:29 <portdirect> itxaka: i'd avoid configmap deps across charts
15:14:31 <evrardjp> it's still not great
15:14:36 <portdirect> it gets super ugly
15:14:45 <evrardjp> convention over configuration can help here IMO
15:14:49 * itxaka is thankful to not have experience there :D
15:14:51 <portdirect> but if you have the names, then you init container can get the ids..
15:15:20 <portdirect> so the info to pass as values becomes super simple
15:15:29 <itxaka> I think that havin the neutron bootstrap creating networks is a really great and clean idea
15:15:41 <itxaka> its useful in a lot of different scenarios
15:15:45 <itxaka> not only for this
15:16:12 <itxaka> then the init containers for tempest can just try to get the existing ids for those values only
15:16:13 <evrardjp> I like the idea of automatically having not a bootstrap but another job to add some networks
15:16:26 <itxaka> so its simpler to maintain
15:16:29 <evrardjp> and those can indeed contain tempest networks
15:16:44 <evrardjp> and with convention, re-use those in tempest helm chart
15:16:51 <portdirect> this is what we did for ironic
15:16:54 <portdirect> eg: https://github.com/openstack/openstack-helm/blob/master/ironic/templates/bin/_retreive-cleaning-network.sh.tpl#L21-L25
15:17:06 <portdirect> and a job to create if not already in existance
15:17:51 <portdirect> https://github.com/openstack/openstack-helm/blob/master/ironic/templates/bin/_manage-cleaning-network.sh.tpl
15:18:08 <portdirect> which could be transfered over to tempest pretty simply
15:18:27 <evrardjp> so you mean a change in the tempest helm chart
15:18:32 <itxaka> I dont even think we would need a job to create, that seems to fall out of a chart territory....
15:18:45 <itxaka> like neutron chart should do network stuff, tempest should do tempest
15:18:51 <evrardjp> I am fine with either neutron or tempest tbh
15:18:56 <portdirect> itxaka: i agree
15:19:13 <evrardjp> but not as a separate chart which shares the data between with the help of configmaps
15:19:15 <evrardjp> or others
15:19:17 <portdirect> though in some places the neutron boostrap job etc has limitations - eg with visibility
15:19:36 <evrardjp> could you clarify this?
15:19:40 <portdirect> hence why im suggesting that perhaps we build a openstack-object cart that just drives a heat template
15:20:01 <portdirect> evrardjp: yeah - people dont know to look for it :)
15:20:14 <portdirect> and also inter service work is hard
15:20:18 <evrardjp> portdirect: oh I misunderstood you -- you want to regroup ALL the activities under one roof?
15:20:23 <portdirect> no
15:20:27 <evrardjp> and conditionally trigger things?
15:20:29 <evrardjp> ok
15:20:35 <portdirect> use the boostrap job for boostrapping the service
15:20:41 <evrardjp> I didn't quite catch what's your end state you desire then
15:20:45 <itxaka> bootstrap seems like a win win to me
15:20:48 <portdirect> and have a chart that can creqte higher level objects
15:21:52 <itxaka> now its me the one that got lost
15:22:14 <itxaka> so no bootstrap jobs but a chart that creates all desired objects in teh deployment?
15:22:49 <portdirect> both - use boostrap where required
15:23:08 <portdirect> but a chart that can then be used above that to create deployed artifacts scross the cloud
15:23:35 <portdirect> eg - boostrap can not create say a collection of glance images, neutron networks and nova vms sanely
15:23:47 <portdirect> but can prepare all those services to do so
15:24:06 <portdirect> and a higher level chart then has value of orchestrating these 'primed' services
15:24:11 <itxaka> why cant bootstrap create a collection of objects related to its own chart?
15:24:18 <portdirect> it can
15:24:21 <itxaka> it also sounds like reinventing heat :p
15:24:30 <portdirect> no - its using heat ;)
15:24:50 <portdirect> <portdirect> Pete Birley or a generic heat driven 'openstack objects chart'
15:25:08 <itxaka> should chats be self contained? like everything from glance should eb done on the glance chart?
15:25:12 <itxaka> chats lmao
15:25:17 <itxaka> *charts :D
15:25:35 <srwilkers> yeah, i think that's the point being made here
15:25:38 <evrardjp> I like the idea of being self-contained too
15:25:39 <itxaka> maybe someone isnt deploying heat but still wants some images to be there
15:25:57 <portdirect> charts have to be self contained
15:26:03 <evrardjp> srwilkers: I think the message is not crossing the atlantic
15:26:20 <evrardjp> :)
15:26:48 <evrardjp> portdirect: I guess the question is where to put the line on "when/how" to use this chart leveraging heat cli
15:26:58 <portdirect> i can boil it down to this i think:
15:27:17 <portdirect> 1) bootstrap script to boosstrap a service
15:27:17 <portdirect> 2) heat template in a chart to bootstrap a cloud
15:27:22 <portdirect> we have 1 today
15:27:30 <portdirect> we could do with 2
15:27:44 <evrardjp> ok let me rephrase this
15:27:48 <evrardjp> see if I got it
15:28:00 <srwilkers> ultimately, the chart portdirect is talking about would be to do things that require creating objects that span the multiple services in our cloud (ie: i need to create some networks, some images, and whatever else for people/tempest/whatever to consume).  it doesn't make sense to have a job specific to tempest create things like networks and images that it needs to consume
15:28:13 <srwilkers> which is why it'd be nice to have a chart that simply leverages heat to do this for us
15:28:14 <evrardjp> 1) bootstrap will only be used to "get started" with the service, and anything happening after service setup is delegated elsewhere.
15:28:54 <evrardjp> I don't agree on the fact that "it doesn't make sense to have a job specific to tempest to create things tempest needs"
15:29:02 <evrardjp> that's exactly the point of the bootstrap jobs?
15:29:15 <portdirect> re 1 - yes
15:29:47 <evrardjp> ok I understood the line/border
15:29:53 <evrardjp> the border makes sense to me
15:29:59 <evrardjp> it's clear to explain to people and document
15:30:03 <portdirect> re 2 - on the fence, i'd not block it - but here it would make more sens i think to seperate the concerns - as tempest is a test suite
15:30:22 <portdirect> 2 being `it doesn't make sense to have a job specific to tempest to create things tempest needs` in this case
15:30:57 * srwilkers shrugs
15:31:21 <alanmeadows> The other item worth mentioning
15:31:23 <itxaka> ok, so then its kind-of agreed that for creating objects in a cloud, an extra chart would be needed that leverages heat to create those objects
15:31:42 <itxaka> with the only goal of that chart is bringing the deployed cloud to a user required state
15:31:51 <portdirect> itxaka: that would be the best path i think
15:31:58 <evrardjp> alanmeadows: yes?
15:31:59 <itxaka> not just to use tempest but as a boostrap cloud kind of chart
15:32:01 <alanmeadows> is if you use bootstrap to actually "set up your cloud" rather than just do whats absolutely necessary to have that service up you'll end up with piecemeal data fed into dozens of different charts that will be hard to visualize/manage
15:32:23 <portdirect> hence my visibility comment earlier
15:32:28 <alanmeadows> that actually probably blongs in one place (e.g. how I setup openstack for my users)
15:32:43 <evrardjp> I think we were in agreement to not set things to "set up your cloud" in bootstrap :)
15:32:51 <itxaka> agreed
15:33:46 <portdirect> just to throw things off again - the one q i have is heat or shade?
15:34:04 <evrardjp> the focus point here was to setup the requirements for tempest. If we consider the setup the requirements part of the "set up your cloud" or if tempest chart itself would be setting up its requirements itself
15:34:07 <portdirect> i think proabably heat, though it does depend in it - but a q worth asking
15:34:25 <evrardjp> you probably mean sdk
15:34:31 <evrardjp> anything idempotent works for me
15:35:04 <itxaka> heat I think althougth falling back to openstackclient in case of no heat may be worth discussing
15:35:10 <itxaka> but who doesnt install heat!
15:35:25 <srwilkers> the same people who thought the game of thrones ending was good
15:35:26 <evrardjp> this is why SDK would not be a bad idea
15:35:33 <itxaka> oh lmao
15:35:35 <gagehugo> haha
15:35:37 <itxaka> shots fired
15:35:39 <mfuller_> hahaha
15:35:42 <evrardjp> srwilkers: haha
15:35:43 <portdirect> itxaka: here i dont think we want to offer a choice - if we go heat - inject raw heat as values.
15:35:52 * itxaka adds srwilkers to good guys list
15:36:08 <srwilkers> why do you think i came all this way itxaka?
15:36:09 <itxaka> portdirect: sounds good!
15:36:12 <portdirect> evrardjp: i ment shade here
15:36:17 <portdirect> but favor heat
15:36:49 <itxaka> I think that making it clear that this requires heat would make the chart simpler
15:36:51 <portdirect> evrardjp: actually i take that back - but still favor heat
15:36:52 <portdirect> :)
15:37:09 <portdirect> so shall we do it itxaka ?
15:37:13 <portdirect> im down.
15:37:18 <alanmeadows> probably the i-configure-openstack-chert should just support a few values? provide your uber heat stack here, openstackclient values here, shade values here?
15:37:24 <alanmeadows> then the end user can do what they please?
15:37:31 <evrardjp> I think the openstacksdk is more self contained and provide the same value, but it's up to the committer to decide
15:37:34 <alanmeadows> as long as, as evrardjp says, its idempotent
15:38:11 <portdirect> alanmeadows: wfm
15:38:13 <evrardjp> alanmeadows: you could have been belgian with your sense of the compromise there
15:38:20 <itxaka> lmao
15:38:32 <alanmeadows> well in this case, supporting "all the things" isn't that much work
15:38:34 <alanmeadows> ;-)
15:38:37 <itxaka> wfm
15:38:49 <evrardjp> things can also grow organically
15:38:59 <itxaka> so heat first then openstackclient then shade?
15:39:06 <portdirect> itxaka: sounds good
15:39:09 <itxaka> yaya
15:39:18 <portdirect> heat > bash > python ;P
15:39:21 <evrardjp> openstackclient will be tedious to make idempotent, but up to you
15:39:30 <evrardjp> portdirect: cannot compute that line
15:39:40 <portdirect> no one can - its why its perfect
15:39:41 <evrardjp> I cannot compute*
15:39:54 * itxaka adds portdirect to the bad guys list
15:40:09 * portdirect strokes white cat
15:40:19 * evrardjp would like to see s/Arya/itxaka/ lists
15:40:54 <portdirect> ok to move on?
15:40:57 <evrardjp> are the action points taken?
15:41:02 <evrardjp> ok for me to move on
15:41:08 <itxaka> last item, if you have a good name for it please write it down
15:41:12 <itxaka> Im bad at names
15:41:26 <itxaka> so I will use alanmeadows one if there is no better suggestions :P
15:41:28 <srwilkers> a chert has no name
15:41:32 <itxaka> i-configure-openstack-chert
15:41:37 <portdirect> cloudstrap
15:41:40 <evrardjp> srwilkers: stop it! :p
15:42:00 <evrardjp> portdirect: that's a trademark?
15:42:11 <evrardjp> if not I should totally make taht a trademark
15:42:26 <portdirect> might be :(
15:42:43 <itxaka> already taken
15:42:46 <portdirect> openstack-objects may be the best, though most borning name
15:43:01 <evrardjp> the project name is openstack-helm
15:43:08 <evrardjp> for those who don't follow
15:43:40 <evrardjp> should we move on though?
15:43:56 <itxaka> +1
15:44:06 <itxaka> oh wow that took a long time
15:44:17 <portdirect> #topic meeting times ( http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006381.html )
15:44:39 <itxaka> so I put that in there but was just because the mail is on the list and I didnt see too many replies
15:44:48 <itxaka> maily a heads up to people
15:44:52 <itxaka> mainly*
15:44:58 <itxaka> +1 to move on
15:45:06 <evrardjp> good reminder
15:45:14 <portdirect> thx itxaka
15:45:15 <evrardjp> jayahn: will thank you for it :)
15:45:34 <portdirect> #topic versioning of helm charts -- spec status
15:45:48 <evrardjp> just wondering where we are there
15:45:51 <portdirect> so im behind here - and need to get a draft of this up
15:46:03 <portdirect> but have something thats close to being push-able
15:46:23 <evrardjp> ppl can only start to work on things when the spec will be merged
15:46:44 <evrardjp> it's just a reminder
15:46:50 <evrardjp> next topic
15:46:50 <evrardjp> ?
15:47:29 <portdirect> #topic gates changes -- spec status
15:47:54 <portdirect> evrardjp: you added this one i think
15:48:11 <evrardjp> yes
15:48:56 <evrardjp> I think we discussed some improvements in gates in the past. I had a series of patches long ago that always ended up in merge conflicts, so I dropped them all. IIRC, there was a spec that was supposed to be out, what's the status?
15:49:20 <portdirect> theres not been discussion of a spec that i can rememebr
15:49:39 <portdirect> but srwilkers and I were going to look at the gates prior to ptg - and have still to do so
15:49:47 <portdirect> https://etherpad.openstack.org/p/openstack-helm-meeting-2019-04-23
15:49:56 <portdirect> so we should get back on that
15:49:57 <evrardjp> ok, I thought the result of that look was a spec
15:50:02 <evrardjp> my bad.
15:50:25 <portdirect> no worries - a spec would certainly help this process
15:50:35 <portdirect> so we should consider one following the poc
15:50:55 <itxaka> portdirect: is the poc close or far away? maily to write it down on the etherpad
15:51:14 <portdirect> i think its just a case of srwilkers and i finding 4 hours to do it
15:51:16 <evrardjp> no need to really have specs for the purpose of having specs though, but documenting the reasons we decided to do that way, is useful
15:51:31 <portdirect> evrardjp: exactly++
15:52:29 <evrardjp> reviews? :D
15:52:51 <evrardjp> ovs-dpdk could deserve some love! :p
15:53:00 <portdirect> #topic reviews
15:53:01 <evrardjp> but I am getting ahead of myself here
15:53:24 <portdirect> as always a long list of things that could do with some love:
15:53:26 <portdirect> Reviews:
15:53:27 <portdirect> - https://pasted.tech/l/1701e85c296214b4 (or long url: https://review.opendev.org/#/q/project:%255Eopenstack/openstack-helm.*+owner:%22Jean-Philippe+Evrard+%253Cjean-philippe%2540evrard.me%253E%22++is:mergeable++is:open )
15:53:27 <portdirect> OVS-DPDK
15:53:27 <portdirect> - https://review.opendev.org/#/c/659581/ - workflow
15:53:27 <portdirect> - https://review.opendev.org/#/c/626894/ - reviews from cores appreciated
15:53:27 <portdirect> - https://review.opendev.org/#/c/643284/ - reviews from cores appreciated
15:53:27 <portdirect> - https://review.opendev.org/#/c/651060/ - reviews from cores appreciated
15:53:28 <portdirect> - https://review.opendev.org/#/c/659351/ - reviews from cores appreciated
15:53:28 <portdirect> - https://review.opendev.org/#/c/659787/ - mysql: set hostpath for mysql-data (fixes zuul jobs)
15:53:29 <portdirect> - https://review.opendev.org/#/c/659557/ - ceph: bump kubernetes version for ceph images
15:53:29 <portdirect> - https://review.opendev.org/#/c/650523/ - Migrate ubuntu jobs from Xenial to Bionic (important for gating)
15:54:02 <evrardjp> the first one is very important to us, but not as important as fixing the zuul jobs :D
15:54:03 <portdirect> for the dpdk work - it would be good to see this gated
15:54:26 <georgk> yes, evrardjp mentioned this already
15:54:53 <georgk> i need to see if I get DPDK working in the virtualized Openstack infra
15:54:59 <evrardjp> portdirect: maybe not gated, but at least CI tested first? It still need some things to merfe first
15:55:01 <portdirect> that would be great georgk
15:55:07 <evrardjp> merge*
15:55:16 <georgk> is that a prerequisite for gettting this merged?
15:55:23 <portdirect> ci tested would be
15:55:31 <evrardjp> portdirect: but this is very special
15:55:53 <evrardjp> (if we don't have the hardware)
15:56:16 <portdirect> evrardjp: we can get 3rd party ci i think for this
15:56:26 <evrardjp> 3rd party ci will never gate
15:56:34 <evrardjp> it can vote but that's the max
15:56:53 <evrardjp> (on purpose)
15:56:59 <portdirect> but im not sure why we could not run in -infra?
15:57:14 <georgk> as I said, I need to check
15:57:45 <evrardjp> let me rephrase this -- we can have 3rd party CI voting, and I think that's an option we should follow if we can't do things in infra
15:57:47 <georgk> I'll probably poke some of you who have better insight into this
15:58:07 <portdirect> georgk: i think deepak would be great here
15:58:13 <evrardjp> but I also believe that this has value merged even without the testing yet. (Because there could be chicken and egg things)
15:58:44 <evrardjp> (for example, the image building of dpdk should be merged before)
15:58:55 <evrardjp> but that's my personal opinion, I am no one here :p
15:59:42 <georgk> right, we need to address the images
15:59:59 <evrardjp> out of time
16:00:25 <portdirect> thanks everyone!
16:00:32 <portdirect> see you all in the channel
16:00:36 <portdirect> #endmeeting