15:00:17 <srwilkers> #startmeeting openstack-helm
15:00:18 <openstack> Meeting started Tue Sep 18 15:00:17 2018 UTC and is due to finish in 60 minutes.  The chair is srwilkers. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:22 <openstack> The meeting name has been set to 'openstack_helm'
15:00:29 <srwilkers> #topic rollcall
15:00:30 <srwilkers> o/
15:00:39 <mattmceuen> o/
15:01:30 <lamt> o/
15:01:40 <jayahn> o/
15:01:41 <jamesgu__> o/
15:01:53 <srwilkers> let's take a few minutes and add any last minute items to the agenda
15:01:55 <jayahn> this switching in hurry is confusing me.. :)
15:01:59 <openstack> srwilkers: Error: Can't start another meeting, one is in progress.  Use #endmeeting first.
15:02:03 <srwilkers> oops
15:02:07 <srwilkers> https://etherpad.openstack.org/p/openstack-helm-meeting-2018-09-18
15:02:09 <srwilkers> there we go
15:02:24 <john_W> o/
15:02:26 <srwilkers> lets get started at 5 after -- gives us a few minutes to add things here
15:02:36 <ianychoi> o/
15:03:40 <jayahn> hey ianychoi
15:04:06 <ianychoi> jayahn, hello :)
15:05:04 <srwilkers> #topic PTG recap
15:05:19 <portdirect> o/
15:05:22 <srwilkers> here's the etherpad we used at the PTG https://etherpad.openstack.org/p/openstack-helm-ptg-stein
15:05:27 <evrardjp> o/
15:05:49 <srwilkers> i was really happy with what we were able to chat through, and was especially happy to see some new faces as well as the usual suspects (like jayahn ;) )
15:06:12 <portdirect> yeah - over the three days we had some really productive sessions
15:06:32 <srwilkers> portdirect: ill let you lead this topic and the others until you get your new smile ;)
15:06:51 <portdirect> and its always great to both find out what else is going on in openstack - and put faces to irc names
15:07:05 <ianychoi> So nice - although I couldn't stay during Wed-Fri at PTG
15:07:31 <portdirect> for me the real highlights were the work that we will kick off this week on docs
15:07:44 <portdirect> thanks so much ianychoi and jayahn for helping with this
15:08:03 <evrardjp> irc nickname/face resolution service is so underrated
15:08:16 <mattmceuen> +1
15:08:20 <portdirect> and also getting some much needed thinking about our gating, and im really looking forward to evrardjp 's help here!
15:08:25 <jayahn> oh. no. it is co-working, not helping. :) thanks to you.
15:08:38 <jayahn> +1 on evrardjp!
15:08:43 <ianychoi> portdirect, it was so great pleasure to see u, and thanks a lot all :)
15:09:10 <ianychoi> evrardjp, so glad that you were also deeply involved with openstack-helm team :)
15:09:17 <evrardjp> :)
15:09:32 <evrardjp> gating is on my pipeline, 5th currently
15:09:33 <portdirect> it was also great to see some new faces appear and hear about use cases we had not imagined for osh
15:09:54 <evrardjp> sorry if I can't land that today :p
15:10:13 <portdirect> in particular, the turnout for airship was great, so i hope to see some tighter colab there
15:10:30 <portdirect> and also people wanting to use ironic and senlin for k8s cluster management
15:11:12 <portdirect> so it seems like whatever olstion the edge working group comes up with for hardware and k8s provisioning, osh has a home for now in the mix :D
15:11:17 <portdirect> *solution
15:11:41 <portdirect> I'll try to get a proper write up for the next meeting
15:11:49 <portdirect> anything anyone else want to add?
15:12:40 <srwilkers> also, looking at exploring getting monasca's helm charts working with openstack-helm to show it as a potential option
15:13:03 <evrardjp> portdirect: nope, nice to have met you all!
15:13:12 <jayahn> ditto
15:13:24 <srwilkers> good to move on portdirect?
15:13:45 <portdirect> yeah
15:13:46 <srwilkers> #topic Docs Repo
15:14:05 <srwilkers> portdirect: take it away
15:14:19 <portdirect> I'll bea ble to start this, prob not finish
15:14:25 <srwilkers> roger
15:14:34 <portdirect> which in many ways sounds like the docs themselves
15:14:35 <portdirect> :)
15:14:44 * mattmceuen ba-dum-ching
15:14:51 <portdirect> at the ptg we discussed creating a new repo
15:14:57 <portdirect> openstack-helm-docs
15:15:19 <portdirect> which we will use to home all the documentation for osh(-infra/-addons) etc
15:15:45 <portdirect> this week we plan to seed this repo with some cores spanning multiple languages
15:16:07 <portdirect> and then once we have a outline of the desired docs in place
15:16:34 <portdirect> use the great work that our korean speaking contributors have done to seed this repo
15:16:50 <jayahn> https://etherpad.openstack.org/p/openstack-helm-docs-outline
15:16:54 <portdirect> ^^
15:17:04 <jayahn> this was the etherpad link we created for gathering idea on outline
15:17:19 <portdirect> yeah - we still need to finish hashing that out
15:17:23 <jayahn> yeap
15:17:57 <portdirect> I think looking at other projects for inspriation, eg OSA/Kolla/TripleO/K8s/Istio would really pay dividends here
15:18:08 * portdirect has to leave
15:18:22 <portdirect> but I think we really have a good path forward
15:18:27 <srwilkers> portdirect: i agree
15:18:32 <portdirect> you ok to take it from here jayahn ?
15:19:01 <jayahn> well, portdirect said most of it.
15:19:26 <jayahn> we just need to get inspriation, and put it into our own osh docs outline.
15:20:18 <srwilkers> yep, i think we should make it a point to try and have an idea of what things look like before next week's meeting
15:20:27 <srwilkers> that way we can get started :)
15:20:50 <ianychoi> I am not sure other project cases which repo only has docs but IMO it is good direction forward
15:21:03 <jayahn> fyi, next week is korean-thanksgiving holiday week in korea
15:21:24 <srwilkers> jayahn: oh, let's shoot for the week after next then, so as not to disrupt the holiday :)
15:21:55 <jayahn> I will try to put some thought into outline etherpad though
15:22:13 <srwilkers> sounds good to me
15:22:25 <srwilkers> any other points on this topic?
15:22:38 <evrardjp> none
15:22:45 <jayahn> not really, ianychoi will help us go through this path as well.
15:22:47 <jayahn> thanks in advance.
15:22:48 <evrardjp> thanks jayahn and ianychoi
15:22:56 <srwilkers> :)
15:23:06 <ianychoi> jayahn, definitely :) evrardjp also thanks!
15:23:24 <srwilkers> #topic Gates
15:23:33 <srwilkers> this one will cover a few things
15:23:56 <evrardjp> :)
15:24:17 <srwilkers> first, wanted to just call general awareness to the fact the checks in openstack-helm are all very unhappy.  the glance bootstrap job is failing, and we need to find the root cause
15:24:47 <evrardjp> if someone asks why it's broken: I didn't do it
15:24:50 <evrardjp> (yet)
15:24:50 <srwilkers> i've got a few changes in flight to help us achieve that.  first, ive proposed disabling all check/gate jobs except for the single node nfs gates to help ease the load on openstack-infra
15:25:01 <srwilkers> https://review.openstack.org/#/c/603368/
15:25:36 <srwilkers> so we're not constantly launching multinode checks we know are going to fail, while we sort things out.  once we can fix the issue, we'll add the jobs we disabled
15:26:11 <mattmceuen> sigh, of course the single node nfs gate failed on that PS...
15:26:24 <evrardjp> yeah but let's try to fix it earlier rather than later. Merging things with reduced test coverage is risky
15:26:35 <evrardjp> mattmceuen: yeah :(
15:26:45 <srwilkers> yeah, we'll need to disable the rest of them mattmceuen
15:27:04 <srwilkers> evrardjp: agreed.  this is my highest priority right now
15:27:53 <evrardjp> srwilkers: when I will have a little time to clean up the gates, I will also make sure the log files gathering is not so verbose
15:28:00 <srwilkers> or make them nonvoting for the moment, to get the change to disable them in
15:28:15 <srwilkers> evrardjp: that's been one of the nice-to-have's on my list for awhile
15:28:30 <evrardjp> srwilkers: I will ping you for reviews when I will be on it
15:28:51 <srwilkers> as the number of things we gathered originally made sense, but now accounts for a significant portion of the post run job's time
15:28:52 <evrardjp> thanks for the work on gates srwilkers
15:29:17 <srwilkers> evrardjp: awesome, will be happy to review whatever you need.  i secretly like working on the gates -- it's like a puzzle
15:29:27 <evrardjp> srwilkers: I know the feeling :)
15:30:07 <srwilkers> but since you brought it up, the other portion of this topic was to announce/discuss some of the ideas you had for making our gates more efficient
15:30:11 <evrardjp> for those who weren't there in the gating conversation about the PTG ...
15:30:16 <srwilkers> evrardjp: take it away ;)
15:30:22 <evrardjp> haha
15:30:24 <evrardjp> nice thanks
15:30:49 <evrardjp> yeah so, I have seen quite a series of duplicate jobs, some redundant and missing coverage
15:31:04 <evrardjp> for example jamesgu__ would love to see opensuse support I guess :)
15:31:24 <jamesgu__> ah yes
15:31:26 <evrardjp> so the plan is to slowly simplify what we can, making everything more modular.
15:31:50 <evrardjp> I will first combine playbooks, then make sure each of those are enough modular to trigger parts of them depending on the jobs
15:32:04 <evrardjp> one base job will be tested on multiple distros
15:32:21 <evrardjp> further (longer) jobs will be moved to periodics
15:32:28 <evrardjp> Which means two things:
15:32:46 <evrardjp> 1) help will be needed for those reviews, so that we can have shared ownership of gating
15:33:16 <evrardjp> 2) attention will be requested on periodic logs, so that we have shared knowledge of failures, and focus on those before merging new things (that's the hardest part)
15:33:55 <evrardjp> Well that's my opinion :)
15:34:04 <evrardjp> ofc people can/will disagree :D
15:34:08 <evrardjp> hahaha
15:34:34 <evrardjp> does that plan look okay?
15:34:38 <jayahn> understood. especially, i totally agreed on "hardest part".. will need everyone's attention and support
15:34:47 <srwilkers> evrardjp: i think that's a good plan
15:35:04 <mattmceuen> That's awesome evrardjp - thanks for taking this on
15:35:04 <evrardjp> I guess another point we might add, is that periodics are cheaper than check+gates
15:35:12 <srwilkers> the shared ownership for gating will be awesome
15:35:16 <mattmceuen> How will the "periodic" part be managed?
15:35:40 <evrardjp> so if some specific case should arise, it could become an experimental check, and then, if we have more people interested by it, go periodic
15:35:49 <evrardjp> mattmceuen: well that's up to us to decide
15:36:14 <evrardjp> I would like to discuss this in another meeting, to be better prepared
15:36:23 <evrardjp> I have a few ideas and experience around that
15:36:36 <mattmceuen> awesome.
15:36:44 <srwilkers> evrardjp: yeah, sorry.  i ambushed you on this one, although i didnt expect us to get this detailed :)
15:36:46 <evrardjp> but it will take a certain amount of time in the discussion, and I don't want to monopolize the conversation
15:37:00 <srwilkers> i agree we should discuss this more formally once you've had some more time to get your thoughts down :)
15:37:31 <evrardjp> srwilkers: yeah didn't get the chance to work on it yet, sorry :(
15:37:50 <srwilkers> evrardjp: that's okay :)
15:38:18 <srwilkers> any other comments before moving on?
15:38:44 <evrardjp> none
15:38:54 <srwilkers> #topic Move to uWSGI
15:39:27 <srwilkers> portdirect set this on the agenda, but since he's unavailable at the moment do we want to move this topic to the next meeting or discuss this in the openstack-helm channel when hes back around?
15:39:58 <srwilkers> because thankfully, i cant read his mind
15:40:02 <srwilkers> :)
15:40:02 <mattmceuen> one of those - I don't have a preference
15:40:26 <srwilkers> let's try to get his attention later in the channel
15:41:06 <srwilkers> #topic Values ordering spec
15:41:31 <srwilkers> this was something we talked about at the PTG.  jayahn and his team started the spec here and did most of the work
15:41:54 <srwilkers> i've just simply worked to move it the last little bit: https://review.openstack.org/#/c/552485/
15:42:26 <srwilkers> one of the items we mentioned at the PTG was having some sort of mechanism to enforce this ordering, so as to avoid the overhead of relying on just reviews to prevent divergence
15:43:08 <srwilkers> evrardjp suggested yamllint to achieve that, and i threw together a quick change to include it in the linting job in osh-infra at the moment: https://review.openstack.org/#/c/603222/
15:43:34 <evrardjp> yeah it's kinda ugly -- and yamllint is not so flexible in terms of custom rules -- sorry for that
15:43:42 <evrardjp> but I guess it's better than the jungle?
15:43:42 <srwilkers> the only grievance i have is that to prevent the key-ordering rule from analyzing top level keys, we have to disable/reenable the key-ordering rule via comments around each top level key
15:44:17 <srwilkers> evrardjp: yeah, it's better than nothing at the moment.  also saved us from having to hand-cook some bash to do it, at least in the interim
15:44:19 <evrardjp> I have briefly checked on jsonschema
15:44:37 <evrardjp> but I guess when I will have more time I will write a proper linter
15:45:01 <evrardjp> that's just far down on my list, I am not sure we'll ever see it : p
15:45:01 <portdirect> jsonschema may be a fut here?
15:45:05 <portdirect> fit
15:45:09 <srwilkers> yeah, it would be nice to have a future state to work towards
15:45:09 <mattmceuen> evrardjp, just looking at the yamllint PS real quick just now, I must have missed it - where is the key ordering actually defined?
15:45:44 <evrardjp> mattmceuen: I can check the patchset, but generally yamllint take a default configuration, and you extend it based on your needs with small modifications
15:45:55 <evrardjp> I think the config key for that is keyordering
15:45:58 <evrardjp> iirc
15:46:36 <evrardjp> portdirect: it might, but it doesn't have some of the linting features (and json is unordered)
15:47:43 <evrardjp> I have to check all of that. In the meantime, I appreciate the consitency, as it helps not forgetting anything and brings consistency
15:47:59 <evrardjp> and brings ease of reviewing*
15:48:00 <mattmceuen> So evrardjap you're saying we're relying on the default key ordrering now, but can extend that to define our own?  If so sounds good to me :)
15:48:11 <srwilkers> mattmceuen: the key-ordering rule uses unicode point to determine alphabetic ordering of the keys.  if we disable it for top level keys, it ensures all our sub-level keys are ordered correctly
15:48:28 <evrardjp> mattmceuen: exactly what srwilkers just said
15:48:40 <srwilkers> while allowing us to, in a hacky fashion, ensure our top level keys are ordered in the way we define in our spec
15:48:41 <evrardjp> it's not possible to extend the configuration or rules for this yet.
15:48:49 <mattmceuen> Oh, so the "yamllint disables" would need to be long lived across all charts
15:48:51 <evrardjp> it's just a boolean: care or not care
15:49:05 <srwilkers> mattmceuen: yeah, until we determine a more thorough way to do it
15:49:10 <evrardjp> mattmceuen: yes, that's why it's ugly :p
15:49:11 <mattmceuen> gotcha
15:49:13 <mattmceuen> :D
15:49:26 <srwilkers> its not ideal, but neither is having sprawl across the charts
15:49:47 <srwilkers> ive got some other ideas, but we can save those for the openstack-helm channel
15:49:54 <srwilkers> we've got one more item i'd like to tackle in the time we have left
15:49:56 <evrardjp> sounds nice
15:50:11 <srwilkers> #topic Dropping newton support
15:50:47 <srwilkers> at the ptg, we discussed whether it was time to move on from newton as the default as it's a bit long in the tooth
15:51:14 <jamesgu> +1
15:51:17 <srwilkers> the general thought was "yes", but we didnt have a quorum in attendance so i wanted to discuss this here and vote now
15:51:18 <mattmceuen> +1
15:51:20 <jayahn> +1
15:51:26 <srwilkers> +!
15:51:31 <jamesgu> what will be the new default?
15:51:32 <lamt> +1
15:51:34 <srwilkers> (yes, im that excited)
15:51:37 <evrardjp> +1
15:51:46 <evrardjp> jamesgu: let's ask for rocky
15:51:48 <evrardjp> :D
15:52:00 <evrardjp> or master
15:52:02 <jamesgu> you know what exactly I was thinking!
15:52:04 <srwilkers> master or bust!
15:52:09 <srwilkers> just kidding (maybe)
15:52:14 <evrardjp> srwilkers: haha
15:52:18 <evrardjp> jamesgu: ;)
15:52:22 <jayahn> i am good with that. ;)
15:52:24 <ianychoi> I don't know the details on openstack-helm but also +1 for same direction :)
15:53:04 <srwilkers> so it seems we have quorum for moving on from newton -- i'll follow up with portdirect on discussing next steps here
15:53:04 <mattmceuen> Note we only have overrides defined in the repo for Newton, Ocata, and Pike:  https://github.com/openstack/openstack-helm/tree/master/tools/overrides/releases/pike
15:53:13 <evrardjp> it's simpler and in alignment with upstream expectations to be tracking master, but I understand it's a delicate game, and the latest stable is an "easier to track" and understandable goal for deploy projects
15:53:25 <evrardjp> mattmceuen: good point
15:53:29 <srwilkers> #action srwilkers to follow up with portdirect on moving on from newton as default
15:53:48 <mattmceuen> First step is we need to get overrides defined, then gating established, then we can pick good defaults from among the battle-hardened gated versions
15:53:51 <srwilkers> mattmceuen: that's true, but think portdirect also had work to support queens awhile back
15:53:54 <evrardjp> srwilkers: there were people that weren't at the ptg, could you explain the plan?
15:54:12 <srwilkers> evrardjp: sure
15:54:15 <jayahn> queens will not be that hard.
15:54:28 <portdirect> jayahn: queens is done, just not merged
15:54:34 <jayahn> i know. :)
15:54:45 <jayahn> that is what i mean by not be that hard
15:54:46 <srwilkers> portdirect: you care to elaborate on the plan here, if you're afoot?
15:55:14 <portdirect> i may be missibng context
15:55:24 <portdirect> but we intend to move to ocata by default for now
15:55:25 <mattmceuen> sounds like the anesthetic hasn't worn off
15:55:34 <evrardjp> haha
15:55:41 <mattmceuen> ++ ocata for now
15:55:42 <portdirect> and then also add upgrade gating as a periodical (thx evrardjp !)
15:55:59 <portdirect> once thats in place, too the moon! or rocky...
15:56:06 <jamesgu> could I ask why not even Pike for example?
15:56:22 <evrardjp> jamesgu: I guess the intent is to slowly move towards latest
15:56:31 <srwilkers> evrardjp: jamesgu: yep
15:56:33 <evrardjp> by having a path to upgrade that's periodically tested
15:56:35 <portdirect> ^ this, though i hope slow becomes very quick
15:56:51 <jamesgu> got it
15:56:59 <portdirect> as i (and alanmeadows ) have tested upgrades a few times
15:57:05 <evrardjp> portdirect: I also have dreams :)
15:57:11 <srwilkers> lol
15:57:17 <srwilkers> dont let your dreams be dreams evrardjp
15:57:25 <evrardjp> haha good point!
15:57:40 <srwilkers> alright, one last thing before we adjourn
15:57:46 <srwilkers> #topic reviews
15:57:59 <srwilkers> i've listed some things that would be nice to get some reviews on
15:58:12 <srwilkers> S3 manifests, snippets, scripts for HTK: https://review.openstack.org/#/c/588066/
15:58:12 <srwilkers> Elasticsearch S3 repository support: https://review.openstack.org/#/c/559417/56
15:58:12 <srwilkers> Trim OSH-Infra dev deploy gates: https://review.openstack.org/#/c/602833/
15:58:12 <srwilkers> Update fluentbit version, config template: https://review.openstack.org/#/c/602784/
15:58:12 <srwilkers> Add kernel, dockerd, kubelet logs: https://review.openstack.org/#/c/602786/
15:58:12 <srwilkers> RFC: Add htk snippet for tolerations https://review.openstack.org/#/c/595021/5
15:58:37 <srwilkers> oh, i just noticed the gnocchi and ceilometer item on the etherpad -- im sorry whoever put that down there
15:58:52 <srwilkers> let's discuss that in openstack-helm once portdirect is back around, as he's the last one to work on those i think
15:59:40 <srwilkers> that puts us at time -- let's carry over anything we want to discuss in openstack-helm
16:00:01 <srwilkers> thanks everyone
16:00:04 <srwilkers> #endmeeting