15:00:44 <mattmceuen> #startmeeting openstack-helm
15:00:44 <openstack> Meeting started Tue Apr 10 15:00:44 2018 UTC and is due to finish in 60 minutes.  The chair is mattmceuen. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:45 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:48 <openstack> The meeting name has been set to 'openstack_helm'
15:00:58 <mattmceuen> #topic Rollcall
15:01:04 <mattmceuen> Hey roman_g!
15:01:07 <srwilkers> o/
15:01:09 <lamt> o/
15:01:15 <d|k> o
15:01:16 <roman_g> o/
15:01:28 <gmmaha> o/
15:01:30 <rwellum> o/
15:01:32 <mattmceuen> https://etherpad.openstack.org/p/openstack-helm-meeting-2018-04-10
15:02:10 <mattmceuen> Please add in all the burning topics, questions, etc you would like to discuss today
15:03:24 <roman_g> I'm listening in. Mostly.
15:03:40 <mattmceuen> #topic Multiline LMA functionality (srwilkers and jayahn)
15:03:59 <mattmceuen> srwilkers and jayahn -- anything left to discuss on this one or did you level set offline?
15:05:00 <srwilkers> ive been working towards something that works here
15:05:40 <srwilkers> initially, i thought using something like the journald logging driver would close the gap here.  after some digging though, using anything but the json-file logging driver results in a loss of the ability to query for logs via kubectl
15:05:43 <srwilkers> which is a regression
15:06:23 <portdirect> this makes me sad panda
15:06:30 <srwilkers> portdirect: tres sad
15:07:10 <srwilkers> there are open issues in kubernetes that im watching for any updates, but seems the ability to use journald along with kubectl was removed when CRI was introduced
15:07:10 <mattmceuen> what's the downside of json-file approach?
15:07:45 <srwilkers> downside of the json-file approach is that we need to define regex's for each log format we may encounter with osh/osh-infra
15:08:01 <srwilkers> fluentbit will need a regex to identify the first line in a multiline entry (like stacktraces, etc)
15:08:14 <srwilkers> just requires some additional work on our end
15:08:50 <mattmceuen> gotcha
15:08:57 <srwilkers> there was some work done a long time ago to try and address this (dulek started it forever ago) that we might be able to leverage as a base if necessary
15:09:40 <srwilkers> but really, that's all ive got to say about it
15:09:50 <mattmceuen> thanks srwilkers
15:10:11 <roman_g> can we stream logs to the network and capture/process them on separate logging host? kubectl is kinda not supposed to be used for digging into logs. Only for fast lookups.
15:10:48 <roman_g> I'm not very competent in the topic, you may ignore question ))
15:10:57 <cristicalin> roman_g, that's what fluenbit is doint actually
15:11:03 <srwilkers> roman_g: thats the idea of having a centralized logging tool like fluentbit.
15:11:25 <cristicalin> srwilkers, does the journald driver account for multiline though ?
15:11:50 <roman_g> yes, journald works well with multiline logs
15:11:51 <srwilkers> cristicalin: it seemed to handle it better in the initial work i was doing
15:12:00 <cristicalin> as the stdout/err from containers still gets broken into multiple lines
15:12:27 <cristicalin> so docker would still see multiple lines and not know they are from the same message
15:12:38 <cristicalin> or you use openstack loggins to send to journald ?
15:13:14 <cristicalin> on our end we force openstack log formatting and have looked into using a json formatter
15:13:45 <cristicalin> works nicely for openstack originated logs but goes into a recursive crash loop for things like sqlalchemy
15:14:26 <mattmceuen> yikes
15:15:24 <cristicalin> here's what we use now, ignore the consul-template bit http://paste.openstack.org/show/718845/
15:16:54 <mattmceuen> Just curious cristicalin, what drove you guys to the journald approach?
15:17:16 <cristicalin> mattmceuen, we don't use journald, just experimented with it
15:17:22 <cristicalin> we use json-file
15:17:58 <mattmceuen> ahh gotcha
15:19:13 <srwilkers> cristicalin: neat.  will take a gander later today
15:19:22 <mattmceuen> srwilkers do we have a good list of all the multiline formats we'll need to support regexes for?
15:19:33 <mattmceuen> and/or just address them as we run accross them?
15:20:25 <srwilkers> mattmceuen: not at the moment.  the openstack services would be the ones to tackle first, then can address the other ones bit by bit
15:20:36 <mattmceuen> Cool - sounds good to me, thanks
15:20:44 <mattmceuen> #topic Storyboard for 1.0 integration items
15:21:15 <mattmceuen> As you guys know, the Foundation is working to move most projects to use Storyboard instead of Launchpad this cycle
15:21:49 <mattmceuen> At the PTG we decided we would go with the flow here, rather than being a trailblazer or a laggard
15:22:15 <mattmceuen> That still means we should start planning for it and find the right time to migrate
15:22:51 <mattmceuen> We have our 1.0 requirements spec in progress: https://review.openstack.org/#/c/556325/
15:23:14 <mattmceuen> That will result in a lot of granular bits that need to be coordinated well and worked quickly across our entire team
15:23:20 <mattmceuen> So we can get to 1.0 :D
15:23:52 <mattmceuen> I'm wondering, would it make sense to move to storyboard before we create all those work items, so we don't create them and then immediately migrate them from one system to another?
15:24:05 <rwellum> How is that related to storyboard? Or storyboard will be used to get to 1.0?
15:24:21 <mattmceuen> storyboard in this context is just a place to put the work - no direct tie in
15:24:25 <rwellum> ok
15:26:18 <mattmceuen> opinions welcome :)  If you're not familiar with storyboard yet, it's an openstack scope management tool created within openstack with openstack-like workflows and collaboration in mind
15:26:19 <mattmceuen> https://wiki.openstack.org/wiki/StoryBoard
15:26:38 <rwellum> Be interesting to see which projects have converted over. I know I set it up for kolla, but we didn't go to town on it - mainly because bug-tracking seemed way off.
15:27:12 <cristicalin> octavia has added a few things on their storyboard but not much
15:27:17 <mattmceuen> https://storyboard.openstack.org/#!/project_group/list
15:27:26 <mattmceuen> ^ projects that have migrated so far
15:28:16 <rwellum> Cool.
15:29:10 <mattmceuen> I think one thing that would be valuable would be for one of our team to do something of a deep dive into storyboard, see how other projects use it, and help guide OSH into using it well
15:29:36 <mattmceuen> Anyone interested in taking that on?
15:29:49 <rwellum> Has it been enabled for osh yet?
15:30:33 <rwellum> mattmceuen: be happy to take that on, seems like busy-work that could get me more involved in osh community
15:30:37 <mattmceuen> Somehow an osh project was already created, it's a mystery... I think we'll just recreate it upon migration I think
15:31:05 <mattmceuen> lol rwellum - that would be much appreciated and I think it'll actually be extremely valuable
15:31:12 <rwellum> np
15:31:35 <mattmceuen> I've gotten feedback from multiple folks that OSH would really benefit from a better-defined community roadmap
15:31:49 <mattmceuen> And I think migrating to storyboard would be something of a fresh start
15:32:03 <rwellum> +1
15:32:21 <rwellum> Although you've just described all of OpenStack :)
15:32:47 <mattmceuen> Well in that case, maybe we can define some good conventions that others can benefit from ;-)
15:33:01 <mattmceuen> I have some things I will share with ya offline too to give more background
15:33:14 <rwellum> Talking about roadmap - I did add a relevant topic to the etherpad.
15:33:22 <rwellum> Sure sounds good - I need to talk to you anyway :)
15:33:50 <mattmceuen> nice -- let's segueway right into it, I just have one short thing for later
15:33:53 <mattmceuen> #topic Support for other releases? Pike, Queens etc
15:34:04 <portdirect> we do already
15:34:23 <roman_g> Nancy created storyboard for OSH
15:34:26 <portdirect> but still need to demonstrate upgrade in the gate i think before moving this into the repo
15:34:36 <portdirect> as agreed at the ptg
15:34:42 <portdirect> 'the sugarcubes'
15:34:51 <mattmceuen> 'the sugarcubes'
15:34:51 <rwellum> So I am demoing osh to my company tomorrow - you're talking 60+ people, one of the things they are going to ask is releases, and upgrades.
15:35:07 <rwellum> So that's good news portdirect
15:35:26 <portdirect> I've been running master again recently...
15:35:45 <rwellum> https://en.wikipedia.org/wiki/The_Sugarcubes ??
15:36:02 <portdirect> almost as cool
15:36:20 <portdirect> I'm working on writing up the versioning spec for OSH
15:36:34 <rwellum> roman_g: cool, glad we know who did it - I'll look into it more.
15:36:49 <rwellum> portdirect: great. Let me know if you need a guinea pig?
15:36:52 <portdirect> and the simplest way to hash it out it was using sugar cubes on a desk
15:37:03 <mattmceuen> I'm expecting diagrams of sugarcubes in the spec portdirect
15:37:11 <mattmceuen> ascii
15:37:16 <portdirect> there was also a teaspoon
15:37:31 <portdirect> (to represent upgrade paths)
15:38:07 <rwellum> A nice cup of tea to represent falling into an upgrade whirlpool
15:38:18 <portdirect> no!
15:38:26 <mattmceuen> Our position overall rwellum is that we want to forge our way to the current release quickly, with gated upgrade support, and non-voting gates for master
15:38:30 <portdirect> openstack helm makes for nice calming tea
15:38:49 <portdirect> that may be drunk with the pinky out
15:38:57 <rwellum> :)
15:38:57 <mattmceuen> Feel free to share that with your crew and good luck with the presentation :)
15:39:04 <portdirect> ++
15:39:05 <rwellum> mattmceuen:+1
15:39:33 <mattmceuen> Last thing on the agenda:
15:39:39 <mattmceuen> #topic IRC - Slack integration revisited
15:40:06 <mattmceuen> So a few weeks ago we lamented the fact that Slack was removing its gateway for IRC (etc) integration
15:40:21 <mattmceuen> Believeing our Slack-IRC integration would be SOL
15:40:49 <mattmceuen> I chatted with cw since then, and he says that the tooling we use for Slack-IRC integration should actually not be impacted by this, that it doesn't use that particular gateway
15:41:00 <mattmceuen> So that's excellent news :)
15:41:57 <mattmceuen> #topic Reviews Needed
15:44:20 <mattmceuen> Anyone have any PS that they'd like to share to get some eyeballs on them?
15:45:03 <srwilkers> not at the moment
15:45:10 <cristicalin> active/passive maria ?
15:45:39 <srwilkers> actually
15:46:04 <srwilkers> wouldnt mind some eyes on: https://review.openstack.org/#/c/498929/
15:47:22 <mattmceuen> active/passive maria - was chatting w/ portdirect on that and he may have a thought on how to simplify the approach and get it done fast.  Can we plan on that for a topic of discussion next meeting cristicalin?
15:47:44 <cristicalin> yep, will try to make it
15:48:19 <mattmceuen> awesome - thanks man
15:48:28 <cristicalin> I'll ping portdirect later during the week as we are also looking into the problem
15:49:03 <mattmceuen> perfect
15:49:15 <mattmceuen> #topic Roundtable
15:49:26 <mattmceuen> open discussion, anything else on your minds?
15:51:08 <mattmceuen> Alright guys - good meeting, and have a good week!  See you in #openstack-helm
15:51:11 <mattmceuen> #endmeeting