16:27:06 <rhallisey> #startmeeting kolla
16:27:08 <openstack> Meeting started Wed Jun 15 16:27:06 2016 UTC and is due to finish in 60 minutes.  The chair is rhallisey. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:27:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:27:12 <openstack> The meeting name has been set to 'kolla'
16:27:26 <rhallisey> #topic Kolla-kubernetes
16:27:42 <rhallisey> did anyone have any further comments
16:27:50 <rhallisey> I appreciate any review guys
16:28:11 <wirehead_> Yeah, I think I'm obsessed with getting us to the point where we're not carrying 7 patches on the Kolla side.
16:28:22 <coolsvap> rhallisey, i will also do
16:28:38 <rhallisey> wirehead_, it's because kolla-kube is new, we're sending out patches like crazy
16:28:42 <rhallisey> it won't happen forever
16:28:46 <wirehead_> :)
16:28:48 <rhallisey> :)
16:28:53 <rhallisey> thanks coolsvap
16:29:20 <Mech422> is there anything non-committers can do to help ?
16:29:34 <rhallisey> Mech422, you can try it out :)
16:29:44 <rhallisey> so looking at the topics..
16:29:54 <rhallisey> # topic Refactoring of build.py
16:29:55 <wirehead_> I think there was some convo about the Kolla gates.
16:29:57 <Mech422> rhallisey: yeah - I'll prolly fire up a test run this weekend :-)
16:30:06 <wirehead_> On the openstack-dev mailing list this morning.
16:30:07 <rhallisey> #topic Refactoring of build.py
16:30:26 <rhallisey> let's keep moving we're 30min in
16:30:44 <rhallisey> no inc0?
16:30:56 <Mech422> rhallisey: he said he couldn't attend today
16:31:00 <rhallisey> ok
16:31:04 <rhallisey> I'll give a quick summary
16:31:53 <rhallisey> inc0 has a bunch of patches up that also need review (sorry for nagging).  The patches will making everything much more configurable
16:32:07 <rhallisey> they should be easy reviews
16:32:28 <rhallisey> #topic do we need tqdm?
16:32:37 <rhallisey> #link https://review.openstack.org/#/c/325529/
16:32:41 <coolsvap> rhallisey, i added it
16:32:46 <rhallisey> coolsvap, go for it
16:32:55 <coolsvap> i was in the requirements meeting earlier today
16:33:07 <coolsvap> where they wanted to know if we need tqdm
16:33:12 <rhallisey> #link https://review.openstack.org/#/c/325527/
16:33:14 <coolsvap> i was looking for harlowja_
16:33:32 <coolsvap> and i need cores looking at the review
16:33:41 <coolsvap> proposed by harlowja_
16:34:03 <rhallisey> I know inc0 tried it out. I haven't yet
16:34:14 <coolsvap> and whether we want them in or not
16:34:27 <coolsvap> and provide update on the requirements review
16:35:01 <rhallisey> awsome, thanks coolsvap
16:35:22 <rhallisey> #topic Non ini config
16:35:40 <rhallisey> this might have been a inc0 topic also
16:36:02 <pbourke> this came up a few weeks ago on the mailing list
16:36:04 <Jeffrey4l> are we talking the merge_config?
16:36:09 <pbourke> yeah
16:36:20 <Jeffrey4l> we talked this in the ML.
16:36:35 <pbourke> i suggested custom merge modules which sdake seemed to like, never went any further
16:37:07 <Jeffrey4l> 1. solution is using more merge stragy, like override, append
16:37:23 <rhallisey> do you have a link to the thread
16:37:26 * rhallisey is looking
16:37:53 <pbourke> http://lists.openstack.org/pipermail/openstack-dev/2016-May/093936.html
16:38:00 <pbourke> #link http://lists.openstack.org/pipermail/openstack-dev/2016-May/093936.html
16:38:04 <pbourke> is that right?
16:38:15 <rhallisey> perfect thx
16:38:17 <Jeffrey4l> http://lists.openstack.org/pipermail/openstack-dev/2016-May/093936.html
16:38:23 <Jeffrey4l> correct
16:39:05 <pbourke> unless someone is prepared to write those merge modules I would say we need an easier solution
16:39:14 <rhallisey> pbourke, I didn't follow the thread closely, was there a solution agreed upon?
16:39:18 <rhallisey> ok
16:39:20 <pbourke> rhallisey: not really
16:39:20 <Mech422> stupid question...doesn't that awesome config file tool support OS configs yet..
16:39:28 <Mech422> dam - let me get name real quick
16:39:31 <pbourke> Mech422: crudini?
16:40:00 <rhallisey> pbourke, so we may need to bump that thread and get the discussion going again
16:40:19 <Mech422> pbourke: no - RH did one that parses like every config file in the system
16:40:27 <rhallisey> I read though and see if I have an idea
16:41:07 <Mech422> pbourke: http://augeas.net/
16:41:11 <Mech422> augeas
16:41:32 <pbourke> interesting
16:41:52 <Jeffrey4l> i think the apache conf file is the hardest one. it is hard to tokenized.
16:41:54 <Mech422> pbourke: you have to 'teach' it the file format - but then its REALLY nice for updatign stuff
16:42:26 <Jeffrey4l> is augeas support apache config file ?
16:42:30 <Mech422> pbourke: Here's what it can handle already: http://augeas.net/stock_lenses.html
16:42:40 <pbourke> #link http://multivax.blogspot.co.uk/2011/02/managing-apache-config-with-augeas.html
16:42:55 <pbourke> might be a runner
16:43:06 <pbourke> we should look into it and follow up on the mailing list
16:43:11 <pbourke> thanks Mech422!
16:43:16 <rhallisey> pbourke Mech422 how about we follow up on the ML
16:43:25 <Mech422> Jeffrey4l: I think so - its RH tool - so apache is under 'http' in that list
16:43:27 <rhallisey> breath some life into that thread
16:43:40 <rhallisey> thanks for the pointer Mech422
16:43:48 <Mech422> np
16:44:11 <Jeffrey4l> seems cool
16:44:30 <rhallisey> k let's move to gates
16:44:32 <Mech422> it would be useful to more then just Kolla too...
16:44:49 <rhallisey> #topic Gate stability and voting
16:44:56 <rhallisey> #link http://lists.openstack.org/pipermail/openstack-dev/2016-June/097401.html
16:45:18 <rhallisey> dmsimard made a good point about our gates
16:45:26 <Jeffrey4l> 1. gate is buggy now. need more eye on it.
16:45:30 <rhallisey> there are a bit flaky and are not voting
16:45:39 <rhallisey> I agree Jeffrey4l
16:46:04 <Jeffrey4l> 2. i think it is time to remove the `non-voting` flag to the ubuntu/centos gate job.
16:46:15 <pbourke> +1
16:46:29 <rhallisey> considering kolla's priorities for N, this is definitely up there.  Gating has been a weakness for kolla
16:46:29 <coolsvap> +1
16:46:31 <Jeffrey4l> in this way, it will enforce the commiter to notice the build/deploy gate result.
16:46:55 <mandre> that will force us to fix the flaky gates
16:47:05 <rhallisey> mandre, that's a good point
16:47:18 <pbourke> Sam used to mention mirrors as a major blocker for making the gates voting
16:47:33 <pbourke> I dont fully understand it though tbh
16:47:47 <mandre> so we're already using mirrors for pip and ubuntu packages
16:48:05 <mandre> there is apparently mirrors for centos, since a few weeks ago
16:48:14 <Jeffrey4l> kolla failed most not because the repo mirror.
16:48:19 <pbourke> there's centos mirrors all around the world though no?
16:48:35 <dcwangmit01> hi
16:48:40 <mandre> pbourke: centos mirrors hosted in infra
16:48:41 <rhallisey> dcwangmit01, hey :)
16:48:48 <Jeffrey4l> yes. centos mirrors around the world and it will use the fastest ones.
16:49:03 <pbourke> mandre: that would be faster sure, but I dont see how lack of infra mirrors could ever block kolla dev
16:49:46 <Jeffrey4l> mandre, what's the url? kolla do not use the centos mirror from infra.
16:49:51 <pbourke> will say one thing, making flaky mirrors voting will put off new contributors
16:50:09 <mandre> let me find the announce email
16:50:21 <wirehead_> Yeah.  I know that dcwangmit01 tried several times to get a full passing set of gates and every time a different set failed.
16:50:50 <mandre> http://permalink.gmane.org/gmane.comp.cloud.openstack.infrastructure/4286
16:50:51 <rhallisey> pbourke, it could, but so will broken code that gets merged because of flaky gates
16:50:56 <Jeffrey4l> pbourke, but it is one thing we must do. otherwise, new commit may introduce more and more bugs.
16:50:57 <wirehead_> Then again, flaky non-voting gates is will also spoil the average noob dev's tea.
16:51:08 <pbourke> rhallisey: 95% of failures are not code related
16:51:09 <pbourke> imo
16:51:21 <pbourke> they will rage quit
16:51:33 <dcwangmit01> I was obsessive about "recheck"'ing the tests, each time getting a different set of failures for different reasons
16:51:44 <pbourke> I think a better approach is to start collecting a list of all failures from now on
16:51:47 <pbourke> create bugs for them
16:51:48 <rhallisey> I do agree, but then that's lame to think our gate isn't passing 95% of the time
16:51:59 <pbourke> Im not saying they fail 95% of the time
16:51:59 <wirehead_> Well, we can always split up the gates a bit more.
16:52:11 <Jeffrey4l> mandre, cool we can enable the centos infra mirror in kolla.
16:52:14 <pbourke> im saying when the do fail 95% of that is not related to the patch
16:53:10 <pbourke> 9 mins left, now people are aware of the issue I suggest they reply to the ML thread
16:53:17 <pbourke> and we come up with some solutions
16:53:23 <rhallisey> ya agreed
16:53:37 <rhallisey> pbourke, do you want to start a thread or shall I?
16:53:43 <rhallisey> or we could add onto daves
16:53:51 <pbourke> rhallisey: say we just use daves
16:54:07 <rhallisey> k let's continue the discussion there
16:54:13 <Jeffrey4l> ok.
16:54:15 <rhallisey> #topic Open Discussion
16:54:39 <Jeffrey4l> https://review.openstack.org/329808
16:54:39 <Jeffrey4l> here.
16:54:50 <Jeffrey4l> need review :D
16:55:06 <pbourke> is there any reason we wanted to enforce the new kernel
16:55:23 <pbourke> Jeffrey4l: did you check the log for the original commit?
16:55:30 <wirehead_> Oh, so I noticed a few +2's on our Kolla patches already, but if someone could review https://review.openstack.org/#/c/327925/ that blocks a lot of other patches from being merged.
16:56:14 <Jeffrey4l> pbourke, the doc say aufs not works very well on the old kernel ( < 3.19 )
16:56:18 <rhallisey> didn't 14.04 not work for a time?
16:56:24 <rhallisey> maybe that's why?
16:56:29 <wirehead_> I've had issues building Centos for roughly the same reason.
16:56:55 <Jeffrey4l> wirehead_, you mean build centos on the ubuntu? it is a bug from aufs.
16:57:22 <Jeffrey4l> rhallisey, 14.04 works. Our CI is use 14.04 for host os.
16:57:35 <wirehead_> No, building Centos images from stock Docker because of the same issue with AUFS.
16:57:42 <rhallisey> not now, but a few months ago
16:57:55 <rhallisey> I thought there was some discussion about it
16:57:58 <rhallisey> could be wrong
16:58:13 <rhallisey> oh one more thing everyone
16:58:17 <Jeffrey4l> even though aufs do not works. We should recommend the user to use other storage driver, rather than upgrade the kernel.
16:58:23 <Jeffrey4l> rhallisey, no idea for that.
16:58:30 <rhallisey> since I know some people were here at 16:30 UTC
16:58:49 <wirehead_> Jeffrey4l: I can get behind that.
16:58:52 <rhallisey> according to the ML proposal the meeting is moved to 16:00 UTC
16:59:09 <rhallisey> just a heads up for those that were late. We are at 16:00 UTC now
16:59:18 <rhallisey> Jeffrey4l, kk
16:59:24 <rhallisey> 1 minute left!
16:59:29 <Jeffrey4l> rhallisey, we may need re-claim the meeting time and update the wiki page.
16:59:45 <coolsvap> Jeffrey4l, its already up for review
16:59:50 <coolsvap> i think might have been merged
16:59:51 <Jeffrey4l> cool
16:59:55 <Jeffrey4l> wirehead_, thanks.
17:00:02 <rhallisey> Jeffrey4l, ya I think sdake is already on top of it
17:00:07 <rhallisey> ok thanks everyone!
17:00:10 <coolsvap> but i am still not sure how it came half n hour early for me
17:00:15 <rhallisey> #endmeeting kolla