16:00:25 #startmeeting kolla 16:00:25 Meeting started Wed Sep 28 16:00:25 2016 UTC and is due to finish in 60 minutes. The chair is sdake. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:26 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:28 The meeting name has been set to 'kolla' 16:00:38 #topic rollcall 16:00:39 o/ 16:00:43 o/ 16:00:46 hello 16:00:54 \o/ 16:00:57 o/ 16:01:01 o/ 16:01:06 \o 16:01:32 hi 16:01:36 o/ 16:02:00 hey there folks ;) 16:02:05 o/ 16:02:06 o/ 16:02:10 o/ 16:02:11 where be inc0 16:02:28 here 16:02:32 cool 16:02:54 #topic announcements 16:02:57 o/ 16:03:28 as you are probably aware, inc0 is my successor as ptl - say grats ;) 16:03:35 congrats inc0 16:03:40 congrats inc0 !! 16:03:41 contrats inc0! 16:03:41 \o/ 16:03:42 congratulation inc0 16:03:44 congrats! 16:03:49 thank you all:) 16:03:54 sdake: and thanks for all your hard work! :) 16:03:55 congratulations, Michał! 16:04:03 thanks kfox1111 16:04:15 how the rest of this cycle will opertate is as follows 16:04:22 thank you Steven for your hard work! 16:04:25 woo 16:04:34 uring ptl transitions, typically at election time the successor ptl takes on rsponsibilities or summitand master 16:04:35 yay inc0 16:04:36 congrats inc0! good sailing! 16:04:50 and my responsibility for newton stays in place 16:04:54 and ofcourse nice work sdake 16:05:00 thanks rhallisey 16:05:04 thank steak for bootstrap kolla in 1st and 2nd cycle 16:05:16 duonghq its been 3 years dude ;) 16:05:28 since Juno 16:05:31 so back on track 16:05:48 look to inc0 for leadership for master/summit/ocata/etc 16:06:14 look to both of us for 3.0.0 - as far as I'm concerned its a bit of a joint responsibiity 16:06:34 finally, I want to thank everyone in the community for supporting kolla 16:06:39 without you, kolla wouldn't exist 16:06:46 or would exist in vastly different ways 16:06:58 kolla has created a turning point for openstack 16:07:02 thats a good thing ;) 16:07:09 #chair inc0 16:07:10 Current chairs: inc0 sdake 16:07:14 ok inc0 - take it away 16:07:23 ok:) thanks sdake! 16:07:26 also, i have a conflict right at this time, so I have to jet ;) 16:07:35 but i'll be back in 45-50 mins 16:07:45 for all the hard work, I'll make sure to continue making kolla great, so we won't ever have to make it great again 16:07:57 ha 16:07:57 sbezverk also one point of clarificaiton, I am remaining on the core teeam going forward 16:08:02 #topic rc2 16:08:25 ok, rc2 means we branch stable/netwon and open master to Ocata 16:08:45 this happends 10-13 Oct 16:09:02 sdake: not sure the reason of your comments to me, I had no doubt you would stays as core ;-) 16:09:15 I'd like to target 10, but that depends on how many bugs and stuff we'll be able to fix 16:09:26 or more importantly, how many will stay 16:10:02 #link https://launchpad.net/kolla/+milestone/newton-rc2 16:10:05 o/ 16:10:08 o/ 16:10:35 o/ 16:10:36 o/ 16:10:50 Sorry, what is this for? 16:10:55 take a look at this list, let's try to fix as much as we can in following 2 weeks 16:11:22 Daviey, these are bugs outstanding in Kolla targeted for RC-2 16:11:25 Ah 16:11:37 reviewers, please prioritize reviews as bug priority goes 16:12:06 besies that, we need lots of testing 16:12:33 if there is any bug you feel is important to be fixed in N, shout out to any of cores and let's triage/target it to RC2 16:12:45 after that we release 3.0.0 16:13:02 we still will backport bugfixes to stable branch, but let's make sure release is solid 16:13:16 Not just kolla, but i kinda feel a few bugs have been introduced in the last couple of weeks across OpenStack.. We really need to bash out testing hard 16:13:23 one point on this also, please do not merge any patches that are not high/critical for newton-2 16:14:03 especially take a look at our release notes and 1. test what's there, new features and 2. if you see any feature missing from release notes, put a patch up please 16:14:06 inc0: I asked sdake, and I'll ask you. right before cutting the final rc2, can you please let me know and I'll run some regression tests against kolla-kubernetes just to make sure we don't introduce any bugs last minute. 16:14:22 we really need a gate test for that, and its in progress, but its kind of manual right now. :/ 16:14:58 kfox1111, sure, be extra careful when you put any patch to kolla itself 16:15:15 kfox1111, we don't have a release though 16:15:23 also please test it out as much as you can yourself:) deployment and such of kolla-ansible with your patch 16:15:33 inc0: sure. 16:15:33 even though we depend on kolla, we can just go off master 16:15:40 well we have an N release 16:15:45 eh not really 16:15:46 rhallisey: yeah, but I think we want to make sure trunk kolla-kubernetes works with released kolla 3.0.0, 16:15:53 let's skip a release for N 16:15:57 make the first release O 16:16:03 rhallisey, but if some regression gets introduced due to kolla-k8s required changes before we branch out, it's gonna be critical 16:16:03 as ocata is going to have a bunch of refactoring goin on to do the ansible split. 16:16:12 so we will want a stable target for a little while, which 3.0.0 can be. 16:16:19 that's find 16:16:21 fine 16:16:27 I think it would be a problem 16:16:38 split will happen after we branch out stable 16:16:38 and issue we hit will likely be hit on the kolla side 16:16:47 amy* 16:16:50 any* 16:17:04 yeah. so if we can target 3.0 and let the ansible split happen and settle out for a month or two, 16:17:14 then we can retarget to trunk again and both parts can make progress. 16:17:16 sure that works 16:17:18 bottom line everyone - extra careful testing 16:17:27 +1. :) 16:17:29 anyone wants to add anything? 16:17:30 kfox1111, it's going to be a bump road :) 16:17:36 i think it makes sense to make the split in or a few days after barcelona 16:17:53 Even knowing "Hey, this worked in rc2 but broke" is going to eliminate wasted effort. 16:18:01 berendt, I'll talk with infra folks to make it seamless 16:18:13 berendt: hard to say... some advantage to getting it done before the summit so folks wont keep talking about it then. 16:18:39 kfox1111, interesting opinion. Either way works for me 16:18:46 kfox1111 makes sense 16:18:53 kfox1111, decision has been made, when we branch it out is just a workload issue 16:19:03 yeah. 16:19:09 it might require some git magic to retain history 16:19:15 anyway, moving on 16:19:21 inc0, could be worth disccusing over beers or something our process for that 16:19:26 #topic summit session schedule 16:19:47 So thank you for answers in poll 16:20:08 https://etherpad.openstack.org/p/kolla-o-summit-schedule I took liberty of copying all the sessions from poll in order of votes 16:20:24 we have total of 9 design session (WR in etherpad) and 3 fishbowl 16:20:55 inc0: I quite wanted a session about version numbering... I think there is potentially enough content to cover most of a session at least 16:20:58 * britthouser is late 16:20:58 0/ 16:21:25 as you can see, I didn't follow poll results 100%, also took liberty of adding 2 more sessions to the list, one about deprecation and stuff and second session for kolla-k8s 16:22:06 Daviey, we always have contributor meetup and beer track, we can discuss then too 16:22:26 Daviey, a doc for that would be awesome. Agreed though 16:22:53 so this is what I'm going to propose unless somebody is opposed to these ideas, so shoot away 16:23:22 or if somebody have question regarding this schedule, please ask 16:23:44 I was really hoping for baremetal to make the list 16:23:56 it was nearly in the top9, and its a pretty new feature we just added. 16:24:11 britthouser, yes, although I'm not sure what we should talk about there 16:24:34 that we do not work on monitoring is not good 16:24:52 ok, fact that we're not doing session doesn't mean we don't work on it 16:25:12 yes of course, but i think it needs a wr.. 16:25:17 sessions are for things we need to discuss, are contentious, aren't clear 16:25:17 we can cover some of these topics in the roadmapping 16:25:28 I think the roadmapping session will be a bit open 16:25:46 we can also have backup topics of one of those topics don't use the whole time 16:25:49 but I bet they will :) 16:26:00 what is the focus of Security VMT threat? 16:26:11 berendt, so VMT is a team in OpenStack 16:26:26 yes.. but what will we do in the wr? 16:26:35 they started to analyze our projecxt in Austin 16:26:47 they will come to us and work with us to finish up this analyze 16:26:52 ok 16:27:41 I wanted to keep it on schedule even tho it was pretty low in poll, because it's good to have them locally to finish up this work and get us vulnerability-managed tag 16:28:04 ya it's good to get that tag 16:28:10 yes that makes sense 16:28:38 Monitoring and Bare metal were ones that I thought are extremally important, but not much to discuss really (if you don't agree, do tell) 16:28:54 we have both features in Kolla in N, we need to fix it up, gather feedback and such 16:29:25 but what are questions we need to answer on sessions? 16:29:36 i think the schedule is fine. we have to cover a lot of topics... 16:29:55 for monitoring e.g. which monitoring tool we want to use, if we want to keep telegraf, ... 16:29:59 we always have couple unassigned hours in Friday 16:30:01 I guess my thought as far as baremetal goes, is since its a new feature - go over what was done, get some feedback, discuss next steps, etc. maybe that can be rolled into roadmap though? 16:30:31 Yeah I think pencil it in for friday is fine with me. 16:30:33 britthouser, ya agreed 16:30:36 britthouser, so what I'm sayin is that I doubt there will be much feedback to get 16:31:01 there may not be, but we could talk about what to do next or somethign 16:31:08 * rhallisey hasn't used it yet 16:31:21 ccccccfnnvlltgvvfcrllgbgthlhnubclilhbjeeciig 16:31:25 sure, so let's discuss what to drop for bare metal then 16:31:30 anyway there will be plenty of *open time* 16:31:34 ccccccfnnvllfnvhltlffkhbnddgijcdhvntfgbvnfjv 16:31:36 we voted and i think we should go with this schedule 16:31:38 lol 16:31:45 inc0, nah keep the schedule 16:31:46 we can discuss other topics on friday 16:32:12 inc0, let's have baremetal and monitoring be the #1 and #2 backup topics 16:32:15 we 16:32:22 +1 16:32:25 ok, so let's keep it for now, we don't need to submit it just now, so it still is open to slight changes (slight please) 16:32:31 it will be apart of the *beer summit* 16:33:25 ok, moving on 16:33:36 #topic deprecation 16:33:54 so I wanted to move this topic in summit, hence my addition to schedule 16:34:07 we decided to deprecate fedora in N and keep debian in 16:34:37 inc0: how about deprication of heka? 16:34:37 (can someone please take action item to add warning during build if fedora then deprecated?) 16:34:44 heka is another thing 16:34:48 inc0 already did it 16:34:53 oh thank you 16:35:03 #link https://review.openstack.org/#/c/369184/ 16:35:11 so here is thing, I'd like to put some sort of process behind decisions like that 16:35:47 it might be as simple as documentation matrix of "what has CI and is super supported and what is not" 16:35:54 as distros and services goes 16:36:07 heka is different matter, let's talk about it after that 16:36:27 I would use the word “maintained” instead of supported, but that sounds like a good idea 16:36:32 ok, but we will talk right ;-) 16:36:38 yes, agree britthouser 16:36:48 yes sbezverk thanks for bringing it up 16:37:09 hi, sorry I've been away and just noticed this was going on :/ 16:37:26 and then have some rule, if bugs against that distro aren’t worked on in X months, then vote to deprecate. something along those lines? 16:37:58 it will be more complicated than this 16:38:24 for example for distros we might require CI, but RHEL, although maintained and healthy won't have CI for license reasons 16:38:25 how about some distro is out of maintain for 1-2 cycle. 16:38:43 Jeffrey4l, what I mean is how do we define "maintain" 16:38:55 anyway, I don't expect us to answer it today 16:38:58 Yeah…I think we can work out that definition at summit. 16:39:10 just wanted to throw an idea for you to think about 16:39:15 and make a call at summit 16:39:17 ok 16:39:19 +1 16:39:41 we also need this defintion also for services, not only for distros 16:39:51 I get lots of questions like "do you support ironic", answer is yes, but we'd lie if we'd say that we have same level of quality across every service we have 16:39:58 berendt, totally agree 16:40:03 at the moment we do not have a CI for most of our services, this has to be changed 16:40:33 I don't think we'll ever reach a spot when we'll have CI for everything we can deploy, Big tent is growing all the time 16:40:34 ci is one of the most thing we need set up in O cycle. 16:40:44 +100 16:40:49 but we can just say openly what has CI and what doesn't 16:40:49 +infinity ;) 16:40:52 +1 16:41:10 CI is in summit, agree totally;) 16:41:27 ok, moving on 16:41:31 #topic heka 16:41:48 sbezverk, can you please describe issue to everyone? 16:43:00 ok, I'll do it:) https://github.com/mozilla-services/heka 16:43:15 if you click the link there is big DEPRECATED work in the title 16:43:17 that's bad. 16:43:23 we need to look for alternatives 16:43:31 logstash 16:43:37 nuts! 16:43:56 Well, the promise for Hekka was that you'd get one fancy foofy daemon to do logs and metrics and alerts. 16:44:01 I think logstash is deprectated too in preference to filebeat? 16:44:04 As opposed to logstash + monitoring + metrics. 16:44:15 fluentd's an option too. 16:44:23 s/logstash/filebeat/... 16:45:09 let's open a blueprint "replace-heka"..? 16:45:18 +1 berendt 16:45:18 how about "kill-heka"? 16:45:29 I'll write the bp 16:45:31 This is that intersection between the modern notion of a "12 factor app" and the way that OpenStack presently is. From the Kolla-kube world, I'm kinda a bigger fan of not using a logging daemon inside of the contaienr at all. 16:45:32 :) 16:45:35 because we use elasticsearch for our logs i think it makes sense to go with them... 16:45:41 #action inc0 to write kill-heka bp 16:45:51 inc0, hehe 16:46:03 wirehead_, 12 factor app is ideal and reality is not. 16:46:08 what about https://review.openstack.org/#/c/375615/7 ? 16:46:12 fluentd 16:46:13 So are there things that we need for Heka to change, and we wont’ get those changes? 16:46:24 i think it makes no sense to replace heka by something and to add fluentd as well 16:46:31 or Heka works fine as is, but we don’t want to be on a dead code? 16:46:39 we already used fluentd in kolla-kubernetes 16:46:53 why not to have more things in common than different ;-) 16:47:00 sbezverk, link to you fluentd patch?? 16:47:03 also don't forget we'll need upgrade procedure for it 16:47:22 there could be multiple ones here 16:47:23 with the way kolla lays out its log files, 16:47:28 https://review.openstack.org/375615 16:47:33 I guess what I’m asking is, is heka broken today? or just a dead-end? 16:47:40 it really shouldn't matter if you use a heka container, or fluentd, or filebeat to do the uploading. 16:47:42 britthouser: dead end 16:47:44 dead end 16:47:48 anyway, bp will be essential in Ocata, if somebody feel like taking it, we can discuss pros and cons of alternatives 16:47:59 no one maintain the heka's code 16:48:07 ok if it's a dead end how about we leave it and provide an upgrade path 16:48:10 Ok good to know. So its not hair of fire, but the sooner we make a u-turn the better. 16:48:15 sdake: Just reading scrollback... RH *could* do 3rd-party-ci, if they cared to test kolla. 16:48:16 but it does work so it's not *that* bad 16:48:17 heka has to be dropped, i think this has not to be discusses 16:48:18 Security patches, etc. 16:48:18 they don't support it so let's not in the upcoming cycle 16:48:49 Daviey, well, 3rd party ci is more than just code 16:49:02 so let's not put them on the line 16:49:03 yeah. I think the discussion is not about keepign heka alive. just what to replace it with. 16:49:04 berendt, +1 the really concern is which we choose. 16:49:10 we would need, however, call that one out 16:49:11 inc0: I mean, if we wanted to have RH as a flavour we supported.. it could be 3rd party 16:49:38 but not having 3rd party ci shouldn't make them deprecated 16:49:49 Yes it should :) 16:49:59 If it isn't tested, we have to assume it doesn't wrk 16:50:01 work* 16:50:04 so we might create something in between, like we have it, we think it works but not really tested by our CI 16:50:17 anyway, session on summit, we need to call these things out 16:50:18 that way leads to madness 16:50:20 can we finish the heka discussion first? 16:50:21 lots of moving parts 16:50:25 you could say its vendor supported, if thats the case? 16:50:29 we’re on to a different topic Daviey, and we decided to define all that at summit 16:50:34 berendt, sure, sorry 16:50:40 ok, back to heka 16:50:53 We are sticking with heka for Newton, right(!?) 16:50:57 I'm just going to propose bp, think of alternatives 16:51:01 we do 16:51:09 Neutron is done, only bugfixes 16:51:10 what about https://review.openstack.org/#/c/375615? 16:51:14 heka goes away in mitaka 16:51:19 i would prefer to -2 this one until we have a decision 16:51:22 we might however mark it as deprecated 16:51:54 done berendt 16:51:55 inc0 we should mark it as deprecated right now, i think we do not need a mailing list vote for this deprecation 16:52:05 berendt, I think we don't need to vote 16:52:25 so yes, let's mark is as deprecated now 16:52:26 inc0 do you add a deprecation release note after opening the blueprint ? 16:52:26 inc0: I filed bp centralized-logging which could be used for logging discussion activities 16:52:47 sbezverk centralized logging bp makes no sense because we already have centralized logging 16:53:10 sbezverk, yes that name is confused. 16:53:27 Jeffrey4l ok that can be changed 16:53:50 let's move on? next topic? 16:54:03 sbezverk but this bp is not correct.. you wrote "bp to track activities around introducing fluentd as a replacement of heka in kolla" 16:54:11 I'll publish new bp 16:54:12 sbezverk we have not yet decided to use fluentd... 16:54:12 about heka 16:54:42 kill-heka is nice ;) 16:54:42 and yes, I'd also would like to have some alternatives before we jump to fluentd 16:54:43 berendt: but we are :-) it is already running in kolla-kubernetes 16:54:59 and we need that image in kolla, if kolla continues provide us images 16:55:10 sbezverk but this does not mean that you need an ansible role for it, right? 16:55:29 sbezverk, berendt I'd love to have same logging architecture in both kollas:) 16:55:36 but let's make this educated decision 16:55:42 even if this means changing it in kolla-k8s 16:56:02 inc0: that is my point exactly why complicates things if we can have somehting common 16:56:05 sbezverk, i think kolla can provide fluentd image. But Kolla may not use fluentd for collection logs. 16:56:17 yes but we have to discuss it first... we use E and K at the moment and they have there own tool 16:56:23 Jeffrey4l: and that is perfectly fine at least for now 16:56:35 sbezverk yes.. but your review provides docker + ansible.. 16:56:49 berendt: you do not have to use it 16:56:56 ok, hold on. We acknowledged problem, we discuss it now 16:57:00 it does not hart to have it right? 16:57:06 so let's first gather alternatives 16:57:08 sbezverk it does, we have to maintain it 16:57:20 I've heard at least 2 - fluentd and filebeat 16:57:26 why should we implement an ansible role for fluentd when we decide to use filebeat? 16:57:30 sbezverk, please split the docker file and ansible playbook. 16:57:49 Jeffrey4l: sure will do that 16:57:54 let's see if there is anything else, weigth pros and cons and push patches up when we make a decision 16:58:06 and ideally both kolla-ansible and kolla-k8s should have same logging stack 16:58:12 inc0, agreed 16:58:22 we have whole release to sort this out 16:58:28 does kolla-k8s uses elasticsearch and kibana? 16:58:37 yes 16:58:40 yar 16:58:48 rhallisey, I think we ran out of time 16:58:50 sorry:( 16:58:53 inc0: I wanted to squeeze in before the end of the meeting..... the performance monitoring stack seems to be in a sad state and not ready for release. Is anyone else looking at it? 16:58:55 no worries 16:59:02 we can overflow to our channel 16:59:03 imho, there is no alternative for elasticsearch and kibana 16:59:05 sure 16:59:16 what topics are left? 16:59:17 Jeffrey4l +1 16:59:23 Daviey, what do you mean? 16:59:32 timeout... 16:59:35 I'll quickly type kolla-kube update: 16:59:35 ok, we ran out of time:) 16:59:44 Daviey, let's talk on #openstac-kolla 16:59:46 -> #openstack-kolla 16:59:51 ya let's jump over 16:59:59 thank you guys! I wanted to thank you again for your trust in me:) 17:00:01 and votes 17:00:06 +1 inc0 :smiley: 17:00:08 woot for kolla? 17:00:21 thanks Ryan. ofc woot for kolla please:) 17:00:23 woot:) 17:00:24 you promised! 17:00:26 woot!! 17:00:26 :) 17:00:28 woot :) 17:00:31 woot!! 17:00:31 woot! 17:00:35 wh00000000t 17:00:35 #endmeeting kolla