16:30:45 #startmeeting kolla 16:30:46 Meeting started Wed Jun 1 16:30:45 2016 UTC and is due to finish in 60 minutes. The chair is inc0. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:30:47 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:30:50 The meeting name has been set to 'kolla' 16:30:56 hello everyone, show of hands please 16:31:04 hi 16:31:06 o/ 16:31:08 o/ 16:31:09 hello o/ 16:31:12 o/ 16:31:14 #topic rolll-call ? 16:31:25 o/ 16:31:27 hi! 16:31:37 damn 16:31:40 o/ 16:31:42 #topic show-hands 16:31:45 #topic rollcall 16:31:46 :) 16:31:52 hi 16:31:56 hi again! 16:32:12 hi 16:32:13 please re-show of hands;) 16:32:21 o/ 16:32:21 \o 16:32:23 I'm new to running meetings 16:32:23 \o 16:32:25 \o 16:32:27 hi 16:32:33 hello |o/ :) 16:32:50 #topic agenda 16:33:02 agena in wiki is outdated unfortunately 16:33:10 but we'll go through usual stuff 16:33:13 o/ 16:33:23 #topic Announcements 16:33:32 1. we're past n-1 16:33:39 https://launchpad.net/kolla/+milestone/newton-1 16:33:40 _o_ \o\ /o/ _o| |o_ 16:33:58 lots and lots of stuff there 16:34:13 actually newton 1 releases this week 16:34:29 well its May 30 - Jun 3 16:34:35 vhosakot is it somehitng like YMCA ;-) ?? 16:34:41 :) 16:34:44 so yeah, we still have few days 16:34:47 hello \o 16:34:56 please, let everyone take a bug from n-1 list and try to close it 16:35:10 we have tons of bugs confirmed, we need to get this managable 16:35:30 any other announcements? 16:35:33 yes 16:35:45 the most important thing we can do in the next 2 days is make sure mitaka-1 is stable 16:35:59 rather newton-1 16:36:09 tia :) 16:36:21 so "tons of bugs" and "take at least one";) 16:36:30 small announcement 16:36:38 oraclelinux gates are now active and green :) 16:36:42 yay 16:36:44 wouhou 16:36:46 thanks all for help with that 16:36:47 cool! 16:37:00 pbourke-home, (y) 16:37:10 than kyou pbourke-home for doing the majority of the work :) 16:37:35 anything else? can we move on? 16:37:46 #topic DSL 16:38:04 since sdake is up and about, let's have quick discussion about it please 16:38:18 #link https://review.openstack.org/#/c/323612/5 16:38:44 i need to leave on the hour 16:38:45 I'm going to paste what I said previously: The problem with the DSL is that it requires a certain structure for the generated Dockerfiles but it will be much harder to write inscruitable Elemental. The problem with jinja2 is that if you aren't carefull, you can cross over into being inscruitable, but you won't worry about things not representable in Elemental down the road. 16:39:07 sdake, let's try to get as much discussed as possible 16:39:19 I will review the PS 16:39:34 * rhallisey needs to also :) 16:39:36 so my take on it - imho we can't deprecate jinja2 without at least full cycle of deprecation 16:39:36 what i'd like is for people to review it on the patch 16:39:59 deprecation policies don't apply to internal implementations 16:40:10 it's not internal implementation 16:40:15 it's external sdake 16:40:17 it's somethign that people use and extend 16:40:33 'people' here being 1 or 2 people 16:40:36 or maybe 10 16:40:44 the longer we wait the more pain there is in their transition 16:40:46 every operator that I heard from and runs kolla on prod either plan or did changes to dockerfiles or created their own 16:40:59 sdake, that's not true 16:41:20 lots of deployments use things like vendor storage or vendor networking 16:41:29 deprecation poliices ONLY apply to external interfaces 16:41:29 they will go through pains of writing their dockerfiles 16:41:37 and with upgrade we'll break their env 16:41:46 this IS external interface 16:42:08 so we NEED a deprecation period for it 16:42:33 and that's even if we agree that it's good idea at all, which I'm not convinced about tbh 16:42:34 i disagree with your assertion that it is an external interface, just because people are using it as such 16:42:34 i agree on that tbh 16:42:48 you agree on which pbourke-home 16:42:54 It's an internal 'accidentally external' interface. 16:43:00 we are planning to rebase soon and if everything changes to dsl I am pretty certain it will be a blocker 16:43:01 we are not providing dockerfiles we are providing containers 16:43:13 coolsvap, that's a false 16:43:22 we provide dockerfiles which can be built 16:43:33 and really, practiaclity over philosophyu 16:43:34 people use our tools to build their images 16:43:47 if we lose deployments "because you shouldn't do this in the first place" we did lose a deployment 16:43:54 https://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml#n2107 16:43:54 im not totally against the dsl as you know, but the backwards compatibility/stability argument inc0 makes is ringing true with me 16:44:08 are we not a little beyond sweeping architectural changes 16:44:10 at this point 16:44:25 and we really CANT make this argument because we didn't provide good way to meet use cases of vendor stuff in mitaka 16:44:38 Being overly afraid of sweeping architectural changes has caused much pain and suffering in the OpenStack world. 16:44:59 wirehead_: I would argue the ones made cause more pain 16:45:01 for the operators 16:45:16 wirehead_, we are operator tool 16:45:20 and a deployment tool 16:45:31 if we break something, it's not just kolla thats broken 16:45:36 its literally everything 16:46:10 the only thing that even rings viably in your argument is that operators that have cusotmized their dockerfiles 16:46:11 i think the argument here should be whether we think DSL is a good way to generate Dockerfiles or not 16:46:12 our first and foremost priority should be stability of a deployment 16:46:19 wont be able to easily move their customizations to a new implementation of the dsl 16:46:28 no sdake that's not only thing 16:46:40 That still trebles to the pain of realizing we should have done something now a few years down the road. 16:46:43 I just don't want to say that you're overly confident in "this is easy" part 16:46:49 why it is and why it is not, what are the potential blockers 16:47:20 ok lets table the deprecation policy now, seems like putting cart before horse 16:47:22 let me give you an example of build.py 16:47:27 there will be no need for deprecation if we dont do a dsl 16:47:33 so lets move on to coolsvap 's lline of reasoning 16:47:49 well, if we agree on doing dsl and add deprecation policy, that means we will support both jinja2 16:47:52 and dsl 16:47:54 for time being 16:47:59 and see if dsl flies 16:48:02 is there an etherpad for DSL ? where do we write our views ? 16:48:12 vhosakot in the review inc0 linked above 16:48:13 vhosakot, there's a blueprint spec. 16:48:16 vhosakot, let's use review 16:48:20 ah ok cool 16:48:21 vhosakot, patch 323612 16:48:37 there will come a day 16:48:41 if we dont do the dsl now 16:48:44 blocker for me: maturity of code is important 16:48:56 where we will be forced to do it because of the crushing weight of our existing implementation as we scale out the big tent 16:48:57 we have fairly mature stuff now, needs refactoring but it's mature 16:49:05 (it needs refactoring because it matured) 16:49:29 history revisionism 16:49:41 I'd call it an experience 16:49:48 what really happened is people bolted a bunch of stuff onto builld.py to customize stuff without any concern for consistency 16:50:00 right now we have one dockerfile per image, and we will be replacing that with one yaml file per image 16:50:03 we need to make the customization step of build.py separate from building the images 16:50:16 pbourke-home i was thinking one yaml file per container set 16:50:18 the add on files will only be for customisation right 16:50:18 but whatever works 16:50:30 sdake, agree on that we need to refactor build.py 16:50:35 pbourke-home the new dsl will be the way to generate the dsl 16:50:36 so are we better off number of file wise 16:50:37 but that doesn't mean throwing it away 16:50:41 Also, providing a superior experience for add-on Kolla functionality is probably going to make the pain of the DSL less, IMHO. 16:50:48 That seems to be something not well thought out so far. 16:50:50 there are 40 container sets 16:50:54 there are 115 containers 16:51:00 pbourke-home, we can achieve same thing in jinja 16:51:04 with the method in the spec, there would be 40 yml files 16:51:13 it's a matter of what we want, technologically we can pull off a lot 16:51:25 inc0: i know im just addressing the 'we're going to be overwhelmed by big tent' arg 16:51:49 there are 40 j2 files now though 16:51:54 there are 115 j2 files 16:51:59 there woud lbe 40 yml files 16:52:06 yeah, we were limited by "only use if" 16:52:09 because heat has heat-api, heat-api-cfn, heat-engine, etc 16:52:13 ah I see 16:52:13 these are a container set 16:52:22 you will compress them 16:52:25 yeah, we have 100+ j2 and 100+ Dockerfiles 16:52:26 yeah that's one nice thing 16:52:26 the top level directory contains elemental.yml" 16:52:41 If we're going to make an issue out of this, a non-sdake representative needs to actually determine how hard it is to translate a service from j2 to yml. 16:52:44 i could compress them into one file but i think per container set is fine 16:53:03 wirehead_ i htink we all think that is trivial 16:53:05 ut i could be wrong 16:53:18 it takes about 20 minutes per service 16:53:20 inc0: how would you feel about using a less specific form of dsl 16:53:21 so pbourke-home we can easily refactor our Dockerfiles to make use of full jinja2 potential and compress everything as we like 16:53:38 so I just fail to see any real value in DSL in any form 16:53:44 since we have perfectly working tool now 16:53:51 inc0 in that cae, jinja2 is a dsl, lets throw it out! 16:53:55 I think there's value in it 16:54:00 inc0 it is perfectly workign for existing requirements 16:54:00 but the question is is it enough 16:54:05 not new requirements 16:54:06 inc0, my opinion is similar 16:54:07 it's a dsl that's mature, documented, stable and we already use it 16:54:07 i think we can always achieve a whole lot technologically but we need to find the best fit of technology and user experience 16:54:27 Putting on my operator hat, I suspect I'd be able to glean more information faster out of sdake's yaml file than a fully optimized jinja2 template. 16:54:27 yes recall kolla is a tool targeted at operators 16:54:36 not jinja2 rocket scientists 16:54:36 when you look at j2 files and yml files, yaml looks more clean, I think operators will appreciate that if in addition they will be provided documentation 16:54:36 so let's throw it out and create somethign immature, new, buggy and screw everyone who dared to create their own dockerfiles in the process 16:54:46 wirehead_: operators wont want to learn the dsl 16:54:49 pbourke-home, correct me if I'm wrong, that includes Oracle too right? 16:54:59 inc0: yes 16:55:04 inc0 i will address everything bu tthe lat part about people who made their own dockerfiles 16:55:24 let me rephrase, is there any operator who has prod deployments who DID not edit any dockerfile? 16:55:33 inc0 we will have a full testing period of 6 weeks in repo 16:55:36 plus the rest o fthe cycle 16:55:48 well, I've put 7 points in total in review 16:55:53 5 months is plenty of time to harden any mistakes we made 16:56:09 no, it is not, we have tons of bugs, other prority items and such to address 16:56:14 pbourke-home, I could say that on the other way around. Operators wont want to learn our particular set of jinja2 either. 16:56:17 sdake, ya but you're talking about hardening mistakes and adding new features 16:56:23 this kinda halts all new features 16:56:25 pbourke-home to that argument, you could label the custom plugin shit we have all over build.py 16:56:41 pbourke-home yet people learn how to use that stuff - necessity is mother of invention and all that 16:56:43 There's also creating a concept of an 'external builder'. 16:56:48 wirehead_, jinja2 is really close to what Dockerfile looks like 16:56:58 rhallisey it doesn't halt any new features 16:57:11 well it is how Dockerfile looks like with if statements, for loops and includes 16:57:11 i suspect, during development and review, atleast one docker file will get messed up during dev 16:57:14 and few other 16:57:15 and that will need repair 16:57:21 The present jinja2 is really close to what Dockerfile looks like. As the jinja2 complexity increases that may not be the case. 16:57:35 it's always just a templating language 16:57:36 I think it will, because we will be strengthening dsl 16:57:38 wirehead_ bingo im right there with you 16:57:39 nothing more 16:57:58 mandre, you here? 16:58:02 so sdake let say you start making dsl a mianstream 16:58:02 PHP is "just" HTML. :) 16:58:17 did people see what mandre's method for third party plugins was? 16:58:25 I'm pretty much of inc0's opinion 16:58:33 wirehead_, so let's remove php and write our websites in yaml that will generate html 16:58:53 There's at least 10 implemtations of that for each popular language in Github. 16:59:13 yeah, I don't feel adding another one 16:59:24 none of them were really good ideas, we can iterate examples 16:59:44 mandre of which opinion, that dsl is a bad idea? 16:59:45 so tell me this sdake: let say we agree on DSL 16:59:54 how do we add new container until all DSL is ready? 17:00:03 I believe we can clean up our code base with advanced jinja2, and implement plugins in a nicer way 17:00:08 how do you merge anything without breaking master untill all dsl is ready? 17:00:15 that the dsl is overkill and not flexible enouth 17:00:23 by all I mean every single container for every single distro and install type 17:00:24 mandre its not just plugins, we need customized binaries installed and customized repos PER COTNAINER 17:00:38 sdake, doable with jinja 17:00:41 easily 17:00:42 mandre your reaching at not flexible enough - you haven't provided any example where it can't work 17:00:43 it's doable 17:00:55 either way is doable 17:00:56 sdake, his third party plugin patch 17:00:59 sdake, install a package, run a command, install a package 17:01:02 in that order 17:01:06 the quetion is what is the best way for operators to engage with the project 17:01:09 how do you do it in yaml? 17:01:14 exactly what inc0 said 17:01:23 mandre higher up a level 17:01:26 inc0 higher up a level 17:01:29 nop 17:01:33 how does the operator integrate with it 17:01:45 install a c compiler, compile requirement lib, install a software needing compiled lib 17:02:04 real world example when you might need this order 17:02:17 yaml lis ordered 17:02:26 sdake, you can't control order in which the instructions are run with your dsl 17:02:28 its not a random ordering of things 17:02:32 but you group install packages together 17:02:39 mandre that is incorrect 17:02:46 so unless you want to do command that install stuff, it's impossible 17:02:55 and if you do then you'll do yaml that has nested dockerfile 17:03:00 i really need to jet soon 17:03:07 and this discussion needs to take place on the review 17:03:16 sdake, indulge me, how do you do it in your yaml? 17:03:17 but, here is the bottom line from my end 17:03:28 i gotta go I ontt have time at this point to explain it to you 17:03:33 remember, I said I have to leave ont he hour 17:03:36 I wrote a spec 17:03:53 I expect mandre and inc0 to write a spec as well that meets all the requirements I sset out in the use casees 17:04:11 I expect the core reviewers to review both of these specs and provide voting 17:04:19 if no spec wins, then we will have no customization 17:04:19 i know sdake has to go but I'll just throw this out and he can read later on 17:04:31 if both specs win, well what can I say, I dont know what to do with this 17:04:34 I personally would not bother with specs and just do pocs 17:04:41 pbourke-home: +1 17:04:47 pocs are there really 17:04:55 well give me a bit to polish jinja2 17:04:56 inc0: not enough 17:04:56 pbourke-home, sdague +1 17:04:59 pbourke-home the reason we do specs is to break logjams 17:05:00 dunno whether doing pocs instead of specs is "openstack way" 17:05:08 the dsl is barely functional 17:05:17 specs blow donkey 17:05:19 so it doesn't offer answers to the questions been asked here 17:05:20 nihilifer, whichever works, one poc is better than 100 specs imho 17:05:29 agree 17:05:29 the reason is becaue they add a bunch of tiem and complexity to development 17:05:40 let's get the pocs in "workable" state 17:05:43 I say this as someone who contributed some code to the dsl poc so its nothing personal 17:05:51 2 pocs trial by combat? 17:05:55 yes 17:05:55 in this case, because inc0 and mandre disagree, and possibly some others (its unclear) we require a spec with a majority vote 17:05:57 that's why we only do specs if we want to throw out our fundamental architecture 17:06:14 let the spec come after 17:06:24 or in paralell 17:06:26 i want all discussion in one spec 17:06:29 well, i'd like to have 100% working pocs first, then spec voting 17:06:33 or two specs 17:06:37 but not all spread out over 20 patches 17:06:44 one spec and one poc for each 17:06:45 then vote 17:06:52 hmmm... ok 17:06:53 ok, agree 17:07:00 wfm 17:07:07 they should prob have the same requirements though 17:07:08 thanks pbourke-home 17:07:10 they need to be listed 17:07:13 sounds good 17:07:15 as long as use cases are ALL met in the spec i wrote 17:07:16 so they aren't solving different problems 17:07:22 and then saying theirs is the best :) 17:07:22 there is a list in sdake spec 17:07:23 awesome :) 17:07:27 agreed 17:07:31 I'll add few of my own 17:07:34 pbourke-home good thinking 17:07:44 inc0 if you want to add stuff thats fine, but lets keep it implementation agnostic 17:07:51 ofc 17:07:54 i am interested in use cases here 17:08:06 these are just hte use cases coming out of "my sales channel" 17:08:06 I don't do troyan horses...most of the time 17:08:18 ok brainstorm the midcycle 17:08:19 sessions 17:08:22 I need a list of sessions 17:08:25 I have to go now 17:08:29 bbi1hr or so 17:08:57 lgtm, i agree 17:08:57 ok 17:09:06 #topic midcycle brainstorm 17:09:12 etherpad: https://etherpad.openstack.org/p/kolla-newton-midcycle 17:09:19 so, question guys, do we have venue? 17:09:34 or a date, although I might just be behind the power curve there. 17:09:36 inc0, I think still unknown 17:10:08 Are we doing it cohosted with an existing midcycle (c.f. operator?) 17:10:27 nothing that I know of 17:10:38 I thought Boston and Rayleigh, NC were discussed as venues 17:10:43 well, let's do a brainstorm for sessions 17:10:44 I can try for Westford,MA 17:10:52 vhosakot, ya 17:10:58 Raleigh* 17:10:59 cool 17:11:00 we also had option for Canada 17:11:09 Lyncos is trying to do that 17:11:31 so let's explore our options, I'll try Texas;) 17:11:39 anyone have anything to add to the etherpad for topics :) 17:11:43 although TX in the middle of summer might be hard to bear 17:12:08 Kolla-Kubernetes 17:12:17 wirehead_, add it :) 17:12:22 added it* 17:14:18 I've gone biking in TX in the middle of summer. I fear nothing except running over armadillos. 17:16:30 let's do this till 12.20 17:16:35 so 3 more minutes 17:17:45 So, minor rant: I started using Kolla-Kubernetes and Kolla a week or so ago and there have been a bunch of cases where the mainline was broken. 17:17:46 #link https://etherpad.openstack.org/p/kolla-newton-midcycle 17:18:06 wirehead_, let's leave it for open discussion 2 minutes from now;) 17:18:12 I'd like to hear how it's broken 17:20:06 kk, let's leave it for now 17:20:13 we can review it again next week 17:20:31 #topic Open discussion 17:20:39 wirehead_, you have a floor 17:20:54 K, so two things I observed. First, ansible 2.0 being changed in mainline but the docs not being changed. 17:21:22 I think there was patch upstream 17:21:30 not sure tho, but sure, we need to add docs for it 17:21:34 the docs will lag behind to some extent 17:21:44 as they're not required as part of a change 17:21:46 it's a master tho so that's understandable 17:21:50 https://review.openstack.org/#/c/322361/2 17:22:31 So, the docs were saying Ansible 1.9.4. The actual working build at the moment was Ansible 2.0, with a doc patch that said 2.x was OK. 17:23:40 yeah, in master it's expected to have some period between api change and doc change 17:23:57 I would disagree with that assertion. 17:24:05 well, that's our policy 17:24:17 since we don't do docs and feature in same patch 17:24:17 well we could be faster 17:24:24 that's always true;) 17:24:42 maybe shouldve required the doc change with the 2.0 change 17:24:50 anything else on your mind wirehead_ ? 17:24:57 95% of the time, it's OK to lag the docs. Not for things of the Ansible magnitude. 17:25:15 well, special cases are not special enough 17:25:16 Ive always wondered about the doc policy tbh 17:25:25 as to how much sense it makes 17:25:33 if you change something important, you should document it 17:25:33 me too, but reason is that we won't hold back code for typos in docs 17:25:37 true 17:25:40 as code might be blocker for other stuff 17:25:40 So, second observation was that I had a regression on Kolla-kubernetes over the weekend. I'm mostly giving it a pass because month-old-project. 17:25:47 also I get some people are good devs but just can't document 17:25:53 it is what it is 17:26:24 wirehead_, ya there's no gating yet so it's going to happen here and there 17:26:39 well it as, as yhou said, month old project which we still work on architectural level 17:27:00 Yeah, that's mostly a "we should start working on gating" comment. 17:27:33 that is also always true;) 17:27:39 and it holds true to kolla ansible too 17:27:43 our gates sux 17:27:51 not as much as it used to, but still 17:27:52 Projects that implement gating earlier move faster. 17:28:13 well, we have some openstack infra constrains 17:28:25 i just added a note to the midcycle to dicuss why we still can't make ours gate vote 17:28:25 anyway, 2 minutes left 17:29:23 ok, thank you guys! 17:29:35 cya all in #openstack-kolla 17:29:40 #endmeeting kolla