15:59:14 #startmeeting kolla 15:59:15 Meeting started Wed Apr 19 15:59:14 2017 UTC and is due to finish in 60 minutes. The chair is inc0. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:59:16 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:59:18 The meeting name has been set to 'kolla' 15:59:19 o/ 15:59:21 o/ 15:59:23 o/ 15:59:24 meeting 15:59:24 o/ 15:59:24 o/ 15:59:24 o/ 15:59:26 #topic rollcall 15:59:32 o/ woot 15:59:34 howdy folks 15:59:38 o/ 15:59:44 how y'all doin' today 15:59:45 o/ 15:59:53 o/ 16:00:00 o/ 16:00:01 doing swell 16:00:05 o/ 16:00:11 wo0t 16:00:16 woot 16:00:17 o/ 16:00:21 o/ 16:00:35 o/ 16:00:44 woot o/ 16:00:47 o/ 16:01:39 #topic announcements 16:02:18 Another plea for help from PTL - we'll host hands on workshops in Boston for kolla-ansible 16:02:25 and we'll need mentors in room 16:02:31 ;) 16:02:32 inc0 who else you got lined up 16:02:38 and what time and day is the sesion 16:02:40 * berendt will not be in Boston :( 16:03:02 o/ if you want, I can give you a hand 16:03:15 duonghq visa approved then? 16:03:21 https://www.openstack.org/summit/boston-2017/summit-schedule/events/18177/easier-openstack-deployment-novice-installation-using-kolla?BackURL=https%3A%2F%2Fwww.openstack.org%2Fsummit%2Fboston-2017%2Fsummit-schedule%2Fglobal-search%3Ft%3Djastrzebski%23eventid%3D18177 16:03:29 sdake, yes, it should be delivered to me tomorrow or Friday 16:03:33 duonghq: would appreciate it greatly 16:03:34 duonghq nice! 16:03:45 Tue 4:40-6:10 16:04:03 ;) 16:04:20 inc0 if I dont have a conflict, I'll participate - I'll let you know if I have a conflict prior to summit 16:04:36 yeah, just throwing it out there 16:04:56 who will prepare the session? 16:04:59 room is big, it's going to be recorded and we can show once for all that you can deploy fully fledged openstack within an hour 16:05:03 that's on me berendt 16:05:30 inc0: any help needed for the preparation? 16:05:51 I might, I'll get back to you, thank you! 16:05:57 sounds like a nice way to end tuesday, ill be there granted there's no meeting that show up 16:06:24 would be great:) 16:06:30 ok, let's move on 16:06:38 moment 16:06:38 any announcements from community? 16:06:41 yes :) 16:06:45 go ahead 16:07:11 Jeffrey4l tagged kolla-kubernetes 0.6.0 - released blueprints are here: https://launchpad.net/kolla-kubernetes/+milestone/0.6.0 16:07:24 thanks for making 0.6.0 a success :) 16:07:25 w00t :) 16:08:14 ok, well, moving to actual agenda;) 16:08:16 16:08:28 #topic Core mentorship (inc0) 16:08:42 ok, I want to kick off mentorship program 16:09:11 it's a first for me so bear with me and let's work together to get this show on the road:) 16:09:35 inc0 what did you hvae in mind ? 16:10:07 #link https://etherpad.openstack.org/p/kolla-mentorship-signup 16:10:27 I'd like to start pairing people together - core and person who want's to become core:) 16:10:35 i really like this idea 16:10:40 +1 16:11:01 idea is, core mentor will review reviews of his mantee 16:11:05 spsurya_ you at all interested? if so - etherpad ftw :) 16:11:15 manatee? :) 16:11:21 manatee 16:11:23 ;) 16:11:25 well 16:11:29 the correct word is mentee :) 16:11:32 mentee? 16:11:36 my god 16:11:39 ive studied for years and am still not a manatee 16:11:42 student 16:12:00 jascott1 i feel the same way :) 16:12:09 :) 16:12:23 padawan 16:12:25 so mentor will make sure to provide feedback to student if feedback is needed 16:12:28 padawan, yes 16:12:35 apprentice 16:12:38 there are always two, one master and one apprentice 16:12:40 srwilkers: :) 16:12:48 I'm first on that list - enthusiasm not desperation.. 16:12:55 lol 16:13:05 i think what would work best i a mentor pool 16:13:14 +1 16:13:15 also student will have trusted person to ask all the question student might want to ask;) 16:13:21 we all have different strengths to bring to mentoring 16:13:39 well at the end of the day 16:13:41 I like the idea 16:13:48 everyone is part of same community 16:13:55 it's more of a kickoff 16:14:18 also, mentor will either talk to me or propose core voting himself once student is ready:) 16:14:23 inc0: agreed 1:1 is more managable for both participants 16:14:28 sbezverk, +1 16:14:29 core voting is end of mentorship 16:14:41 kolla-k8s is hot ;) 16:14:42 Good part is if the student emerges as a Core candidate - he or she will have a mentor who can speak for them. 16:14:46 and also will make my job easier to determine who should be core candidate;) 16:15:09 Jeffrey4l that is why the PTL bag is red ;) 16:15:11 inc0: suggest matching timezones if possible 16:15:20 ofc pbourke 16:15:38 so let's do first match of metnors-students right now:) 16:15:50 since we have both lists filled 16:16:02 hmm k8s is popular 16:16:03 jascott1: I chose you:P 16:16:20 for reasons obvious;) 16:16:21 awww 16:16:25 thanks! 16:17:18 so please, let's pair up all the mentees with mentor right now 16:17:21 Wow this is middle school picking teams all over again... :( 16:17:46 kfox1111 your utc -7 :) 16:17:50 rather utc -8 16:18:04 rwellum ya - i am not big on the pick one idea 16:18:13 pbourke can you take blallau under your wing? 16:18:24 inc0: sure thing 16:18:35 thank you :) 16:18:36 sdake: mind caring for rwellum ?:) 16:18:42 rwellum: please use UTC instead of European Summer Timezone 16:18:55 ha - yeah hrw 16:18:59 inc0 yes, although I am also mentoring spsurya_ atm - so need two spots :) 16:19:00 :) 16:19:39 kfox1111 then?:) 16:20:05 perhaps kfox1111 can assist mentoring rwellum and spsurya_ with me :) 16:20:26 well, figure it out guys yourself:) I don't want to interfee 16:20:29 guys I assume this list is also always open, suggest linking it from the kolla wiki and sending to the ML 16:20:39 ? 16:20:41 pbourke: good point! I'll handle that 16:20:46 thanks inc0 16:20:47 ofc it's always open 16:20:54 +1, I'd like to maybe next round 16:21:01 krtaylor there is no next round 16:21:06 krtaylor if your interested, sign up 16:21:08 krtaylor: it's one constant round:) 16:21:18 there are still mentors available, myself included:) 16:21:31 Many of these mentors have been super helpful in the past so I'm happy with whoever. 16:21:54 so general interaction doesn't change, we still are same community 16:22:09 one thing that changes is part of mentors job is to initiate core voting 16:22:19 or ask me to do so 16:22:23 what does that mean 16:22:37 if your student is ready in your eyes 16:22:41 ah 16:22:43 got it 16:22:50 send proposal email or ask me to do it, I don't mind either way 16:23:23 sdake: yeah 16:23:41 kfox1111 - you interested in taking point on one of spsurya_ or rwellum and i'll play backup :) 16:23:48 I'm also going to talk about it during summit project update session 16:23:54 so it will be all official:) 16:24:15 we need some cores for k8s in Asia timezones 16:24:15 * berendt has to leave earlier, will read backlogs later 16:24:28 and Euro 16:24:29 sdake: works for me. 16:24:39 ok let's move on, we'll get back to this topic in few weeks 16:24:47 kfox1111 k - put your name first and second on which one you want to take main contact with :) 16:24:50 #topic Demystifying the gate (sdake) http://doodle.com/poll/bee7umevf43nwi6y 16:24:55 sdake you have the floor 16:25:00 thanks inc0 16:25:05 in support of the mentorship program 16:25:09 sdake: I don't want to play favorites. flip a coin? 16:25:13 I am holding a two week 6 session 45 minute workshop 16:25:18 kfox1111 flip a coin ftw :) 16:25:27 kfox1111 that is actually how our mascot was decided :) 16:25:54 these workshops will be recorded and published 16:25:59 (on youtube) 16:26:06 sdake, thats awesome 16:26:16 I would encourage anyone that is interested in learning how the gate works in kolla-kubernetes to join in 16:26:22 the first step is voting for timing 16:26:34 I recognize the timing isnt' ideal for apac 16:26:42 although I need to sleep sometime ;) 16:26:48 thanks srwilkers 16:27:23 one thing I've learned in my ramp up on kolla-kubernetes is understanding the gate is cruicial to being an effective developer 16:27:34 jascott1 - we sort of had one of thes ethis week ad-hoc - what were your thoughts? 16:27:40 wow... google has "virtual coin flip" built in now... 16:27:42 scary. 16:27:43 it was good 16:28:00 spsurya_ ? 16:28:00 sdake: who is going to share "kube gate" working knowledge? I hope you sign up kfox1111 for it ;) 16:28:16 sdake: can we move on?:) 16:28:16 sdake: here 16:28:20 sbezverk i have a basic enough understanding of the gate to get people to a competent stage 16:28:37 if your interested, please vote on the doodle poll 16:28:45 http://doodle.com/poll/bee7umevf43nwi6y 16:28:51 I second sdake on that 16:28:54 i am sending out the announcement to the ml friday 16:29:02 (on times selected) 16:29:02 gates are hard to understand and intimidating, but really important 16:29:13 thank you Steve for taking this on 16:29:14 sbezverk: I think we both shoudl work on spreading that knowlege. 16:29:23 inc0 np :) 16:29:31 the more folks that knows the gate the better we all will be. 16:29:51 sbezverk kfox1111 your welcome to attend the sessions if you like - could use the help :) 16:30:07 kfox1111: would love to, it is just bad timing for me until the second week of June 16:30:44 sdake: anything else? 16:30:50 inc0 should do it thanks 16:30:58 thanks:) 16:31:10 #topic Mixing binary and source image (daidv) 16:31:22 hmm, daidv is not around it seems 16:31:43 ok, let's move on then 16:31:56 #topic Kolla-Kubernetes development (srwilkers) 16:32:02 Steve, shoot 16:32:45 so i threw this on the agenda because i saw some conversation in passing, and im really supportive of finding a dev path that works for kolla-kubernetes 16:33:09 inc0, I'll take over daidv's topic after this topic 16:33:12 this was something i worked on for awhile but got sidetracked after some priorities took over 16:33:19 ok duonghq 16:33:25 what do you mean by 'dev path'? 16:33:34 path, environment, whatever 16:33:46 'world?' 16:33:53 like, a set of instructions for setting up a dev system? 16:33:55 srwilkers, you mean learning curve? 16:34:04 no more that kfox1111 is getting at 16:34:35 there have been a couple options mooted recently - and they all sound pretty good 16:34:41 i think has we demystify the gate, a development environment will shake out of that work 16:34:49 but would be great to pick one that we can run with and support longterm 16:35:02 sdake: yeah. 16:35:17 ya, after demystify the gate, dev should straight out 16:35:25 duonghq ++ 16:35:30 ah ok - so this is not being looked at? https://blueprints.launchpad.net/kolla-kubernetes/+spec/devenv-taketwo 16:35:30 portdirect: mostly agree. but I'm afraid devs are cats. they often do whatever they want. :) 16:35:44 so we can provide a reference, but its likely a lot of folks wont follow. 16:35:57 #link https://review.openstack.org/#/c/457317/ one way to do it 16:36:03 kfox1111, then that's their problem when they say "it worked locally but wont pass the gate" 16:36:08 I've never seen two dev envs that were the same. :/ 16:36:17 srwilkers: right. 16:36:24 kfox1111, as a dev, i agree entirely with you 16:36:46 kfox1111, so thats why id like to see a reference. and sdakes right: the gate sessions will shake out that reference 16:36:51 inc0: that certainly is one way to skin this cat - looks nice 16:37:04 it depends what we are going for but minikube has those limitations that worry me 16:37:10 srwilkers: if it is not passig the gate there is a good reason and in 99% it is not the gate as people like to think 16:37:36 sbezverk, yep. 16:37:37 sbezverk i tend to agree, the gate is typically correct unless it thorws a flase negative 16:37:40 jascott1: yeah - minikube is great for webapps but is a bit limiting for what we need 16:37:42 sbezverk: but gates != dev 16:37:51 inc0 for sbezverk the gates are dev 16:38:03 sdake: for every failure people need to identify root cause and then address it accordingly 16:38:07 well, not for me;) 16:38:11 sbezverk: +2 16:38:13 inc0 i understand sbezverk 'z workflow in detail as I have had to emulate it to get any real work done 16:38:14 also as newer peeps study the gate it would be great to annotate it 16:38:34 gate has a lot of noise that is required by infra 16:38:41 lots of hacks to get it up and running 16:38:49 but the core should be the same i think 16:38:49 that's why I dislike reading gate code 16:38:51 jascott1: big plus 1. :) 16:38:59 regardless of the mothod chosen to implement it 16:39:02 agree portdirect - and core is kubernetes 16:39:18 yeah. I think we can segregate the parts of the gate that really are gate specific and those that are generic too. 16:39:19 k8s gives us nice lowest common denominator 16:39:26 kfox1111 ftw :) 16:39:26 make it easier to reuse those parts for dev env. 16:39:34 inc0: its why we are all here :D 16:39:35 kfox1111: that alone would go a long way 16:39:38 kfox1111, yep. agree 16:39:53 I took an initial stab at that, but haven't had the time to complete it. 16:39:55 it would go 99% of the way 16:40:03 kfox1111: I am ok for dev env to reuse, but not ok when gate is getting changed to accomodate dev env.. 16:40:11 thats why stuff like setup_kubernetes.sh is a seperate script. shouldn't be too gate specific anymore. 16:40:13 this is something i'd like to help with. ive got some bandwidth, and tackling some gate work would be a good way for me to dip my toes back in 16:40:21 and something ive been wanting to understand more anyway 16:40:22 we just need to figure out how to completely pull it out and make it rerunable. 16:40:22 same here 16:40:23 inc0: patches welcome to make it more readable. We definitely try to strike a balance of collecting necessary info and being too verbose 16:40:26 we can create ansible playbook to deploy it too, it's easy win and single command to run this would be cool 16:40:32 kinda like stack.sh for devstack 16:40:46 clarkb its not your code - its kolla's code :) 16:40:53 clarkb: with zuulv3 it will be much easier to handle 16:41:06 since we can ditch shell in favor of ansible 16:41:15 * kfox1111 is really looking forward to zuulv3 16:41:16 (if I understand zuulv3 correctly) 16:41:23 sdake: inc0 said the gate is unreadablr due to infra hacks 16:41:43 clarkb i see - well i can't defend that - i dont know what hacks are being spoken of 16:41:49 clarkb: mostly because of various regions 16:41:58 not infra itself 16:42:00 but at any rate inc0, thats pretty much it. happy to see sdake's gate sessions, since that took a lot of my talking points away. also gives me something to listen in on next week 16:42:10 don't get me wrong, I'm not attacking infra :( 16:42:20 I'd happily revuew changes to make things more readable if there are issues 16:42:41 yeah. there is some weird stuff around infra's firewalls/dns caching/rax. 16:42:43 inc0: most of hack we have in the gate is because of multiple versions we currently support 16:42:57 its stuff that really is specific to our use case though. most other openstack projects don't deploy sdn in vm's. 16:42:58 as soon as it is sorted out, the gate code will be much much cleaner 16:42:59 yeah that's what I'm saying 16:43:10 we want to make it run on all infra nodepools 16:43:24 that produces code that's not useful for anyone outside gates 16:43:34 I think we should be able to safely tuck that away into their own scripts though. 16:43:40 if we can separate that kind of stuff from actual devenv deployment, that'd be great 16:43:46 "don't look at this file unless you are debugging the gate env itself." 16:43:53 exactly 16:44:08 personal request of PTL, can we do this asap and make it a priority? 16:44:37 well, I'll work on it myself;) 16:44:41 a priority above getting v4 support? 16:44:43 but will need your help 16:45:01 I can help explain things at least. 16:45:06 kfox1111: the only place comes to mind is iptables script, the rest is kolla gate specific, no? 16:45:14 k, let's do that then, I'll focus on this part 16:45:41 sbezverk: there's some iptables rewriting stuff that is very gate specific. there's stuff to change unbind to be accessable from k8s as well as the host (for dns) 16:45:58 I think there are a couple of other minor things but can't remember off hand. 16:46:20 also I'd remove all the multi-branch things really 16:46:25 kfox1111: right two then ;) iptables and dns, does not sound like a disaster to me 16:46:38 inc0: multi-branch things? 16:46:51 support for 2/3/4/t? 16:47:06 yeah 16:47:17 -2. 16:47:18 that WILL simplify the gate for sure 16:47:23 some day, yes. 16:47:28 kfox1111: nto from gates 16:47:29 not at least until 1.0. 16:47:39 hold on, I'm not removing this from gate logic 16:47:47 oh. you mean, move the multibranch stuff into the gate specific code? 16:47:52 yes 16:47:57 ah. ok. 16:48:04 well.... 90% agree. 16:48:16 I think a suer might want to checkout and test a build on 2 or 3. 16:48:20 once you know your ropes, feel free to shuffle it around 16:48:30 so, some script that versions the checkout for the version they want to test. 16:48:34 it's about getting it up and running 16:48:47 but, yeah, generally agree. its mostly gate specific today. 16:49:34 #action inc0: segregate gate specific code from general kolla-k8s 16:49:57 portdirect srwilkers it's your topic;) anything else you wanted to mention? 16:50:23 A Dev Env is a chicken-egg situation I've discovered. On one hand I want a quick environment to get me up and running and contributing to the project, on the other hand if it's too automated then the I have no idea of the underlying technologies involved, and I don't think I can reach Core that way. The former is great for demo's though. On the other hand, 16:50:23 if/when the former fails, it's hard to get anyone interested in helping because it's all a black-box, and the Cores who could help won't use the dev environment necessarily. This has been my experience so far. 16:50:38 rwellum right- which is why we need to learn together 16:50:48 rwellum that is why the deployment guide was written 16:50:56 rwellum thank to atleasat 10+ people in the community 16:50:58 Agreed sdake 16:51:08 if we learn together - we experience it 16:51:10 rwellum: yeah. agreed. tahts why I think adapting and breaking up the gate stuff to support a dev env would help. 16:51:14 that is why we are demystifyign the gate 16:51:16 make it moduler with shell like,, 16:51:27 +1 definitely. 16:51:35 setup_kubernetes. setup_helm. setup_sdn, setup_compute_kit. 16:51:41 its easy to get off the ground. 16:51:41 if we have one magic script that runs, that isn't learning together ;) 16:51:49 when someone wants to know something more about how ti works, 16:51:56 then can look at the one script that they are most interested in. 16:52:13 and will learn how both the gate and the dev env work at the same time. 16:53:01 i agreee one set of scripts 16:53:09 coming out of the demystifying the gate 16:53:12 is winning 16:53:12 +1 sdake 16:53:16 +1 rwellum 16:53:57 then everyone leans how the gate works, everyone learns how the dev env works, everyone learns 16:53:59 one thing I think would help is genericizing the "wait_for" scripts. make one that takes in a few arguments. might make it a lot less code for people to look through to understand. 16:54:02 cool, I think we have a plan 16:55:18 i guess we are out of time 16:55:21 opens? 16:55:25 or what 16:55:30 ya, 5mins left 16:56:07 opens. 16:56:11 #topic open discussion 16:56:30 I will throw https://docs.openstack.org/infra/manual/testing.html out there as docs on how our test envs are set up and what you can rely on (if you think you've run into a cloud or region specific issue that may be helpful) 16:56:56 clarkb - thanks dude - didn't know that doc existed and been working on openstack for 5+ years :) 16:57:08 clarkb: thanks. I do need to spend some more time with the gate and rax. currently the multinode is still failing always on it. 16:57:09 i think all the cores could spend the time to read it 16:57:12 sdake: its relatively new, came out of barcelona as something we should have so put it together 16:57:23 it started failing when we want to k8s 1.6. 16:57:27 clarkb ++ :) 16:57:29 great 16:57:57 any more opens? 16:58:01 kfox1111 - lets figure out with infra how to actually root cause the rax problems 16:58:14 kfox1111 rather then rechecking a million times and blaming infra for what ostensibly is our gating code :) 16:58:53 also happy to look at logs again if you were able to gather more info off the subnode(s) 16:59:46 clarkb I think a thread on the ml about this topic would be helpful - so we can figure out what logs to gather 16:59:49 ok we're out of time 16:59:51 atm we don't know what to record 16:59:51 thank you all! 16:59:56 #endmeeting kolla