16:00:02 <Jeffrey4l> #startmeeting kolla
16:00:07 <openstack> Meeting started Wed Mar  7 16:00:02 2018 UTC and is due to finish in 60 minutes.  The chair is Jeffrey4l. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:10 <openstack> The meeting name has been set to 'kolla'
16:00:11 <Jeffrey4l> #topic roll-call
16:00:23 <spsurya__> w00t
16:00:29 <pbourke> o/
16:00:32 <hrw> o/
16:00:41 <rwellum> o/
16:00:44 <chason> o/
16:01:21 <hrw> I will be partially as have other confcall at same time
16:01:38 <Jeffrey4l> #topic Announcements
16:02:07 <caoyuan> hi guys
16:02:09 <Jeffrey4l> have a nice trip at dublin gus
16:02:35 <Jeffrey4l> 1. we have release kolla queens rc1 and rc2 will be released next weekly.
16:02:56 <Jeffrey4l> so this is the last time to fix the bug and test upgrade.
16:03:34 <Jeffrey4l> feel free to call the kolla cores to review the emergency patches.
16:04:03 <Jeffrey4l> 2. along with the rc1 release, queens branch is created too. So the master branch is opened for pb.
16:04:43 <Jeffrey4l> if you get a -2 for feature freeze reason, you can ask core to remove it.
16:04:56 <Jeffrey4l> any other annoucements?
16:05:05 <pbourke> minor announcement, I'm currently in possession of a stack of kolla stickers from the PTG. I missed the chance to hand them out so will bring them to the next event :)
16:05:20 <duonghq> o/
16:05:46 <Jeffrey4l> cool thanks pbourke
16:06:00 <Jeffrey4l> any others?
16:06:11 <rwellum> If we send you a SAE will you send them over pbourke :)
16:06:47 <Jeffrey4l> lol
16:06:50 <pbourke> if anyone wants to paypal me postage I'll happily post them to you
16:07:17 <rwellum> +1
16:07:23 <Jeffrey4l> let us move on, we have lots of things to do today ;D
16:07:29 <Jeffrey4l> # topic kolla queens release requirements
16:07:34 <Jeffrey4l> #topic kolla queens release requirements
16:08:00 <Jeffrey4l> since we don't have much time before release dead line
16:08:13 <Jeffrey4l> and we have several critical issue to resolve.
16:08:25 <Jeffrey4l> so i want to post them here.
16:08:36 <Jeffrey4l> first is https://review.openstack.org/530208
16:08:47 <Jeffrey4l> chason is working on it.
16:08:58 <Jeffrey4l> and it break queens branch image building.
16:09:38 <Jeffrey4l> vpnnas images does not exist and it work as a l3 extenstion.
16:09:40 <pbourke> is there no option to leave things as is for queens?
16:10:15 <pbourke> seems late for an architectural change like that
16:10:34 <Jeffrey4l> pbourke, no. either drop the vpnass support in queens for kolla. or backport the bp
16:10:48 <Jeffrey4l> yeah, this is a general issue for kolla now.
16:11:22 <Jeffrey4l> when the poroject marked some feature as deprected. narmally kolla doesn't aware this until this feature is remove.
16:11:45 <Jeffrey4l> this happen on keyston uuid token too.
16:11:58 <Jeffrey4l> no good idea to resolve such kinda issue. :(
16:12:29 <Jeffrey4l> anyway, i think we haven't to backport this pb to queens branch like a FFE.
16:12:40 <Jeffrey4l> any question for this? ^
16:13:09 <pbourke> sounds ok to me
16:13:35 <Jeffrey4l> cool, thanks
16:13:40 <caoyuan> it ok for me
16:14:08 <Jeffrey4l> there is also some other patches, like bump repo or openstack service version here
16:14:08 <Jeffrey4l> https://review.openstack.org/#/q/status:open+project:openstack/kolla+branch:stable/queens
16:14:43 <Jeffrey4l> mostly it should be merged before release, so please keep eyes on it cores.  :D
16:14:47 <hrw> review https://review.openstack.org/#/c/550378/ https://review.openstack.org/#/c/550367/ to have centos/aarch64 repos fixed properly (in master) and then backport one to have ceph/luminous for aarch64/centos
16:15:12 <Jeffrey4l> yes ^^ some are still on master
16:15:54 <Jeffrey4l> ok, that's all for queens release.
16:15:57 <Jeffrey4l> let us move on.
16:16:03 <Jeffrey4l> #topic snowpenstack recap / debrief
16:16:30 <pbourke> one or two things on this then we can move onto the goals etc.
16:16:42 <Jeffrey4l> first of all, thanks pbourke to hold the meeting at ptg, since i am absent.
16:16:55 <pbourke> Id like to apologise on behalf of Ireland for the weather :)
16:17:01 <gema> LOL
16:17:08 <Jeffrey4l> could you take this pbourke ?
16:17:11 <pbourke> sure
16:17:29 <pbourke> so we didn't get to discuss everything we wanted, or at least as well as we might have liked
16:17:38 <pbourke> but we did cover a good bit of ground given the circumstances
16:17:43 <spsurya__> pbourke: i am unhappy ;)
16:17:46 <Jeffrey4l> #link https://etherpad.openstack.org/p/kolla-rocky-ptg-planning
16:17:54 <pbourke> I'm writing up a summary report that I can send round on the mailing list
16:18:02 <pbourke> to save people trying to parse the etherpads
16:18:13 <pbourke> that said there is some good info in the pads so its worth looking at them as well
16:18:14 <duonghq> thank Paul
16:18:37 <pbourke> I think the main thing we need now is to take what was discussed and distill it into a list of goals for rocky
16:18:53 <pbourke> as we wont be able to cover everything and some have more priority
16:19:12 <pbourke> Jeffrey4l: does sound ok?
16:19:14 <pbourke> *that sound
16:19:18 <spsurya__> +1 pbourke
16:19:21 <Jeffrey4l> yes
16:19:37 <rwellum> thanks pbourke
16:19:43 <pbourke> I cant remember how we've done this in the past
16:19:53 <pbourke> etherpad or just let the PTL decide
16:20:17 <gema> I guess you put out the goals and see if there is someone willing to work on them?
16:20:21 <gema> take volunteers?
16:20:35 <pbourke> true, no point in having goals if there's no one to take them up
16:20:37 <Jeffrey4l> iirc, we need review the current actions. and convert them to a bp.
16:20:56 <Jeffrey4l> then priority the bp and take volunteers
16:21:20 <pbourke> how about I'll send out the summary mail and Jeffrey4l can take it from there?
16:21:58 <Jeffrey4l> pbourke, sure. i think we can do this in ML. after you sent the mail.
16:22:06 <pbourke> sounds good
16:22:26 <Jeffrey4l> yeah.
16:22:30 <pbourke> the other thing from the PTG that we didn't get to discuss before things were called off was kolla-k8s
16:22:35 <pbourke> did the call on friday go ahead?
16:23:12 <Jeffrey4l> since all the attendee have no idea for kolla-k8s, it is ended soon.
16:23:34 <pbourke> ok, good to know either way
16:23:44 <Jeffrey4l> we are agree re-scheduler another meeting to talk about it when k8s active contributor is in
16:23:57 <Jeffrey4l> rwellum, around?
16:24:24 <rwellum> Yeah - apologies for my remoteness, some changes with my $$job etc.
16:24:35 <gema> Jeffrey4l: you may want to sync up with kevin zhang from my team
16:24:43 <rwellum> The meeting on Friday was 3am for me I think
16:24:55 <Jeffrey4l> rwellum, yes. the time is bad
16:25:03 <Jeffrey4l> gema, i will , thanks.
16:25:04 <gema> kevin zhao , sorry
16:25:22 <Jeffrey4l> spsurya__ have add some info on the ptg planing etherpad.
16:25:46 <Jeffrey4l> rwellum, could you talk about kolla-k8s ?
16:26:15 <spsurya__> Jeffrey4l: also had discussion with OSH PTL and flapper87 of TripleO team
16:26:31 <spsurya__> about Openstack on kubernetes
16:26:51 <Jeffrey4l> spsurya__, any conclusion?
16:27:03 <spsurya__> but would be better if we have discussion with few key contributors og kk8s
16:27:11 <rwellum> Sure but I don't have too much to add, I think the same questions we had pre-PTG are still the same. I have noticed that I no longer can run kolla-k8s, seem to be various image issues. I know ansible seeing some of the same.
16:27:28 <spsurya__> Jeffrey4l: it is kind of biased
16:27:38 <spsurya__> from each project side
16:27:53 <spsurya__> evry project is quite similar
16:28:02 <Jeffrey4l> spsurya__, yeah
16:28:16 <spsurya__> as per discussion with other project contributors
16:29:17 <spsurya__> Jeffrey4l: some kk8s has more control over services but complex deployment
16:29:29 <spsurya__> w.r.t to OSH
16:29:38 <Jeffrey4l> spsurya__, and i will check the current status of different openstack over k8s project.
16:30:05 <Jeffrey4l> if you are intereting this, welcome join us.
16:30:06 <spsurya__> Jeffrey4l: i already added few in eatherpad
16:30:14 <spsurya__> rocky PTG
16:30:47 <Jeffrey4l> yeah, i saw them. But i will try to run the project and compare it ;D
16:31:02 <spsurya__> Jeffrey4l: +1
16:31:20 <spsurya__> I will also like to run the project
16:31:32 <rwellum> I think one question is, do you want a kolla-k8s project that is uniquely tied to kolla?
16:31:47 <Jeffrey4l> rwellum, are you saying the image?
16:31:57 <rwellum> If the answer is yes then as a team we should commit to it.
16:32:59 <rwellum> No I mean currently it's: kolla->kolla-k8s and kolla->ansible. They are both sub-projects of kolla. If we decide that osh is the way forward then we would break that relationship.
16:33:01 <Jeffrey4l> yes. we hope kolla-k8s to be success as kolla-ansible.
16:34:03 <duonghq> but we are lacking of contributor?
16:34:06 <spsurya__> +1 Jeffrey4l & rwellum
16:34:15 <duonghq> aren't we?
16:34:21 <pbourke> duonghq: yes
16:34:49 <Jeffrey4l> rwellum, any suggestion how? the issue is which way (or deploy philosophy) is correct and we will follow.
16:35:05 <pbourke> a problem I see is there is currently confusion from new comers to kolla as to the differences between kolla-k8s and openstack-helm
16:35:08 <spsurya__> we shouldn't compare with OSH if we want to make kk8s successful as kolla-ansible
16:35:19 <pbourke> also when people ask for help around kolla-k8s they rarely get a reply
16:35:25 <pbourke> so overall its making kolla look bad
16:35:38 <Jeffrey4l> pbourke, yes.
16:35:54 <gema> Jeffrey4l, pbourke: maybe we should refocus kolla-k8s
16:35:58 <spsurya__> pbourke: agree
16:36:00 <rwellum> I agree. I think kfox and sbezverk have dropped out a lot and no-one is answering
16:36:01 <pbourke> there are also people wasting time investigating kolla-k8s without the knowledge it may not be around much longer
16:36:03 <gema> state clear goals, how long there is to go
16:36:06 <caoyuan> agree
16:36:10 <gema> and whether it makes sense to continue
16:36:17 <gema> and ask for volunteers
16:36:18 <rwellum> We had fairly clear goals for Rocky
16:36:23 <gema> if there is nobody
16:36:23 <gkadam> duonghq, pbourke  hi, I would like to contribute here. this is my second kolla meeting. I missed last week, was down with viral fever and cough. but for many again a questions arises is , how a newbie can start?
16:36:29 <inc0> hello everyone (I'm alive!) one thing I was considering, instead of killing
16:36:30 <inc0> project
16:36:37 <inc0> was to just separate it
16:36:44 <pbourke> inc0: +1
16:36:45 <caoyuan> when contributor commit a ps, seems no body review it..
16:36:49 <gema> rwellum: we didn't get to discuss them because of snow
16:36:50 <inc0> and let it live it's own life
16:37:08 <Jeffrey4l> gkadam, welcome
16:37:13 <pbourke> if poeple want to work on it of course that's fine. but currently it doesn't seem in a good position to be under the kolla namespace
16:37:16 <gkadam> Jeffrey4l, :)
16:37:29 <inc0> separation would mean separate core team (already exists), separate governance (team meetings etc) and separate PTL
16:37:35 <ktibi> hum kolla-k8s success as kolla-ansible ? and people like me with a production platform on kolla-ansible ? I need to redeploy ? what is the plan ?
16:37:53 <pbourke> ktibi: you don't have to do anything if you dont  want to :)
16:37:59 <gema> ktibi: TBD
16:38:03 <Jeffrey4l> ktibi, that is far from current goal i think.
16:38:05 <inc0> ktibi: as far as I know kolla-ansible is not going anywhere in forseeable future
16:38:29 <Jeffrey4l> inc0, +1 for separate
16:38:32 <inc0> both can and will live in pararell
16:38:39 <ktibi> what is for you "far" ? because I work for 1-2 year :/
16:38:43 <inc0> parallel, hate this word
16:39:17 <gkadam> inc0, stranger things had parallel worlds too :D
16:39:17 <inc0> ktibi: since I'm no longer PTL I don't want to make any statements, but kolla-ansible was always meant to be stable and prod
16:39:31 <spsurya__> +1 inc0 for governance thing for kk8s
16:39:32 <Jeffrey4l> i understand you in wrong. kolla-ansible will not retire
16:39:39 <inc0> that means it's going to be maintained and I don't think anyone ever considered just blank move to k8s
16:39:58 <rwellum> Guys I have to leave unfortunately - but will check the thread when I am back. I am available to discuss k8s more this afternoon.
16:39:59 <Jeffrey4l> inc0, i will start to do the seperate thing recently, but may need you help :D
16:40:08 <inc0> Jeffrey4l: I'm here if you need me
16:40:08 <pbourke> rwellum: thanks for checking in
16:40:13 <Jeffrey4l> rwellum, thanks
16:40:50 <inc0> so ktibi don't worry, we won't leave you and other opearators;)
16:41:09 <ktibi> inc0 ok thx ;)
16:41:21 <Jeffrey4l> so what we need now is:  1. seperate   2. clear goals  and 3. volunteers
16:41:38 <spsurya__> rwellum: thanks for coming and update
16:41:52 <duonghq> I like inc0 idea
16:42:19 <gema> will be a hard sell for me if it is outside kolla
16:42:26 <gema> we may not be able to contribute anymore
16:42:29 <inc0> it wouldn't be bad to consider kolla-ansible to be separate as well, both projects have enough scope for one PTL
16:42:48 <duonghq> over and over, I got question about kolla-k8s vs ovh as much as kolla-ansible vs tripleo (especially when tripleo undercloud is moving to containerized fully)
16:42:48 <Jeffrey4l> gema, it is not outstid of kolla. just another team. you can also join it .
16:43:14 <Jeffrey4l> the deep reason is: kolla-ansible and kolla-k8s are totally different on tech arch.
16:43:14 <inc0> I had a lot of kolla-ansible vs openstack-ansible
16:43:14 <gema> Jeffrey4l: we work for a committee, it's easier to be part of an established team than to go and join a new one
16:43:21 <inc0> confusion will always be there
16:43:35 <gema> inc0: +1
16:43:48 <inc0> gema: thing is, if one person can focus just on kolla-k8s
16:43:53 <inc0> (as in separate PTL)
16:44:15 <inc0> and separate governance
16:44:30 <inc0> this one person can be more focused on re-invigorating community
16:44:45 <gema> inc0: true, I understand that
16:44:45 <Jeffrey4l> inc0, +1
16:45:07 <duonghq> it'll be hard time for kolla-k8s when people come to questioning about this on its feet vs ovh, etc,
16:45:07 <inc0> it's already separate team effectively
16:45:44 <Jeffrey4l> duonghq, ovh? osh?
16:45:54 <duonghq> Jeffrey4l, osh, sorry for my typo
16:46:31 <Jeffrey4l> so, the next action for kolla-k8s will be:  1. seperate   2. clear goals  and 3. volunteers
16:46:45 <Jeffrey4l> i will report the progress in next meeting.
16:46:48 <spsurya__> +1 inc0 Jeffrey4l
16:47:04 <Jeffrey4l> let move on.
16:47:09 <duonghq> Jeffrey4l, inc0 do you think we can (should) bring our goal from last cycle(s) to the-new-kolla-k8s?
16:47:35 <Jeffrey4l> hrm? that's that mean?
16:47:43 <spsurya__> duonghq: yes
16:47:50 <inc0> there is quite clear list of goals for k8s
16:48:06 <Jeffrey4l> ah, duonghq sure.
16:48:20 <spsurya__> https://etherpad.openstack.org/p/kolla-kubernetes-todo
16:48:35 <spsurya__> https://etherpad.openstack.org/p/kolla-queens-ptg-k8s-release-roadmap
16:48:40 <spsurya__> duonghq:
16:48:41 <inc0> separation will be quite interesting, as I don't think anyone ever done that
16:48:58 <duonghq> spsurya__, thanks
16:49:02 <inc0> we should discuss it with TC how to do it best
16:49:02 <pbourke> inc0: nova? ;)
16:49:15 <inc0> ok. yes it has been done;)
16:49:23 <duonghq> spsurya__, I think we can move it to the-new project
16:49:50 <duonghq> hmm, project separation doesn't too rare in OpenStack world?
16:49:50 <spsurya__> duonghq: yes inc0 idea is good
16:50:37 <Jeffrey4l> ok, time to move on.
16:51:01 <Jeffrey4l> #topic Security bug in gates
16:51:07 <Jeffrey4l> #link https://bugs.launchpad.net/kolla-ansible/+bug/1749326
16:51:08 <openstack> Launchpad bug 1749326 in kolla-ansible "Exploitable services exposed on community test nodes" [Critical,Confirmed]
16:51:18 <Jeffrey4l> pbourke, do you have any update for this?
16:51:32 <pbourke> Jeffrey4l: unfortunately not, it totally slipped my mind last week
16:51:46 <pbourke> that's why I added it to the agenda
16:52:00 <pbourke> looks like you have a patch?
16:52:16 <Jeffrey4l> i think the only possible solution is add iptables rules and only allow konwn vm to access
16:52:21 <Jeffrey4l> yes.
16:52:26 <Jeffrey4l> #link https://review.openstack.org/550325
16:52:44 <Jeffrey4l> recently there a is memcached ddos attack,
16:52:54 <Jeffrey4l> you can get more from http://www.openwall.com/lists/oss-security/2018/03/02/1
16:53:01 <ktibi> ho T_T
16:53:14 <ktibi> https://bugs.launchpad.net/kolla-ansible/+bug/1753425 https://review.openstack.org/#/c/549715/
16:53:15 <Jeffrey4l> i just disabled the udp protocol support in memcached. since we don't use it.
16:53:15 <openstack> ktibi: Error: malone bug 1753425 not found
16:53:25 <pbourke> so does this fix the problem or just partially?
16:53:50 <inc0> it doens't really fix problem
16:53:51 <Jeffrey4l> partically. or not, it not solve the problem.
16:53:59 <inc0> it lowers potential of exploitation
16:54:13 <Jeffrey4l> memcached port is still open for whole intenet world
16:54:49 <Jeffrey4l> the only way i think is: " add iptables rules and only allow konwn vm to access "
16:54:59 <pbourke> that or do we need it in the gates?
16:55:02 <Jeffrey4l> so need implement this in or ci jobs.
16:55:16 <pbourke> what services currently need memcache
16:55:48 <Jeffrey4l> nova-consoleauth
16:55:59 <pbourke> hmm ok so we need it
16:56:02 <ktibi> I apply my patch for udp and all works for me. UDP is not need to be enable.
16:56:26 <Jeffrey4l> there are also palce use memcached as cache tier. it is helpful for performance.
16:56:38 <Jeffrey4l> so we shouldn't remove it.
16:56:56 <Jeffrey4l> ktibi, your patch? which patch?
16:57:16 <ktibi> Jeffrey4l, I make a bug security and a review for this issue since 2 day
16:57:24 <ktibi> I think you miss it and recreate a review with same change
16:57:32 <pbourke> i'll look at Jeremy's iptables rules
16:57:54 <Jeffrey4l> cool. i am think there should be the same solution in devstack gate.
16:58:09 <Jeffrey4l> but i am not sure where the code is.
16:58:34 <pbourke> looks like he linked it in the bug
16:58:39 <ktibi> I used the solution of redhat https://access.redhat.com/solutions/3369081
16:58:48 <Jeffrey4l> ok. i will check it later.
16:58:55 <Jeffrey4l> time is up.
16:59:14 <Jeffrey4l> ktibi, thanks
16:59:23 <Jeffrey4l> thanks for coming.
16:59:27 <Jeffrey4l> have a good day
16:59:35 <Jeffrey4l> #endmeeting