23:00:47 <sdake> #startmeeting kolla
23:00:49 <openstack> Meeting started Wed Mar 30 23:00:47 2016 UTC and is due to finish in 60 minutes.  The chair is sdake. Information about MeetBot at http://wiki.debian.org/MeetBot.
23:00:50 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
23:00:52 <openstack> The meeting name has been set to 'kolla'
23:00:55 <sdake> #topic rollcall
23:00:56 <asalkeld_> o/
23:00:57 <sdake> \o/
23:00:58 <akwasnie> hi
23:01:00 <inc0> o/
23:01:11 <Jeffrey4l> o/
23:03:06 <vhosakot> o/
23:03:58 <sdake> #topic annouoncements
23:04:00 <sdake> our agenda is here:
23:04:04 <sdake> #link https://wiki.openstack.org/wiki/Meetings/Kolla#Agenda_for_next_meeting
23:05:19 <sdake> 1. sam has dropped off the core reviewer team of his own accord.  If he is lilke most peple he probably values his privacy and may not want to be questioned about it.
23:05:40 <inc0> he will be missed :(
23:06:06 <sdake> sam's contributions have been super valuable to kolla and I wish him well in the future
23:06:23 <sdake> If the core team wants to wish him well as well, I thnk that would be appropriate.
23:06:47 <sdake> 2. the irc channel has moved to #openstack-kolla
23:06:49 <Jeffrey4l> miss him.
23:07:19 <sdake> 3. the gate which was busted is now fixed for the most part in correct fashions for master and soon mitaka - keep the patches coming
23:07:56 <vhosakot> the doc gate is still flaky
23:08:00 <sdake> 4. we have a vote on how to manage the backports underway - please vote.
23:08:09 <sdake> 5. we have a core reviewer nomination under way - please vote.
23:08:18 <sdake> any announcements from the community?
23:09:01 <sdake> #topic Liberty backport planning
23:09:05 <sdake> inc0 all yours
23:09:07 <inc0> well
23:09:09 <inc0> https://review.openstack.org/#/c/299563/
23:09:14 <inc0> so we need a plan
23:09:26 <inc0> I'm working on getting mitaka code deploying liberty
23:09:38 <inc0> and got it to work in prototype fashion
23:10:06 <inc0> obviously this is ugly and I need to work on it a bit more, but here it is
23:10:19 <inc0> question - what are we doing with this code?
23:10:23 <inc0> I see 2 options
23:10:35 <inc0> 1. We keep it master and effectively abandon mitaka
23:10:39 <inc0> liberty* branch
23:11:02 <inc0> 2. we backport it to liberty stable and we can remove this "deploy_liberty" nonsense
23:11:23 <inc0> which means that we'll have 2 branches that will be virtually identical
23:11:38 <inc0> and patches that lands in mitaka will have to be backported
23:11:38 <sdake> sounds like what the current vote is
23:11:44 <inc0> I would guess almost all of them
23:12:07 <sdake> i htink what this means is backporting is effectively blocked until mitaka is released
23:12:10 <inc0> I'm personally in favor of second options
23:12:11 <asalkeld_> can't we keep templates for major openstack version in tree
23:12:13 <sdake> to make a nice tidy backport process
23:12:23 <asalkeld_> templates/mitaka/*
23:13:03 <asalkeld_> so that one branch can at least deploy 2 or 3 releases
23:13:09 <inc0> well asalkeld_ truth is, our templates aren't super sophisticated (not sure if I wrote this word correctly, you get the idea)
23:13:17 <inc0> which means we're good at most part
23:13:22 <sdake> asalkeld  so you mean templates ike heat-mitaka.j2 heat-liberty.j2?
23:13:45 <asalkeld_> sdake: yeah (or just add a new directory for each release
23:13:45 <inc0> well, I wouldn't make a habit out of it
23:13:52 <vhosakot> yes.. I like 2 option as well.. why should we "make Mitaka deploy Liberty" ? does other OpenStack projects do this ? one release is made to deployed an older relase ?
23:14:09 <inc0> we may get lucky now, next release mariadb will make breaking change and we're screwed
23:14:34 <inc0> vhosakot, so other deployment tools usually deploy previous version
23:14:50 <inc0> I think Fuel for example is still on Kilo
23:15:06 <vhosakot> Mikaka Fuel dpeloys Kila ?
23:15:08 <inc0> osad I think deploys master
23:15:11 <asalkeld_> hard to test upgrades if you have to keep checking out different versions of the code
23:15:40 <inc0> asalkeld_, but it will be more stable this way
23:15:42 <sdake> well we do want stable branches per release
23:16:00 <sdake> stable/mitaka = atleast mitaka
23:16:01 <asalkeld_> ok, just an idea
23:16:05 <sdake> stable/liberty = atleast liberty
23:16:13 <inc0> well, I think we may get lucky now
23:16:26 <inc0> but I'd liek to keep 1 release per release
23:16:29 <inc0> and mitaka - mitaka
23:17:09 <inc0> only reason this backport is even a thing is data_containers which were super critically horrible nuke in your own datacenter
23:17:17 <inc0> and people don't like nukes in dc
23:17:59 <sdake> ok well here is my thinking on the matter
23:18:13 <sdake> inc0 should continue to work on defining HOW we backport
23:18:21 <inc0> so...my personal favorite is to ditch what we now call liberty, and make liberty out of master + my code once I'm done with it
23:18:28 <sdake> he has taken that on that responsibility of his own accord
23:18:43 <sdake> but as far as tming goes
23:18:53 <sdake> I'd really like to wait until Mitaka is tagged
23:19:00 <inc0> fair enough
23:19:19 <sdake> after Mitaka is tagged, Id' like to distribute the backport work throughout the core reviewr team and our respective community
23:19:27 <inc0> so what I need from you guys is to help me testing
23:19:37 <sdake> so we can get it done in la week or two
23:19:40 <sdake> rather then 6 months ;)
23:19:43 <inc0> confirm that serviced deploys correctly and test centos
23:20:11 <sdake> we need to teset ubuntu as well
23:20:18 <vhosakot> can the backported code to Liberty be tested in gate ?
23:20:22 <inc0> I'm doing everything on buntu
23:20:28 <sdake> our gate testing is weak vhosakot
23:20:34 <vhosakot> compute kit
23:20:46 <inc0> vhosakot, we need to make gate out of it
23:20:46 <sdake> only copute kit and barely that :)
23:20:57 <sdake> stable/lbierty hs a gate
23:21:06 <inc0> which will need to change
23:21:15 <sdake> if its the same as mitaka why change it?
23:21:27 <inc0> if it's the same, then no, but is it
23:21:29 <inc0> ?
23:21:43 <inc0> well, in any case
23:21:44 <sdake> it will be - take mitaka cp to liberty
23:22:09 <inc0> we'll get it done
23:22:27 <inc0> all I'm saying is, we need to make a decision on HOW
23:22:31 <sdake> yup have full confidence in inc0, he got us upgrades rolling with help from the community
23:22:46 <sdake> inc0 i prposed in the vote on the mailing list how
23:22:55 <sdake> if you think how should be different can you repropose a different thing
23:22:57 <inc0> wonderful, thank you
23:23:12 <inc0> I'll do that
23:23:39 <sdake> ok any Q&A?
23:23:57 <vhosakot> I have Q...
23:24:01 <inc0> for everyone who helps with backport, drinks are on PTL;
23:24:02 <inc0> ;)
23:24:10 <sdake> drinks are on inc0 he means :)
23:24:26 <vhosakot> will steps to test backport be posted on etherpad ? what to test ? how to test ?
23:24:32 * sdake needs to abuse the corproate card more
23:24:46 <inc0> vhosakot, so test will be - deploy and do stuff
23:25:05 <sdake> the deal is we want liberty to be the same as mitaka
23:25:08 <rhallisey> hello
23:25:09 <vhosakot> is this like downloading inc0's patch set, deploy kolla, check basic sanity ?
23:25:12 <sdake> so we can leverage all the testing we have gone through
23:25:14 <sdake> hey rhallisey
23:25:15 <rhallisey> srry was at the sstore
23:25:19 <inc0> but I'll make sure to get it testable (is this even a word?) before we tag mitaka
23:25:45 <inc0> vhosakot, yeah, pretty much, basic or more advanced, as much as you can
23:26:05 <sdake> inc0 perhaps your proposal is different then what was proposed on the ml
23:26:11 <sdake> i think we need to have further discussion there
23:26:32 <sdake> there being the ml
23:26:35 <rhallisey> does that mean there more needed to be backported?
23:26:49 <rhallisey> s/there/there is
23:26:55 <sdake> rhallisey yes - read your email!! ;)
23:26:56 <inc0> rhallisey, everything will be backported eally
23:27:08 <inc0> and I mean everything
23:27:14 <rhallisey> ya I saw that
23:27:17 <rhallisey> I +1'd it
23:27:25 <rhallisey> unless something changed in the last 1hour
23:27:28 * sdake reads his mail :)
23:27:52 <sdake> i think inc0 hasn't read my proposal yet
23:28:12 <Jeffrey4l_> what the final solution? deploy liberty using mitaka branch Or backport mitaka branch to liberty?
23:28:15 <inc0> didnt get mail yet
23:28:25 <inc0> I'm for the latter
23:28:28 <sdake> Jeffrey4l_ i prefer second choice
23:28:29 <Jeffrey4l_> I think inc0 using the latter.
23:28:53 <Jeffrey4l_> I think inc0 using the former. ( sorry for typo)
23:29:06 <Jeffrey4l_> I perfer second, too.
23:29:07 <inc0> Jeffrey4l_, for demo purposes
23:29:33 <inc0> but I'd rather ditch conditional and make liberty a copy of mitaka with slight changes
23:29:34 <vhosakot> inc0 said 2nd option  -  2. we backport it to liberty stable and we can remove this "deploy_liberty" nonsense
23:30:28 <sdake> ok well unfortunately we have a poor turnout at our irc meetings at the 23:00 utc tiemslot
23:30:40 <sdake> so we really need to have this discussion on the mailing list
23:31:13 <sdake> so inc0 slurp your mail and get us going ;)
23:31:17 <vhosakot> is there a list of things going to be backported to Liberty.... etherpad link ?
23:31:17 <sdake> ok moving on :)
23:31:37 <sdake> #topic Liberty rc3 bug triage [timebox 25 minutes]
23:31:58 <vhosakot> sdake: is there a list of things going to be backported to Liberty
23:31:59 <sdake> vhosakot perhaps we need to start that - inc0 can yout ake that up ini yuor response?
23:31:59 <inc0> mail no there, bits slow, instructions unclear
23:32:06 <vhosakot> cool
23:32:30 <sdake> our deadline for rc3 is friday April 1sst
23:32:33 <sdake> as in 2 days
23:32:34 <Jeffrey4l__> ( sorry, offlined just now)
23:32:52 <sdake> I am tagging at the last possible moment for folks to get bug fixes in
23:33:10 <sdake> After that, Mitaka final is April 8th, 2016
23:33:40 <sdake> so we are going to go through the painful process of bug categorization for confirmed bugs
23:34:24 <sdake> #link https://launchpad.net/kolla/+milestone/mitaka-rc3
23:34:41 <sdake> any bug that is New is probably a backport
23:34:43 <sdake> lets ignore those
23:34:52 <sdake> i'll sort those out after the meeting
23:34:58 <sdake> anythign confirmed, needs to be fixed asap
23:35:02 <asalkeld_> lots there
23:35:10 <sdake> ya we have a slew of bugs
23:35:23 <sdake> I really need people to focus on fixxing bugs rather then developing features
23:35:58 <sdake> we can handle about 25 bugs a week if we coordinate well
23:36:03 <sdake> that is our community capacity
23:36:22 <vhosakot> that is good bug-fixing rate
23:36:37 <sdake> yes - we have 1 week left until release
23:36:45 <sdake> we can't possibly fix all these bugs in the time frame allotted
23:37:04 <sdake> please work on critical bugs
23:37:23 <sdake> a critical bug is something that stops kolla from functioning properly in our 7 major kolla-ansible operational mdoes
23:37:30 <sdake> reconfigure, ugprade, deploy, precheck, etc
23:37:45 <sdake> a high bug is something that is a real probem
23:37:57 <vhosakot> yes, I'm hardening the prechecks
23:38:06 <sdake> I am going to look at a ll the low bugs an bounce them on the 1st if there are not solutiosn for them
23:38:19 <sdake> I am going t o look at all the medium bugs and see if they are really low prior to that
23:38:36 <sdake> what i need is help prioirtizing bugs
23:39:01 <sdake> the second peice of help i need is finishign the job on bugs in the in progress state
23:39:50 <sdake> the third peice of help i need is fixing bugs in the critical or high states that are unassigned
23:40:06 <sdake> i will send a request to the mailing list about all of this need for help
23:40:17 <sdake> but can anyone point out any holes in this approach?
23:40:32 <sdake> i'd like to make sure we deliver something that works properly
23:41:59 <vhosakot> sdake: what about scale bugs related connection limit, nova scheculer with 100 VMs... ?
23:42:14 <sdake> vhosakot i am looking at that now
23:42:17 <vhosakot> cool
23:42:18 <sdake> i would call that critical
23:42:36 <sdake> or high at minimum
23:42:53 <sdake> what i'e seen so far with 1000 vm launch is it can't be done
23:43:00 <sdake> without changing koll aaround
23:43:11 <sdake> at most on a 64 node clusery you can launch 50 vms
23:43:16 <sdake> that is totaly unacceptable to me ;)
23:43:40 <vhosakot> how many are compute nodes ?
23:43:42 <sdake> we need  to make sure a 100 node cluster works ocrrectly for a fully loaded set of compute nodes
23:43:58 <sdake> 3 controller 10 stroage 87 compute at 256gb ram
23:44:05 <sdake> is our arbitrary pie in the sky target
23:44:12 <sdake> this leads to open discussion
23:44:16 <sdake> #topic open discussion
23:44:57 <inc0> sorry battery died
23:45:06 <vhosakot> we'll need continued help from manjeets and gmmaha to debug/test scale fixes...
23:45:44 <sdake> it takes me 6-12 hours to even dig into one ebug vhosakot
23:45:52 <sdake> because i dont hve access to hte metal
23:46:00 <sdake> meanwhile i could be fixing 3-4 critical bugs
23:46:01 <vhosakot> agreed, scale issues are different/tough
23:46:22 <gmmaha> vhosakot: we can definitely help as best as we can
23:46:48 <vhosakot> thanks gmmaha
23:47:28 <inc0> I hope to get us some serious hardware soon, for a time at least
23:48:59 <vhosakot> I need two my reviews to be merged.. they are in queue long
23:49:12 <sdake> ya folks
23:49:15 <sdake> stay on top of the queue
23:49:28 <sdake> i wnt eveyrone to have an opportunity to be first to review a patch
23:49:34 <sdake> dont wait for someon else to review it first
23:49:34 <vhosakot> https://review.openstack.org/#/c/296012/    and    https://review.openstack.org/#/c/296649/
23:50:18 <sdake> anything else?
23:51:10 <vhosakot> so, is inc0 going to send email about backport
23:51:26 <inc0> I thought sdake will do it?
23:51:28 <sdake> he said he would follow up on the thread
23:51:32 <vhosakot> ah ok ok
23:51:32 <inc0> ah
23:51:33 <sdake> email already sent
23:51:33 <inc0> well
23:51:38 <vhosakot> yes
23:51:39 <inc0> ok, I'll follow up
23:51:41 <inc0> tomorrow
23:52:16 <inc0> I don't trust my skill of constructing correct sentecnes any more today
23:52:45 <vhosakot> haha
23:53:36 <sdake> ok well i'll gie folks back 7 minutes of their lives
23:53:49 <sdake> please stay focused on milestone 3 and mitaak final
23:54:09 <sdake> #endmeeting