16:31:58 <rhallisey> #startmeeting kolla
16:31:59 <openstack> Meeting started Wed Dec  9 16:31:58 2015 UTC and is due to finish in 60 minutes.  The chair is rhallisey. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:32:00 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:32:03 <openstack> The meeting name has been set to 'kolla'
16:32:11 <rhallisey> #topic rollcall
16:32:18 <rhallisey> hello
16:32:19 <akwasnie> hi
16:32:19 <pbourke> here
16:32:26 <nihilifer> hi
16:32:31 <britthouser> o/
16:32:32 <rhallisey> inc0, around?
16:32:43 * jpeeler is here
16:32:48 <rhallisey> jpeeler, oh hai!
16:33:27 <kproskurin> hi
16:33:47 <inc0> yeah
16:33:54 <rhallisey> #topic Announcements
16:34:38 <rhallisey> I don't really any announcements, I figured this meeting will be focused on upgrades
16:34:44 <inc0> yay
16:34:50 <rhallisey> I know we need to tag M-1 soon
16:34:59 <rhallisey> like yesterday ).0
16:35:06 <rhallisey> O.o
16:35:10 <nihilifer> like 4 Dec?
16:35:20 <rhallisey> #topic Upgrade playbook
16:35:39 <rhallisey> inc0, you added this, why don't you lead the discussion
16:35:41 <rhallisey> :)
16:36:13 <rhallisey> #link https://review.openstack.org/#/c/254395/
16:36:19 <inc0> ok, so https://review.openstack.org/#/c/254395/
16:36:53 <inc0> I started to make playbook for upgrades
16:37:16 <inc0> I took slightly different approach than we agreed on summit
16:37:55 <inc0> it seems it makes more sense to break stuff per project
16:38:27 <inc0> every project is different and procedure will be different
16:38:38 <britthouser> That seems to match what we had drawn on the whiteboard?
16:38:48 <inc0> so to get any upgrades asap, I'll just ask community per project and do what they tell me to do
16:39:07 <inc0> well  we won't have "kolla version" this way
16:39:17 <inc0> it will be nova version and keystone version
16:39:39 <pbourke> my only concern is how will that fit in with current ops expectations
16:39:50 <pbourke> as most do not do rolling upgrades
16:40:03 <pbourke> they will want to go from kolla x to kolla y
16:40:09 <britthouser> So the "kolla version" will just be a conglomerate of each project's versions?  that's kinda how I understood things were going to work in Tokyo
16:40:37 <inc0> yeah, and it might be snowflake
16:40:54 <inc0> I'm ok with ks liberty and rest mitaka, thats ok
16:42:18 <rhallisey> inc0, I got a question since I missed this meeting.  This would be upgrade from liberty to M
16:42:26 <pbourke> will other distrubutions such as rdo be moving to this mechanism I wonder?
16:42:27 <inc0> kolla version will be just version of code you have, not version of deployed stuff
16:42:46 <inc0> yeah, this will be L->M
16:42:52 <inc0> or rather L->master
16:42:55 <inc0> at any point
16:42:59 <inc0> until we tag
16:44:42 <britthouser> Maybe I'm missing the obvious, but this sounds like what we discussed in Tokyo.  At the point we tag, then that is when the "version" of kolla is defined.  and it will just be the sum of the versions of each project at that point in time?
16:44:49 <nihilifer> i'm not aware what exactly you agreed about in tokio and how's your current idea different
16:45:01 <nihilifer> but the current idea makes sense IMO
16:46:29 <rhallisey> #link https://etherpad.openstack.org/p/kolla-mitaka-upgrade
16:47:20 <rhallisey> we can take a minute to read here
16:49:36 <pbourke> i think the key difference is the etherpad was to provide a monolithic playbook to move between atomic versions of kolla
16:50:04 <pbourke> a "version" of kolla would be comprised of multiple versions of it's components
16:50:07 <rhallisey> '    - Most likely will supply a playbook or script (last resort) for each upgrade'
16:50:24 <pbourke> but there would be no option to move indiviviual ones forward
16:51:08 <pbourke> does that make sense based on what others remember?
16:51:27 <britthouser> Maybe this is just micro/macro viewpoint.  Micro: each project will have its own playbook.  I remmeber we sketched this out on the white board.
16:51:28 <rhallisey> so in inc0's case can one upgrade just nova?
16:51:36 <inc0> yeah
16:51:48 <rhallisey> ok I like that
16:51:51 <britthouser> macro: a set of playbooks tied together and tagged ?
16:52:41 <rhallisey> inc0, I like it, but we will need to doc our recommendation. Did you guys talk about the case someone upgrade only one service
16:52:42 <britthouser> i.e. the two methods don't exclude each other.  One just makes up the other?
16:52:45 <rhallisey> and just leaves it that way
16:53:20 <rhallisey> britthouser, I think so
16:53:34 <rhallisey> that's the way I understand it at least
16:54:05 <britthouser> ok cool. Me too.
16:54:19 <britthouser> @inc0 does that jive with your understanding too?
16:54:30 <rhallisey> since we have the ability to do a rolling update is there any concern for say running a mitaka nova and a liberty glance
16:54:34 <inc0> yeah, so if you look at patchset
16:54:43 <inc0> there is upgrade.yml master playbook
16:54:50 <inc0> this will call one upgrade after anothert
16:55:49 <inc0> but you can to -t nova to upgrade just nova
16:55:51 <jokke_> rhallisey: that specific combo will hopefully have a problem :P
16:56:02 <rhallisey> jokke_, not necessary, I've done it
16:56:36 <rhallisey> I did it wil kilo & liberty
16:56:38 <inc0> hey sdake we're talking about upgrades so you might be interested
16:56:39 <jokke_> rhallisey: we are currently working to get mitaka Nova consuming v2 Images API and we are doing some changes in Glance side to facilitate that
16:56:54 <britthouser> to rhallisey' point, I could see operators wanting to pick and choose specific versions of each project.
16:57:03 <jokke_> rhallisey: so as said we're hopefully in a state where we get that work done this cycle and it might have a problem
16:57:15 <inc0> well jokke_ in this case upgrade of glance is straightforward enough to be done before you upgrade nova
16:57:25 <britthouser> but I think that's a diffrent discussion.  if the upgrades for each project are tehre, then that functionatliy could happen down the road.
16:57:26 <rhallisey> jokke_, I mean it may not work now, but that would be cool to have.  In the past it worked for me
16:57:32 <jokke_> inc0: and that way it should work perfectly fine
16:57:34 <inc0> I expect most of people will try to upgrade whole thing anyway
16:57:47 <inc0> this approach would let them do that in steps
16:57:57 <inc0> which in my mind is valuable ability
16:58:02 <rhallisey> inc0, I expect there to be a decent amount of people who do a by service upgrade
16:58:06 <britthouser> agreed inc0
16:58:07 <inc0> today glance, tomorrow  nova
16:58:14 <rhallisey> and keep some around that they don't want to touch
16:58:24 <rhallisey> 'mix and match'
16:58:27 <inc0> upgrade glance -> validate if it's ok and only after that you do next step
16:58:33 <jokke_> my point was to bring that up because I think these things might happen between project so you might not be able to count that every variation of upgrade order will work
16:58:34 <inc0> and you can stop anywhere you want
16:58:37 <rhallisey> you can do it properly if it's documented correctly
16:58:52 <rhallisey> jokke_, for sure. I've tried XD
16:58:57 <rhallisey> jokke_, some do though
16:59:04 <inc0> jokke_, we'll need to carefuly create upgrade guide
16:59:11 <inc0> and do order well
16:59:20 <inc0> but that's part of this task anyway
16:59:22 <rhallisey> inc0, the docs for ordering/mix&match would need to be spot on
16:59:36 <inc0> yup
16:59:49 <rhallisey> and they may not work from M->N
16:59:51 <rhallisey> you never know
16:59:52 <inc0> I intend to do lots of experiments with this, and I'd appreciate help
16:59:56 <rhallisey> so it's a tricky subject
16:59:58 <inc0> as I'll probably focus on ubuntu
17:00:22 <inc0> so our playbook might change each release
17:00:23 <rhallisey> inc0, I did a bunch of research in the past around this.  I'll try and help it along
17:00:44 <rhallisey> inc0, correct, the playbook and docs would change based on the intended upgrade
17:00:45 <inc0> but general idea is it should always upgrade from last stable to current master
17:01:22 <inc0> best case scenerio - doc will just need to explain what ansible-playbook upgrade.yml does
17:01:36 <inc0> and all you need to do is to call this master playbook and it works
17:01:54 <inc0> that would be my goal
17:02:05 <inc0> and upgrade.yml always upgrades stable->master
17:02:15 <rhallisey> cool
17:02:17 <inc0> therefore when we tag out Mitaka it will upgrade liberty->mitaka
17:02:22 <britthouser> So each time a new stable is cut, the playbooks would be rewritten? or appended?
17:02:32 <inc0> and change to be mitaka->master
17:02:50 <rhallisey> britthouser, Appended
17:02:52 <inc0> so I don't expect upgrade procedure will be different per release really
17:03:08 <inc0> there might be some differences, but we'll work on those case by case
17:03:13 <nihilifer> but if it will?
17:03:15 <inc0> nova upgrade procedure hasn't changed since I
17:03:21 <nihilifer> then i'm +1 for idea od appendinf
17:03:25 <nihilifer> appending*
17:03:30 <inc0> we will adjust plays accordingly
17:03:34 <inc0> maybe append, maybe not
17:03:42 <inc0> there might be steps that needs to be deleted
17:03:47 <inc0> or replaced
17:04:02 <rhallisey> for sure
17:04:08 <inc0> all I'm saying is these plays should always upgrade stable->master
17:04:22 <inc0> and we need to keep working on these forever and ever
17:04:32 <inc0> later on we should gate on this
17:04:43 <pbourke> it sounds good :)
17:04:44 <britthouser> So lets say I"m running liberty, in two years I decide to upgrade to N release.
17:04:55 <rhallisey> britthouser, O.o
17:04:58 <inc0> britthouser, then you download mitaka
17:04:59 <britthouser> will the playbooks to upgrade L->M still be around?
17:05:02 <inc0> upgrae to mitaka
17:05:02 <rhallisey> hehe
17:05:09 <inc0> then you download N and upgrade to N
17:05:19 <inc0> as long as code will be
17:05:21 <britthouser> ok...yeah that's what I'm getting at.  Will the N versions have the playbooks for L->M?
17:05:25 <rhallisey> britthouser, so I was thinking we have a playbook set per release
17:05:25 <britthouser> sounds like no?
17:05:31 <rhallisey> major release
17:05:34 <inc0> britthouser, no, they will not
17:05:44 <inc0> you'll need to do step by step
17:05:47 <britthouser> Ok yeah that would be a pain to maintain.
17:05:49 <inc0> but plays will be available on git
17:05:55 <britthouser> right
17:05:57 <nihilifer> so they will be in stable branches, right?
17:06:02 <rhallisey> yes
17:06:08 <nihilifer> ok, makes sense
17:06:08 <inc0> so you can do that, it will be harder, but well...your fault really
17:06:34 <inc0> so, my approach has one other advantage
17:06:54 <inc0> if any of you want to pick a project and write upgrade play for it, I'll appreciate help;)
17:07:27 <rhallisey> inc0, are their BP's for each service?
17:07:30 <inc0> we can do this in pararell and more project will get upgrade coverage
17:07:31 <britthouser> So we're basically telling operators not to get more than one release behind, or they'll upgrades will require multi-steps
17:07:32 <rhallisey> doesn't look like it
17:07:35 <pbourke> do work items
17:07:38 <inc0> rhallisey, I made one for nova
17:07:52 <rhallisey> ok I'll make BPs for the rest of the services
17:08:15 <inc0> work items also work, but each play will be big enough for a bp I think
17:08:36 <inc0> and might require discussion on itself as projects differ and upgrade procedures differ
17:08:48 <rhallisey> I'd rather do BP's so we can track who has each one
17:09:07 <inc0> yeah, agree rhallisey
17:09:34 <rhallisey> ok thanks inc0.  Any more points of emphasis on the topic of upgrades?
17:09:52 <inc0> if you agree with this approach, then let's just get to work I guess:)
17:10:01 <rhallisey> I like it
17:10:15 <rhallisey> #topic kolla-mesos
17:10:25 <rhallisey> Just wanted to give them a shotout
17:10:40 <rhallisey> and remind the cores that if you are interested in reviewing that work
17:10:45 <rhallisey> be sure to filter for it
17:10:59 <rhallisey> because I noticed they were 1 core short on like 10 patches
17:11:33 <rhallisey> so if you are looking to do some reviews add kolla-mesos to your watchlist
17:11:45 <rhallisey> #topic Open Discussion
17:11:57 <rhallisey> who's got some cool stuff they want to talk about?
17:12:03 <britthouser> I missed last week's meeting, but was there an update on mid-cycle?
17:12:11 <inc0> britthouser, nope
17:12:18 <britthouser> Ok. =)
17:12:19 <nihilifer> last week freenode had splitbrain
17:12:24 <rhallisey> sdake, ^ anything?
17:12:48 <rhallisey> I have something if I may
17:12:51 <rhallisey> https://blueprints.launchpad.net/kolla/+spec/kolla-kubernetes
17:13:03 <rhallisey> I saw kube now support the stuff we need to use it
17:13:08 <sdake> root canal - on pto
17:13:13 <inc0> no net=host in k8s right?
17:13:15 <rhallisey> figured i'd just throw that out there
17:13:18 <rhallisey> inc0, no it is
17:13:21 <nihilifer> rhallisey: nice
17:13:24 <nihilifer> but one question
17:13:30 <nihilifer> what about bootstrap tasks?
17:13:34 <nihilifer> does k8s support them?
17:13:39 <rhallisey> net=host pid=host and --privileged are in v1.1
17:13:54 <nihilifer> that's what we use chronos for in kolla-mesos
17:13:54 <rhallisey> nihilifer, good question.  I'll get back to you on that
17:14:01 <inc0> well, whoever would take on k8s will need to figure this out
17:14:04 <jpeeler> rhallisey: glad you asked about those caps
17:14:07 <sdake> no bbootstrp tasks
17:14:08 <nihilifer> rhallisey: i ask, because from what i know, it doesn't
17:14:10 <nihilifer> but my idea
17:14:15 <inc0> we can do mixture ansible->k8s
17:14:20 <nihilifer> is to use k8s as mesos framework then
17:14:33 <nihilifer> so opposite mixture that inc0 proposes ;)
17:14:39 <sdake> comonguys lets just stick to what we have on our plates
17:14:39 <sdake> whih is massive
17:14:44 <sdake> we don't need to invent more work for no good reason
17:15:04 <rhallisey> I'm not saying this is happening or anything but that I"m just looking at it
17:15:05 <inc0> yeah, upgrades will take lot of work
17:15:14 <rhallisey> +1 focus on upgrades
17:15:23 <inc0> let's wait for someone who needs k8s;)
17:15:34 <nihilifer> +1 for postponing k8s
17:15:38 <nihilifer> but i;m just saying
17:15:41 <inc0> if someone needs it, we'll help
17:15:43 <rhallisey> it's just show and tell :D
17:15:52 <nihilifer> that we may use it as mesos framweork along with chronos
17:16:04 <rhallisey> nihilifer, kk noted
17:16:25 <inc0> yeah once we have mesos there might be little value added by k8s...
17:16:54 <rhallisey> inc0, unless someone wanted to make it. I'm not opposed to that
17:17:09 <nihilifer> k8s would be de facto alternative for marathon
17:17:16 <inc0> yup, it's possible and it's job for the taking;)
17:17:35 <inc0> we could do kolla-on-magnum
17:17:41 <inc0> like tripleo but more;)
17:17:47 <rhallisey> O.O
17:17:48 <inc0> kom
17:17:50 <nihilifer> lol
17:18:00 <britthouser> OCOO
17:18:27 <inc0> koalla with a magnum gun would be logo
17:18:35 <rhallisey> ok well I think we're all set guys thanks for coming!
17:18:40 <britthouser> thx!
17:18:45 <rhallisey> #endmeeting kolla