16:30:08 <sdake> #startmeeting kolla
16:30:09 <openstack> Meeting started Wed Nov 18 16:30:08 2015 UTC and is due to finish in 60 minutes.  The chair is sdake. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:30:12 <openstack> The meeting name has been set to 'kolla'
16:30:13 <sdake> #topic rollcall
16:30:21 <sdake> \O/ !!
16:30:21 <SamYaple> o/
16:30:21 <rhallisey> hello
16:30:24 <nihilifer> hi
16:30:26 <jpeeler> hey
16:30:35 <akwasnie> hi
16:30:37 <stvnoyes> hi
16:31:01 <sdake> hey stvnoyes welcome - folks stvnoyes is one o f the  cats who wrote the cli along with bmace
16:31:26 <stvnoyes> nice to be here
16:31:32 <sdake> I'll give it a couiple mor emins
16:31:42 <bmace> hi
16:31:54 <SamYaple> stvnoyes: good work on the cli btw. dont know if I have told you personally
16:32:22 <stvnoyes> thx
16:33:01 <pbourke> hi
16:33:04 <sdake> #topic 1.1.0 release
16:33:20 <sdake> ok folks I'd like to discuss what features we need for 1.1.0 which are not in the tracker
16:33:23 <sdake> sec for tracker link:
16:33:51 <sdake> #link https://launchpad.net/kolla/+milestone/mitaka-1
16:33:57 <sdake> anything marked "Essential"
16:34:05 <sdake> please open it up
16:34:36 <SamYaple> sdake: thats for mitaka-1
16:34:43 <SamYaple> the 1.1.0 is for liberty
16:34:55 <sdake> well yes we develop in mitaka-1 firt then backport
16:35:18 <sdake> i was thinking our essential items for mitaka 1 would make up our 1.1.0 releae
16:36:00 <SamYaple> well things like zookeeper and drop root we arent backporting
16:36:13 <sdake> thoswe aren't essential
16:36:15 <sdake> those are high
16:36:23 <SamYaple> ah i follow
16:36:29 <SamYaple> sorry about the confusion
16:36:33 <sdake> np
16:36:52 <sdake> ok now that everyone understands, is there anything misssing?
16:37:03 <sdake> so i can gte er in the tracker now
16:37:56 <sdake> hey britthouser
16:38:02 <britthouser> sorry...late
16:38:05 <sdake> we are looking at the 1.1.0 feature list
16:38:21 <sdake> #link https://launchpad.net/kolla/+milestone/mitaka-1 (ESSENTIAL marked items)
16:38:44 <sdake> so I think we probably need someething around dupgrade of the openstack serices
16:38:46 <sdake> do folks agree or disagree with that?
16:39:08 <sdake> (not infra services, but openstack-specific services like neutron, nova, glance, etc)
16:39:27 <SamYaple> sdake: yea but that doesnt need to land before 1.1.0
16:39:45 <sdake> cool well if it doesn't needt o land for 1.1.0 then i'm good with leaving it out for now
16:40:02 <sdake> so sounds like we have th elist of essential features for mitaka-1
16:40:05 <britthouser> we don't need it for 1.1.0 b/c we're not changing the pinning of any services in 1.1.0?
16:40:13 <sdake> can folks please take 1 blueprint each?
16:40:16 <nihilifer> sdake: but you mean creating new bp(s) for the upradability of OS services or what?
16:40:26 <sdake> nihilifer yup
16:40:33 <SamYaple> britthouser: we are pinning the services to specific version in 1.1.0 (we arent currently doing that)
16:40:41 <dmsimard> sdake: I'd love to see support for aodh in there
16:40:49 <sdake> aodh is what?
16:40:57 <dmsimard> the third component of telemetry
16:41:12 <dmsimard> there's ceilometer (collection), gnocchi (datastore) and aodh (alarming)
16:41:19 <sdake> i see
16:41:29 <sdake> so our general lan is to finish th ebig tent in mitaka-2 and mitaka-3
16:41:34 <dmsimard> sure
16:41:36 <sdake> and reserve mitaka-1 for big ticket items
16:41:46 <dmsimard> works for me
16:41:47 <sdake> stuf that could potentially take the entire cyclee
16:42:13 <sdake> dmsimard does gnocchi replace mongodb?
16:42:24 <dmsimard> gnocchi interfaces mongodb
16:42:33 * sdake sadfaces
16:42:34 <dmsimard> gnocchi is the api between ceilometer and mongodb
16:42:36 <dmsimard> kinda
16:42:53 <SamYaple> dmsimard: yea. the holdup for ceilometer is HA mongodb
16:42:56 <sdake> ok well after this meeting  iwill record all of the big tent services a blueprints
16:43:04 <dmsimard> sdake: +1
16:43:13 <sdake> SamYaple that isn't the ony holdup but one of them
16:43:37 <sdake> dmsimard the nfolks can take ownership of the big tent sesrvices as they like
16:44:03 <sdake> ok so lets go through each of these individually
16:44:23 <sdake> #link https://blueprints.launchpad.net/kolla/+spec/pin-ceph
16:44:36 <sdake> the job here is iterally installing a specific version of rpms
16:44:44 <sdake> instead of yum install ceph
16:44:49 <sdake> yum install ceph-0.84.5-0.1
16:44:52 <sdake> or something like that
16:45:04 <sdake> i have checked upstream, they keep all copies and never deeleete old versions
16:45:12 <SamYaple> for the record, this is only affects centos
16:45:14 <sdake> so once we pin, we r egood to go
16:45:17 <sdake> SamYaple ack
16:45:32 <dmsimard> sdake: that might not be true for delorean repos
16:45:33 <sdake> any takers on this super easy blueprint?
16:45:45 <dmsimard> unless I'm mitaken
16:45:47 <sdake> dmsimard w edodnt use delorean for ceph we use the ceph repos
16:45:47 <dmsimard> mistaken*
16:45:57 <dmsimard> right, we don't mirror ceph
16:46:00 <dmsimard> my bad
16:46:20 <sdake> #link https://blueprints.launchpad.net/kolla/+spec/mariadb-lights-out
16:46:44 <sdake> if I wan't mistaken, I think Sam Yaple fearleessly agreeed to take this super complex blueprint on
16:46:45 <sdake> was that accurate?
16:47:06 <sdake> SamYaple ^^ ?
16:47:28 <SamYaple> yea ill work on it. its just a special playbook instead of by hand stuff
16:47:43 <SamYaple> this means that if a lights out situation occurs then the operator must do special things
16:47:44 <sdake> cool can you assign yourself
16:48:17 <sdake> #link https://blueprints.launchpad.net/kolla/+spec/record-version
16:48:28 <sdake> was it pbourke who was working on recording the version tag?
16:49:06 <SamYaple> sdake: that was me
16:49:07 <pbourke> no
16:49:09 <inc0> how do we do it? did you agree on any approach?
16:49:13 <sdake> nihilifer since you are likely to be our newest core, interested in that ceph hammer pinning blueprint? :)
16:49:31 <sdake> we have not agreed on  any approach but i'm not picky
16:49:33 <SamYaple> inc0: no we didnt there is a WIP patch for the pinning
16:49:33 <nihilifer> yep, you can assign this one to me
16:49:44 <sdake> nihilifer pelae asign it to yourself
16:49:54 <sdake> if i assign it to you its not thee same a you assigning to yourself in launchpad tracker
16:50:06 <sdake> i think you dont get credit for it in stackalytics and stuff
16:50:10 <nihilifer> sdake: i have no rights, you probably need to add me to kolla-drivers
16:50:19 <sdake> nih will do after meeting ok?
16:50:23 <sdake> anyone can be in kolla-drivers
16:50:23 <nihilifer> sure
16:50:44 <inc0> nihilifer, you should be able to assign yourself regardless of this
16:50:53 <sdake> inc0 not to bllueprints i think
16:51:09 <inc0> launchpad is hard
16:51:13 <sdake> so the rest of mitaka-1 looks really solid
16:51:18 <sdake> we are making really good progressz
16:51:20 <nihilifer> exactly, for bugs it's possible, for bps not
16:51:26 <inc0> if noone works on version pinning I can take on it
16:51:29 <sdake> if I hadn't just jammed a bunch of work in at the last minute we would be on track :)
16:51:44 <inc0> I'm back in action now;)
16:51:59 <sdake> inc0 you have #link https://blueprints.launchpad.net/kolla/+spec/sanity-check-container
16:52:03 <sdake> which is "Not started"
16:52:18 <inc0> ah so we pinned it to mitaka-1
16:52:18 <inc0> ?
16:52:23 <inc0> ok, I'll work on this one then
16:52:26 <sdake> pinned what
16:52:32 <inc0> bp for sanity
16:52:49 <sdake> oh you mean assignedto mitaka-1?
16:52:52 <inc0> yeah
16:52:54 <sdake> I dont know who assigned it to mitak1
16:53:00 <sdake> if it shoudl be in a different delivery chane it
16:53:04 <inc0> I'm ok with that, I'll manage
16:53:06 <sdake> I just go off whats inthe tracker
16:53:44 <inc0> let's keep it m-1, I'll start it right away
16:53:49 <sdake> cool
16:54:13 <sdake> #topic using an extra base image instead of virtualenv
16:54:26 <sdake> nihilifer this topic is for you to share your picture
16:54:44 <sdake> for th ezookeeper dependency
16:54:45 <sdake> kazoo
16:55:07 <SamYaple> sdake: nihilifer started on the virtualenv work
16:55:23 <SamYaple> that virtualenv stuff was a requirement from before kazoo
16:55:26 <sdake> nihilifer did you decide to do virtualenv instead of the second thing?
16:55:29 <nihilifer> sdake: yeah, my first idea was https://drive.google.com/file/d/0B_WVU0_0Qd1rSUF6VUk3OXFrMGc/view?usp=sharing
16:55:45 <sdake> they are two orthogonal tasks
16:55:50 <nihilifer> but SamYaple convinced me to venvs
16:55:55 <sdake> got it
16:55:56 <nihilifer> because kazoo is not the only problem
16:56:01 <sdake> so your not doing that anymore then?
16:56:24 <nihilifer> yep, i'll go with venvs, mostly because ceph-common also brings conflicts to python libs
16:56:24 <inc0> how about adding a hook for includes (here we go again..;))
16:56:36 <SamYaple> and well have more as we add more features
16:56:41 <SamYaple> conflicts that is
16:56:44 <sdake> ok, nihilifer  can you please make a blueprint
16:56:47 <sdake> and dget it put in mitaka-1
16:56:52 <inc0> virtualenvs will only work for python packages
16:56:54 <sdake> we will probably slip it o mitaka-2
16:56:56 <nihilifer> i created b
16:56:58 <sdake> (medium priority pelaes)
16:57:00 <nihilifer> bp*
16:57:06 <nihilifer> and i can move it to milestone
16:57:20 <sdake> good pelase do that - to d o that you need to be part of the drivers group
16:57:20 <SamYaple> sdake: the work is already started
16:57:26 <sdake> good
16:57:29 <SamYaple> i imagine itll merge in less than 2 weeks
16:57:37 <sdake> whatever works
16:57:51 <sdake> december 4th is deadline for mitaka-1
16:58:13 <sdake> since our gates are workign well now (yay) i'm going to stay pretty strict on the release dates
16:58:48 <inc0> damn, no more merge-fests-after-release-like-there-is-no-tomorrow
16:58:55 <sdake> inc0 lol
16:58:59 <sdake> #topic open discussion
16:59:07 <SamYaple> ok got two things to talk about here
16:59:08 <inc0> midcycle?
16:59:13 <sdake> ok folks, any one want the foor?
16:59:17 <sdake> sam has the floor
16:59:29 <SamYaple> thanks, first is bugs/backports
16:59:34 <SamYaple> #link https://bugs.launchpad.net/kolla/+bug/1517554
16:59:34 <openstack> Launchpad bug 1517554 in kolla liberty "test bug" [Undecided,New]
16:59:47 <SamYaple> if youll open that youll see it has a Liberty branch as affects
17:00:11 <SamYaple> if a bug affects both master and liberty, select the "target to series" button and add liberty
17:00:34 <SamYaple> then when you backport the patch (or do an all new one if backport is not an option) the closes-bug tag willwork properly
17:00:48 <SamYaple> for bugs you do _not_ need the 'backport: liberty' tag
17:01:01 <SamYaple> that is for things that we cant track through the bug tracker
17:01:08 <SamYaple> like trivial fixes
17:01:25 <SamYaple> does this make sense to everyone?
17:01:29 <britthouser> And in general do we backport all bugs?
17:01:31 <pbourke> yup
17:01:43 <SamYaple> britthouser: well if a bug affets the liberty branch then yes
17:01:44 <sdake> britthouser yes we backport all bugs
17:01:57 <sdake> but not features
17:02:00 <SamYaple> now when mitaka comes out and liberty is old-stable we may adopt a securityf-ix only policy
17:02:04 <sdake> we only backporting the essential mitaka-1 features
17:02:50 <SamYaple> ok if everyone is good with the bugs thing then im going to move on
17:03:08 <sdake> SamYaple
17:03:11 <SamYaple> yes
17:03:16 <sdake> please send an email to the mailing list
17:03:25 <sdake> since we have many missing cores from this meeting
17:03:38 <SamYaple> ok ill try to figure out how to start an email. i dont email
17:03:43 <SamYaple> ill get it done
17:03:47 <sdake> i dont expect this wil go off without a hitch
17:03:56 <sdake> i expect there wil be a learning curve here
17:04:02 <SamYaple> not a big deal. it takes 3 seconds to fix a mistake
17:04:08 <sdake> the reason we are doing this folks is becasue we need to be able to handle backports properly
17:04:13 <SamYaple> nothing on the patchside even needs to change
17:04:36 <sdake> ok #2 SamYaple
17:04:40 <SamYaple> ok next order of business
17:04:52 <SamYaple> #link https://blueprints.launchpad.net/kolla/+spec/binary-ubuntu
17:05:05 <SamYaple> I dont think this is going to happen in Mitaka
17:05:15 <SamYaple> *maybe* cloud-archive will be doing this for the N release
17:05:44 <sdake> why not get prepped for it now
17:05:55 <sdake> besides maybe nobody is around to o the work
17:05:57 <SamYaple> because cloud-archive has liberty packages right now
17:06:12 <nihilifer> do we have to use cloud-archive? can't we use just debian?
17:06:15 <SamYaple> package names change from kilo to liberty to mitaka
17:06:21 <SamYaple> nihilifer: no this is ubuntu
17:06:23 <sdake> yes but it should be a simple change to backport to mitaka at the end of the cycle
17:06:30 <nihilifer> we're installing delorean on all red hat famili distros
17:06:30 <SamYaple> the debian ones arent built for ubuntu (there are issues)
17:06:40 <nihilifer> so why we can't install debian on debian family?
17:07:06 <SamYaple> nihilifer: ubuntu packages have canonical support, debian ones do not
17:07:25 <sdake> ok so lets just assume for a moment nihilifer  we use cloud archived
17:07:27 <nihilifer> delorean don't have support from oracle for delorean
17:07:38 <nihilifer> and we're using it for oracle linux already
17:07:41 <nihilifer> afaik
17:07:44 <sdake> what is the harm in packaging liberty in the mitaka releae of kolla from binary packaging
17:07:56 <SamYaple> i dont really care about centos though, ubuntu packages for ubuntu get support. this is important for busniess
17:08:18 <SamYaple> besides, as i stated before the debian openstack packages dont work on ubuntu. ive tried them before
17:08:34 <SamYaple> sdake: yes the package names change
17:08:51 <sdake> the proposal on the table as I read it is to ship liberty paackages in the mitaka dockerfiles
17:09:00 <sdake> and do some rework at the nd of the cycle
17:09:02 <SamYaple> i said this when this came up for liberty, if cloud-archive releases only after openstack i am not ok with including it
17:09:12 <nihilifer> can we continue this topic on openstack-dev list? maybe if guys from debian and ubuntu packaging teams will see the topic, it will be better
17:09:13 <SamYaple> because then we cant tag stable with the rest of openstack
17:09:36 <sdake> nihilifer ca nyu start a thread please ssince sam is anti-mailing-list )
17:09:45 <nihilifer> sdake: lol, ok
17:10:00 <SamYaple> i dont want to new fangled email
17:10:01 <SamYaple> irc for me
17:10:13 <sdake> ok anyone else want the floor now?
17:10:36 <britthouser> britthouser
17:10:41 <SamYaple> lol
17:10:44 <sdake> shoot
17:10:49 <sdake> britthouser ure up
17:11:33 <britthouser> Just a quick question/survey.  I asked this earlier but figured I might get larger response here.  I'm curious what everyone uses for the dev environment?  Baremetal or VM?  VM in fusion/virtualbox/libvirt? VM in OpenStack?
17:11:57 <sdake> 4 node bare metal
17:12:04 <rhallisey> baremetal
17:12:08 <britthouser> Just wanting to get a feel for what is used the most, so I don't start blazing any trails on my own..at least not yet. =)
17:12:08 <inc0> mixture of all for me, but mostly vms
17:12:11 <pbourke> libvirt
17:12:26 <sdake> inc0 basically does baremetal in vms :)
17:12:31 <sdake> (follows baremetal instructions)
17:12:35 <pbourke> bug not with vagrant as vagrant-libvirt is a pile of @*!*
17:12:47 <jmccarthy> libvirt, some virtualbox also
17:13:00 <inc0> yeah we could look at fixing vagrant
17:13:16 <inc0> I got lots of flame coming my way with "I ran vagrant and it failed"
17:13:21 <britthouser> Ok...so majority is baremetal, which is what I was planning to.  So that gives me warm fuzzies. =)
17:13:31 <britthouser> thx!
17:13:42 <pbourke> i have a quick question
17:13:44 <pbourke> if that's ok
17:13:44 <sdake> inc0 pelase do - i would but I dont know how to usse vagrant
17:13:45 <sdake> I thought it got fixed but it did not
17:13:46 <jpeeler> i happen to use vagrant-libvirt... but haven't deployed in a while
17:13:51 <sdake> pbourke has floor
17:13:55 <inc0> I don't use it as well
17:14:09 <pbourke> so Im back looking at adding plugin support for kolla
17:14:21 <pbourke> and would like to move kolla-build.conf to yaml
17:14:27 <pbourke> to support this
17:14:37 <pbourke> but, it would cause some backwards compat concerns
17:14:40 <pbourke> how do people feel about this
17:14:45 <pbourke> https://review.openstack.org/#/c/246942/
17:14:51 <sdake> yaml dependencies on deploy node are ok imo
17:14:56 <SamYaple> im neutral
17:15:00 <sdake> if you make it work backward compat im good
17:15:01 <pbourke> sdake: its more that existing build.confs will break
17:15:05 <pbourke> if people have their own
17:15:07 <nihilifer> i use vagrant-libvirt mostly
17:15:16 <sdake> pbourke use two different file names dude
17:15:24 <sdake> and implement both
17:15:30 <sdake> (only get plugins with yaml)
17:15:39 <SamYaple> sdake: im not ok with that
17:15:40 <inc0> and mark ini as deprecated
17:15:40 <SamYaple> thats worse
17:15:49 <SamYaple> mark ini as deprecate is ok
17:16:13 <sdake> y amark a deprecate but you can't jus tremove it in a cycle
17:16:17 <inc0> lets keep one release of support of both and say ini stops working with N
17:16:21 <sdake> it needs to be deprecated for one entire cycle
17:16:45 <inc0> for a moment we can check "if ini then do stuff, else do yaml stuff"
17:16:45 <pbourke> there are some alternatives but each have cons
17:16:47 <sdake> this is standard openstack practice
17:16:58 <pbourke> e.g. put the plugin config in a separate file and keep the ini
17:17:05 <pbourke> but id rather just have one build conf
17:17:17 <inc0> I dislike having 2 configs with 2 formats for one thing
17:17:21 <pbourke> yeah
17:17:28 <sdake> pbourke lets go for one file, keep ini for one cycle, mark depreecated in docs/build.py
17:17:35 <sdake> and remove at start of N cycle
17:17:36 <inc0> support both for a release and remove ini in N?
17:17:43 <pbourke> I hate to introduce a deprecation but...
17:17:50 <pbourke> its probably the best option
17:17:56 <pbourke> ok thanks guys
17:18:05 <rhallisey> got a question also
17:18:22 <rhallisey> do you think we're ready to make the gate voting?
17:18:29 <sdake> holy christ no
17:18:37 <inc0> haha
17:18:41 <pbourke> :p
17:18:45 <sdake> we need local mirrors of evereything beefor the gate can vote
17:18:48 <pbourke> SamYaple is about to refactor the whole thing anyway
17:18:52 <rhallisey> uh it's been pretty solid though
17:18:55 <inc0> we don't want delorean to paralyze our dev
17:19:11 <rhallisey> ya that's true it will change later in the cycle
17:19:19 <sdake> we also need delorean going through upstream CI before we can gate on its deployment
17:19:28 <sdake> because atm it breaks reguarlaly
17:19:40 <inc0> we could somehow mirror it in CI
17:19:48 <sdake> the big problem with voting gates atm is local mirrors
17:20:11 <inc0> but until we get 100% dependable dependencies (well...meh), I'm -1 on voting
17:20:17 <SamYaple> its unlikely well ever make an binary packages master voting
17:20:34 <SamYaple> we may be able to maek the source gates voting after some mirrors are in place
17:20:43 <rhallisey> ya it would be for source
17:20:50 <sdake> i tend to agree with SamYaple it wil be difficult to make something that doesn't change per commit (delorean changfes every 3 days) voting
17:20:51 <inc0> we can mirror binary as well
17:21:04 <SamYaple> mirroring is not the issue inc0
17:21:14 <SamYaple> its the 'delorean is broken' is the issue
17:21:30 <sdake> here is what happens
17:21:33 <sdake> bad commit hits master
17:21:36 <sdake> delorean rebuildds
17:21:44 <sdake> bad commit in master reverted
17:21:49 <sdake> delorean bits stick around for 3 days
17:21:51 <sdake> while the gate falls over
17:21:58 <sdake> until new delorean build is available
17:22:03 <pbourke> could just make source gates voting?
17:22:06 <pbourke> for now
17:22:15 <sdake> the source gates fail atm often
17:22:20 <pbourke> doh
17:22:30 <sdake> i htink becasue of mirroring but i am not certain
17:22:31 <inc0> it will fail a lot more often if we introduce deployment
17:22:38 <SamYaple> source gates do not fail often
17:22:43 <SamYaple> unless there is an issue
17:22:58 <inc0> also source gates have little impact tbh
17:23:09 <pbourke> inc0: how so
17:23:14 <inc0> if it builds, it doesn't mean it works
17:23:20 <pbourke> inc0: we also deploy
17:23:24 <rhallisey> well gat deploys too
17:23:41 <SamYaple> yea but if the playbooks finish then it says ok
17:23:43 <inc0> and that will be super problematic because we'll become CI for openstack
17:23:46 <SamYaple> it doesnt currently check containers are up
17:23:49 <SamYaple> working on it
17:23:53 <inc0> glance breaks packaging, we have our work paralyzed
17:24:13 <SamYaple> inc0: if glance doesnt deploy, then they cant commit
17:24:18 <SamYaple> they run it through devstack too
17:24:27 <SamYaple> any problems we feel others will
17:24:33 <inc0> but devstack doesnt have lib separation
17:24:44 <sdake> if other projects gated on kolla in a voting way
17:24:49 <SamYaple> sure but they dont have a lib stomping issue anymore inc0
17:24:52 <sdake> then thatwould make wense :)
17:25:13 <inc0> it would then
17:25:28 <inc0> but "it worked on devstack" doesn't mean it works with us
17:25:30 <sdake> ok quick info on midcycle
17:25:31 <inc0> that's my point
17:25:44 <sdake> SamYaple has secured us a facility
17:25:52 <inc0> so south carolina?;)
17:25:54 <sdake> i will pubilsh a voting of the schedule
17:25:59 <rhallisey> sweet
17:26:00 <sdake> peoplease, pleae vote
17:26:18 <sdake> there will be soda coffee lunch
17:26:37 <sdake> remote access will be possible but not ideal
17:27:13 <sdake> i may try to get u ssome bagels for the morning
17:27:22 <sdake> depends on how my flights 2work out and rental car situation
17:27:35 <inc0> let's cook together;)
17:27:42 <SamYaple> too many cooks
17:27:44 <sdake> it wil be 2 days
17:27:51 <rhallisey> inc0, bring us some world famous texas BBQ
17:27:54 <sdake> tuesday/wednedsay
17:28:05 <inc0> rhallisey, I'm yet to try it tbh
17:28:08 <SamYaple> rhallisey: carolina BBQ is also good ;)
17:28:13 <sdake> next meeting we will start working on the agenda
17:28:14 <rhallisey> I know!
17:28:17 <rhallisey> :)
17:28:19 <britthouser> please no carolina BBQ
17:28:23 <britthouser> =P
17:28:24 <SamYaple> dry rub vs tomato based
17:28:25 <inc0> haha
17:28:35 <SamYaple> NC has vinegar
17:28:46 <SamYaple> silly north carolina
17:28:50 <SamYaple> ok times up
17:28:54 <SamYaple> back to #kolla
17:28:55 <rhallisey> bye
17:28:56 <sdake> ya times up
17:29:00 <sdake> enjooy
17:29:02 <sdake> #endmeeting