20:01:56 <ttx> #startmeeting tc
20:01:57 <openstack> Meeting started Tue Sep 24 20:01:56 2013 UTC and is due to finish in 60 minutes.  The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:01:59 <jd__> o/
20:02:02 <openstack> The meeting name has been set to 'tc'
20:02:02 <ttx> Our agenda:
20:02:07 <ttx> #link https://wiki.openstack.org/wiki/Governance/TechnicalCommittee
20:02:09 <mikal> Morning
20:02:14 <ttx> This should be the last meeting for the current members !
20:02:23 <ttx> Let's break everything
20:02:30 <jgriffith> ttx: +1
20:02:31 <ttx> #topic Savanna incubation request: final discussion / vote
20:02:32 <mikal> Heh
20:02:36 <russellb> well then
20:02:37 <annegentle> wow sad to see the band break up
20:02:41 <ttx> #link http://lists.openstack.org/pipermail/openstack-dev/2013-September/014623.html
20:02:47 <ttx> #link https://wiki.openstack.org/wiki/Savanna/Incubation
20:02:55 <ttx> Two weeks ago we had the initial discussion on Savanna. Several issues were raised...
20:03:05 <ttx> Including the fact that Savanna was both providing a hadoop-cluster-provisioning API and a mapreduce data API
20:03:14 <ttx> And that the cluster provisioning part still had to see where it fits, compared to Heat and Trove
20:03:20 <ttx> Sergey posted a few answers recently at:
20:03:28 <ttx> #link https://wiki.openstack.org/wiki/Savanna/Incubation#Raised_Questions_.2B_Answers
20:03:41 <ttx> The discussion last week (and on the mailing-list since) shows that a lot of discussion is still needed to come up with a final integration plan
20:03:52 <ttx> Some TC members think that incubation would be a way to encourage that discussion
20:04:05 <ttx> Other TC members would prefer that the integration roadmap is finalized prior to incubation, the incubation period being about the execution of that plan (with graduation when the plan is executed)
20:04:05 * annegentle is listening to/watching the video at http://www.youtube.com/watch?v=SrlHM0-q5zI
20:04:31 <ttx> Personally I was on the first group but I think the second group's position makes a lot of sense... no real need to rush this. But I just want to make sure that discussion happens
20:04:58 <russellb> well, and I think incubation further encourages the discussion
20:04:59 * mordred is in the first group - doesn't think that we've required the integration plan pre-incubation for anyone else
20:05:01 <ttx> So I guess I would be fine with deferring incubation for a few weeks/months, as long as we still give time to Savanna at the design summit so that they can discuss with other incubated/integrated projects and come up with a plan
20:05:22 <russellb> i think incubating them gives even more reason for everyone else to participate in the needed discussion
20:05:28 <russellb> so i guess i'm in the first group
20:05:33 <jgriffith> mordred: I think the bigger questions were around overlap
20:05:57 <mordred> yup. I'd also like to say thank you to SergeyLukjanov for responding to last time's questions
20:05:58 * jgriffith appears to be the only one still in the second group
20:06:12 <markwash> we haven't required a full integration plan in the past, but integration hasn't felt as troubling to me personally in the past
20:06:25 <annegentle> I'm with markwash, it's a timing thing
20:06:30 <jgriffith> integration means something very different in this case IMO
20:06:31 <ttx> jgriffith: you're not alone.
20:06:46 <russellb> may mean that it's incubating for longer than one release then
20:06:49 <russellb> but i don't think that's bad
20:06:51 <ttx> It's perfectly fine to defer the request until the concerns of scope overlap are solved
20:06:56 <mordred> russellb: ++
20:07:00 <dolphm> jgriffith: i'm in the second group with you
20:07:00 * markmcclain is in 2nd too
20:07:02 <annegentle> I was trying to think of any "loss" from deferring incubation, and I don't see any other than fairness, that they are getting more scrutinized due to the recent growth.
20:07:26 <jgriffith> annegentle: IMO that needs to happen at some point
20:07:37 <jeblair> i think that whatever plan you go with should not result in the sort of situation we were in with trove where we graduated a project from incubation because we "had not been clear at the outset what the requirements should be"
20:07:38 <jgriffith> annegentle: and I really think they're a *new* type of project
20:07:39 <markmc> for me, the project looks like it's on the right track and has a sustainable developer team around it
20:07:43 <annegentle> and I'm not always in the business of fairness as my kids will tell you :)
20:07:43 <ttx> mordred, russellb: the trick is, we bless the end goal here and the team to deliver it. here the end goal still looks a bit fuzzy to me, with both clustering and data API
20:07:56 <jgriffith> annegentle: haha1
20:08:09 <mikal> It is important to have this discussion now I think, because its a lot harder when people have put in all the incubation work and think its a done deal and then we're concerned.
20:08:17 <mordred> ttx: the https://wiki.openstack.org/wiki/Savanna/Incubation#Raised_Questions_.2B_Answers has an aim towards most of the provisioning going to heat
20:08:17 <jgriffith> mikal: +1
20:08:19 <russellb> ttx: well can't we just make that clear now then, that there will be expectations, and those need to be formed?
20:08:25 <mordred> mikal: ++
20:08:31 <markwash> I'm concerned that, based on the proposed integration specifically re: Trove, the savannah group may have a vested interest against what seems to me to be a likely "best" integration path
20:08:45 <markwash> which is why bestowing incubation status feels like it may be premature
20:08:56 <vishy> o/
20:09:03 <markmc> what are we saying is the "best" integration path?
20:09:38 <jgriffith> markmc: personally I don't know that we can say that, other than address the overlap and shared concerns
20:09:39 <markwash> markmc. . not we, just me. . I'm considering "savannah uses trove to deploy hadoop" to be a potential answer
20:09:52 * hub_cap stays quiet
20:10:08 <mordred> I still don't know what "uses trove to deploy hadoop" really means - the two don't seem related to me at all
20:10:15 <markmc> mordred, agree
20:10:15 <russellb> mordred: +1
20:10:27 <russellb> both of them using heat, yes
20:10:29 <jd__> incubation should be about "Does this could ever be an integrated project for OpenStack?", that's it, IMHO, the rest is the purpose of incubation
20:10:33 <mordred> russellb: ++
20:10:36 <markmc> the cluster provisioning *implementation* overlapping with heat makes sense to me
20:10:46 <mordred> markmc: ++
20:10:47 <jgriffith> jd__: if that's the case it's an easier question IMO
20:10:49 <markmc> which is similar feedback we gave to trove
20:11:03 <mordred> yah.
20:11:13 <markmc> jd__, we've required a sustainable developer team in the past too - e.g. for designate
20:11:16 <hub_cap> fwiw, trove has no clustering solution, so integration will likely take a while whatever path is chosen ;)
20:11:20 <shardy> markmc: the cluster provisioning implementation could be a new (shared) Heat resource
20:11:25 <markmc> but it looks to me like savannah has that
20:11:27 <ttx> mordred, russellb: I sit on the fence. I feel like we'll see a lot more clearly after the summit and be in a lot better position to decide. On the other hand, incubation might be a way to make sure that discussion happens :)
20:11:40 <ErikB> Keep in mind that provisioning VMs is a very small part of what Savanna brings to the table.
20:12:01 <russellb> ttx: but if they don't incubate, they get lost in the noise
20:12:02 <dolphm> jd__: ++
20:12:10 <jd__> markmc: well because we probably don't envision a project with 0 or 1 developer being part of integrated OpenStack ;)
20:12:12 <mikal> ErikB: that's a good point
20:12:21 <jgriffith> russellb: if they poof that easily better now than later
20:12:24 <mordred> ErikB: yeah. this sentence is what I think did it for me: "Hadoop isn't a database or just data storage, but a huge ecosystem with tons of data processing related tools."
20:12:26 <markmc> jd__, right
20:12:34 <ttx> russellb: in particular, I would like to clarify if Savanna's goal is to provide a provisioning API, a data API, or both.
20:12:50 <jgriffith> ttx: +1
20:13:03 <ttx> Currently it's "both", and it doesn't make real sense to me as an openstack  project
20:13:03 <mordred> which helped me realize I was framing hadoop a bit wrong in my head
20:13:16 <ErikB> mordred - exactly. Deploying Hadoop is complex and often times vendor specific not lending itself well to a generic approach.
20:13:18 <markmc> ttx, what's the big issue with "both" as an answer?
20:13:22 * markmc isn't seeing it
20:13:25 <ErikB> ttx - both what?
20:13:26 <mordred> ttx: I've been struggling with that view since you said it originally ...
20:13:28 <mordred> what markmc said
20:13:44 <mordred> I'm not sure what it's a problem - given the nature of hadoop/map reduce processing
20:13:52 <gabrielhurley> I'm still very on the fence about this entire type of project... The "data processing" program seems like a big grey area of user/customer-oriented workflows and best practices and I have a hard time seeing the right fit for integration, but maybe that's just me.
20:13:54 <markwash> the issue for me is that we already have similar provisioning apis
20:13:55 <ttx> markmc: I see those two APIs as having a completely different audience
20:14:13 <jgriffith> gabrielhurley: not just you FWIW
20:14:23 <ttx> markmc: that's really two projects lumped together
20:14:24 <jmaron> it's goal isn't to provide a provisioning api or a data api.  it's goal is to make hadoop viable in an openstack, virtualized environment.  provisioning is part of that process, and it's MUCH more than a data api
20:14:38 <mordred> I see it as "as a user of a cloud, I would like to map/reduce some data, but I don't want to have to understand the ops aspect of running a hadoop cluster"
20:14:43 <markmc> ttx, for hadoop? I think I see it as the same audience
20:14:47 <markwash> maybe I just need more time to find my way to the Kool-Aid table :-)
20:14:48 <markmc> mordred, right
20:14:49 <mordred> I _do_ grok the map/reduce job I want to run
20:14:54 <gabrielhurley> jmaron: if you say that, then I have to say "why hadoop and not X?"
20:14:59 <mordred> this is why I think it's a good fit for a cloud
20:15:03 <ErikB> ttx: you can't have one without the other. Provisioning is incidental it's the provisioning of the hadoop services on the virtualized resources where the value is. This is complex, vendor specific.
20:15:20 <mordred> either a) you hav ea job and you want to run it on your own private hadoop, and still don't wnat to manage the details of running the hadoop
20:15:30 <IlyaE> Hadoop is a huge ecosystem and Savanna goal is to provide integrated experience of running Hadoop solutions seamless on top of OpenStack
20:15:33 <mordred> or b) you have a job and you don't mind running it on a potentially shared hadoop
20:15:57 <mordred> but in either case, you want the workload taken care of, and you don't really care about the details of how the cluster operates
20:15:59 <ErikB> gabrielhurley: Savanna is specifically focused on hadoop and nothing else at the moment
20:16:08 <gabrielhurley> yes, but why should OpenStack bless Hadoop?
20:16:09 <IlyaE> providing integration with existent tooling to properly configure and operate Hadoop clusters in integral part of the all process
20:16:15 <ttx> mordred: interesting
20:16:26 <IlyaE> and Elastic Data Processing or Data API is built on top
20:16:27 <mordred> gabrielhurley: same reason we bless MySQL
20:16:33 <gabrielhurley> except we don't
20:16:36 <mordred> sure we do
20:16:42 <mordred> trove is MySQL aaS out of the box
20:16:42 <gabrielhurley> we implicitly use mysql and are trying to add more backends
20:16:52 <gabrielhurley> trove is also working to support postgres and nonrel
20:16:53 <jd__> mordred: it has abstraction to provide for others
20:16:53 <SergeyLukjanov> gabrielhurley, Hadoop is the largest open source eco in area of Big Data
20:17:01 <jmaron> gabriellehurley: I would assume openstack welcoming hadoop would be a win-win for both
20:17:02 <jd__> I'm with gabrielhurley on this one :)
20:17:15 <mikal> SergeyLukjanov: are they any viable competitors at the moment?
20:17:16 <IlyaE> gabrielhurley - it's not about what is on backend of OpenStack - it's about apps running on OpenStack
20:17:19 <aignatov> jmaron: ++
20:17:21 <ttx> IlyaE, mordred: I guess I can see your point of providing a "portfolio of solutions for people wanting to do mapreduce stuff"
20:17:26 <gabrielhurley> Savanna (as I understand it) has no intention of being pluggable, or agnostic in any way
20:17:27 <mikal> SergeyLukjanov: or is it so big its the only game in town?
20:17:36 <hub_cap> trove wil be supporting posgres soon fwiw, and redis
20:17:39 <ErikB> gabrielhurley: OpenStack provides IaaS presumably to do something useful, one of the use cases that users are screaming for is to deploy hadoop on OpenStack. This is not easy todo today, as a matter of fact it's quite difficult. Savanna solves this exact problem and brings hadoop to the masses on top of OpenStack.
20:17:44 <gabrielhurley> Hadoop has a great ecosystem, but it's far from the only game in town
20:17:46 <hub_cap> and cassandra / mongo will come after :)
20:17:49 <ttx> IlyaE, mordred: that would address my fear of "two different audiences"
20:17:52 <markwash> why wouldn't I want to use an off-the-shelf hadoop job management system, and just deploy my hadoop on openstack with trove or heat?
20:17:56 <IlyaE> gabrielhurley Savanna has pluggable mechanism implemented already
20:18:00 <mordred> ttx:
20:18:02 <mordred> ttx: ++
20:18:04 <SergeyLukjanov> mikal, there are a lot of other tools in area of data processing, for example, Twitter Storm (that is now on the go to the Apache Foundation)
20:18:08 <annegentle> Yeah I'm a conservationist for OpenStack resources and having to support hadoop/savanna in our support channels is not win to me
20:18:11 <mordred> markwash: because that's really hard
20:18:17 <gabrielhurley> FWIW, I'm not arguing the value of savanna as solving a real problem. I'm questioning if that's a problem that integrated OpenStack is trying to solve for it's users.
20:18:45 <IlyaE> ttx In terms of audience it's absolutely the same - people who need to run analytic/computational workloads on cloud platform
20:18:51 * mordred thinks that IaaS means "help me do things via APIs that I normally have to hire admins to do for me"
20:19:00 <ttx> IlyaE: point taken
20:19:29 <jmaron> annegentle:  you are not interested in running viable, sought after application soltuions on openstack?
20:19:33 <mikal> SergeyLukjanov: would it be possible to have an API which expresses botha Hadoop map reduce or a Storm one in a sensible manner?
20:19:39 <mordred> so that as a dev, I can focus on the task at hand, and as an operator, I can consolidate admin knowledge into an efficiency-by-scale solution
20:19:47 <dolphm> mordred: but admins can do ANYTHING
20:19:48 <mikal> SergeyLukjanov: or are their assumptions so fundamentally different that API wouldn't make sense?
20:19:48 <annegentle> jmaron: not at all, but what resources are they entitled to?
20:20:00 <notmyname> mordred: wordpress shared blogging platforms ;-)
20:20:11 <akuznetsov> mikal  yes Twitter Storm can be deployed to Hadoop via Yarn
20:20:22 <markwash> notmyname: incubation approved
20:20:24 <gabrielhurley> Hadoop clustering is hard. I wouldn't really want to do it from scratch personally. If I were gonna do it on OpenStack I *might* use Savanna for that. But I don't know that I would expect to find Savanna as a batteries-included part of OpenStack. The more I think about it the more it feels like an essential and important part of the ecosystem...
20:20:32 <SergeyLukjanov> mikal, in terms of data processing api there are the same - run some workloads with specified configs, inputs and outputs, etc.
20:20:34 * markwash only kids
20:20:56 <ruhe> mikal, there are different data processing tools - Pig, Hive, raw MapReduce jobs. we already have API to use any of them
20:21:06 <mordred> gabrielhurley: I'm not sure what "important part of the ecosystem" means
20:21:15 <mordred> gabrielhurley: or what that looks like to me as a consumer
20:21:19 <ttx> gabrielhurley: I don't always install Hadoop, but when I do, I use Savanna.
20:21:25 <mordred> ttx: hahaha
20:21:37 <gabrielhurley> mordred: if OpenStack doesn't find a way to promote projects that are useful and well-written without integrating them, we're doomed.
20:21:39 <mikal> SergeyLukjanov: so you don't see a large _technical_ barrier to addding storm to savannah later if there was demand?
20:21:50 <markwash> dooooooooooooooooomed
20:21:55 <gabrielhurley> doooooooooooooooooooooooooooooooooooooooomed
20:21:55 <markwash> sorry
20:22:00 <ttx> So I guess we have three options for this meeting's vote: approve incubation now, defer the application until after summit and give some time at the summit to clarify it, or just plain deny it
20:22:01 <ErikB> gabrielhurley: exactly.
20:22:01 <gabrielhurley> I'm not
20:22:13 <SergeyLukjanov> mikal, no, there are no problems to add Twitter Storm to Savanna
20:22:18 <mordred> gabrielhurley: I'd say if the ecosystem doesn't find a way for those to make sense for consumers, the ecosystem is doomed
20:22:22 <gabrielhurley> I'm open to summit time and more thoguht
20:22:24 <mordred> righ tnow "the ecosystem" is vaporware"
20:22:31 <mikal> ttx: that sounds like two votes: do we defer; and do we incubate?
20:22:33 <gabrielhurley> mordred: it's a two-way street
20:22:35 <gabrielhurley> but yes
20:22:36 <dmitryme> gabrielhurley: I don't think there is a silver bullet for every app
20:22:39 <mikal> ttx: the second one only being needed for some outcomes of the first
20:22:42 <ttx> (note that solution (2) is also about punting to the next TC membership)
20:22:50 <mordred> gabrielhurley: if any of our public clouds would do anything other than NIH, we might get somewhere
20:22:59 <gabrielhurley> :-/
20:23:02 <ttx> mikal: (2) is like (3) but granting some official summit time
20:23:15 <mordred> gabrielhurley: thus far, the ONLY way any of them EVER work together on ANYTHING is when we're involved
20:23:32 <hub_cap> ttx either way the integration vote is w a new group :)
20:23:34 <mordred> so, as soon as I see that behvior even start to change a little bit, I'll be less of a maximalist
20:23:34 <dolphm> ttx: is there something that says a project that has been denied incubation by one TC can't re-apply to another TC?
20:23:35 <ttx> mikal: I guess so, yes
20:23:41 <ttx> dolphm: no
20:23:51 <dolphm> ttx: then what's the difference between 2 and 3?
20:24:01 <dolphm> ttx: either way it's a vote for "not now"
20:24:01 <gabrielhurley> agreed. that's partly their fault for not self-organizing more, and partly ours for not guiding more and providing more framework/structure/tools that aren't stackforge->integration
20:24:14 <ttx> dolphm: you allocate official summit time for savanna, which would otherwise have to fight for space in the unconference or wherever
20:24:16 <mikal> So, I think deferring is a cop out. We've had several weeks to discuss this, and I'm not clear on how the small amount of free time each of us has at the summit will help things be clearer. We're here to make a technical call, and we should do that.
20:24:18 <mordred> gabrielhurley: fair enough
20:24:35 <dolphm> ttx: ah, important distinction then
20:24:47 <mordred> gabrielhurley: (altohugh I do think hadoop is normal enough that I want to see clouds support it across the board)
20:24:47 <hub_cap> fwiw, regardless of tc vote, myself, heat core members and savanna need to talk
20:24:53 <hub_cap> myself=trove
20:24:59 <hub_cap> there is a lot of overlap
20:25:03 <markwash> le trove, c'est moi
20:25:09 <mordred> gabrielhurley: so I agre with you in the large, but on this particular thing, I think a hadoop is something I expect a cloud to be able to give me
20:25:16 <annegentle> I agree with gabrielhurley and mordred, "the ecosystem" is fuzzy and we should make that better, but is incubation "the ecosystem" is also fuzzy to me.
20:25:17 <mordred> hub_cap: ++
20:25:24 <russellb> hub_cap: yes, and i think incubating a project is the best way to encourage that to happen
20:25:26 <ttx> It looks like there is no consensus right now, and a bit more savanna exposure could help solve that
20:25:51 <jmaron> as well as hadoop exposure
20:26:12 <mikal> ttx: a lot of TC members aren't talking at the moment though. I feel like a vote is the best way to test for concensus.
20:26:17 <mordred> mikal: ++
20:26:20 <russellb> ++
20:26:24 <mordred> that is, after all, what voting is for
20:26:34 <markmc> yeah
20:26:38 <ttx> true words
20:26:42 <markmc> it would seem unfair to the savannah folks not to vote
20:26:43 <annegentle> I'm studying savanna a ton, and I think they're on the right track, and I'd like to encourage them, but I'm not sure the timing of incubation before summit is quite right.
20:26:46 <gabrielhurley> mordred: Hadoop's kind of a "least-common denominator" right now. it's often not the *right* thing, but people use to to kick the tires a bit, or to do some job that doesn't deserve to be specialized...
20:26:49 <IlyaE> while voting keep in mind, that Savanna is fairly advanced technically, with cluster provisioning up and working, plugins allowing different Hadoop distributions integration, stable and growing community team
20:26:51 <markmc> they've done everything we've asked to prepare for this
20:26:57 <IlyaE> we have bunch of videos published
20:27:06 <SergeyLukjanov> btw we've already have a bunch of working code and a great team that is working on project for long time, dashboard plugin, python bindings, diskimage-builder elements
20:27:12 <ErikB> annegentle: What timing would be better?
20:27:13 <russellb> markmc: +1
20:27:14 <mordred> markmc: ++
20:27:25 <mordred> I actually think they've done an excellent job in both keeping up with infra things
20:27:28 <ttx> IlyaE: paradoxically I feel like it's that advancement that plays a bit against it :)
20:27:29 <mordred> and being proactive about it
20:27:31 <jgriffith> IlyaE: SergeyLukjanov so here's the thing... YOu're way ahead and things look great IMO
20:27:41 <IlyaE> and incubation status would really help to accelerate the further integration of Savanna to OpenStack
20:27:42 <dolphm> annegentle: ++
20:27:49 <jgriffith> the confusion I have is how to reconcile with the overlap of tasks
20:27:52 <IlyaE> that would be the key goal for IceHouse
20:28:02 <dolphm> IlyaE: i'm not convinced that's true
20:28:03 <ErikB> IlyaE: ++
20:28:05 <jgriffith> You keep saying "cluster provisioning is only a small part"
20:28:05 <russellb> i think it's clear that the overlap is an issue and it's the major expectation we have to be worked on
20:28:07 <jgriffith> but it is a part
20:28:15 <jgriffith> and it's not that small else we wouldn't need heat
20:28:26 <jgriffith> I'm fine with incubation
20:28:31 <ttx> ok, we can do a plain yes/no vote, and in case no wins we can decide of any special measure like summit exposure
20:28:40 <jgriffith> I just think it would be great to have better coordination
20:28:41 <jd__> if everybody thinks Hadoop as a service fits OpenStack, the rest is up for incubation
20:28:42 <mordred> ttx: ++
20:28:46 <markmcclain> ttx: ++
20:28:48 <jd__> overlap resolving etc
20:28:56 <russellb> jd__: that's my take too
20:28:58 <SergeyLukjanov> jgriffith, we would like to move provisioning code out as much as possible by Heat integration
20:28:59 <gabrielhurley> jd__: I can agree with that
20:28:59 <annegentle> ErikB: probably just conflating my discomfort with the havana release less than a month away
20:29:04 <mordred> jd__: ++
20:29:13 <ttx> I definitely think a mapreduce API belongs
20:29:23 <markwash> resolving overlap is a major issue, but what guarantees do we have that incubation encourages resolving it in the right way?
20:29:31 <jmaron> "small" isn't a slight or a comment on its importance.  it's an indication that provisioning is only a portion of the tasks executed as part of provisioning hadoop on openstack.
20:29:39 <markwash> I'm actually afraid it makes it worse
20:29:39 <ErikB> annegentle: I see. Although I see them as orthogonal with no impact on each other.
20:29:40 <mikal> markwash: not granting graduation?
20:29:43 <annegentle> markwash: good point, but we don't have other mechanisms
20:29:47 <ttx> I'm slightly skeptical about the provisioning stuff, and would like it to be extremely thin and reuse existing bits, but everyone agrees with that
20:29:52 <IlyaE> markwash there is gating factor in form of graduation
20:29:59 <mordred> markwash: I've been beating marconi over the head about things since they got incubated
20:30:01 <jd__> I've no problem with incubation during 3 years if they can't resolve the issues -- though the project may be kicked out before :)
20:30:01 <russellb> markwash: i don't think we have a guarantee, but my gut says it's better than saying no.
20:30:02 <SergeyLukjanov> markwash, graduation is the guarantee
20:30:03 <dolphm> my opinion is simply that resolving overlap is outside the scope of incubation, which should just be the last few steps towards an integrated release
20:30:04 <mordred> markwash: and they have to take it :)
20:30:08 <markwash> I think the crucial thing is "in the right way"
20:30:14 <markwash> it will certainly be resolved before graduation
20:30:27 <ttx> I had trouble with the "dual audience" question, but IlyaE and mordred kind of fixed that
20:30:32 <jgriffith> russellb: I'm good with that
20:30:34 <mordred> yay! I was helpful!
20:30:36 <markwash> but I don't yet buy a lot of the arguments that "hadoop is just different"
20:31:09 <ttx> jd__: +1
20:31:09 <markwash> so i would want to see more flexibiliity from advocates on that position
20:31:14 <markwash> before voting yes to incubation
20:31:34 <nadya> markwash, what does "in the right way" mean?
20:32:24 <ttx> markwash: I'd say that data transformation is a basic application building block, like a queue or a database
20:33:00 <markwash> nadya: there are lots of ways to resolve the overlap. . we could decide it doesn't exist, we could delete everything provisioning and replace it with heat, we could use trove to provision more things clustery, we could build clustering as its own component
20:33:01 <ttx> markwash: that's how I solve the "hadoop is ok, but wordpress is not" question in my mind
20:33:02 <ErikB> markwash: hadoop is different in two perspectives I believe: scale of deployment (100's to 1000's of nodes) , coordination of configuring and startup of services. My point of reference being JavaEE servers and databases on Hadoop.
20:33:23 <ErikB> correction on Hadoop = on OpenStack
20:33:39 <jgriffith> ummm... you can run single node hadoop.  Just saying
20:33:41 <markwash> nadya: but the advanced state and the corporate connection with savannah spells "vested interests" to me
20:33:47 <jgriffith> but anyway, I think we're getting off track
20:34:03 <markwash> so I just want to be sure before voting yes
20:34:04 <ErikB> jgriffith: this is not what we are solving. We are solving the real world use cases of large scale deployments.
20:34:17 <gabrielhurley> What about things like Storm, Spark, Azkaban, etc. that aren't "hadoop ecosystem"?
20:34:20 <rnirmal> also keep in mind from an openstack perspective a managed service(similar to trove) for provisioning and managing clustering services would be a good separated project. i.e it doesn't have to be Hadoop specific and can support other clustered applications as well.
20:34:23 <ttx> fwiw incubation is a lot less... definitive than graduation to integrated is.
20:34:32 <jgriffith> ErikB: settle, wasn't criticizing or challenging you there
20:34:43 <gabrielhurley> rnirmal: I would vote for that in a heartbeat ;-)
20:34:48 <jgriffith> ErikB: I understand what you're saying
20:34:54 <SergeyLukjanov> gabrielhurley, some of them could be installed at Hadoop 2 using YARN (for example, Twitter Storm)
20:34:59 <ttx> the next TC can remove it if it doesn't like what it sees, and it can stay in incubation for 3 years if that's what it takes to integrate properly
20:35:14 <jgriffith> let's vote!
20:35:15 <gabrielhurley> SergeyLukjanov: is that something Savanna would want to add to their roadmap?
20:35:41 <nadya> markwash, I think all of us agree that Clustering API is important question and we are ready to discuss it. it's too complicated. several project are involved
20:36:10 * annegentle has to get to an appointment, mikal has my proxy vote
20:36:17 <ttx> TC members: please indicate when you're ready to vote, so that I know when to start it
20:36:23 <markmcclain> ready to vote
20:36:24 <mikal> ttx: ready
20:36:26 <ttx> ready
20:36:27 <jd__> ready
20:36:27 <jgriffith> ready
20:36:29 <russellb> ready
20:36:32 <shardy> ready
20:36:34 <markwash> ready
20:36:36 <mordred> ready
20:36:37 <notmyname> ready
20:36:41 <gabrielhurley> ready
20:36:44 <SergeyLukjanov> gabrielhurley, technically it's possible, but it's depend on approved project scope
20:36:49 <markmc> ready
20:36:53 <gabrielhurley> wait
20:37:02 <gabrielhurley> SergeyLukjanov: I think this is essential to the idea of scope
20:37:03 * ttx freezes
20:37:07 <gabrielhurley> is this part of your scope or not?
20:37:18 <SergeyLukjanov> gabrielhurley, currently no
20:37:21 <gabrielhurley> is your scope "hadoop" or "map reduce clustering"?
20:37:23 <gabrielhurley> okay
20:37:27 <gabrielhurley> thanks for clarifying
20:37:39 <gabrielhurley> now ready
20:37:45 * mordred actually just held his breath tensely
20:37:49 <gabrielhurley> lol
20:38:05 <ttx> (only TC members vote, thx)
20:38:06 <ttx> #startvote Accept Savanna in incubation for the Icehouse cycle? yes, no, abstain
20:38:07 <openstack> Begin voting on: Accept Savanna in incubation for the Icehouse cycle? Valid vote options are yes, no, abstain.
20:38:08 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
20:38:13 <russellb> #vote yes
20:38:14 <mikal> #vote yes
20:38:15 <mordred> #vote yes
20:38:16 <ttx> #vote abstain
20:38:16 <notmyname> #vote no
20:38:19 <jgriffith> #vote yes
20:38:19 <markwash> #vote no
20:38:21 <gabrielhurley> #vote no
20:38:22 <markmc> #vote yes
20:38:22 <markmcclain> #vote no
20:38:24 <shardy> #vote yes
20:38:25 <dolphm> #vote no
20:38:25 <annegentle_proxy> #vote yes
20:38:28 <jd__> #vote yes
20:38:36 <ttx> 30 more seconds
20:38:42 <vishy> #vote yes
20:39:14 <ttx> #endvote
20:39:15 <openstack> Voted on "Accept Savanna in incubation for the Icehouse cycle?" Results are
20:39:16 <openstack> yes (9): markmc, vishy, shardy, jd__, russellb, jgriffith, mikal, mordred, annegentle_proxy
20:39:17 <openstack> abstain (1): ttx
20:39:18 <openstack> no (5): gabrielhurley, dolphm, notmyname, markwash, markmcclain
20:39:23 <ttx> That's a yes
20:39:31 <mattf> <3
20:39:33 <ErikB> good call
20:39:35 <markmc> congrats savannah folks
20:39:39 <mattf> thank you
20:39:41 <markmc> (too many to call out individually :)
20:39:41 <vishy> ok now integrate better!
20:39:42 <mordred> wow. we actually had a close vote! I'm proud of us!
20:39:43 <ErikB> thank you
20:39:45 <SergeyLukjanov> thank you
20:39:47 <vishy> :)
20:39:48 <aignatov> thank you a lot!
20:39:50 <russellb> vishy: :)
20:39:56 <IlyaE> thanks!
20:39:57 <jgriffith> :)
20:39:58 <jcooley> sweet!
20:40:00 <akuznetsov> thank you
20:40:00 <nadya> :) !
20:40:03 <russellb> thanks guys, i look forward to seeing what you do over the next 6 months
20:40:03 <mordred> SergeyLukjanov: now you get to migrate to testr :)
20:40:05 <gabrielhurley> indeed, congratulations and no hard feelings, I hope. I would still like to see more agnostic data processing and clustering instead of hadoop-specific.
20:40:06 <jd__> :)
20:40:17 <ttx> Explanation for my abstain: I would actually have preferred to vote on this after the next design summit. But then I'm selfish.
20:40:34 <mordred> ttx: I was going to give you a hard time about that
20:40:42 <ttx> anyway, this is devil's advocate club
20:40:46 <ttx> #topic Open discussion
20:40:50 <hub_cap> gratz savanna! hope to see you in the summit to chat trove!
20:40:53 <hub_cap> (and heat)
20:40:55 <ttx> The initial governance repo commit is still awaiting your reviews at:
20:40:57 <mordred> ttx: so - sum up for me the state of the TC after this meeting
20:40:59 <ttx> #link https://review.openstack.org/#/c/44489/
20:41:12 <SergeyLukjanov> hub_cap, yep, absolutely
20:41:14 <ttx> Also PTL self-nomination in progress, closes Thursday
20:41:19 <ttx> #link https://wiki.openstack.org/wiki/PTL_Elections_Fall_2013
20:41:23 <mikal> The PTL nominations make me sad
20:41:24 <mattf> gabrielhurley, i'd like to see that too
20:41:39 <russellb> notmyname: you running?  :-)
20:41:44 <markwash> mikal: please elaborate
20:41:46 <mikal> I think all of those candidates are great, but I worry that we haven't managed to build a larger group of potential leaders
20:41:47 <mordred> ttx: do we have a TC between now and the TC elections?
20:41:53 <hub_cap> u mean the ptl single person per category cept for horizon mikal?
20:41:56 <jgriffith> mikal: you're not running?
20:41:59 <mikal> markwash: it just feels unhealthy
20:42:00 <mordred> mikal: yah. there is only one contested electoin
20:42:05 <notmyname> russellb: ya, just been doing fun stuff like debugging and airline bookings ;-)
20:42:08 <ttx> mordred: the current TC is still in charge until October 17
20:42:12 * mordred considered nominating myself as a second candidate in all elections
20:42:12 <mikal> jgriffith: that's a fair point, and I may yet. I did run last election...
20:42:27 <mordred> just for funsies
20:42:29 <jgriffith> I guess I'd rather see 1 candidate than 0 candidates
20:42:34 <mordred> jgriffith: ++
20:42:38 <markwash> mordred: I think we should elect you for all of them if you do :-)
20:42:39 <hub_cap> u know has anyone thoguht of that?
20:42:41 <ttx> mordred: although it's good practice to avoid taking decisions while the vote is in progress... unless we are forced to
20:42:44 <mordred> markwash: oh god.
20:42:50 <hub_cap> what happens if u have 0 interested in a ptl for a 6mo?
20:42:50 <mikal> jgriffith: that's true, but what happens when all our incubent PTLs go and work somewhere else?
20:42:59 <ttx> markmcclain: ISTR you had a topic to raise ?
20:43:01 <mordred> hub_cap: most active reviewer
20:43:10 <mordred> hub_cap: gets autonominated/becomes PTL
20:43:14 <markwash> mikal: fwiw I agree. . but I can see some of the reasons for it too
20:43:22 <hub_cap> hah srsly? is that in the "bylaws" (/me throws out lawyerspeak)
20:43:23 <russellb> i think a project with nobody interested would probably be a dead project
20:43:26 <markmcclain> ttx: yeah.. yesterday there was a thread on -infra about marconi and moving to pecan
20:43:33 <hub_cap> russellb: fair enough...
20:43:36 <notmyname> mikal: a lot of us have moved around already, with no major changes ;-)
20:43:36 <russellb> i suspect in many cases, if the incumbent stepped down, there would be multiple ready to jump on the election
20:43:37 <jgriffith> mikal: incubents do step down
20:43:42 <markmc> hub_cap, I think we'd be left with a ptl-less project
20:43:44 <mordred> markmcclain: we had a NICE LONG discussion
20:43:45 <markmcclain> do feel like we need to make it more official that moving to pecan is expected for graduation?
20:43:55 <markmc> hub_cap, which could be an interesting governance experiment
20:44:12 <hub_cap> maybe seeing only 1x per program means the ptls are doing a great job? :)
20:44:15 <mordred> markmcclain: so far all we have is a design summit decision - the TC has not made such a graduation vote
20:44:15 <markmc> hub_cap, personally, I think it would be totally sane to run a project based on consensus amongst core team members
20:44:20 <mikal> Yeah, my concern isn't about reality, its about preception. A large block of single candidate elections _looks_ unhealthy.
20:44:29 <hub_cap> markmc i agree
20:44:40 <russellb> mikal: perception to who?
20:44:41 <mikal> And might also leave new people with the impression that they can't run because there's some cabal blessing process they missed out on.
20:44:43 <mordred> markmcclain: I'm approaching it like the other "align on this" things - let's just beat heads together for a while until something comes out of it
20:44:54 <shardy> markmc: +1
20:44:54 <hub_cap> mikal: i think that most people know what it takes to ptl things and maybe they are ok actually being commiters lol :)
20:44:57 <mikal> russellb: potential adopters of openstack
20:44:59 <jgriffith> markmc: hub_cap personally that's how I think it works anyway
20:45:18 <markmc> heat, for example, rotates the ptl role
20:45:19 <russellb> by all means, i'm not trying to discourage anyone from running, just didn't see it as alarming
20:45:19 <mikal> One of the reasons people pick openstack is our healthy community
20:45:24 <markmc> not because they want to, but because they have to
20:45:28 <markmc> (AFAICT)
20:45:34 <jgriffith> mikal: is there any sort of a proposal you have here to feel better?
20:45:35 <gabrielhurley> FWIW, I think having competition in PTL elections is a really good thing. It's a sign people want the job and are passionate and engaged.
20:45:36 <mikal> russellb: I am "sad" not "freaked out"
20:45:39 <hub_cap> i like the heat approach, itll be interesting to see it as an experiment
20:45:48 <jgriffith> mikal: ie term limits, required number of candidates etc
20:45:53 <jd__> elected or not, there's almost always someone acting like a PTL anyway if there's no such mechanism
20:45:54 <mikal> jgriffith: not really
20:45:56 <ttx> could we focus on markmcclain's question for a minute ?
20:46:02 <mikal> jgriffith: just encouraging people to consider running
20:46:08 <ttx> <markmcclain> ttx: yeah.. yesterday there was a thread on -infra about marconi and moving to pecan
20:46:09 <mordred> jgriffith: term limits  would mean someone other than notmyname would have to run swift!!!
20:46:12 <shardy> markmc: because we have to?
20:46:14 <ttx> <markmcclain> do feel like we need to make it more official that moving to pecan is expected for graduation?
20:46:17 <markwash> mikal: I'm afraid that people are worried that losing an election might damage their ability work with the team moving forward
20:46:31 <mordred> ttx: ++
20:46:34 <jgriffith> mikal: got ya, and I'd agree
20:46:41 <markmc> shardy, well, you feel that there has to be a PTL - a PTLless heat doesn't feel like a realistic option, right?
20:46:44 * mikal defers responsing until after the pecan thing
20:46:48 <jd__> ttx: markmcclain: that could be a request to get my vote for example, yes
20:46:49 <jgriffith> mikal: but I'd also say PTL isn't exactly easy/fun
20:46:51 <markmc> shardy, given our current governance structure
20:46:55 <hub_cap> do we want to pull in the people who are opposed to pecan? doesnt marconi say no to pecan?
20:46:57 <shardy> markmc: It's what we discussed and agreed, because we all wanted to maintain some productivity ;)
20:47:08 <mordred> hub_cap: they do not - it's on their todo list
20:47:12 <jgriffith> mikal: I don't blame anyone for not wanting to do it
20:47:18 <hub_cap> oh great mordred!!!!
20:47:18 <shardy> markmc: Yes, basically we have no need for prescriptive leadership of the Heat team
20:47:23 * hub_cap shuts up
20:47:26 <mordred> I'd actually like to see if we can get moving on that without having the TC make a prescriptive call
20:47:35 <shardy> markmc: but we do need someone to click all-the-things in launchpad I guess ;)
20:47:38 <mordred> shardy: did we both just use the word prescriptive
20:47:40 <mordred> ?
20:47:58 <shardy> mordred: I think I stole it from stevebakers's nomination ;)
20:48:36 <jgriffith> wow... all PTL does is click buttons in LaunchPad?
20:48:52 <mordred> jgriffith: you can spend entire days doing that my friend
20:48:53 <markmc> well, here's to such a nice group of people being elected to the next TC :)
20:49:03 <mordred> markmc: +100
20:49:04 <jgriffith> mordred: you don't have to tell me...
20:49:09 <shardy> jgriffith: well I was joking, but there are a lot of non-technical overhead tasks
20:49:09 <russellb> jgriffith: if only
20:49:20 <jgriffith> but I was getting at the suggestion that that's *all* it is
20:49:26 <mordred> jgriffith: hopefully storyboard will fix that ...
20:49:29 <hub_cap> ya ttx has scripts for all that jgriffith :)
20:49:29 <notmyname> I think the kind of people who like or put up with the requirements of PTL are different than the skills that make a good contributor. and so perhaps many contributors are happy no being (or running) for PTL
20:49:31 <mordred> right ttx ?
20:49:34 <mikal> I think the way we show people that they can lose a PTL election and remain in the community is by doing it a bunch of times. I lost the nova ptl election last time around, and I haven't rage quite (yet). I still even sarcasticly goad russellb in back channels.
20:49:37 <jgriffith> hub_cap: ha!!
20:49:37 <shardy> jgriffith: like I said, poor attempt at humor ;)
20:49:53 <hub_cap> +1 notmyname
20:49:54 <jgriffith> shardy: nah... I get it, it was funny
20:50:05 <mordred> mikal: you should rage more
20:50:06 <dolphm> notmyname: +++
20:50:06 <ttx> mordred: trying to see what was the question
20:50:19 <mordred> ttx: mordred | jgriffith: hopefully storyboard will fix that ...
20:50:28 <mordred> ttx: in response to spending all day clicking buttons in launchpad
20:50:34 <jd__> storyboard?
20:50:38 <markmc> heh
20:50:41 <russellb> i have so much launchpad karma now, haha
20:50:42 <jeblair> i believe mordred was suggesting that storyboard will eliminate the need for ptls ;)
20:50:48 <russellb> where do i redeem it for cash
20:50:48 <mordred> jeblair: ++
20:51:03 <mordred> russellb: shuttleworth's house
20:51:15 <markmc> ttx, are you doing a design summit session on storyboard?
20:51:16 <ttx> mordred: +0 :)
20:51:16 <jgriffith> mordred: LOL
20:51:24 <ttx> markmc: yes I will
20:51:27 <mordred> ttx: you probably should
20:51:29 <markmc> ttx, coolness
20:51:41 <hub_cap> +1 to story board session
20:51:59 <ttx> I like to submit more sessions that I can actually do work in 6 months
20:52:05 <ttx> Note that my "Meet the TC" proposal for the OpenStack Conference in Hong-Kong got accepted
20:52:09 <ttx> It's a panel where anyone on the future TC will be able to participate (if available)
20:52:14 <ttx> I think Monty has been working on securing the TC dinner too.
20:52:19 <ttx> Just to encourage you all to run again :)
20:52:37 <gabrielhurley> I expect good things for dinner
20:52:43 <mordred> yes. I have official HP approval to pay for dinner this time
20:52:43 <gabrielhurley> (if I'm re-elected)
20:52:46 <markmc> we need bodyguards at the TC dinner?
20:52:47 <mordred> if I'm re-elected
20:52:49 <markmc> (lame joke)
20:52:57 <mordred> markmc: wow man
20:52:59 <hub_cap> oh great now mordred is a shoein
20:53:01 <mikal> Ok, so people's election platform is now "I will buy you dinner"?
20:53:03 <markmc> mordred, yeah :(
20:53:13 <mordred> mikal: that's ALWAYS my platform
20:53:14 * jgriffith will buy drinks!
20:53:19 <devananda> mikal: i thought it was "I will buy you a ${drink}"?
20:53:22 <markwash> mikal: I was going to promise free pizza and no homework
20:53:23 * russellb won't buy you anything
20:53:25 <jgriffith> if the bribe method is effective
20:53:25 <russellb> VOTE FOR ME
20:53:27 <mikal> I should announce my platform -- vote for me or I shall kick you in the shin
20:53:29 <jgriffith> russellb: lol
20:53:33 <hub_cap> lol russellb
20:53:33 <lifeless> mikal: nice
20:53:37 <gabrielhurley> mikal: ++
20:53:43 <devananda> lol
20:53:44 <mordred> mikal: remind me to not bring my shins to Oz
20:53:45 * hub_cap sees what open discussion really means
20:53:48 <mikal> Heh
20:54:02 <markmc> hub_cap, "kill time until the next meeting" ?
20:54:10 <ttx> mikal: well, HP will pay dinner EVEN if mordred is not reelected, that's the beauty of it
20:54:18 <hub_cap> i was thinking "pretend we are in -infra for 5 min"
20:54:28 <mikal> ttx: OMG, HP rock!
20:54:38 <ttx> mikal: they are so great. Nice printers, too
20:54:48 <hub_cap> boss monitors for their employees :)
20:54:54 <mikal> ttx: do they have a corporate song?
20:55:07 <mikal> Every company should have a song
20:55:24 <hub_cap> +1 and a danceline
20:55:25 <russellb> do we have an openstack song?
20:55:38 <gabrielhurley> do you really have to ask?
20:55:41 <russellb> i suppose we have a rap duo!
20:55:42 <russellb> :)
20:55:42 <mikal> russellb: we have several bad rap videos...
20:55:46 <gabrielhurley> http://www.youtube.com/watch?v=3jUQ09Jf4GU
20:55:49 <russellb> you mean awesome rap videos?
20:55:50 <hub_cap> guys.... hip hop
20:55:52 <gabrielhurley> mikal: I resemble that remark
20:55:53 <hub_cap> CMON
20:55:59 <jgriffith> geesh russellb !!
20:55:59 <hub_cap> rap is so 90's
20:56:08 <gabrielhurley> also, just wait... change is coming
20:56:10 <mikal> I want an openstack boy band
20:56:10 <russellb> yeah that was poor form for me to even ask
20:56:13 <russellb> gabrielhurley: lol
20:56:15 <russellb> <3
20:56:18 <jeblair> mikal: you are an openstack boy band
20:56:26 <ttx> gabrielhurley: any hint ?
20:56:35 <russellb> heh, we could even #endmeeting first
20:56:36 <mikal> jeblair: I can be the charmingly non-threatening one
20:56:38 <russellb> so it's not on the record :-p
20:56:48 <hub_cap> i dunno this is a interesting logged meeting lol
20:56:53 <jeblair> mikal: that kicks people in the shin
20:56:54 <gabrielhurley> several titles have been tossed around... one of the proposals is "How Can I Live (Without OpenStack)?"
20:56:59 <hub_cap> specially jeblair's boy band comment lol
20:57:06 <mikal> jeblair: gotta do what you gotta do
20:57:08 <hub_cap> ttx: ICEHOUSE!!!!
20:57:27 <lifeless> baby
20:57:38 <hub_cap> nice
20:57:40 <ttx> gabrielhurley: http://www.youtube.com/watch?v=lPPDuWAcMcc ?
20:57:47 <russellb> i'm glad the Icehouse summit is out of the US ... someone might try to serve Icehouse beer if it were here
20:57:49 <hub_cap> yes ttx ^ ^  perfect
20:57:52 <mikal> We can model our boy band off De Jour
20:57:56 <gabrielhurley> lol
20:58:10 <ttx> can't get that music out of my head
20:58:15 <gabrielhurley> let's just say you're closer than you think
20:58:22 <hub_cap> youve just infected the tc ttx
20:58:24 <gabrielhurley> but not that. never that.
20:58:53 <markwash> ttx: we can set up that lightshow cube thing in the dev lounge
20:59:10 <hub_cap> +1 markwash
20:59:25 <ttx> markwash: comes with a horse though*
20:59:45 <ttx> ok, time to close this
20:59:48 <markwash> ttx srsly this video is fantastic
20:59:54 <markwash> naming choice completely redeemed
21:00:06 <hub_cap> lol
21:00:11 <ttx> #endmeeting