20:02:41 #startmeeting tc 20:02:42 Meeting started Tue Apr 30 20:02:41 2013 UTC. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:02:43 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:02:45 The meeting name has been set to 'tc' 20:03:02 Agenda @ https://wiki.openstack.org/wiki/Governance/TechnicalCommittee 20:03:13 Please all welcome shardy who will be Heat PTL for Havana 20:03:20 welcome shardy :) 20:03:22 Heya! 20:03:26 hey shardy 20:03:33 welcome and congrats! 20:03:40 thanks all :) 20:03:50 #topic RedDwarf Application for Incubation - introduction discussion 20:03:57 RedDwarf folks filed for formal incubation at: 20:04:02 #link https://wiki.openstack.org/wiki/ReddwarfAppliesForIncubation 20:04:12 We'd like to kick off the discussion at this meeting and ideally make a decision by next week meeting 20:04:16 hi 20:04:24 So the idea for today is to raise the concerns we have and the questions we'd like answered before next week 20:04:25 hello and thx for the consideration 20:04:35 hub_cap: care to summarize the project so far and why you think it's ready for incubation ? 20:04:42 sure thang 20:05:13 the project contains the basics for a relational database as a service, with 2 implementations being tested and dev'd on, percona and oracle mysql 20:05:31 at present the api does not preclude us from tackling postgres, or nosql solutions 20:05:52 its got instance creation (which will eventually be instrumented by Heat), and user / schema creation 20:06:01 as well as resizes, root password enabling 20:06:15 quotas, limits, standard OpenStack stuffs 20:06:29 all the features can be found at 20:06:45 #link https://github.com/stackforge/database-api/blob/master/openstack-database-api/src/markdown/database-api-v1.md 20:06:57 so why do we feel we are ready? 20:06:59 a few things 20:07:12 weve been following, and interested in being incubated for a while now 20:07:35 we have support from > 1 company, and we are getting more support from vendors who are interested in getting into openstack from a db perspective 20:08:00 weve been following all the guidelines, using stackforge, gating, running ci tests as gates as well as pep / unit 20:08:15 keeping inline w/ blueprints / bugs, and the release schedule OpenStack follows 20:08:27 #link https://launchpad.net/reddwarf 20:08:37 so we feel like we are at critical mass, effectively 20:08:49 and would like the help and support from OpenStack community 20:09:02 ot make this a world class Database as a Service project 20:09:15 * hub_cap gets off soapbox 20:09:34 OK, questions time 20:09:43 The most obvious one is the need to rename the code name of the project into something more... hmm less... pre-existing 20:10:00 but I'm pretty sure there are other questions 20:10:07 yeah, here's one 20:10:09 less preexisting than a 'cool star' ;) 20:10:14 You've mentioned that RD has been deployed in production already. How will this affect openness to significant changes? I suspect that entering incubation will greatly increase visbility of the project, and people interested in providing input. Is that something you have concerns about? 20:10:26 #link http://www.imdb.com/title/tt0094535/ 20:10:35 aww markwash :P 20:10:47 russellb: at rax we are well aware of what this will do to the product, from a openness perspective 20:10:59 ditto for HP. 20:11:02 we are embracing the change as a good thing, and i am devoted 100% to the public 20:11:23 my title @ rax is community dev lead, i focus on the product in the open. period. 20:11:29 hub_cap: A lot of instance-specific stuff seems to have been defined in the API, do you expect that to remain, e.g after you abstract the instance behind a heat template? 20:11:54 Seems like implementation has bled through rather than being a pure DB API abstraction 20:11:55 on the name, fwiw - I don't see it "creating confusion in the market place" - but IANAL, and we should get real advice 20:12:10 I actually agree with markmc 20:12:20 shardy: i think thats a good question. we havent begun to look @ how heat will help us, and if it changes our api, then thats something we have to roll with 20:12:20 yeah, i was surprised to see security group management in your API, for example. seems like a duplicate of something that lives in other apis 20:12:27 hub_cap: have you reached out to mirantis about Savanna (or have they come to you?) 20:12:30 there is some mysql specifics, and id love to take those out 20:12:39 annegentle: not at present 20:12:59 russellb: the security groups is a pass thru, so people dont have to go to nova for some of the functionality 20:13:00 russellb: agree 20:13:03 the security groups impl is there to abstract away the VM from the end user, but also allow them to manage some things 20:13:06 one of the incubation questions is "how likely is the architecture to change drastically?" or something to that effect ... this question about heat seems to be high risk in this area 20:13:27 mordred/markmc: well, we can certainly push it to the Foundation and see how that rolls, before manding a name change. 20:13:31 russellb: its possible that some of the backend will change but we will strive to make this as painles as possible to deployers and users 20:13:35 hub_cap: yeah but i don't agree that passthrough is a solution, only a temporary thing if needed 20:13:37 agree on the issue of future major architecture changes re heat 20:14:00 russellb: thats fine wrt secgroups 20:14:07 Personally I see huge value in Red Dwarf, and everything that was discussed in the email thread on RD's incubation was spot on. But this vote comes down to "are we now openly admitting that OpenStack will accept IaaS++ projects into Integrated?" We're already moving that way, but if we go there for real it's opening the doors for many others. This isn't really a question to anyone; it's merely what I'm grappling with in 20:14:09 I'm not sure I agree wrt heat 20:14:11 i think we should stick to the heat topic, that's a much bigger issue 20:14:12 it'd be nice to at least understand how re-architecting to use heat will change things 20:14:41 mordred, agree that it should use heat? or agree we should know in advance how it will change things? 20:14:49 so i see heat as a way, under the covers to help us create instances, and clusters 20:14:50 russelb: We've already had to change a lot of architecture in the past (back when we were a fork of Nova well over a year ago and changed to be a stand-alone project making REST calls), and we managed to do it while maintaining our API. 20:14:53 I think that heat is a new wrinkle for many of the projects that they'll need to deal with and part of the overall project evolving 20:14:53 russellb: yes 20:15:09 I'd like to see the Heat work as a condition of successful graduation 20:15:12 I think exposing details of how the DB is deployed via the API is wrong, e.g in our RDS nested stack resource implementation all of that is hidden 20:15:15 gabrielhurley: I think having Swift is already answering the question of "are we going IasS+" :-) 20:15:27 users shouldn't care about those details IMO 20:15:39 shardy: agreed 20:15:44 The only potential barrier to incubation that I can see would be "shouldn't it be rearchitected to use Heat prior to be accepted for incubation" 20:15:45 I agree with jd__ 20:15:49 shardy: right. thast why they will create a "instance" and we will use heat to plug it all in 20:16:05 hub_cap: regarding mysql specifics, has there at least been a proof of concept to manage nosql? i don't want to see us hit an openstack release with a blocker for something we're claiming future support for 20:16:06 i dont see that significantly altering our api 20:16:21 dolphm: im working on a redis POC now 20:16:26 personally, I think part of the incubation process is to create the incentive to align with other integrated projects 20:16:37 hub_cap: I think the whole usage of the word "instance" wrt providing a database service needs to be reconsidered 20:16:40 ttx: that's interesting perspective 20:16:55 calling it "instance" if != "instance" in nova is very confusing 20:16:55 one wrinkle: if red warf uses Heat under the hood (which we all want) then we've moved Heat into a dependency tree whereas it was independently optionaly before... 20:16:56 imho of course ;) 20:16:57 shardy: thats fine, we went back and forth for a LONG time to try to come up w/ something that made sense to us 20:17:03 so that question could be used against RD for graduation, not for incubatuion ? 20:17:20 ttx: i like that :) 20:17:26 gabrielhurley, reddwarf will be optional :) 20:17:35 i still think it's worth considering the heat issue now vs later 20:17:39 markmc: yeah, that's about the only solution I can see 20:17:40 I'm trying to decide if it's preservationist or conservationist to require use of Heat API for integration? Does it preserve resources or actually require more? 20:17:47 because what if someone looks into this and decides it's easier to just start over to do a heat based approach? 20:17:52 "it" means the heat req. 20:18:15 i dont think it alters the api annegentle. things that would alter the api woudl be deciding to call it something diff than instances moreso... 20:18:25 russellb: words are difficult to preserve :) 20:18:26 annegentle: It's just about avoiding huge duplication of effort IMO 20:18:26 we are going to use heat behind the scenes 20:18:27 ttx: makes sense to me 20:18:28 annegentle, architectural sanity; avoiding having two things in the architecture that overlap 20:18:29 to me, Heat integration would be contained within the implementation and does not mean that it would bleed through and cause the API to change... 20:18:41 hub_cap, great :) 20:18:54 demorris: +1 20:19:01 demorris: that's what I'd expect too 20:19:01 this is more about blessing an API, though ... 20:19:03 hub_cap: demorris: okay. 20:19:15 err, more than blessing an API 20:19:16 the "using heat won't change the API" seems a bit hand-wavey to me, though 20:19:28 dmorris: also +1, the only concern may be around terminology. but has Heat settled on terminology yet or is that still in flux? 20:19:30 I want to imagine that Heat would be on the implementation rather than api side of RD 20:19:31 demorris: that would be true if the API didn't already expose details of the implementation 20:19:33 i.e. is that based on looking at how you'd use heat, or a vague understanding of what heat is? 20:19:37 sure, but does the history of incubated projects mean that they all have solid, complete apis, before becoming incubated? 20:19:39 yeah my okay was more like oh-kayyy. 20:19:58 markmc: ^^ yeah 20:20:00 i think the process to becoming integrated will be to solidify the api around openstack 20:20:16 yall didnt have input before we were incubated cuz we wre just loosely affiliated 20:20:23 now we are asking for input 20:20:25 markmc: the hand-wavvy part is that we may need additional parameters, but the operations are the same... right? CRUD model. 20:20:25 shardy, am curious in what contexts we are showing impl details in the API 20:20:27 question is, is RD as it is today actually where you would start for something like this based on heat, or would you start over? 20:20:46 and during this process we will be _fixing_ this stuff :) 20:20:49 unless someone has seriously looked into this, i don't think anyone can answer that 20:20:50 russellb: i see no need to start over 20:20:56 do you know that? 20:21:02 i cant predict the future ;) 20:21:11 well, would e.g. the taskmanager service be required if you were using heat ? 20:21:11 nor have i looked into it deeply 20:21:15 but someone can spend some in depth time considering what this would look like 20:21:21 demorris: security groups and rules, instance flavors, instance-orientated terminology 20:21:22 or would the api talk directly to heat and hand off 20:21:28 well, I think since rax and hp have this in front of customers, I doubt they'll start completely from scratch 20:21:38 markmc: task mgr also takes care of long running db tasks 20:21:42 markmc: yes, it would. 20:21:44 backups, restores, replication etc... 20:21:57 ah, ok 20:22:00 changing or building a new API doesn't sound like something bad as long as it's well handled, which is a criteria to judge for accepting a project to core, not really the API itself, IMHO 20:22:01 I think RD has flexed along the way and proven flexible, but my concern is about centralized resource sharing (docs, qa, etc.) It's my usual line of questioning. 20:22:08 btw, i'm disappointed that we have to consider the fact that companies put this into production before integrating with the community in our decision process 20:22:09 markmc: we still want to factor the front end API tier from the backend. 20:22:11 but taskmanager might not be required for instance management tasks? 20:22:33 shardy: I guess I was thinking you were meaning we were exposing DB specific implementation, which I think is what matters in this case 20:22:34 markmc: my guess is "probably not?" wiht a ? 20:22:43 we can remove those code paths :D 20:22:45 russellb: do we ? 20:23:00 ttx: well it has been brought up multiple times in this conversation 20:23:00 russellb: I am as well, but I see this as a path to getting some of that back under control personally 20:23:01 one interesting chicken/egg thing is we'd like to use RD instead of our nested-stack RDS implementation 20:23:10 ttx: I think it is a salient data point 20:23:16 markmc: no, anything that could cause latency on the front-end would want to be moved to the taskmanager/backend. 20:23:24 mordred: yeah, guess so ... but I also want to make it clear that this was the wrong order of operations. 20:23:32 russellb: ++ 20:23:37 shardy, is that really a problem ? seems like it would work fine 20:24:05 markmc: probably not, was just thinking of the bi-directional dependencies 20:24:13 shardy: I had the same question - but I haven't wrapped my head around it fully yet - but couldn't heat call rd which then calls heat ? 20:24:28 I don't see any reason why it couldn't 20:24:39 yeah, would really like someone to do some in depth investigation on how you'd design something like this to work with heat, we're doing a lot of guessing on it 20:24:41 maybe you can work on some clear Heat integration plan that you would follow if you were to be incubated, so that we can judge how realistic it sounds 20:24:46 mordred: Yep, that would work, quite similar to our existing nested stack mechanism but defined via RD 20:24:50 russellb: ++ 20:24:51 ttx: ++ 20:24:58 heh, mordred of OpenStack on Openstack claiming *this* is hard to get his head around :) 20:25:03 we can do that ttx 20:25:09 markmc: hehe 20:25:18 lol 20:25:21 that would be great, seems to be one of the most pressing concerns at this point 20:25:34 anything else RD needs to work on ? 20:25:37 agreed 20:25:45 did we get any answers around docs/ci/etc? 20:25:45 thanks hub_cap and co 20:25:50 welcome 20:25:53 (did I just miss that?) 20:25:55 for the record, you guys seem to have a tonne right 20:26:07 apologies, but I need some help here. I am still struggling with why a Heat plan, or use of Heat needs to be a pre-condition for incubation or graduation.. 20:26:09 gabrielhurley: i dont think we have talked bout it 20:26:09 also for the record, I did a quick license check - all code is apache licensed 20:26:10 given that rax / hp have products based on this, how flexible is the api at this point? 20:26:19 #info RD should investigate and document transition to use Heat if it were to be accepted in incubation 20:26:26 and deps check - nothing new, except factory_boy which looks fine 20:26:30 yeah, love the usage of openstack processes 20:26:37 hub_cap: gothca. annegentle asked about it but it fell through the cracks. 20:26:37 also happy with the scope/fit question 20:26:41 hub_cap: so you linked to a markdown API doc but I thought you were working on API docs that are more like the rest of OpenStack API docs? 20:26:41 they're doing a good job on CI on stackforge, and are currently running third-party integration tests triggered by our gerrit 20:26:42 hub_cap, do you intend to keep the rsdns client in tree? 20:26:45 to me, it is a matter of implementation on how we orchestrate the creation of database resources, clusters, etc... 20:27:02 I've seen their stackforge work also and it's good. 20:27:06 demorris: yes but the implementation *is* an important question that we do consider as criteria. 20:27:07 dolphm: i say its flexible. worst case we can version and go 20:27:19 demorris: it's not something to be dismissed as trivial and unimportant. 20:27:22 we've got hooks for them in devstack/devstack-gate and know how to hook them in if they become incubated 20:27:28 annegentle: we are, mike from rax has some that hes almost done with 20:27:35 russellb: agree, not trying to dismiss it 20:27:35 hub_cap, dolphm: yep, we can always version if drastic changes are needed 20:27:39 hub_cap: one suggestion is to put a link to your github codebase on the wiki application page 20:27:41 demorris: we want to avoid a project incubated realizing it needs to be completely rewritten to be integrated with other projects 20:27:42 i pushed that out to give people an idea of whats going on 20:27:48 struggling to keep up with all the conversations 20:27:52 annegentle: good call ill do that now 20:27:54 demorris: so investigating that in advnace sounds snae 20:27:54 demorris: lol 20:27:55 hub_cap: so, maintain backwards compatibility on further changes to v1? 20:27:57 *sane 20:27:58 russellb: its flexible enough that rax uses straight mysql and hp uses percona. 20:28:12 jcooley: *nod* 20:28:20 hub_cap, do you intend to keep the rsdns client in tree? 20:28:23 dolphm: ya i think so 20:28:26 ttx: agree, I think some things got lost in translation for me 20:28:32 hub_cap: your bug list seems fairly triaged, are you using test suites now? 20:28:48 markmc: not if we can use a client that moniker has built for us :D 20:28:50 hub_cap: i'm asking because i'm noticing what i'm hoping are bugs on the spec doc, rather than impl :) 20:29:14 hub_cap, well, it seems odd for a project to carry something like that IMHO - would be good to move it out 20:29:17 I'd love to understand why we are pushing Heat for RD as well. . is it that RD is currently duplicating a lot of Heat functionality? 20:29:28 markwash: yes 20:29:30 hub_cap, what in-guest distros do you currently support/use ? 20:29:30 annegentle: im not sure what u mean wrt test suites 20:29:39 hub_cap: how do you test now? 20:29:50 markwash: yes, orchestration basically ;) 20:29:59 ahh, so we have unit and integration tests, the integration is done via jenkins that hp has put out 20:30:00 markwash, yes, dupication of instance management and the like - RD would use Heat as an implementation detail 20:30:06 so, is there a concern that this duplication is divisive? 20:30:13 * hub_cap pulls the next question off the stack 20:30:25 hub_cap: you guys still have proboscis in the tree? or are you totally on to testr yet? 20:30:33 hub_cap: can always come back and read scrollback to catch all the points raised 20:30:45 * hub_cap will have to russellb ;) 20:30:46 mordred: The plan is to make proboscis run on testr so we can migrate. 20:30:47 * markwash will definitely look more into the Heat question for himself, so we can move on 20:31:02 grapex: that hurts my head 20:31:14 dolphm: we dont have any bugs in the spec doc persay at present 20:31:23 mordred: Rather than drop the suite complete. However once we make it run testr, it should run everything fairly trivially the way other projects do and fit into the existing CI systems. 20:31:33 grapex: great 20:31:40 markmc: we support deb based currently, ubuntu/debian 20:31:45 hub_cap: what's the timeframe for that Heat adoption analysis ? trying to see if it's worth it keeping the incubation decision on the agenda for next week 20:31:55 hub_cap, any plans for supporting other distros? any problems anticipated? 20:31:58 i dont think its out of the question to say 1 wk 20:32:11 hub_cap: post to openstack-dev when done 20:32:11 markmc: woudl love to support rpm based, no problems there 20:32:15 ttx: roger 20:32:48 ok i _think_ i got everyones Qs... if u still have one fire or ping me 20:32:56 interestingly, we didn't discuss the scope of the project 20:32:58 s/u/you/ 20:33:07 * markmc thinks the scope is fine FWIW 20:33:14 markmc: +1 20:33:23 Fine discussing it if that's an issue for anyone 20:33:32 ++ 20:33:35 otherwise we'll just go to the next topic 20:33:41 Works for me 20:33:45 I'm still a tiny bit concerned 20:33:55 thx for your consideration peoples 20:34:06 markwash: I don't like to see you concerned. 20:34:13 hub_cap: +1 20:34:13 markwash: we can chat about it if you'd like offline? or online if need be :D 20:34:15 because I would see the inclusiveness of its scope a prereq, but so far it seems like mostly promises and plans 20:34:24 all of which seem lovely 20:34:30 heh 20:34:37 markwash: what, you don't trust them? ;-) 20:34:48 well, I don't trust anybody forward looking, so :-) 20:34:49 ya markwash? 20:34:54 :P 20:34:55 haha... we have to start somewhere 20:34:56 markwash: I definitely like the idea to be more database-agnostic... but so far it's just a plan, true 20:35:37 but to me, honestly, if it fits the AWS bill of what DBaaS means, its probably right for our system 20:35:38 it's also something that can be proved during incubation 20:35:42 Maybe they could also prepare a rationale as to why their architecture will also support other-SQL and NoSQL ? 20:35:44 agree on more agnostic. otoh, adding more backends is a "if someone cares" thing 20:36:01 same as virt drivers 20:36:06 without forcing us to individually look into the code and figure it out 20:36:07 yeah, wouldn't want to mandate more backends only to have them be abandoned after the POC 20:36:09 we only added hyperv because someone showed up who wanted it 20:36:14 speaking of AWS, i noticed that they have nosql as a separate service instead of all in one 20:36:15 ttx: we have blueprints for the API that discuss how the API will be DB agnostic 20:36:16 ttx: that's why i'd like to see a proof of concept of a nosql impl, rather than more wording :) 20:36:28 russellb: oh interesting 20:36:39 we should consider that before we just assume it would be a part of this 20:36:40 russellb: interesting, do you know why? 20:36:45 I like the idea of having fewer services 20:36:47 no, was hoping someone else did :) 20:36:50 dolphm: that sounds a bit harsh to mandate for the incubation decision. maybe something we could require for graduation ? 20:36:51 hub_cap: ? 20:36:54 but the APIs are quite different 20:36:58 I think I'd settle for some convincing proof that the api is compatible with other likely backends (Cassandra, say) 20:37:01 ttx: ++ 20:37:03 the nosql aws service seems to have a lot more ... nosql-isms 20:37:15 that'd be my guess russellb, different apis... 20:37:21 ttx: agree 20:37:23 I don't personally think that reddwarf needs to support nosql 20:37:25 I think scope is an important part of this whole process (incubation, graduation, core etc) 20:37:28 * markmc is fine with the idea that nosql could be a separate service 20:37:28 yeah, but are different apis what we think is ideal? 20:37:31 or not? 20:37:33 if needs be 20:37:38 because I don't think that nosql and sql things really share much in the way of operational semantics 20:37:43 At this point I want them to explain why it should be agnostic. Then we can put them against their words in the following months to prove it 20:37:52 same as how swift and cinder are different 20:37:53 mordred: I'd be fine with that, so long as other agree that there might be room for a simliar NoSQL DBaaS project in OpenStack 20:37:55 because they are DIFFERENT 20:38:05 markwash: I'd be totally open to that 20:38:11 we might find that it doesnt fit, then we focus more on relational :) 20:38:12 markwash, yes, there would be room 20:38:13 same here, i think, based on what i've read so far 20:38:25 well, cool 20:38:32 that might seem more sane to me anyway 20:38:34 markwash: yes the topic around a distributed key-value store service similar to DynamoDB should be had 20:38:41 it shoudl AT LEAST be SQL-db-agnostic 20:38:52 ttx: +1 :) 20:39:01 ttx: def. weve strived for that so far.. maybe i do a postgres impl instead of redis/cassandra? 20:39:15 btw - I would like to congratulate the reddwarf team for not following the aws footsteps in forking mysql to do their work 20:39:17 yes, i think postgres may be a better next step 20:39:17 to prove its db agnosting 20:39:28 ttx: +1 20:39:39 or drizzle :) 20:39:50 * mordred shuts up 20:39:51 sqlite! 20:39:51 wait 20:39:52 :-p 20:39:57 lol 20:39:59 ok so for next week, ill do the following, 1) have a postgres POC, 2) have diagrams / wiki for heat design in reddwarf 20:40:21 are there other outstanding things i should bring ot the table? 20:40:23 #action markwash research the differences between AWS DBaaS offerings for sql-like and nosql-like 20:40:26 just for my own notes 20:40:43 Personally I'd be fine with you explaining why the code supports postgres rather than a POC, but a POC may actually be faster ;) 20:40:59 * jd__ votes postgres 20:41:01 i wonder markwash, do they not expose a "backend" to the serivce, like we would do w/ mongo or cassandra 20:41:11 markwash: would love to share my thoughts with you on this topic if you are up for it 20:41:21 so their api has to have doc retrieval / storage etc 20:41:23 OK, unless there are still people concerned, I'd like to switch to next topic 20:41:26 have studied this area in depth 20:41:30 demorris: cool 20:41:33 ttx: go for it 20:41:38 thank you all again 20:41:41 #topic Discussion: What is our goal with OpenStack WRT API behavior and implementation 20:41:44 jgriffith: Care to introduce this one ? 20:41:58 ttx: sure 20:42:16 So the issue is a question of API and expected behaviors 20:42:35 My take has been that the end user should not know/care what the backend storage is for the most part 20:42:51 That means, when I say "cinder create-snapshot" I get a snapshot 20:42:58 for example 20:43:07 markwash: AWS has developed the offerings with different teams and different, optimized hardware environments... :) 20:43:18 I read the thread... I think our goal is not to let storage backend vendors differentiate with unique features. Our goal is to provide a set of basic API calls that reflects our view of "block storage as a service"... and have drivers for the backends which support that set 20:43:34 Well... 20:43:47 My proposal is that the base/refernce implementation is just that 20:43:53 Also, we need to remember that some deployments might have more than one vendor in them, and those need to feel consistent. 20:43:54 If you can't meet that bar you are out 20:44:06 If you want to exceed functionality that's fine 20:44:13 mikal: +1000000 20:44:20 jgriffith: does the base/reference impl just clone on snapshot? 20:44:24 mikal: So that's my entire argument 20:44:30 remember this also not unique to storage.. networking has this issue too with API compliance 20:44:33 markwash: it can do whatever you want to implement 20:44:43 jgriffith: I'm actually not sure of the value to openstack of allowing differentioation in extensions, at least for the block storage project... 20:44:44 markwash: I don't care what the implementation looks like 20:44:46 jgriffith: how are you sure the reference implementation can cover all cases that another vendor might offer? 20:44:49 We see a similar problem with multiple hypervisors in nova too 20:44:51 markwash: all I care about is the end result 20:44:58 jd__: it can't 20:44:59 jgriffith: but I can be convinced otherwise I guess 20:45:15 ttx: well... yes 20:45:25 ttx: so there are mechanism via things like volume-types 20:45:28 keystone has this concern as well, considering how likely it is for someone to write their own identity driver / auth plugin / etc 20:45:29 ttx: that covers this nicely 20:45:30 IMO 20:45:31 jgriffith: that sounds like a problem to me somehow :) 20:45:36 jd__: which? 20:45:54 jd__: ie what sounds like a problem? 20:46:05 jgriffith: rather than defining an api with a base/ref impl, I'd rather define it in terms of an api with independent testing. . 20:46:11 jgriffith: leveling down all drivers to your base implementation features? 20:46:22 jgriffith: the significance being, I might be willing to cut corners for something that is "just" the base impl 20:46:24 jd__: markwash ok timeout 20:46:29 sorry 20:46:30 you're going the opposite direction 20:46:36 * markwash pauses 20:46:39 :) 20:46:41 I'm stating jus the opposite! 20:46:41 I still say that the "core API" should not by definition include everything that the reference implementation supports. 20:46:52 There are vendors that can't do features that the base offeres 20:46:52 core API needs to look across implementations 20:46:54 offers 20:47:01 and they're suggesting they shouldn't have to 20:47:07 I'm stating that's not an option 20:47:10 jgriffith: +1 20:47:13 what gabrielhurley said about core API is the way I see the current state of affairs on Compute 20:47:20 annegentle: gabrielhurley I agree 20:47:22 that sounds like the bare minimum to me 20:47:39 gabrielhurley: in fact that's my entire proposal in a nut shell really 20:47:43 yep 20:47:49 jgriffith: ok, you mean raising NotImplemented isn't an option? so it's better to not have this driver basically? 20:47:54 jd__: exactly 20:48:08 jd__: LVM isn't exactly a super high bar to begin with 20:48:13 But only for the "core api" right? Drivers can implement functionality over and above that? 20:48:14 my only addendum to that is that any capability that's not core needs to be "discoverable". 20:48:23 high bar or not, its a bottom-up description 20:48:26 mikal: over and above is fine 20:48:29 e.g. you can query /extensions and find out what's supported beyond the "core" 20:48:30 How do I know what's core or not? 20:48:36 mikal: as long as it doesn't change the API 20:48:42 I think that the core API needs to be suported by everything 20:48:46 mikal: I'll publish it 20:48:48 jgriffith: fair enough, I think the real question is where the bar is, this can be different for each case 20:48:51 mikal: or just look at the API today 20:48:53 or else it's a TERRIBLE user exprience 20:48:59 Agreed 20:49:08 mordred: you say that as if this isn't already the case... 20:49:09 mordred: yes, what jgriffith proposes sounds like the bare minimum to me 20:49:10 mordred: so that's what I wanted mind share on 20:49:12 I think we implicitly do this in nova already to be honest 20:49:13 mordred: agreed 20:49:18 mikal: not ture 20:49:19 true 20:49:22 i.e. all drivers can launch a VM, not all can do console output 20:49:23 we're fixing it in nova 20:49:37 in the past IIRc there was a divergence 20:49:39 that's an interesting question even for Ceilometer since we have a few drivers lacking behind the API and not offering all features for now 20:49:51 and I don't think we should artificially keep a feature oout of core just becaus a vendor might no be able to keep up 20:49:52 does anyone disagree with jgriffith that drivers should implement the core set of API features or else be rejected ? 20:49:56 jgriffith: is this about raising NotImplemented for a given function, or raising NotImplemented for a given function depending on the inputs? 20:50:01 if it's a thing that's a fundametal feature 20:50:08 ttx: yes 20:50:16 ttx: we agree since we kind-of decided the same thing during the summit for Ceilometer 20:50:17 ttx: yes 20:50:20 ttx: otherwise they are pointless drivers 20:50:21 markwash: it's about you MUST implement the API 20:50:29 I don't give a rats butt how you do it 20:50:30 the identity api has to be able to raise 501 or 403 as an expected behavior, depending on driver / driver config 20:50:31 mordred: so you don't disagree with jgriffith 20:50:55 looks like veryone agrees 20:50:58 ttx: I find it highly unlikely that I disagree with anyone 20:51:03 mordred: haha!!! 20:51:10 a little lost actually 20:51:10 mordred: i disagree w/ that statement 20:51:10 except my keyboard which I really need to change 20:51:30 markwash: I can discuss after if you like? 20:51:31 let me summarize 20:51:36 or anyone else for that matter 20:51:46 It sounds like you're definition of API is implicitly defined such that NotImplemented is an invalid result 20:51:50 #info block storage drivers should implement the core set of API features or else be rejected 20:51:57 ttx: bingo 20:52:11 but I could define a core API where it was acceptable, and that would completely undermine this discussion 20:52:22 markwash: ? 20:52:30 markwash: what would you accomplish by doing that? 20:52:45 markwash: is that a rhetorical question ? 20:52:46 and what would the user experience be? 20:53:05 ttx: yes, somewhat rhetorical 20:53:06 markwash: I wasn't suggesting a philosophical discussion, I'm just proposing that things like Horizon work 20:53:16 but I"m trying to get down to the specific case in question 20:53:17 markwash: invalid for something that is "core", which should be a smaller subset of "essential" functionality. e.g. core are things that you would not reasonably build a service without (yes, that's still subject to interpretation, but at some point it has to be). 20:53:20 and that and end user get's the expected results regardless of what backend is in use 20:53:22 so, here's a thought - our core API could expose what optional features are available 20:53:31 build the concept of capabilities into the core API 20:53:32 markwash: horrid idea 20:53:44 markmc: yeah i think gabrielhurley has been begging for that :) 20:53:49 markmc: I think I've been saying that for 2 years... ;-) 20:53:49 markwash: if you're suggesting the dynamic API reading? 20:53:50 rather than only having optional features as extensions 20:54:04 markwash: perhaps I'm misunderstanding.. sorry 20:54:15 I think I'm on you all's side here, just trying to understand and consider a specific case 20:54:16 markmc: that's orthogonal to the discussion of requiring core APi featyures to be implemented in drivers 20:54:37 the flipside is the possibility that *nothing* is "core" and *everything* is a capability that must be discoverable. but that makes for really messy clients and wonky UX's. 20:54:47 ttx, no, it's not - if the core API supported optional feature, drivers would not have to implement optional core features 20:54:48 markmc: you're talking about extra API features discovery, right ? 20:54:53 * jgriffith thinks he's regretting bringing this up :) 20:55:28 markmc: oh, I see 20:55:39 So here's the thing, the cleanest way for what markmc is proposing (I think) 20:55:42 is via extensions... 20:55:45 jgriffith, well, would it be ridiculous for snapshots to be optional and for it to be discoverable via the API that a volume isn't snapshotable 20:55:57 markmc: I still think what jgriffith wants mindshare on is a good starting point 20:55:57 reason being there can be multiple backends that have different capabilities 20:55:59 gabrielhurley: sounds beautiful *and* scary :) 20:56:10 jgriffith, notice I say the capability is discoverable *per volume* 20:56:11 i think consistent API behavior across drivers is much more important than whether a feature is wholly available or not -- a single call should mean the same thing to two different drivers, if both implement it 20:56:16 jgriffith, that's not an extension 20:56:20 markmc: indeed and that's what's scary 20:56:25 dolphm: +1 20:56:28 markmc: that volume can be anywhere on any backend 20:56:28 jgriffith, why? 20:56:40 and the end user has to get all the capabilities to know what he/she can/can't do 20:56:46 dolphm: I think that's my angle as well, much better put when you said it 20:56:58 and not only that, most SP's don't want their customers knowing the details 20:56:59 dolphm, markwash: that's the world of "everything must be discoverable", 'cuz otherwise I can't build a UI for that. 20:57:09 jgriffith, I'd say e.g. gabrielhurley would be perfectly happy if he could query per-volume whether a snapshot button should be enabled 20:57:15 gabrielhurley: agree 20:57:25 markmc: ok... well for the record 20:57:29 gabrielhurley: that sounds like a sensible requirement 20:57:32 I violently object to that idea 20:57:33 Running out of time... I think this discussion can safely move to the ML, as I'm not sure we need a cross-project TC call on this 20:57:45 jgriffith, wonderful :) the violence really helps :) 20:57:48 I think that completely kills the user experience 20:57:51 I'd rather have what jgriffith proposes, but honestly I'll take either. What I can't take it the current state of affairs which is neither. :-D 20:57:55 we need drinks for this :-) 20:58:00 markmc: I didn't mean that literally 20:58:17 markwash: I will happily debate this over drinks tomorrow ;-) 20:58:24 I would like for there to be a sane set of things that I don't have to query that I can count on to work 20:58:35 mordred: +1 20:58:40 for things that may or may not work, I'd like to know how to know if they do or don't 20:58:45 jgriffith: I think what you propose is sane. Mark's option is sane too... but in the end whoever does it gets to decide ? 20:58:49 mordred: +1, but if not possible somehow, what dolph said 20:58:58 but really, I want those things to be things I'm likely to not care about in the general case 20:59:01 ttx: markmc works for me 20:59:06 ttx: I think that's why we *do* need a TC decision on this. lol. 20:59:26 last minute question... 20:59:28 I guess I'm not understanding the debate here 20:59:34 ttx: sorry.. go ahead 20:59:36 shardy: you had a question about whether adding AutoScaling to Heat was a formal extension of scope that the TC should give its opinion on 20:59:46 quick answer from TC members ? 20:59:57 ttx: shall I just ping the ML since we're out of time? 20:59:57 think it's worth the TC discussing at least 20:59:58 +1 on adding it 21:00:07 * markmc had always thought auto-scaling would be a separate service 21:00:18 but maybe that's just because aws does it that way 21:00:20 shardy: push to the ML and we'll discuss it next time 21:00:33 ttx: will do 21:00:37 Juju does it in the same service IIRC 21:00:42 Which makes sense to me 21:00:46 or consensus will just be reached on the ML, who knows 21:00:58 the general idea of the TC discussing the widening of scope of projects seems sane, anyway 21:01:05 +1 21:01:07 +1 21:01:10 #endmeeting