22:37:55 #startmeeting 22:37:56 Meeting started Tue Aug 2 22:37:55 2011 UTC. The chair is danwent. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:37:57 Useful Commands: #action #agreed #help #info #idea #link #topic. 22:38:03 thanks Dan! 22:38:07 #topic API sync vs. aysnc 22:38:11 is RamD still here? 22:38:47 oh man, I really hope I didn't start up the meeting bot for nothing :P 22:38:47 Basically we have two choices: 22:39:04 sorry back here 22:39:06 1) the plugin defines the behaviour 22:39:12 2) the API defines the behaviour 22:39:15 salv: go ahead 22:39:30 Pros of approach #1 22:39:58 flexible and simple 22:40:02 Cons of approach #1 22:40:22 Potential duplicated code among plugins (as pointed out by Sumit) 22:40:34 Pros of approach #2 22:40:46 Reliable, well defined behaviour. Clients will be happy. 22:40:51 Cons fo approach #2 22:41:40 Burden on API layers, needs for changing plugin interface, potentially unnecessary if then the plugin implements the async behaviour as well. 22:41:40 salv: by "the API defines the behavior", do you mean the API would return without making a call to the plugin? 22:41:59 salv: The code duplication issue can be solved with code-sharing. Once we have real customers, i.e. people deploying OpenStack complain about it, we can promote the behavior definition to the API. It seems good to keep our current approach flexible. 22:42:06 salv: I suspect plugins that perform complex ops will implement their own degree of asyc as needed. 22:42:07 danwent: the API will call the plugin and return without waiting for it to complete the operation 22:42:22 salv: This is the stance we took on the DB model too, that we can leverage DB models by code sharing. 22:42:29 I thought a bit about it and I'm leaning for approach #1 at the moment 22:42:51 From the API perspective, what we should ensure is consistency towards API users. 22:43:01 I don't feel all that strongly here, other than I don't want something that makes writing a simple plugin complex. 22:43:14 salv: consistency is important as well. 22:43:26 salv: +1 on consitency 22:43:37 #2 will help us here, right? 22:43:46 My proposal is to leave the decision on sync/async behaviour to the plugin 22:43:46 approach #2 guarantees consistency 22:43:59 and modify the API such as it guarantees consistency at the same time 22:44:17 +1 salv 22:44:29 I think we can achieve it by stating that the API "does not guarantee" the completion of an operation when the response is returned 22:44:33 salv: is that explained in the spec? 22:44:44 what does consistency mean in terms of async vs. sync 22:44:53 RamD: not yet, that why we are discussing it :) 22:44:58 is there a specific use-case? 22:45:25 somik: consistency of sync vs async behavior 22:45:25 I think that's the important thing...how does API can ensure consistency for a plug in behaviour not clear for me 22:45:31 Let's look at consistency from the user perspective. The user does might not have any idea about the plugin. 22:46:02 possible confusion in the air.. there's consistency "all apis look the same" and consistency "acid data integrity guarantees" 22:46:17 uptil the client gets the response it wants, why does the client care 22:46:19 i think the original meant #1, i fear some people are reading it as #2 22:46:30 Tv_ I agree that we are after consistency #1 22:46:42 Tv_: right. I'm talking of consistency as "api always look the same despite of what behaviour is actually being employed" 22:47:00 solv: that's given, right? :-) 22:47:27 we should not have # of APIs based on plugin one Quantum core API thats it. 22:47:45 So in the same way in which the user is not bothered by the actual plugin Quantum uses, the user will not care at all whether it is synchronous or asynchronous 22:47:51 i think you should say something like "plugins are expected to respond in <1sec. any part of plugin api that can't consistently do this should be designed to be async, following the calling conventions of the blahblah openstack api" 22:47:59 As long as the API faithfull tells what's happening. 22:48:17 salv: +1 22:48:34 something like, you get handed back a token, you can poll the status with the token, whatever is used by e.g. vm creation and slow operations like that in openstack 22:48:44 so we can add a "resource status" concept similar to what is done in nova 22:48:54 Tv_: exactly. 22:48:55 Tv_ +1 22:49:53 then, if the plugin is sync, then the status will be immediately "available", otherwise it will be in "build" or "provisioning" for a while, but the user can check it with the token (or the resource id itself) 22:50:08 I think this will give the API user the consistency it needs. 22:50:10 i'd hate to force all plugin ops to go through that extra token phase 22:50:14 let simple things be simple 22:50:26 Tv_: sure 22:50:40 that would not apply to GET operations, for instance. 22:50:42 I think we should make things simple and add this complexity in E release if needed, after design summit 22:50:48 having it be a "status" on a resource can help things be simple 22:51:13 the status concept can easily be implemented within diablo timeframe 22:51:13 similar to salvatore's email 22:51:35 oh here's one trap i've seen people stumble in; if you can use an async api as if it were sync, and it works if you're slow enough, people *will* screw it up 22:51:50 as in, "it worked when i typed it in the repl!" 22:52:00 so i prefer it when an async api looks different enough 22:52:25 like, if you "create" something, and you're supposed to poll until it's ready before you use it elsewhere 22:52:29 people will forget to poll 22:53:04 Tv_ agreed, async clients are a lot more complicated therefore we dont want quantum clients to go through that complexity if its not needed. 22:53:29 If its needed by a plugin and the customer needs that plugin, they can still do the complicated plugin 22:54:07 somik: do you recommend we should leave things as they are for the diablo release? 22:54:26 somik: my question as well 22:54:38 I think that would help us polish things out and let some end-customers try Quantum easily and then we can iterate 22:54:38 somik: I think we are agreeing on the "status" level "async" for now..let's start with that. 22:55:03 let me take a look at the spec as well and send in my comments 22:55:11 RamD: that would be adding something to the API that's not agreed upon aka "feature creep" ;) 22:55:35 somik:oops I thought we are converging :-) 22:55:43 somik: reasonable enough 22:56:36 and the plugins can still implement async behavior, the API changes for async we can iterate on after we get some real feedback 22:56:41 let's consider the scenario where the API stays as it is. There could be incosistencies. Let's see if those incosistencies are serious enough. 22:57:28 Example: an asynchronous plugin might do stuff with PHY switches, virtual switches, and other network appliances. When you create a network 22:57:41 salv: I'm for the "resource status" based API behaviour 22:57:49 somik: if left to the plugin to decide whether sync or async what behavior would the client expect? 22:58:04 somik: if we do async call without API change, how can user check whether it's done or not? 22:58:04 you can think your network is there, but actually the plugin is still provisioning it. The API client will try for instance to do a plug interface call and that will fail. 22:58:25 client has to be written with one or the other approach in mind 22:58:30 as a starter we leave that to "plugin" to support the behaviour and as we move forward we can consider on API layer in future 22:58:37 ying: that's my point as well. Client will think resource is ready, but it's still being provisioned 22:59:14 the plugin responds within a reasonable HTTP timeout.. the client doesn't care about the rest. 22:59:35 ying: +1 22:59:49 somik: you mean the plugin responds when the provisioning of the resource is complete? 22:59:53 quit 23:00:07 RamD: bye 23:00:14 the plugin does not necessarily need to do all of its work before returning. 23:00:19 salv: thats one approach in the current framework 23:00:37 for example, on a create network, it need only store the notion that the network exists. 23:01:02 somik: user needs to care, as after creating network, he will use it and need to make the provisioning is done 23:01:16 but isn't there a difference between a network that "exists" and a network that is "available"? 23:01:16 danwent: that implies plugin its implementing async behavior 23:01:26 Sumit: yup 23:01:53 SumitNaiksatam: plugin abstracts the complexity and we expose a simpler model to user for now 23:01:58 my guess is that most complex plugins will do that already. 23:02:01 I think plugin should not be forced to do either async or sync. They can do whatever the developers thinks it's better for them! :) 23:02:19 salv: agreed 23:02:42 salv: in that case framework should implement async behavior 23:02:45 My point actually is not about the plugins at all. My point is that there is a difference between a resource which exists and a resource which can be actually used 23:02:46 salv: +1 23:02:54 good to have that flexibility, but need a way to let user know it's sync or async api 23:03:17 think about a server in nova. You cannot suspend an instance if it is not running, can you? 23:03:17 plus, for async api, needs a way to let user know whether the resource is ready 23:04:00 I would be in favor of exposing some kind of status, which in general indicates whether the plugin is "actively managing" the port. 23:04:16 thats the plugin design issue, one way the plugin can do this is by handling asnc backend with a sync API, which is the current API 23:04:19 this status may be down because it hasn't been provisioned yet, or because a switch was down, or for a host of other reasons. 23:04:48 ying: for "async api" you mean the API layer uses something like a message queue to dispatch a call to the plugin? 23:05:08 networks are pretty dynamic things, and in general we will need a way for someone to check if the logical model is currently correctly mapped to the real world. 23:05:19 * salv needs some more coffee 23:05:20 salv: that would definitely be out of scope for diablo ;) 23:05:39 ok, sounds like there's enough discussion here that we should let salvatore get some sleep 23:05:47 salv: async api means async call, no matter we do it in api layer or plugin layer 23:06:00 * salv had his coffee, now he feels like it's 9AM 23:06:13 I just want to say, for async mechanism, we need a way to let user check whether it's done or not 23:06:39 ying: good, we're on the same page. Do you think the concept of "resource status" will be enough? 23:06:58 salv agree 23:06:58 ying: the current API is sync, but the plugin can handle async implementation, and we can still work with current sync api. 23:07:39 somik: we still need extend current api by having status checking, right? 23:07:40 somik: agreed. But as ying says, we need a mechanism to check whether the operation has completed or not. Do you agree on that? 23:07:53 salv: how about you send out a proposal for the resource status and people see if it handles their use cases? 23:08:33 danwent: will do (actually I think I already did it, but the proposal was not detailed enough probably) 23:08:42 danwent: +1, salv: a high level example will help 23:08:53 take create_network() for instance 23:08:53 +1 23:08:58 #action salvatore to send out more detailed thoughts on resource status 23:08:58 we are over time limit, I dont want to hold people off for long, we can do email discussion, that would be better 23:09:07 agree 23:09:15 k 23:09:24 thanks salv for shepherding this! 23:09:30 ok, this is a great discussion :) 23:09:36 we good to end the log? 23:09:47 yes;-) 23:09:57 #endmeeting