22:37:55 <danwent> #startmeeting
22:37:56 <openstack> Meeting started Tue Aug  2 22:37:55 2011 UTC.  The chair is danwent. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:37:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic.
22:38:03 <salv> thanks Dan!
22:38:07 <danwent> #topic API sync vs. aysnc
22:38:11 <salv> is RamD still here?
22:38:47 <danwent> oh man, I really hope I didn't start up the meeting bot for nothing :P
22:38:47 <salv> Basically we have two choices:
22:39:04 <RamD> sorry back here
22:39:06 <salv> 1) the plugin defines the behaviour
22:39:12 <salv> 2) the API defines the behaviour
22:39:15 <RamD> salv: go ahead
22:39:30 <salv> Pros of approach #1
22:39:58 <salv> flexible and simple
22:40:02 <salv> Cons of approach #1
22:40:22 <salv> Potential duplicated code among plugins (as pointed out by Sumit)
22:40:34 <salv> Pros of approach #2
22:40:46 <salv> Reliable, well defined behaviour. Clients will be happy.
22:40:51 <salv> Cons fo approach #2
22:41:40 <salv> Burden on API layers, needs for changing plugin interface, potentially unnecessary if then the plugin implements the async behaviour as well.
22:41:40 <danwent> salv: by "the API defines the behavior", do you mean the API would return without making a call to the plugin?
22:41:59 <somik> salv: The code duplication issue can be solved with code-sharing. Once we have real customers, i.e. people deploying OpenStack complain about it, we can promote the behavior definition to the API. It seems good to keep our current approach flexible.
22:42:06 <danwent> salv: I suspect plugins that perform complex ops will implement their own degree of asyc as needed.
22:42:07 <salv> danwent: the API will call the plugin and return without waiting for it to complete the operation
22:42:22 <somik> salv: This is the stance we took on the DB model too, that we can leverage DB models by code sharing.
22:42:29 <salv> I thought a bit about it and I'm leaning for approach #1 at the moment
22:42:51 <salv> From the API perspective, what we should ensure is consistency towards API users.
22:43:01 <danwent> I don't feel all that strongly here, other than I don't want something that makes writing a simple plugin complex.
22:43:14 <danwent> salv: consistency is important as well.
22:43:26 <RamD> salv: +1 on consitency
22:43:37 <RamD> #2 will help us here, right?
22:43:46 <salv> My proposal is to leave the decision on sync/async behaviour to the plugin
22:43:46 <SumitNaiksatam> approach #2 guarantees consistency
22:43:59 <salv> and modify the API such as it guarantees consistency at the same time
22:44:17 <somik> +1 salv
22:44:29 <salv> I think we can achieve it by stating that the API "does not guarantee" the completion of an operation when the response is returned
22:44:33 <RamD> salv: is that explained in the spec?
22:44:44 <somik> what does consistency mean in terms of async vs. sync
22:44:53 <salv> RamD: not yet, that why we are discussing it :)
22:44:58 <somik> is there a specific use-case?
22:45:25 <SumitNaiksatam> somik: consistency of sync vs async behavior
22:45:25 <RamD> I think that's the important thing...how does API can ensure consistency for a plug in behaviour not clear for me
22:45:31 <salv> Let's look at consistency from the user perspective. The user does might not have any idea about the plugin.
22:46:02 <Tv_> possible confusion in the air.. there's consistency "all apis look the same" and consistency "acid data integrity guarantees"
22:46:17 <somik> uptil the client gets the response it wants, why does the client care
22:46:19 <Tv_> i think the original meant #1, i fear some people are reading it as #2
22:46:30 <somik> Tv_ I agree that we are after consistency #1
22:46:42 <salv> Tv_: right. I'm talking of consistency as "api always look the same despite of what behaviour is actually being employed"
22:47:00 <RamD> solv: that's given, right? :-)
22:47:27 <RamD> we should not have # of APIs based on plugin one Quantum core API thats it.
22:47:45 <salv> So in the same way in which the user is not bothered by the actual plugin Quantum uses, the user will not care at all whether it is synchronous or asynchronous
22:47:51 <Tv_> i think you should say something like "plugins are expected to respond in <1sec. any part of plugin api that can't consistently do this should be designed to be async, following the calling conventions of the blahblah openstack api"
22:47:59 <salv> As long as the API faithfull tells what's happening.
22:48:17 <somik> salv: +1
22:48:34 <Tv_> something like, you get handed back a token, you can poll the status with the token, whatever is used by e.g. vm creation and slow operations like that in openstack
22:48:44 <salv> so we can add a "resource status" concept similar to what is done in nova
22:48:54 <salv> Tv_: exactly.
22:48:55 <RamD> Tv_ +1
22:49:53 <salv> then, if the plugin is sync, then the status will be immediately "available", otherwise it will be in "build" or "provisioning" for a while, but the user can check it with the token (or the resource id itself)
22:50:08 <salv> I think this will give the API user the consistency it needs.
22:50:10 <Tv_> i'd hate to force all plugin ops to go through that extra token phase
22:50:14 <Tv_> let simple things be simple
22:50:26 <salv> Tv_: sure
22:50:40 <salv> that would not apply to GET operations, for instance.
22:50:42 <somik> I think we should make things simple and add this complexity in E release if needed, after design summit
22:50:48 <danwent> having it be a "status" on a resource can help things be simple
22:51:13 <salv> the status concept can easily be implemented within diablo timeframe
22:51:13 <danwent> similar to salvatore's email
22:51:35 <Tv_> oh here's one trap i've seen people stumble in; if you can use an async api as if it were sync, and it works if you're slow enough, people *will* screw it up
22:51:50 <Tv_> as in, "it worked when i typed it in the repl!"
22:52:00 <Tv_> so i prefer it when an async api looks different enough
22:52:25 <Tv_> like, if you "create" something, and you're supposed to poll until it's ready before you use it elsewhere
22:52:29 <Tv_> people will forget to poll
22:53:04 <somik> Tv_ agreed, async clients are a lot more complicated therefore we dont want quantum clients to go through that complexity if its not needed.
22:53:29 <somik> If its needed by a plugin and the customer needs that plugin, they can still do the complicated plugin
22:54:07 <salv> somik: do you recommend we should leave things as they are for the diablo release?
22:54:26 <SumitNaiksatam> somik: my question as well
22:54:38 <somik> I think that would help us polish things out and let some end-customers try Quantum easily and then we can iterate
22:54:38 <RamD> somik: I think we are agreeing on the "status" level "async" for now..let's start with that.
22:55:03 <RamD> let me take a look at the spec as well and send in my comments
22:55:11 <somik> RamD: that would be adding something to the API that's not agreed upon aka "feature creep" ;)
22:55:35 <RamD> somik:oops I thought we are converging :-)
22:55:43 <salv> somik: reasonable enough
22:56:36 <somik> and the plugins can still implement async behavior, the API changes for async we can iterate on after we get some real feedback
22:56:41 <salv> let's consider the scenario where the API stays as it is. There could be incosistencies. Let's see if those incosistencies are serious enough.
22:57:28 <salv> Example: an asynchronous plugin might do stuff with PHY switches, virtual switches, and other network appliances. When you create a network
22:57:41 <RamD> salv: I'm for the "resource status" based API behaviour
22:57:49 <SumitNaiksatam> somik: if left to the plugin to decide whether sync or async what behavior would the client expect?
22:58:04 <ying> somik: if we do async call without API change, how can user check whether it's done or not?
22:58:04 <salv> you can think your network is there, but actually the plugin is still provisioning it. The API client will try for instance to do a plug interface call and that will fail.
22:58:25 <SumitNaiksatam> client has to be written with one or the other approach in mind
22:58:30 <RamD> as a starter we leave that to "plugin" to support the behaviour and as we move forward we can consider on API layer in future
22:58:37 <salv> ying: that's my point as well. Client will think resource is ready, but it's still being provisioned
22:59:14 <somik> the plugin responds within a reasonable HTTP timeout.. the client doesn't care about the rest.
22:59:35 <RamD> ying: +1
22:59:49 <salv> somik: you mean the plugin responds when the provisioning of the resource is complete?
22:59:53 <RamD> quit
23:00:07 <salv> RamD: bye
23:00:14 <danwent> the plugin does not necessarily need to do all of its work before returning.
23:00:19 <somik> salv:  thats one approach in the current framework
23:00:37 <danwent> for example, on a create network, it need only store the notion that the network exists.
23:01:02 <ying> somik: user needs to care, as after creating network, he will use it and need to make the provisioning is done
23:01:16 <salv> but isn't there a difference between a network that "exists" and a network that is "available"?
23:01:16 <SumitNaiksatam> danwent: that implies plugin its implementing async behavior
23:01:26 <danwent> Sumit: yup
23:01:53 <somik> SumitNaiksatam: plugin abstracts the complexity and we expose a simpler model to user for now
23:01:58 <danwent> my guess is that most complex plugins will do that already.
23:02:01 <salv> I think plugin should not be forced to do either async or sync. They can do whatever the developers thinks it's better for them! :)
23:02:19 <danwent> salv: agreed
23:02:42 <SumitNaiksatam> salv: in that case framework should implement async behavior
23:02:45 <salv> My point actually is not about the plugins at all. My point is that there is a difference between a resource which exists and a resource which can be actually used
23:02:46 <somik> salv: +1
23:02:54 <ying> good to have that flexibility, but need a way to let user know it's sync or async api
23:03:17 <salv> think about a server in nova. You cannot suspend an instance if it is not running, can you?
23:03:17 <ying> plus, for async api, needs a way to let user know whether the resource is ready
23:04:00 <danwent> I would be in favor of exposing some kind of status, which in general indicates whether the plugin is "actively managing" the port.
23:04:16 <somik> thats the plugin design issue, one way the plugin can do this is by handling asnc backend with a sync API, which is the current API
23:04:19 <danwent> this status may be down because it hasn't been provisioned yet, or because a switch was down, or for a host of other reasons.
23:04:48 <salv> ying: for "async api" you mean the API layer uses something like a message queue to dispatch a call to the plugin?
23:05:08 <danwent> networks are pretty dynamic things, and in general we will need a way for someone to check if the logical model is currently correctly mapped to the real world.
23:05:19 * salv needs some more coffee
23:05:20 <somik> salv: that would definitely be out of scope for diablo ;)
23:05:39 <danwent> ok, sounds like there's enough discussion here that we should let salvatore get some sleep
23:05:47 <ying> salv: async api means async call, no matter we do it in api layer or plugin layer
23:06:00 * salv had his coffee, now he feels like it's 9AM
23:06:13 <ying> I just want to say, for async mechanism, we need a way to let user check whether it's done or not
23:06:39 <salv> ying: good, we're on the same page. Do you think the concept of "resource status" will be enough?
23:06:58 <ying> salv agree
23:06:58 <somik> ying: the current API is sync, but the plugin can handle async implementation, and we can still work with current sync api.
23:07:39 <ying> somik: we still need extend current api by having status checking, right?
23:07:40 <salv> somik: agreed. But as ying says, we need a mechanism to check whether the operation has completed or not. Do you agree on that?
23:07:53 <danwent> salv: how about you send out a proposal for the resource status and people see if it handles their use cases?
23:08:33 <salv> danwent: will do (actually I think I already did it, but the proposal was not detailed enough probably)
23:08:42 <SumitNaiksatam> danwent: +1, salv: a high level example will help
23:08:53 <SumitNaiksatam> take create_network() for instance
23:08:53 <ying> +1
23:08:58 <danwent> #action salvatore to send out more detailed thoughts on resource status
23:08:58 <somik> we are over time limit, I dont want to hold people off for long, we can do email discussion, that would be better
23:09:07 <SumitNaiksatam> agree
23:09:15 <ying> k
23:09:24 <SumitNaiksatam> thanks salv for shepherding this!
23:09:30 <danwent> ok, this is a great discussion :)
23:09:36 <danwent> we good to end the log?
23:09:47 <ying> yes;-)
23:09:57 <danwent> #endmeeting