19:05:11 <kgriffs1> #startmeeting Marconi
19:05:12 <openstack> Meeting started Thu Jan 24 19:05:11 2013 UTC.  The chair is kgriffs1. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:05:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:05:15 <openstack> The meeting name has been set to 'marconi'
19:05:37 <kgriffs1> #topic Review last week's actions
19:05:50 <kgriffs1> #info kgriffs to kick the tires on Pecan
19:06:09 <kgriffs1> So, I did play around a bit with Pecan.
19:06:31 <kgriffs1> It has a lot of cool stuff, but is a bit slow for my tastes.
19:07:33 <kgriffs1> We are doing a hack day today in my office and some people are playing with various web frameworks, including Falcon.
19:08:06 <kgriffs1> I'm hoping to get some good feedback from everyone to see what they like and don't like about each framework, to feed into the discussion around what we should use for Marconi.
19:08:58 <treeder> with a tagline like that, i'm surprised. ;)
19:09:25 <kgriffs1> At this point, Falcon is still a bit of a science project, so the jury's still out.
19:09:44 <kgriffs1> heh
19:09:56 <treeder> referring to Pecan's tagline
19:10:02 <kgriffs1> exactly.
19:11:23 <kgriffs1> I did a fairly simple benchmark where I set a body, read a single query parameter, and also have a param as part of the URI path. Pecan and Flask were about the same speed, while Bottle was several times faster per request, and Falcon was about twice as fast as Bottle.
19:11:45 <kgriffs1> Granted, Falcon does less than the other frameworks, but at least you don't have to pay for what you don't use. Or something like that.
19:11:57 <kgriffs1> We'll have to see how things go with more realistic tests.
19:12:34 <kgriffs1> Anyway, that something simmering on the back-burner while we figure out the API.
19:12:38 <treeder> Can you post link to Falcon?
19:12:43 <treeder> and what was yours again?
19:12:50 <kgriffs1> http://github.com/racker/falcon
19:13:54 <kgriffs1> BTW, the benchmark was hitting the WSGI callable directly, not going through any webserver.
19:14:56 <kgriffs1> We're talking about a performance range of 2 thousand req/sec vs 30 for Falcon on my MBP.
19:15:13 <kgriffs1> Kinda crazy. I just don't want the framework to be the bottleneck in terms of latency.
19:15:38 <treeder> ya, that's pretty significant
19:17:18 <kgriffs1> #topic Review the g1 milestone
19:18:07 <kgriffs1> So, I just wanted to take a minute and review what our first milestone looks like. It has "g" in the name for "Grizzly" - I'm hoping to get at least a few of these done before the next summit.
19:18:08 <kgriffs1> https://launchpad.net/marconi/+milestone/grizzly-1
19:18:35 <kgriffs1> #info Set up Marconi wiki, repo, Launchpad, etc.
19:19:03 <kgriffs1> That's pretty much all done at this point. The Wiki could use some refactoring, but it's there.
19:19:15 <kgriffs1> #info Define v1.0 functional and non-functional requirements
19:20:08 <kgriffs1> I've got a rough draft of this that is mostly based on the meeting we had last summit to kick off the project.
19:20:09 <kgriffs1> http://wiki.openstack.org/marconi/specs/grizzly
19:20:48 <kgriffs1> In a future meeting we should refine it, but it's probably best to get some code out there rather than spend too much time writing specs.
19:21:16 <treeder> +1
19:21:40 <treeder> 1) define MVP API 2) code
19:21:47 <kgriffs1> #info Define v1.0 API
19:21:56 <kgriffs1> +1
19:23:15 <kgriffs1> So, the API draft is out there but has a bunch of TBD on it that I'm hoping we can work through pretty quickly. We started last week, and will keep going today.
19:23:48 <kgriffs1> #info Create a skeleton implementation and test suite
19:24:30 <kgriffs1> So, this will probably go into the next milestone. I'm thinking we will just close out g1 once we have the API (mostly) figured out.
19:25:10 <kgriffs1> (also I've got some interviews coming up in the next couple weeks to hire some full-time devs)
19:25:47 <kgriffs1> #action kgriffs1 to move coding to g2
19:26:11 <kgriffs1> Any questions/comments before we move on?
19:27:02 <kgriffs1> #topic Decide whether to support tags or some other fanout mechanism, or just stick with N-queue approach (for v1.0 at least)
19:27:37 <kgriffs1> So, what do you guys think is the way to go for v1.0?
19:28:14 <treeder> what's N-queue approach?
19:28:47 <edsrzf> I think that's your preferred approach
19:29:00 <kgriffs1> What I mean by that is to, say, use two queues rather than nested namespaces.
19:29:08 <treeder> ahh, ya
19:29:36 <kgriffs1> Also, not having tags means all filtering happens either by using multiple queues or doing it in the client firehose-style.
19:29:58 <treeder> N-queue approach is definitely the simplest
19:30:06 <kgriffs1> +1
19:30:11 <treeder> I also think we should have concrete queues regardless of whether tags are added on
19:30:25 <kgriffs1> #agreed Have concrete queues
19:30:41 <treeder> So as a base point, I think the N-queues approach is best
19:30:52 <treeder> fanout and tags can be added on top
19:31:10 <kgriffs1> Any objections?
19:34:50 <kgriffs1> #agreed No tags or fanout for v1
19:34:52 <treeder> brb
19:34:56 <kgriffs1> OK
19:36:37 <kgriffs1> #topic Consider splitting the API into two namespaces, one for work queuing and one for eventing
19:37:27 <kgriffs1> A few Pros off the top of my head:
19:37:34 <kgriffs1> #info Pro: Allows for different scaling architectures, storage backends, etc.
19:37:42 <kgriffs1> #info Pro: Simplifies semantics for the two major use cases
19:38:11 <kgriffs1> #info Con: Remove affordances - clients won't be able to mix/match semantics (we would be effectively removing features)
19:38:25 <kgriffs1> Thoughts?
19:39:42 <kgriffs1> By "eventing", I mean notifications and RPC style communication
19:40:46 <kgriffs1> By work queuing, I mean semantics like those provided by IronMQ and SQS.
19:42:10 <edsrzf> So are we talking about separate semantics, or separate namespaces?
19:43:19 <kgriffs1> The API spec as currently written lets you do all of the above under a single namespace. The way the client interacts with a given queue determines the contract, essentially.
19:43:30 <kgriffs1> http://wiki.openstack.org/marconi/specs/api/v1
19:44:28 <kgriffs1> So, I could have a queue with work items or metering data documents in it, and have consumers pulling those off in a way that hides those documents from other consumers (similar to the way SQS works).
19:45:41 <kgriffs1> But an observer could peek at the messages on the queue if they wanted to - it would be up to the developer to ensure the observer is passive.
19:46:50 <kgriffs1> The alternative would be to basically have two different queue types and clients aren't able to mix-and-match semantics.
19:47:12 <treeder> what's the alternative way?
19:47:14 <kgriffs1> I guess you could even have two completely different services for each type.
19:47:35 <treeder> a consumer could take a message and it wouldn't be hidden?
19:48:02 <kgriffs1> They couldn't take a message per say, but they could peek and promise to only be a passive observer.
19:50:04 <treeder> right, so that's essentially just a different function/endpoint
19:50:09 <treeder> GET vs PEEK
19:50:17 <treeder> peek doesn't reserve the message
19:51:13 <kgriffs1> as written, the spec would allow PEEKing at messages that are reserved AKA hidden/locked.
19:51:43 <treeder> where is that in the spec?
19:51:47 <treeder> can't find it
19:52:16 <kgriffs1> sorry, it's a ways down. Search for "Lock Messages"
19:52:40 <kgriffs1> http://wiki.openstack.org/marconi/specs/api/v1
19:52:57 <treeder> oh, so GET messages is like peek
19:53:09 <treeder> and lock is more like a get?
19:53:15 <kgriffs1> yeah. I was attempting to keep state on the client as much as possible.
19:53:16 <treeder> in terms of SQS/beanstalk/IronMQ terms
19:53:38 <kgriffs1> right. But GET isn't really supposed to change state.
19:55:36 <kgriffs1> I jut had a thought. The body of the post could be a list of queues to pull from. That reduces load on the server.
19:56:36 <kgriffs1> Although that means we would probably want a batch GET across multiple queues, and that could get weird.
19:59:10 <kgriffs1> SQS gives you back some kind of transaction ID or something for each batch of messages, right?
19:59:22 <kgriffs1> (it's been a little while since I looked at it)
19:59:33 <kgriffs1> (looking at docs)
20:00:25 <kgriffs1> oic.
20:00:26 <kgriffs1> ReceiptHandle
20:00:30 <kgriffs1> It's per message
20:00:31 <treeder> don't think there's a transaction
20:00:33 <treeder> ya
20:00:53 <kgriffs1> What does IronMQ do?
20:02:10 <treeder> in terms of?
20:02:19 <treeder> lock vs peek?
20:02:26 <treeder> very similar to SQS
20:02:39 <treeder> GET messages locks/reserves messages for the timeout period
20:02:48 <treeder> we also have a /peek endpoint
20:02:48 <kgriffs1> no, I mean, is their some kind of handle for, e.g., renewing the timeout?
20:02:55 <treeder> each message has an id
20:03:09 <kgriffs1> # IronMQ has a /peek endpoint
20:03:11 <treeder> we have touch/release for renewing the lock
20:03:17 <treeder> or releaseing the lock
20:03:19 <kgriffs1> oic
20:03:20 <treeder> releasing
20:03:45 <kgriffs1> Do you think releasing a lock manually is something Marconi should have?
20:03:59 <treeder> probably
20:04:01 <kgriffs1> #IronMQ has touch/release for renewing the lock
20:04:21 <treeder> use case is: "i got a message that I can't deal with right now so I'll put it back on the queue for someone else"
20:04:28 <kgriffs1> oic
20:04:47 <treeder> and you can release with a delay too meaning it won't be available for a while
20:04:56 <treeder> just as you can post a message with a delay
20:05:03 <kgriffs1> So it seems like doing the locks per-message is better than as a batch?
20:05:08 <treeder> touch resets the timeout
20:05:39 <treeder> hmmmm
20:05:46 <kgriffs1> or does it matter?
20:05:53 <treeder> i don't know if there's any sane way to batch them?
20:06:15 <kgriffs1> I guess you just assign the same lock ID to a group of messages.
20:06:24 <kgriffs1> (not normalized)
20:06:24 <treeder> ok
20:06:59 <treeder> so if I take 100 messages, I have to deal with them all, no matter what?
20:07:06 <treeder> like I can't fail on one message
20:07:15 <treeder> without affecting the whole batch
20:07:29 <kgriffs1> Well, if you fail on one and keep going, that one will eventually time out and go back into the pot
20:07:30 <treeder> just thinking out loud
20:07:45 <kgriffs1> or you could manually release it
20:07:46 <treeder> if the lock is on a batch though
20:07:52 <treeder> wouldn't the whole batch have to go back on the pot
20:07:53 <treeder> ?
20:08:28 <kgriffs1> oic. You would keep renewing until you are done with the entire batch, in the meantime another worker could have been picking up the ones you failed/skipped.
20:09:28 <treeder> eg:
20:09:34 <treeder> server X grabs 100 messages
20:09:42 <treeder> server crashes after processing 50 messages
20:09:58 <treeder> lock expires and all 100 messages would have to go back on?
20:10:19 <kgriffs1> no, the server would presumably have deleted the 50 messages before crashing
20:10:29 <treeder> ahh, right
20:10:30 <kgriffs1> (by ID, not lock)
20:10:45 <treeder> got it, so it deletes by message id?
20:10:50 <kgriffs1> yeah
20:11:03 <treeder> ok
20:11:05 <kgriffs1> maybe also have to include the lock id for safety
20:11:08 <treeder> you say tags in the spec
20:11:13 <kgriffs1> yeah
20:11:14 <treeder> is a tag a message id too?
20:11:36 <kgriffs1> no, that's a holdover from the idea to not use concrete queues
20:11:43 <kgriffs1> I need update it
20:11:53 <kgriffs1> so, would be...
20:12:38 <kgriffs1> POST {base_url}/messages/{queue}/locks{?limit}
20:12:42 <kgriffs1> or something like that
20:12:48 <kgriffs1> oops
20:13:09 <kgriffs1> POST {base_url}/{queue}/locks{?tags,limit}
20:13:19 <kgriffs1> anyway, you get the idea - it will take some refinement
20:13:34 <kgriffs1> dang. Sorry - left tags in again. /me blushes
20:15:09 <kgriffs1> So, I'm thinking that per-message receipt handles or whatever you call them is a little more flexible.
20:16:03 <treeder> ya, I think so
20:16:18 <treeder> message id's to keep it simple. ;)
20:16:35 <kgriffs1> #agreed use per-message lock/receipt IDs or just message IDs directly
20:17:38 <kgriffs1> I guess that's an argument for having separate queue types or namespaces or whatever.
20:18:57 <kgriffs1> On the other hand, having to provide a handle of some sort when deleting a message (in addition to it's ID) does mitigate a race condition between a lock expiring and a message being deleted.
20:19:38 <treeder> handle to a lock?
20:20:24 <kgriffs1> Another option would be to require clients put a UUID string in the request (possibly User-Agent header), then when a client gets a batch, the messages are tagged with that UUID so only they can delete them if they are still locked.
20:20:26 <kgriffs1> yeah
20:21:58 <kgriffs1> #action kgriffs1 to update API spec to remove tags, add concrete queues
20:22:14 <treeder> need to think about that batch lock thing
20:22:18 <kgriffs1> OK
20:22:26 <kgriffs1> Well, the meeting is running pretty long
20:22:46 <kgriffs1> maybe we should sleep on the locking as well as the question of whether to have a unified queue type/namespace.
20:22:52 <treeder> ya, well we're making progress so that's good.
20:23:16 <kgriffs1> definitely.
20:23:39 <treeder> alright, I have to run to another meeting, talk to you next week.
20:23:45 <kgriffs1> sounds like a plan
20:24:10 <treeder> maybe we should get a marconi mailing list going or something?
20:25:06 <kgriffs1> Well, the way OpenStack seems to like to do it is put everything in openstack-dev with a tag in the subject line, i.e., [marconi].
20:25:22 <kgriffs1> #agreed Start some discussion threads on the mailing list
20:25:42 <kgriffs1> appreciate everyone's help.
20:26:25 <kgriffs1> #action everyone think about locking semantics
20:27:00 <kgriffs1> #action everyone think about / discuss  the implications of a unified queue type/namespace and come up with a recommendation for next mtg.
20:27:32 <kgriffs1> #endmeeting