19:05:02 <kgriffs> #startmeeting marconi
19:05:03 <openstack> Meeting started Thu Feb 14 19:05:02 2013 UTC.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:05:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:05:06 <openstack> The meeting name has been set to 'marconi'
19:05:13 <kgriffs> Hey folks
19:05:18 <kgriffs> let's get this party started
19:05:35 <kgriffs> #topic Review actions from last time
19:05:55 <kgriffs> So, did anyone learn about PATCH support in major web servers?
19:06:15 <kgriffs> I'm not seeing wkharold in the room
19:06:51 <cppcabrera> wkharold left this link last time: http://weblog.rubyonrails.org/2012/2/25/edge-rails-patch-is-the-new-primary-http-method-for-updates/
19:07:28 <cppcabrera> "Yes, it should. I have personally tried Apache, nginx, Phusion Passenger, Unicorn, Thin, and WEBrick. They all understood PATCH requests out of the box." - from the link, Feb. 25, 2012
19:07:34 <kgriffs> rock on
19:07:44 <kgriffs> sounds like we are good on that front, then
19:08:04 <flaper87> ₥e got here a bit late
19:08:22 <flaper87> and forgot to use the / in the last message
19:08:26 <kgriffs> #note PATCH is well-supported in servers, client libs TBD
19:08:26 <flaper87> ^^
19:08:28 <cppcabrera> You haven't missed much. We just started. :)
19:08:33 <flaper87> cool
19:09:14 <kgriffs> I'd be surprised if we had issues with any of the common HTTP client libs.
19:09:35 <flaper87> kgriffs: requests supports all that
19:09:51 <kgriffs> sweet. I <3 Requests
19:09:59 * flaper87 loves it too
19:10:11 <kgriffs> httpie is pretty awesome too
19:10:14 <kgriffs> anyway...
19:11:42 <flaper87> so, what were we discussing?
19:12:10 <kgriffs> #topic API - Post Messages
19:12:19 <kgriffs> https://etherpad.openstack.org/queuing-api
19:12:53 <kgriffs> so, last time we left off talking about what the body of a message should be
19:13:47 <kgriffs> As long as it is valid JSON and within some max size (say, 64K), I think we should allow it.
19:14:34 <kgriffs> Any objections?
19:15:17 * kgriffs watches a tumbleweed roll by
19:15:20 <cppcabrera> Sounds good to me. The max size sounds like a good parameter to leave to configuration time.
19:15:20 <flaper87> no
19:15:28 <flaper87> no objections, I mean
19:15:36 <treeder> +1
19:16:14 <kgriffs> #agreed Allow arbitrary bodies, as long as it is valid JSON and under a certain size.
19:16:17 <treeder> cppcabrera: i think the restrictions have to be set
19:16:32 <treeder> as part of the api
19:16:39 <kgriffs> OK. Max size?
19:16:58 <treeder> +1 on 64k
19:17:27 <ametts-atl> Now that we've confirmed PATCH is widely supported, is there anything we need to re-review from last week?
19:17:35 <flaper87> 64kb sounds reasonable
19:17:57 <cppcabrera> +1 on 64KB, as well.
19:18:51 <kgriffs> so, that's about 16K messages per GB
19:19:14 <kgriffs> If clients max out the size
19:19:24 <kgriffs> seems like most messages would be a lot smaller
19:20:19 <kgriffs> #agreed Specify max message size as part of the API, make it 64 KiB
19:20:33 <kgriffs> OK, so one more thing on this before we move on...
19:22:02 <kgriffs> When a client posts a message, the server decorates it with some metadata, such as an ID.
19:23:04 <kgriffs> so, nesting the "body" element in the post document mirrors the nested structure of the message when it is retrieved
19:23:38 <kgriffs> We could get rid of the nesting for POSTs, but I'm not sure how much it matters either way.
19:25:00 <kgriffs> Actually, maybe we should, because the way the spec is written now would imply that a client could include, e.g., "id" if they wanted, but the server should generate that IMHO.
19:25:15 <cppcabrera> I think eliminating the nesting would be cool to avoid having to do something like: msg['body']['content'] vs. msg['content'] from a json parsing POV.
19:25:18 <kgriffs> Any objections to hoisting "body" up one level.
19:25:34 <kgriffs> (in the POST)
19:25:42 <cppcabrera> No.
19:25:43 <kgriffs> let me edit the etherpad
19:25:52 <treeder> looking...
19:26:25 <flaper87> mmh
19:27:35 <treeder> Are you saying just post the content directly with no wrapper?
19:27:59 <kgriffs> almost. The only wrapper is an array.
19:28:10 <treeder> oh, i see
19:28:36 <treeder> my only concern is it doesn't leave much room for improvement
19:29:01 <treeder> can't add new features or anything to the messages
19:29:03 <kgriffs> Or we could say, it has to be an array of objects
19:29:16 <kgriffs> good point
19:29:42 <treeder> and i'm a big fan of future proofing api's
19:29:53 <kgriffs> +1
19:30:16 <flaper87> I think it, as it's in the wiki, is good
19:31:06 <kgriffs> #agreed We say it is an array of these objects and we can even have a JSON schema that we validate against, and technically this is a different media type that what you get back from a GET.
19:31:20 <flaper87> one thin, tho
19:31:22 <flaper87> thing
19:31:24 <kgriffs> shoot
19:31:56 <flaper87> in that example, what does event and backupId mean?
19:32:04 <flaper87> just random parameters used for the example?
19:32:11 <kgriffs> correct
19:32:26 <kgriffs> could be anything that's valid JSON
19:32:29 <flaper87> we should maybe create some context around those examples
19:32:31 <flaper87> ok
19:33:03 <flaper87> what I wanted to be sure is that those paremeteres werent part of a, lets say, header for that body
19:33:08 <kgriffs> yeah, this spec isn't great documentation; I imagine we'll have something a lot friendlier for devs to reference.
19:33:11 <flaper87> like content-length or whatsoever
19:33:14 <kgriffs> oic
19:33:45 <flaper87> because, imho, that wouldn't be consistent with having a "body" key in the root dict
19:34:53 <kgriffs> yep
19:35:08 <kgriffs> OK, so anything else for posting messages?
19:37:45 <flaper87> nopoe
19:37:47 <kgriffs> #topic API - Get Messages
19:37:48 <flaper87> arg
19:38:24 <kgriffs> question, comments, rude remarks? ;)
19:39:14 <kgriffs> One thing that probably isn't clear is why marker is decoupled from message IDs.
19:39:31 <kgriffs> The reason is that it provides more options for storage implementations
19:40:34 <kgriffs> We can chat about it more in a future meeting, when we start talking storage drivers.
19:41:10 <cppcabrera> +1 for the decoupling effort.
19:41:15 <kgriffs> Of course, marker may be a message ID, but client's shouldn't know any better.
19:41:41 <treeder> why the userAgent in response?
19:42:35 <kgriffs> is it helpful to know what client sent the message?
19:42:51 <treeder> hmm, not sure
19:43:32 <treeder> also, href is sort of redundant in the message
19:43:39 <flaper87> mmh, don't think that's useful
19:43:49 <kgriffs> Actually, if we add message signing eventually, it would be necessary to know who signed it to look up the public key
19:44:22 <cppcabrera> Hmmm.. what if the user-agent was just embedded in a message? Leave it up the application to generate it and include it?
19:44:27 <kgriffs> How about taking out userAgent for now and adding it back later if needed?
19:44:55 <cppcabrera> Sounds good to me.
19:45:14 <treeder> +1
19:45:19 <flaper87> +1
19:45:21 <flaper87> makes more sense
19:45:22 <kgriffs> #agreed Drop userAgent in message header, revisit later if needed
19:45:34 <ametts-atl> Is there a potential race condition between getting messages and claiming them?  Would you typically get first, then claim, or vice-versa?  Do we need an option to do both in one operation?
19:45:48 <flaper87> lets try not to overthink from the beggining
19:45:54 <kgriffs> You would claim first
19:45:58 <flaper87> we're doing a good job so far
19:46:14 <kgriffs> claim actually returns a list of affected messages
19:46:32 <kgriffs> doing a regular GET would be more for pub-sub or an auditor
19:47:07 <treeder> Get Messages and Claim should be pretty much the same functionality, but claim "claims" them right?
19:47:23 <ametts-atl> Does claim include the message content, or just the ids?
19:48:04 <kgriffs> Same as GET, you get the whole enchilada
19:48:21 <ametts-atl> gotcha - ok
19:48:21 <flaper87> lol @ enchilada
19:48:35 * kgriffs missed lunch today
19:48:44 <treeder> hehe
19:48:48 <flaper87> that explains everything
19:48:50 <flaper87> :P
19:48:57 <treeder> I think that should be part of the spec
19:49:00 <kgriffs> so, let's finish up with GET
19:49:11 <flaper87> treeder: enchilada ?
19:49:13 <flaper87> :P
19:49:14 <treeder> ya
19:49:16 <treeder> ;)
19:49:52 <treeder> re: href: seems redundant?
19:49:53 <cppcabrera> enchilada - cloud queuing for those that missed the free lunch.
19:50:03 <treeder> haha
19:50:10 <flaper87> hehe
19:51:41 <kgriffs> So, I'm OK with dropping the per-message href, assuming the URI template for getting a single message is defined somewhere, maybe in a home document.
19:52:07 <treeder> you have Get a Specific Message endpoint
19:52:23 <kgriffs> yes
19:52:27 <treeder> user already has queue name and message id, so easy to make that href
19:52:33 <kgriffs> yep
19:52:49 <kgriffs> Any RESTafarians in the room who object?
19:53:33 <kgriffs> #agreed Drop per-message href field
19:54:11 <kgriffs> Anything else for "Get Messages"?
19:54:18 <flaper87> nope
19:54:21 <treeder> nope
19:54:34 <kgriffs> moving on...
19:55:06 <kgriffs> this will be our last topic for today.
19:55:11 <kgriffs> #topic API - Claim Messages
19:58:37 <flaper87> so, basically the claim is a patch with a clain action in it
19:58:41 <flaper87> op*
19:59:47 <kgriffs> right, the client patches x messages with a "claim" and the server filters any messages that already have a claim.
20:01:10 <flaper87> seems to make sense
20:01:24 <kgriffs> As long as all workers use this method rather than GET, it works.
20:01:57 <kgriffs> Of course, an auditor or debugger or whatever could use GET to watch things go by
20:02:55 <kgriffs> well, looks like we are out of time. You guys want to sleep on this one and pick up the discussion next week?
20:03:27 <kgriffs> maybe go have some enchiladas…
20:03:32 <cppcabrera> Any new action items? We've still got the three pending on Trello.
20:03:35 <flaper87> kgriffs: yeap, I'd like to think a bit more on it
20:03:48 <flaper87> and also, should complete my zmq part (I was out of office)
20:03:49 <treeder> ya, sounds good
20:03:57 <treeder> i'm getting hungry. ;)
20:03:59 <kgriffs> oic
20:04:19 <flaper87> treeder: if kgriffs missed his lunch you should do the same
20:04:24 * flaper87 runs away
20:04:26 <flaper87> hehe
20:04:37 <kgriffs> flaper87: I'd appreciate your thoughts on how to abstract out the protocol in the implementation (make it pluggable). Maybe could be it's own blueprint.
20:05:04 <kgriffs> what was the trello link again?
20:05:13 <treeder> https://trello.com/board/openstack-marconi/511403287d138cd6200078e0
20:05:37 <flaper87> kgriffs: sure, I've that queued, sorry I didn't make it for this meeting
20:07:02 <kgriffs> no worries, man. Really appreciate your help. Also, it would be cool if you could help me start brainstorming about storage drivers.
20:07:29 <kgriffs> (specifically, what backend is a good first target)
20:07:38 <flaper87> kgriffs: yep, we could make that in a new wiki page
20:07:59 <flaper87> kgriffs: my first thought is mongodb
20:08:04 <flaper87> but we can dig that a bit more
20:08:25 <kgriffs> Sure thing.
20:08:39 <kgriffs> I'll schedule it for a future meeting.
20:08:44 <flaper87> +1
20:09:27 <kgriffs> OK guys, let's wrap it up.
20:10:15 <kgriffs> next week, same time, same bat-channel. If we can get enough done, we can go back to twice a month chats.
20:10:30 <kgriffs> #endmeeting