16:03:31 <kgriffs> #startmeeting marconi
16:03:32 <openstack> Meeting started Mon Oct  7 16:03:31 2013 UTC and is due to finish in 60 minutes.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:03:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:03:35 <openstack> The meeting name has been set to 'marconi'
16:03:43 <kgriffs> #link https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda
16:04:08 <kgriffs> No actions to speak of from last time
16:04:21 <kgriffs> so, let's just skip that one today
16:04:24 <kgriffs> next up...
16:04:30 <kgriffs> #topic 	•	Review Graduation BPs/Bugs
16:04:42 <kgriffs> #link https://wiki.openstack.org/wiki/Marconi/Incubation/Graduation
16:05:10 <kgriffs> so… I think we are still stalled on the SQLAlchemy driver?
16:05:34 <alcabrera> I believe so. flaper87?
16:05:39 <kgriffs> waiting on a certain someone to get back from holiday?
16:05:59 <alcabrera> that's what I last heard. :)
16:07:22 <kgriffs> ok
16:07:33 <kgriffs> flaper87: client library?
16:07:37 <zyuan> what's the target guraduation date?
16:08:29 <kgriffs> I want to apply for graduation at least 6 weeks prior to Icehouse
16:08:30 <alcabrera> flaper87|afk: must be struggling with vpn again... :(
16:08:45 <zyuan> when si Icehouse... -_-
16:08:49 <fvollero> alcabrera: yeah, most probably
16:08:54 <kgriffs> The date at the top of that wiki page should give a lot of headroom
16:09:07 <kgriffs> I mean, iirc, it is a 7-8 weeks before
16:09:11 <alcabrera> zyuan: icehouse is th ename of the next openstack official release
16:09:25 <zyuan> i know. looking up when
16:09:32 <megan_w> sorry to be late, meeting ran over
16:09:35 <kgriffs> I am guessing when it will be, since the official date is not determined yet
16:09:42 <zyuan> \!!!
16:09:43 <alcabrera> afaik about the client libraries, flaper87|afk has managed to get it to the point where a low-levelm transport-agnostic API exists, and it can send requests.
16:09:45 <zyuan> well...
16:09:52 <kgriffs> https://wiki.openstack.org/wiki/Releases
16:10:12 <kgriffs> In the past it has been mid april
16:10:19 <flaper87> o/
16:10:21 <flaper87> back
16:10:23 <flaper87> alcabrera: exactly
16:10:24 <kgriffs> woot
16:10:25 <flaper87> :(
16:10:53 <kgriffs> flaper87: client progress?
16:11:12 <kgriffs> #link https://blueprints.launchpad.net/python-marconiclient/+spec/python-marconiclient-v1
16:11:36 <flaper87> kgriffs: sooo
16:11:47 <flaper87> I'm already capable fo QUERYING MARCONIIIIII!!!!
16:11:48 <flaper87> w00000000t
16:11:51 * flaper87 jumps
16:12:01 <kgriffs> QUERY ALL THE THINGS!!!!!!
16:12:09 <flaper87> As for now, I've already implemented qet_queue_metadata, queue_exists and queue_create
16:12:10 <kgriffs> :D
16:12:19 <flaper87> I'll finish implementing other queue methods and push this for review
16:12:24 <megan_w> awesome
16:12:41 <alcabrera> flaper87: that's awesome! :D
16:12:46 <flaper87> Today, all pieces started to fall in place and everything fit and worked like a charm
16:12:48 <fvollero> cool!
16:12:51 * flaper87 almost cried
16:13:18 <kgriffs> excellent
16:13:21 <flaper87> so, a more detailed update
16:13:26 <flaper87> We've 2 APIs
16:14:05 <flaper87> A lower one that expects the user to provide everything needed to succeed - Transport instance, request instances, other params based on the operation.
16:14:32 <flaper87> and a higher one that gives a ORMish API that will make it very easy and straightforward
16:14:47 <flaper87> Client(...).queue(queue_id).exists()
16:15:04 <flaper87> The higher API uses the lower API
16:15:08 <alcabrera> sweet, sweet
16:15:14 <flaper87> and they're both implemented
16:15:26 * alcabrera is looking forward to reviewing this
16:15:28 <flaper87> and that's the patch I'm about to submit
16:15:42 <flaper87> I tried to split the whole thing as much as possible
16:15:52 <flaper87> that's why most of the patches so far don't give the impression of real progress
16:16:01 <flaper87> that's it
16:16:07 <kgriffs> nice work
16:16:49 * alcabrera gives flaper87 the nutella prize
16:16:56 <flaper87> w0000000000000000000000000t
16:17:04 * flaper87 jumps 3 times
16:18:22 <kgriffs> #info flaper87 is making awesome progress on the client
16:18:28 <kgriffs> ok
16:18:43 <kgriffs> anything else to note re the remaining todo's?
16:18:56 <kgriffs> #link https://wiki.openstack.org/wiki/Marconi/Incubation/Graduation
16:19:23 <flaper87> yeah, I'm already looking into the Heat thing
16:19:26 <kgriffs> ykaplan is on holiday still?
16:19:37 <flaper87> if someone wants to take it over, I'm very happy to give that away
16:19:41 <flaper87> otherwise I'll work on that
16:19:59 <flaper87> kgriffs: nope, she's back, I was chatting with here few mins ago
16:20:03 <kgriffs> oh, cool
16:20:13 <kgriffs> #info ykaplan is back!
16:20:23 <flaper87> the state of the sqlalchemy thing is that she already defined tables and wrote tests for that, she'll submit the first patch soon
16:20:32 <kgriffs> awesome sauce
16:20:43 <flaper87> (sorry If I'm working as a proxy here, she's gone already)
16:20:52 <kgriffs> #info sqlalchemy driver started, has schema and initial tests
16:21:01 <flaper87> kgriffs: next thing will be working on the QueuesController and then MEssageController
16:21:19 <flaper87> The idea is to split the work in many patches and ease reviews
16:21:25 <kgriffs> ok
16:21:31 <alcabrera> #info sqlalchemy driver - started, good progress
16:21:31 <kgriffs> sounds like a plan
16:21:38 <flaper87> cool beans
16:21:56 <alcabrera> Our review queue is going to start growing very soon. Fun times. :D
16:22:04 <kgriffs> #info flaper87 is starting work on heat template
16:22:16 <flaper87> few words there
16:22:20 <flaper87> can I ?
16:22:24 <kgriffs> sure thing
16:22:29 <flaper87> cool
16:23:07 <flaper87> so, working on swift templates shouldn't be that hard, the hard part is that we have to support AWS Api as well, AFAIU
16:23:56 <flaper87> Meaning, and I'm don't know Heat that well nor have dug that much into it, we need to make sure that the same template used in AWS to create queues
16:24:04 <flaper87> can be used to create queues in Marconi
16:24:18 <kgriffs> oh
16:24:31 <zyuan> what about messages?
16:24:45 <kgriffs> for some reason i was thinking we wanted heat templates to deploy Marconi itself
16:24:45 <flaper87> zyuan: same thing
16:24:57 <alcabrera> kgriffs: that's what I thought, too
16:24:57 <flaper87> kgriffs: yeah, that too
16:25:00 <flaper87> :D
16:25:10 <kgriffs> hmm
16:25:12 <alcabrera> ah, so in addition, AWS-style API support, a mapping of sorts
16:25:16 <flaper87> Like I said, I'm not a Heat expert nor have dug much into it
16:25:19 <kgriffs> ok
16:25:20 <flaper87> so, lets not freak out
16:25:30 <flaper87> that's what I understood from my first overlook at it
16:25:33 <alcabrera> Yeah, that's one weird mapping. :P
16:25:37 <flaper87> lets write an action there
16:25:44 <flaper87> and I'll dig more for our next meeting
16:25:47 <kgriffs> I can connect you with our Heat dev lead if you like
16:25:53 <flaper87> and hten we can freak out
16:25:55 <flaper87> :D
16:25:57 <alcabrera> lol
16:26:32 <flaper87> kgriffs: it'd be awesome if he could help us out writing those plugins :D
16:26:37 * flaper87 tries, at least tries
16:26:39 <flaper87> :D
16:27:02 <kgriffs> flaper87: he has a team under him so there is a fair chance of that
16:27:03 <kgriffs> :D
16:27:14 <flaper87> sounds amazing :P
16:27:17 <kgriffs> ok
16:27:38 <flaper87> I'll dig on that anyway
16:27:42 <kgriffs> sure thing
16:27:46 <flaper87> and bring more info for our next meeting
16:27:51 <flaper87> so we have more to decide on
16:28:00 <alcabrera> +1
16:28:10 <kgriffs> #action flaper87 to research heat
16:28:17 <kgriffs> ok
16:28:24 <kgriffs> anything else on the current topic
16:28:25 <kgriffs> ?
16:28:40 <flaper87> not from me
16:28:50 <kgriffs> #topic Service catalog entry for Marconi
16:29:10 <kgriffs> So, somebody asked a few days ago what the catalog name was going to be
16:29:59 <kgriffs> I mean, what "name" will be
16:30:08 <kgriffs> #info http://docs.openstack.org/developer/keystone/api_curl_examples.html#get-tokens-token-id-endpoints
16:30:31 <kgriffs> and stuff
16:30:32 <amitgandhi> "name: : "marconi" ???
16:30:36 <kgriffs> i guess so
16:30:41 <kgriffs> type?
16:30:45 <kgriffs> "messaging"?
16:30:53 <alcabrera> sounds good to me
16:31:11 <flaper87> kgriffs: mmh
16:31:12 <kgriffs> anyway, I was thinking we should add that to our list of graduation things
16:31:14 <flaper87> wait
16:31:20 <flaper87> I used something for the devstack aptch
16:31:23 <flaper87> 1 sec
16:31:49 <flaper87> I used queueing as a type
16:32:00 <flaper87> which alings with queues
16:32:13 <flaper87> then we can have a notification one for notifications
16:32:32 <flaper87> https://review.openstack.org/#/c/47999/1/files/keystone_data.sh
16:32:56 <kgriffs> ah, you are one step ahead of me!
16:33:05 <flaper87> :D
16:33:10 <amitgandhi> would notifications have a type = "queueing"
16:33:15 <amitgandhi> or notifications would be its own type?
16:33:22 <flaper87> amitgandhi: its own type
16:33:25 <flaper87> IMHO
16:33:28 <flaper87> it's a different service
16:33:32 <flaper87> under the same program
16:33:36 <amitgandhi> but its a similar type
16:34:45 <ametts> If a client gets pub-sub messages from a queue and sends sms messages, is that queuing or notification?
16:35:17 <megan_w> ametts: notifications
16:35:24 <flaper87> ametts: ^
16:35:28 <amitgandhi> i see it as the same "type" of service.  just like swift and blockstorage fall into object-storage type
16:35:36 <ametts> megan_w, flaper87: but the queue itself is the same
16:35:52 <flaper87> ametts: it doesn't matter, you subscribe to the notification service
16:36:00 <kgriffs> flaper87: is there some canonical catalog that we need to contribute to for keystone deployments? Or do operators just configure them from scratch?
16:36:02 <flaper87> If I want to use keystone to discover a queue service
16:36:07 <flaper87> by using the type
16:36:24 <flaper87> and it returns a notificatio-service url, I won't be able to use that
16:36:59 <kgriffs> the only thing you might do is make it more generic, like "messaging" but it seems like that is too abstract
16:37:03 <flaper87> kgriffs: not sure, TBH
16:37:06 <megan_w> does "messaging" cover everything?
16:37:13 <megan_w> flaper87: great minds :)
16:37:56 <flaper87> We can't have a single service type to cover both services because they won't expose the same API nor bound to the same port
16:38:33 <flaper87> nor be bound*
16:38:48 <ametts> flaper87:  Okay, in that case let's revisit my earlier example...
16:38:48 <megan_w> so if they use any of the notifications api, its notifications, if not, its queuing?
16:39:12 <flaper87> megan_w: yup
16:39:21 <flaper87> ametts: cool!
16:39:23 <ametts> Same queue as before.  Different client.  This one gets all messages and OPTIONALLY claims them.
16:39:30 <zyuan> it that possible to have pure client extension to implement notification with marconi?
16:39:40 <kgriffs> I think the notifications API will be standlone
16:39:45 <kgriffs> s/will/should be
16:39:47 <amitgandhi> kgriffs +1
16:39:50 <flaper87> also, one more note, Marconi offers queues as a service, IMHO, it makes more sense to use queue* that message* for the service type
16:40:16 <flaper87> kgriffs: +1
16:40:34 <ametts> So the notifications API wraps the Queues api?
16:40:42 * ametts is confused
16:40:43 <amitgandhi> ok im pivoting towards type=queueing and type=notifications now
16:40:54 <flaper87> ametts: the notification API talks to the queue API to get messages
16:41:07 <flaper87> or at least it will
16:41:16 <flaper87> as for our last meeting with regard to that
16:41:16 <amitgandhi> ametts: notifications api optionally uses the marconi api if thats the queueing mechanism used
16:41:54 <kgriffs> seems like in that sense, notifications will eventually be it's own project/repro?
16:42:01 <ametts> Ok, but the operations are really similar, right?  I can take this offline if necessary.
16:42:02 <amitgandhi> i think so
16:42:23 <flaper87> kgriffs: it could be, yes!
16:42:38 <flaper87> ok, lets tackle this later
16:42:38 <kgriffs> #info notifications may not depend on marconi-queues
16:42:42 <kgriffs> ok, let's discuss this some more next week. I'd like to get to the queues API finalization
16:42:42 <flaper87> we can change it
16:43:18 <kgriffs> #info use type=queueing and name=marconi for service catalog
16:43:46 <kgriffs> #topic Finalize v1 API
16:44:13 <kgriffs> #link https://wiki.openstack.org/wiki/Marconi/specs/api/v1
16:44:43 * flaper87 loves how much that wiki page has grown
16:44:49 <kgriffs> ok, first off
16:45:17 <kgriffs> The last change that we are making is to formally define the type of Client-ID to be a UUID
16:45:31 <kgriffs> This was done due to confusion we have heard from early adopters
16:45:49 <kgriffs> are there any other pending changes that anyone is aware of?
16:46:02 * flaper87 thinks
16:46:03 <kgriffs> (note - I have already updated the spec per UUID change)
16:46:05 * flaper87 thinks harder
16:46:13 <ametts> Did we finally settle the 204 versus 200 with an empty list debate?
16:46:16 * flaper87 thinks even harder... ouch
16:46:26 <kgriffs> afaik, we are sticking with 204?
16:46:55 <alcabrera> 204 was my understanding (for LIST queues, messages, claimed_messages)
16:47:20 <kgriffs> flaper87: in implementing the client, has 204 posed a problem?
16:47:28 <amitgandhi> the complaint was based around clients having to do more work to handle a 204 no content result vs a 200 empty json list result
16:47:29 <kgriffs> (vs. returning an empty array)
16:47:54 <zyuan> client has to do more work to handle 204
16:48:07 <kgriffs> well, not more work, it is just more code
16:48:07 <zyuan> because 204 is not empty, it's "empty yet"
16:48:10 <amitgandhi> programmer has to do more work to make client handle it =P
16:48:11 <flaper87> kgriffs: nope
16:48:52 <kgriffs> parsing JSON vs. instantiating an empty list/array object
16:48:54 <flaper87> no issues so far, that is
16:49:02 <kgriffs> less work, lower latency all around
16:49:12 <kgriffs> but programmer has to add an "if" statement
16:49:14 <kgriffs> so
16:49:19 <amitgandhi> i like the 204 response.  its semantically correct to me
16:49:23 <kgriffs> ok
16:49:28 <alcabrera> +1 for 204
16:49:28 * flaper87 is part of the anti-if campaign
16:49:33 <flaper87> +1 for 204
16:49:38 <zyuan> kgriffs: the client must handle the special case anyway
16:49:40 <kgriffs> anyone want to vehemently object to keeping as-is?
16:50:02 <kgriffs> going once
16:50:07 <ametts> Well if flaper87 is part of the anti-if campaign, he should choose 200
16:50:20 <amitgandhi> another piece of feedback i got (not sure on accuracy of it) was use of PATCH verb
16:50:31 <amitgandhi> comment was not seen in other openstack api's yet
16:50:46 <kgriffs> yes, that is true
16:50:46 <amitgandhi> may have been a noob
16:50:53 <flaper87> ametts: nope, python is... magical :D
16:50:54 <zyuan> well, openstack'd better to have some PATCH
16:50:55 <flaper87> #link http://www.antiifcampaign.com/join-the-campaign.html
16:50:56 <kgriffs> we are more modern there
16:51:11 <kgriffs> PATCH is less ambiguous than doing a PUT and only including a partial document
16:51:19 <zyuan> +N
16:51:25 <amitgandhi> kgriffs: +1
16:51:32 * ametts thinks some people have too much time on their hands....
16:51:38 <kgriffs> that being said, PATCH is supported in all serious web servers and clients
16:51:56 <kgriffs> so...
16:52:03 <malini> & tsung folks added patch support for us in a whiff !!
16:52:05 <zyuan> NAD
16:52:25 <kgriffs> also, JSON Patch is coming
16:52:38 <kgriffs> so, we need to push the state of the art forward
16:52:39 <flaper87> and we like patches
16:52:41 <kgriffs> baby steps. :D
16:52:48 <amitgandhi> patches FTW!
16:53:06 <acabrera> lol
16:53:09 <kgriffs> so, i personally think it is fine to leave in unless a lot of devs are having to spend hours because of it
16:53:13 <malini> this doesnt sound like a meeting..everybody agrees to everything!
16:53:16 <kgriffs> …which is something I find hard to believe
16:53:17 <acabrera> PATCH + jsonschema has worked nicely for the proxy, FWIW
16:53:50 <kgriffs> ok
16:53:50 <zyuan> malini: i don't agree with you!
16:53:51 <amitgandhi> next piece of feedback was the v1/ HOME document not being standard
16:53:58 <malini> zyuan: :D
16:54:07 <amitgandhi> ie draft wrc spec
16:54:16 <amitgandhi> s/wrc/w3c
16:54:22 <kgriffs> yes. I admit there that I probably jumped the gun since the home doc isn't a totally real thing yet
16:54:43 <zyuan> not a big deal; it's fine to have something experimental
16:54:52 <amitgandhi> personally i like it
16:54:55 <kgriffs> but, considering the alternative is WADL...
16:55:00 <amitgandhi> will it pose a problem?
16:55:20 <amitgandhi> will people be like WTF!
16:55:21 <kgriffs> I have heard from several devs who are actually excited about the home doc
16:55:30 <kgriffs> nobody has said "don't do it"
16:55:41 * flaper87 loves the home doc
16:55:52 <zyuan> at least we can say
16:55:56 <kgriffs> so, once the RFC becomes a standard, we can tweak the home doc to be official
16:56:04 <kgriffs> that's my plan, anyway
16:56:09 <zyuan> read home doc and don't build your own queries beyaond home doc
16:56:19 <amitgandhi> does it differ too much from other openstack api's that marconi becomes the odd one out, or just the first one there ;-)
16:56:20 <flaper87> zyuan: +1
16:56:31 <kgriffs> heh
16:56:34 <ametts> So this means the v1 API is 100% final/official at this point?
16:56:42 <flaper87> I'm not doing that in the client yet, though. But that's the final idea
16:57:03 <kgriffs> afaik, the other projects don't have any kind of machine-readable doc, period
16:57:16 <amitgandhi> are they considering it?
16:57:28 <kgriffs> not sure
16:57:29 <flaper87> amitgandhi: keystone is, AFAIK
16:57:30 <amitgandhi> based on what they have seen with the marconi home doc , or other home docs?
16:57:38 <kgriffs> I bet they will if a lot of people find Marconi's doc useful
16:57:39 <flaper87> but, not 100% sure
16:57:40 <amitgandhi> flaper87: nice =)
16:57:48 <zyuan> kgriffs: :)
16:57:53 <amitgandhi> ok im cool with it being there
16:58:06 <kgriffs> we can always rip it out in v2 if nobody uses it
16:58:08 <kgriffs> ok
16:58:18 <amitgandhi> how would you know if nobody uses it?
16:58:23 <kgriffs> heh
16:58:33 <kgriffs> we ask public cloud operators nicely to give us some stats
16:58:34 <kgriffs> ;)
16:58:36 <amitgandhi> well it would be a breaking change to v2
16:58:38 <flaper87> btw, re versions, I think we need to re-structure the transport package and make it version aware
16:58:48 <ametts> Or you could rip it out and prove that people DO use it. :)
16:59:00 <kgriffs> eh
16:59:00 <zyuan> v2 the name already indicated breaking change
16:59:01 <kgriffs> heh
16:59:09 <flaper87> there has to be a v1 package under the wsgi package the holds wsgi v1 related modules
16:59:12 <amitgandhi> ametts: we dont want to be like another team we will leave unnamed =P
16:59:13 <flaper87> IMHO
16:59:24 <flaper87> not saying we have to do it *right now*
16:59:27 <flaper87> just something to think about
16:59:33 <kgriffs> flaper87: I think v2 is a long ways off. We should strive to introduce non-breaking changes in v1, possibly leveraging media type versioning
16:59:51 <kgriffs> but yeah, it will need to be done
16:59:52 <kgriffs> ok
16:59:53 <kgriffs> so...
16:59:57 <amitgandhi> ok so api v1 frozen?
16:59:58 <flaper87> :D
17:00:10 <zyuan> i think so
17:00:15 * ametts feels like we should vote to make it all official-like
17:00:18 * flaper87 takes his freezing gun out and shoots API v1
17:00:29 <flaper87> vote vote vote vote
17:00:36 <alcabrera> vote~
17:00:37 * amitgandhi is picturing flavio as dr freeze from batman
17:00:52 <flaper87> amitgandhi: LOOOL, that's what I had in mind
17:00:54 <flaper87> :D
17:01:18 <kgriffs> #startvote Freeze v1 API? yes, no, abstain
17:01:19 <openstack> Begin voting on: Freeze v1 API? Valid vote options are yes, no, abstain.
17:01:20 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
17:01:27 <amitgandhi> #vote yes
17:01:31 <zyuan> #vote yes
17:01:33 <ametts> #vote yes
17:01:36 <flaper87> #vote yes
17:01:40 <malini> #vote yes
17:01:57 * flaper87 notices that kgriffs finally remembered to add the abstain option
17:02:10 <alcabrera> #vote yes
17:02:11 <ametts> Wouldn't it be funny if kgriffs voted no?
17:02:18 <flaper87> LOOOL
17:02:21 <kgriffs> #vote yes
17:02:23 <kgriffs> too late
17:02:24 <kgriffs> :p
17:02:35 <kgriffs> ok
17:02:38 <kgriffs> going once...
17:02:43 <flaper87> it would have been even funnier if there was just a yes option
17:02:49 <megan_w> #vote yes
17:02:49 <kgriffs> going twice...
17:03:02 * kgriffs remembers that for next time
17:03:07 <flaper87> :P
17:03:09 <kgriffs> going three times...
17:03:22 <kgriffs> #endvote
17:03:23 <openstack> Voted on "Freeze v1 API?" Results are
17:03:25 <openstack> yes (8): alcabrera, megan_w, kgriffs, ametts, malini, amitgandhi, flaper87, zyuan
17:03:45 <kgriffs> I can't tell for certain, but I think the API is now frozen
17:04:08 <kgriffs> #info v1 API is now frozen
17:04:13 * kgriffs cheers
17:04:17 <megan_w> yay!
17:04:17 <flaper87> w000t
17:04:20 * amitgandhi feels cold
17:04:21 <alcabrera> cool. :)
17:04:22 <kgriffs> thanks everyone
17:04:30 <kgriffs> #endmeeting