18:59:35 <amitgandhinz> #startmeeting Poppy Weekly Meeting
18:59:36 <openstack> Meeting started Thu Feb 12 18:59:35 2015 UTC and is due to finish in 60 minutes.  The chair is amitgandhinz. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:59:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:59:39 <openstack> The meeting name has been set to 'poppy_weekly_meeting'
18:59:48 <amitgandhinz> #topic RollCall
18:59:55 <amitgandhinz> \o/
19:00:02 <sriram> _/\_
19:00:10 <amitgandhinz> lol
19:00:55 <megan_w_> hey everyone!
19:01:16 <dmitry> howdy folks!
19:01:25 <amitgandhinz> dmitry: howdy!
19:01:33 <obulpathi> o/
19:01:37 <dmitry> hey Amit & Megan!
19:01:39 <miqui> o/
19:01:42 <miqui> hello all...
19:01:46 <cpowell> o\
19:02:22 <amitgandhinz> #link https://wiki.openstack.org/wiki/Meetings/Poppy#Agenda
19:02:30 <amitgandhinz> #topic Review Last Week
19:02:42 <amitgandhinz> #link http://eavesdrop.openstack.org/meetings/poppy_weekly_meeting/2015/poppy_weekly_meeting.2015-02-05-19.00.html
19:02:50 * sriram wonders how the meeting would go if everything was ascii art :P
19:02:58 <amitgandhinz> no action items from last week
19:03:10 <amitgandhinz> fyi i did submit the proposal for the vancouver talk
19:03:30 <amitgandhinz> i think they will get published soon for people to start voting on
19:03:30 <obulpathi> sriram: we at poppy, use unicode for everything :)
19:03:36 <sriram> amitgandhinz:
19:03:39 <sriram> cool
19:03:45 <miqui> +1
19:03:47 <sriram> obulpathi: heh
19:04:14 <amitgandhinz> but you can see what i submitted on the etherpad here: https://etherpad.openstack.org/p/Poppy_Vancouver
19:05:42 <amitgandhinz> any questions on this?
19:05:58 <amitgandhinz> if not we can move on to reviewing the current board
19:06:15 <amitgandhinz> #topic Kilo3 Updates
19:06:21 <amitgandhinz> #link https://launchpad.net/poppy/+milestone/kilo-3
19:06:40 <amitgandhinz> lets go through the bp first
19:06:56 <amitgandhinz> sriram: taskflow driver update (tonytan is on vacation right now)
19:07:02 <sriram> sure
19:07:11 <sriram> so there are 2 patches out there.
19:07:35 <sriram> the first one is the base, where everything runs in one task, with a linear flow.
19:07:41 <sriram> thats going to be a first pass.
19:07:51 <sriram> the second patch, is the one I'm currently working on.
19:08:10 <sriram> splits everything into multiple tasks, enclosed within a graph flow.
19:08:24 <sriram> I have managed to get everything working.
19:08:49 <sriram> I need to update the tests, and also add a more sophisticated retry logic while making dns updates.
19:08:57 <sriram> right now we just retry 5 times.
19:09:04 <sriram> thats about it.
19:09:13 <amitgandhinz> cool, coming along nicely
19:09:29 <amitgandhinz> miqui: https://blueprints.launchpad.net/poppy/+spec/home-doc
19:09:42 <miqui> i did some updates....
19:09:58 <miqui> but need team feedback if anything is missing...
19:10:18 <amitgandhinz> submit a patch to gerrit and we can take a look
19:10:18 <miqui> basically does it reflect all of the routes and responses...
19:10:31 <sriram> amitgandhinz: +1
19:10:43 <sriram> miqui: please feel free to submit a patch :)
19:10:46 <miqui> hmmm... i was actually assuming that if you look at the apiari doc
19:10:54 <miqui> there would be the answer....
19:11:05 <amitgandhinz> oh you made the update in apiary?
19:11:09 <miqui> yes
19:11:14 <sriram> oh
19:11:23 <miqui> what am doing is changing the apiary to match what is there...
19:11:27 <miqui> there in the code...
19:11:32 <miqui> not the other way around...
19:11:44 <miqui> i.e. docs match the code....
19:12:10 <amitgandhinz> are you updating it here: http://docs.cloudcdn.apiary.io/#get-%2F
19:12:18 <miqui> yes
19:12:32 <miqui> changing it to reflect the current code
19:12:39 <amitgandhinz> hmm i dont see the changes yet
19:12:47 <miqui> it is a challenge since the code keeps changing....
19:13:09 <amitgandhinz> the endpoints are not changing so shouldnt be impactful
19:13:32 <miqui> amitgandhinz: then i think i am still missing something...
19:13:48 <miqui> i did some minor updates that reflect the models
19:13:52 <miqui> and some typos
19:14:40 <amitgandhinz> ok, lets chat afterwards about this
19:14:41 <miqui> also services seems done
19:14:44 <miqui> k
19:15:07 <amitgandhinz> may need to clarify exactly what is required here
19:15:31 <amitgandhinz> obulpathi: https://blueprints.launchpad.net/poppy/+spec/mimic-racksapce-dns
19:15:42 <obulpathi> I am not getting time to work on this
19:15:46 <obulpathi> I will remove my name from this
19:15:56 <obulpathi> and will pick it up, when I have bandwidth
19:16:21 <amitgandhinz> ok
19:17:04 <amitgandhinz> alright, moving on to bugs
19:17:23 <amitgandhinz> nothing looks like its currently in progress
19:17:38 <obulpathi> https://bugs.launchpad.net/poppy/+bug/1420833
19:17:38 <openstack> Launchpad bug 1420833 in Poppy "service stays in update_in_progress state" [Undecided,Fix committed] - Assigned to Obulapathi (obulpathi)
19:17:42 <obulpathi> https://bugs.launchpad.net/poppy/+bug/1420515
19:17:42 <openstack> Launchpad bug 1420515 in Poppy "Change PATCH to do UPSERT" [Undecided,Fix committed] - Assigned to Obulapathi (obulpathi)
19:17:50 <obulpathi> we have got these bugs fixed
19:17:55 <obulpathi> can we mark it for kilo-3?
19:18:17 <obulpathi> https://bugs.launchpad.net/poppy/+bug/1421352
19:18:17 <openstack> Launchpad bug 1421352 in Poppy "Race condition in PATCH endpoint" [Undecided,New]
19:18:30 <obulpathi> https://bugs.launchpad.net/poppy/+bug/1421352
19:18:41 <amitgandhinz> targeted
19:18:51 <obulpathi> the two bugs related to race conditions
19:19:17 <obulpathi> can you also mark them for kilo-3 please
19:19:25 <obulpathi> cool .. thanks :)
19:19:31 <amitgandhinz> yes i marked them to k3
19:19:34 <amitgandhinz> we also finally got all the api tests passing now too
19:19:42 <obulpathi> yep
19:19:44 <sriram> wooo nice.
19:20:29 <obulpathi> https://bugs.launchpad.net/poppy/+bug/1417368
19:20:29 <openstack> Launchpad bug 1417368 in Poppy "Cassandra storage driver docstring refers to MongoDB" [Undecided,Fix committed] - Assigned to Robert Morgan (robert-thomas-morgan)
19:20:40 <obulpathi> needs to marked for kilo-3
19:20:48 <amitgandhinz> done
19:21:07 <obulpathi> thanks :)
19:21:40 <amitgandhinz> #topic New Items
19:22:00 <amitgandhinz> so i have been documenting some api changes in apiary
19:22:06 <amitgandhinz> http://docs.cloudcdn.apiary.io
19:22:26 <amitgandhinz> i want the team to review them before the work starts on adding those features
19:22:38 <amitgandhinz> i will summarize what the features are here so that everyone has context
19:22:43 <sriram> mark as action item?
19:22:48 <miqui> oh wait....ami
19:22:50 <miqui> amitgandhinz:
19:23:09 <amitgandhinz> #action ALL - review api service configuration changes in apiary
19:23:15 <amitgandhinz> yes miqui?
19:23:27 <miqui> just want to clarify something...
19:23:35 <amitgandhinz> sure
19:23:56 <miqui> so the apiary tool is basically used to document apis,etc..etc.. then we go off to implement as per that spec/doc
19:24:06 <amitgandhinz> generally yes
19:24:16 <miqui> , but the have been treating it as the homedoc
19:24:25 <amitgandhinz> we review the api before implementing it
19:24:32 <miqui> which in turn is modified to match whats in the code today
19:24:41 <miqui> am i doing the wrong thing...
19:25:03 <amitgandhinz> at the end of the day the code and the api docs should match
19:25:09 <miqui> sure sure...
19:25:17 <miqui> but i think  i been doing things backwards
19:25:18 <amitgandhinz> ideally the team agrees on the api doc first, so we dont end up building the wrong thing
19:25:31 <sriram> AFAIK, the home doc should reflect all available endpoints in the api
19:25:36 <amitgandhinz> sriram: +1
19:25:41 <sriram> and needs to be implemented here: https://github.com/stackforge/poppy/blob/181a0216ec1f0dd596a017bf5c257fac96c9f641/poppy/manager/default/home.py
19:26:10 <amitgandhinz> so in the case of poppy, it should reflect the endpoints available for /services and /flavors, and the methods possible (get, put, post, delete), etc
19:26:24 <sriram> yes
19:26:29 <amitgandhinz> a user can then discover individual services by doing a list of /services
19:26:36 <miqui> right... but i think am doing it backwards
19:26:44 <amitgandhinz> how are you doing it?
19:26:59 <miqui> am modifying the homedoc to reflect the code as per the bp
19:27:40 <miqui> instead the correct way is to document what the api is goign top be etcetc and then do the code
19:27:42 <amitgandhinz> so, right now the homedoc as shown in apiary is also incomplete
19:28:01 <miqui> its missing endpoints?
19:28:16 <amitgandhinz> so if you can update apiary for the GET home document endpoint to what it should be....and then update the code to reflect that
19:28:34 <sriram> miqui: yep
19:28:35 <amitgandhinz> homedoc is currently missing the /flavors endpoint
19:28:40 <amitgandhinz> its also missing the hints for the other verbs
19:28:45 <sriram> it lists just the services endpoint right now.
19:28:55 <miqui> hmm missing?
19:29:02 <amitgandhinz> as in its not documented
19:29:02 <miqui> i see it here: xotl: so you are saying that you cannot access sails.config.env.xxx.xxx ?
19:29:06 <miqui> oops
19:29:14 <miqui> http://docs.cloudcdn.apiary.io/#flavors
19:29:26 <sriram> miqui: https://gist.github.com/anonymous/ff02c8d1b81146323db8
19:29:43 <sriram> under href/template, you see that it just lists services
19:29:50 <sriram> we need to add flavors
19:29:52 <amitgandhinz> so if a user of poppy doesnt read the api docs, they have no idea the flavors endpoint exists
19:29:55 <sriram> and all other endpoints.
19:29:57 <miqui> aaaahhhhhhhhhh wait..
19:29:58 <amitgandhinz> and where it is
19:30:06 <amitgandhinz> but if the make a call to /
19:30:06 <miqui> the homedoc IS NOT the apiary doc
19:30:17 <sriram> miqui: nope :)
19:30:20 <amitgandhinz> then they will discover that there is a /services endpoint and a /flavors endpoint
19:30:23 <miqui> the homedoc is just another endpoint
19:30:31 <sriram> GET v1.0/
19:30:34 <amitgandhinz> if they call GET /services, then they discover the list of services created
19:30:46 <amitgandhinz> sriram: +1
19:30:46 <miqui> right so the homedoc is another endpoint....
19:30:47 <obulpathi> miqui: you can fin f the code related to homedoc here: https://github.com/stackforge/poppy/blob/master/poppy%2Fmanager%2Fdefault%2Fhome.py
19:31:03 <miqui> but my question ... am i correct?
19:31:19 <amitgandhinz> the home doc is NOT the apiary doc
19:31:37 * miqui a lightbulb lights up finally
19:31:38 <amitgandhinz> the homedoc needs to be documented in apiary under the / endpoint as per here...
19:31:46 <sriram> sorry, If my answer was confusing :P
19:31:51 <amitgandhinz> http://docs.cloudcdn.apiary.io/#get-%2F
19:32:00 <miqui> all this freaking time i treated the apiary as the homedoc
19:32:06 <miqui> gees
19:32:14 <miqui> its all in the terminology....
19:32:17 <amitgandhinz> hmm....sorry if we created confusion
19:32:24 <miqui> sorry sorry sorry sorry guys
19:32:39 <sriram> miqui: dont sweat it :)
19:32:41 <miqui> home endpoint
19:32:48 <amitgandhinz> yip =)
19:32:50 <miqui> might been clearer to me....
19:32:53 <malini> its just an endpoint after all :)
19:33:01 <sriram> heh
19:33:03 <miqui> when i see 'doc' hey its doc not code
19:33:06 <miqui> gees sorry again...
19:33:25 <miqui> technically malini that is true...
19:33:41 <miqui> k... amitgandhinz i will have another look...
19:33:53 <miqui> thats my update....hahahah
19:33:59 <amitgandhinz> and we can chat more in the channel on this, definately ask questions.  especially if the bp is not clear
19:34:04 <amitgandhinz> ok....
19:34:09 <miqui> sure thanks
19:34:16 <amitgandhinz> http://docs.cloudcdn.apiary.io/#post-%2Fservices
19:34:25 <amitgandhinz> so i have added a section for "content"
19:34:45 <amitgandhinz> the idea here is to implement things like header forwarding, querystring forwarding, and cookie forwarding to origin
19:34:53 <amitgandhinz> those items also become part of the cachekey
19:35:12 <amitgandhinz> that way users can be served content based on their sessions/headers/ etc
19:35:42 <amitgandhinz> the second one i added was under restrictions[]
19:36:02 <amitgandhinz> i added a rule to allow geography restrictions to allow requests to be restricted to certain geo's only
19:36:23 <sriram> cool, I guess thats a country code?
19:36:29 <sriram> or regions?
19:36:33 <amitgandhinz> country code
19:36:43 <sriram> just wondering about the granularity of the region
19:36:52 <amitgandhinz> and regions are subjective
19:37:07 <amitgandhinz> we may need to come up with an enumerated list that we support
19:37:13 <sriram> just wondering.. if lets say I want the content to be accessible from asia
19:37:17 <amitgandhinz> and then map items in that list to ones that providers support
19:37:23 <obulpathi> also, the list that goes into geo, is it whitelist or blacklist?
19:37:28 <amitgandhinz> as im sure each provider refer to countries/regions differently
19:37:31 <sriram> do I need to send all the country codes in asia?
19:37:56 <amitgandhinz> maybe we do each country, but also have the option for regions in our enumerated list of options
19:38:05 <obulpathi> +1
19:38:10 <amitgandhinz> so you can say "asia" or  each country
19:38:25 <sriram> amitgandhinz: yeah, then again, it also depends on what the providers support.
19:38:26 <obulpathi> is this a whitelist or blacklist?
19:38:36 <sriram> but, I like the idea.
19:38:41 <amitgandhinz> restrictions are a blacklist generally
19:38:54 <obulpathi> amitgandhinz: cool
19:39:04 <amitgandhinz> hmmm
19:39:15 <amitgandhinz> we may need to introduce a way to indicate it huh
19:39:43 <amitgandhinz> eg if i want to limit to USA, then i wouldnt want to restrict 250 countries
19:39:51 <obulpathi> yeah
19:39:52 <sriram> amitgandhinz: +1
19:39:57 <amitgandhinz> feedback noted
19:40:11 <amitgandhinz> the third item i added was the ability to enable log_delivery
19:40:30 <sriram> amitgandhinz: cool, that makes sense.
19:40:41 <amitgandhinz> log delivery will basically take the logs delivered from the provider, reformat it into a standardized format, and put it inthe customer's swift contianer
19:41:03 <obulpathi> is there a way to generalize to any container?
19:41:11 <obulpathi> swift/S3/google?
19:41:24 <amitgandhinz> we can make the container name configurable by the operator
19:41:32 <obulpathi> ok
19:41:44 <sriram> obulpathi: we could. just need different drivers.
19:41:51 <amitgandhinz> hmmm....lets start with swift only, and then generalize later with different drivers
19:41:55 <obulpathi> also, only the config option is exposed, right?
19:42:08 <amitgandhinz> yes, only enabled: true/false
19:42:19 <sriram> just make sure that we do the service as abstract base class, as its easily extensible later.
19:42:26 <amitgandhinz> we could make this an object so its easy to add features later to extend it
19:42:35 <obulpathi> the code for pushing logs into containers, is it going to live under poppy?
19:42:54 <amitgandhinz> so instead of log_delivery: true it could be log_delivery : { enabled: true, storage: swift }
19:43:02 <amitgandhinz> obulpathi: yes
19:43:10 <obulpathi> ok
19:44:06 <miqui> what about log format?
19:44:09 <sriram> amitgandhinz: its something thats going to be running in the background?
19:44:14 <miqui> json is pretty popular
19:44:19 <amitgandhinz> if you scroll down, i have it detailed
19:44:26 <sriram> a separate entry point, right?
19:44:41 <amitgandhinz> miqui: the log fromat should follow a standard as users will often parse it into their own systems
19:44:57 <obulpathi> yeah
19:45:04 <miqui> and what is that standard?
19:45:09 <mpanetta> Usually apache
19:45:14 <obulpathi> apache common log format / extended log format support
19:45:14 <amitgandhinz> this one is NCSA combined (apache)
19:45:23 <miqui> hmmm..k
19:46:03 <amitgandhinz> eventually we could add a field to the log_delivery element where the customer can specify the format they want i guess
19:46:10 <amitgandhinz> maybe thats a v2 thing
19:46:17 <amitgandhinz> based on feedback we get on this
19:46:25 <miqui> sure
19:46:42 <sriram> so, again a question the log delivery is an out of band service from the api, correct?
19:46:42 <miqui> some might want it in certain format that fits well with lets say logstash...
19:46:44 <amitgandhinz> ok, the last change i added was the "certificate" field under "domains[]"
19:47:01 <amitgandhinz> the idea here is you can set it to "shared", "san", or "custom"
19:47:20 <amitgandhinz> if you do shared there are validations that kick in on what was entered for the domain
19:47:51 <amitgandhinz> basically shared ssl domains can only have one level of indirection in them
19:48:02 <amitgandhinz> shared will use an operator cert
19:48:26 <amitgandhinz> san will add the user's domain as a Subject Alternate Name to the operator's cert
19:48:43 <amitgandhinz> and "custom" will provision a new cert for the customer's domain at the provider
19:49:24 <amitgandhinz> for custom, we may utilize barbican potentially to generate certs for providers who need it uploaded.  Others (like Akamai) dont trust users to upload a cert and want to provision it themselves to ensure the cert hasnt been compromised
19:49:44 <amitgandhinz> dmitry: how does maxcdn handle dedicated certs?
19:50:04 <amitgandhinz> ie is it custom provisioning? is there an API for it?
19:50:50 <dmitry> Amit: Customers can upload their own certificates or we can purhase on their behalf and provsion.  It's practically real-time and done via our Control Panel :)
19:51:17 <dmitry> dedicated or wildcard certs
19:51:20 <amitgandhinz> cool =)  is there an API/CLI interface to upload it also, or is it only on the control panel?
19:51:31 <amitgandhinz> #link https://www.maxcdn.com/one/tutorial/setup-dedicated-ssl/
19:52:00 <dmitry> this could possible be done via API as well.. I would need to look into the specs
19:52:36 <amitgandhinz> the maxcdn looks somewhat similar (slightly easier) than cloudfronts process
19:53:02 <amitgandhinz> so in these types of examples we can have barbican generate a cert, and then upload it (hopefully via api) to the providers
19:53:16 <obulpathi> +1
19:53:20 <dmitry> Amit, confirmed that certs can be uploaded and provisioned via API
19:53:28 <amitgandhinz> dmitry: awesome =D
19:53:31 <dmitry> via the MaxCDN API
19:53:45 <dmitry> https://docs.maxcdn.com/#ssl-certificate-api
19:54:15 <amitgandhinz> this is nice
19:54:24 <dmitry> we're fancy like that ;)
19:54:34 <amitgandhinz> haha
19:55:18 <amitgandhinz> ok team, so please read through these api spec changes, and give me more feedback on them.  i will incorporate some of the feedback received today into it also
19:55:29 <amitgandhinz> timecheck - 5 min remaining
19:55:45 <amitgandhinz> any questions on any of these suggestions, or are they pretty clear?
19:56:19 <amitgandhinz> #topic Open Discussion
19:56:26 <miqui> one comment..
19:56:28 <amitgandhinz> sure
19:56:42 <miqui> i have seen some code here and there without proper docstrings....
19:56:51 <amitgandhinz> =/
19:57:31 <miqui> so me being a bottom feeder is keeping an eye in this making patches...
19:57:45 <miqui> thats it for me...
19:57:47 <amitgandhinz> please -1 those patches when reviewing
19:57:54 <amitgandhinz> and team, please add docstrings!
19:57:58 <sriram> amitgandhinz: +1
19:58:04 <amitgandhinz> we need them, it makes it easier for others to participate too
19:58:08 <obulpathi> sure
19:58:53 <amitgandhinz> dmitry: also, we started some docs for providers - https://wiki.openstack.org/wiki/Poppy/Provider_-_Getting_Started#Building_a_Provider_Driver
19:59:08 <amitgandhinz> its still a work in progress, but its a start i think
19:59:24 <dmitry> perfect, thanks Amit!
19:59:48 <amitgandhinz> im strill trying to add to it (eg a step by step guide on building a new provider from nothing with code samples)
19:59:56 <amitgandhinz> ok we are at time now
20:00:07 <amitgandhinz> we can discuss more in the #openstack-poppy channel
20:00:10 <amitgandhinz> thanks everyone
20:00:10 <sriram> and thats a wrap.
20:00:11 <obulpathi> see you all next week
20:00:14 <amitgandhinz> #endmeeting