19:00:09 <amitgandhinz> #startmeeting Poppy Weekly Meeting
19:00:09 <openstack> Meeting started Thu Jan  8 19:00:09 2015 UTC and is due to finish in 60 minutes.  The chair is amitgandhinz. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:12 <openstack> The meeting name has been set to 'poppy_weekly_meeting'
19:00:16 <amitgandhinz> #topic RollCall
19:00:23 <amitgandhinz> welcome back everyone
19:00:27 <amitgandhinz> who do we have today?
19:00:37 <catherineR> o/
19:01:02 <tonytan4ever> o/
19:01:14 <miqui_> ..hello ..folks..
19:01:19 <obulpathi> o/
19:01:31 <amitgandhinz> hi miqui_
19:01:50 <megan_w_> o/
19:02:07 <amitgandhinz> #link Agenda https://wiki.openstack.org/wiki/Meetings/Poppy
19:02:13 <mpanetta> o/
19:02:38 <amitgandhinz> #topic Last Year This Year on Poppy
19:02:47 <amitgandhinz> #link Last Meeting - http://eavesdrop.openstack.org/meetings/poppy_weekly_meeting/2014/poppy_weekly_meeting.2014-12-18-19.00.html
19:03:14 <amitgandhinz> okay, so action items from last time
19:03:17 <malini> o/
19:03:19 <amitgandhinz> amitgandhinz to set up a design discussion on the Message Queue Driver in early January
19:03:26 <amitgandhinz> ok lets set this up
19:03:58 <amitgandhinz> how does next Thursday sound?
19:04:07 <obulpathi> good for me!
19:04:11 <amitgandhinz> maybe at 1pm EST
19:04:12 <malini> during usual meeting hrs?
19:04:21 <amitgandhinz> nah lets do it an hour before the usual meeting
19:04:23 <tonytan4ever> I am good.
19:04:35 <obulpathi> +1
19:04:40 <amitgandhinz> miqui_: ?
19:04:41 <malini> I will be in India & will try to stay awake (A)
19:04:43 <miqui> +1
19:04:49 <amitgandhinz> mpanetta: ?
19:04:56 <mpanetta> Sounds good to me
19:05:15 <amitgandhinz> ok i will send out a calendar invite
19:05:21 <amitgandhinz> miqui_: i dont have your email addr
19:05:42 <miqui> migmaqer@gmail.com
19:05:48 <amitgandhinz> thanks
19:06:13 <amitgandhinz> the design discussion will center on what drivers we implement, how workers should work, etc
19:07:08 <amitgandhinz> #agreed amitgandhinz will hold a design discussion with the poppy team on the Message Queue Driver implementation on Thursday Jan 15, 2015 at 1pm EST
19:07:23 <amitgandhinz> ok next action item was malini to add doc to cdn vendor wiki page on how to add a mimic plugin for their api
19:07:40 <malini> :-$
19:07:45 <malini> can we carry it over?
19:07:51 <amitgandhinz> sure
19:07:55 <amitgandhinz> #action malini to add doc to cdn vendor wiki page on how to add a mimic plugin for their api
19:08:21 <amitgandhinz> lets get this done as I want us to make it really obvious for vendors to build out their drivers now
19:08:37 <malini> sure
19:09:00 <amitgandhinz> alright....
19:09:13 <amitgandhinz> #topic Review Kilo2 Activities
19:09:17 <amitgandhinz> #link https://launchpad.net/poppy/+milestone/kilo-2
19:09:47 <amitgandhinz> so first of all congrats everyone, there was an immense number of bugs fixed over the holidays
19:10:07 <obulpathi> yay!
19:10:14 <amitgandhinz> =D
19:10:23 <tonytan4ever> I missed the bug party but still congrats is in order.
19:10:37 <amitgandhinz> we still have some but not many so thats awesome
19:11:19 <amitgandhinz> there is still about 2 weeks left in the kilo 2 cycle
19:11:22 <malini> we ate bugs for holidays
19:11:31 <mpanetta> yum...
19:11:40 <miqui> ..hehehe...
19:11:41 <amitgandhinz> so lets target some of these remaining blueprints
19:11:52 <wbrothers> especially chocoloate covered
19:12:15 <malini> yumm..
19:12:23 <amitgandhinz> 1. Msq Queue Driver - we will discuss this next week and then get started on it
19:12:41 <amitgandhinz> tonytan4ever: Update on shared SSL support?
19:12:58 <tonytan4ever> This is pretty much done, it could use some reviews from everybody though.
19:13:13 <amitgandhinz> so Good progress or In Review?
19:13:13 <tonytan4ever> https://review.openstack.org/#/c/145365/
19:13:29 <tonytan4ever> Should be In Review
19:13:46 <amitgandhinz> ok.  lets get some reviews on this
19:14:14 <obulpathi> got it
19:14:15 <amitgandhinz> miqui_: Update Home Doc
19:15:19 <amitgandhinz> malini: Specify provider url in the conf file
19:15:30 <miqui> hi amitgandhinz, sorry, i have no upda tes..been swamped...
19:15:37 <amitgandhinz> miqui_: thats ok
19:15:38 <miqui> sorry about htis..
19:15:47 <malini> we have this already in place for akamai & fastly..
19:16:06 <malini> havent started for the rest
19:16:07 <amitgandhinz> its just maxcdn and cloudfront that need updating still
19:16:21 <malini> yes
19:16:23 <amitgandhinz> im going to take your name off this for now
19:16:33 <malini> thanks!
19:17:30 <amitgandhinz> obulpathi: 	Create Rackspace DNS plugin for mimic
19:18:03 <obulpathi> it still in same state
19:18:17 <obulpathi> once I fix the caching rules bug, I will finishes this off
19:18:27 <amitgandhinz> ok
19:18:31 <amitgandhinz> malini: End To End Tests for Poppy
19:18:42 <malini> tht is merged
19:18:53 <malini> I wud like to make some minor updates
19:19:02 <amitgandhinz> ill mark it as completed
19:19:02 <malini> around waiting for DNS propagation
19:19:10 <amitgandhinz> you can submit bugs for the updates
19:19:16 <malini> sure
19:20:05 <amitgandhinz> ok so the main items for kilo remaining are the queue driver, ssl support, and fixing the api tests at the gate
19:21:26 <amitgandhinz> does anyone have any questions regarding the bugs?
19:21:47 <malini> nope
19:21:52 <miqui> nope...
19:22:07 <tonytan4ever> no.
19:22:28 <amitgandhinz> #topic New Items
19:22:29 <wbrothers> nope
19:22:37 <amitgandhinz> Nothing new on the agenda
19:22:42 <amitgandhinz> #topic Open Discussion
19:22:55 <amitgandhinz> Anyone have something they want to discuss?
19:22:59 <malini> amitgandhinz: can we talk a bit abt the conformance tests?
19:23:04 <amitgandhinz> ya
19:23:18 <megan_w_> o/
19:23:34 <amitgandhinz> malini: first, then megan_w_
19:23:39 <megan_w_> if anyone from the edgecast/verizon team is here, i'd love to get feedback on how their plugin is coming along
19:23:41 <megan_w_> sounds good
19:24:02 * amitgandhinz watches megan_w_sneak her question in haha
19:24:13 <malini> amitgandhinz: if we were to write a conformance test for an existing provider, say akamai or fastly - what is the o/p we want to compare against in the conformance test?
19:24:41 <amitgandhinz> so we want to make sure they implement the interface correctly and can handle the different inputs
19:24:51 <amitgandhinz> so all providers have to meet the abstract interface we defined
19:25:10 <malini> you mean this one https://github.com/stackforge/poppy/blob/master/poppy/provider/base/services.py ?
19:25:12 <amitgandhinz> i want the conformance tests to be able to pass a range of different values (postive/neg tests) to those interfaces
19:25:15 <amitgandhinz> ya
19:25:30 <amitgandhinz> and the conformance tests should assert the correct responses are returned
19:25:44 <amitgandhinz> that way as a provider builds their driver, they know that they are meeting the contracts
19:26:02 <malini> ok..I'll start working on a POC & solicit feedback as I go
19:26:07 <malini> I am adding a bp for tht now
19:26:21 <amitgandhinz> to answer the question on if the provider driver configures the cdn settings correctly at their end, the api tests and end to ends tests should catch that
19:26:42 <amitgandhinz> we will need a way to run the api tests multiple times, once for each provider we have
19:27:04 <malini> we can just create a big happy flavor with all providers in it :)
19:27:08 <amitgandhinz> that way we can see which providers are working and which ones could be broken
19:27:46 <amitgandhinz> we should be able to just use different test.conf files right?  to test each provider flavor on their own
19:28:07 <malini> yeap..tht will work too
19:28:52 <amitgandhinz> that way we can setup each conf file with its appropriate provider credentials, and also point it to use mimic if necessary
19:29:25 <malini> sounds good..
19:29:30 <malini> megan_w_: your turn
19:29:49 <megan_w_> fantastic.  is there anyone from the edgecast/verizon team here?
19:29:56 <megan_w_> (could be a quick turn)
19:30:14 <megan_w_> wait for it...
19:30:22 <megan_w_> ...
19:30:35 <megan_w_> ok, i'm good.  nothing for me :)
19:31:04 <malini> shud have let megan_w_ go first :)
19:31:09 <miqui_> can i ask a question....
19:31:32 * miqui_ raises hand...
19:31:39 <megan_w_> yes please
19:32:00 <amitgandhinz> go for it miqui_
19:32:13 <miqui_> for the message queue driver design discussion is there a particular messaing backend to consider?
19:32:26 <amitgandhinz> i want to support the following:
19:32:33 <miqui_> just asking so as to t"think ahead " somewhat...
19:32:45 <amitgandhinz> 1. oslo.message - this gives us rabbitmq, zeromq, and qpid, and maybe kafka support
19:32:56 <amitgandhinz> 2.  zaqar support
19:33:09 <amitgandhinz> 3.  maybe celery or taskflow
19:33:19 <amitgandhinz> but 1 and 2 are required
19:33:28 <miqui_> k, thanks...
19:33:52 <amitgandhinz> if we build the driver generic enough we should be able to support many queue platforms
19:34:04 <miqui_> right +
19:34:08 <miqui_> +1
19:34:13 <amitgandhinz> but 1 and 2 tie us into the openstack platform nicely
19:34:49 <amitgandhinz> ok any more questions?
19:35:12 <miqui_> am good for now... thanks...
19:35:17 <malini> do we want to start talking a bit more abt queue driver now?
19:35:24 <malini> since we have 30 minutes left?
19:35:48 <amitgandhinz> lets do some prep first
19:35:48 <malini> I don't want any life back :D
19:35:57 <mpanetta> hah!
19:35:59 <malini> ok
19:36:04 <amitgandhinz> i want to gain a better understanding myself of oslo.messaging
19:36:13 <malini> sure
19:36:17 <amitgandhinz> i think i know zaqar pretty well (one hopes =p)
19:36:48 <amitgandhinz> and i know mpanetta has some concerns around worker processes that need to be heard
19:36:58 <mpanetta> yeah
19:37:12 <mpanetta> mainly that they die properly :P
19:37:26 <amitgandhinz> so lets spend the week building up our notes/concerns, and then we can hit hard at the thur meeting
19:37:37 <amitgandhinz> we can also talk about it during the week too =)
19:37:42 <miqui_> i have some experience with rabbitmq
19:37:51 <amitgandhinz> cool
19:38:12 <obulpathi> regarding the worker mechanisms
19:38:14 <miqui_> ..but likewise need to ramp up on oslo.messaging...
19:38:57 <tonytan4ever> Celery use rabbitmq as a backend queue system, and it has a nice GUI interface to manage and monitor all the worker processes.
19:39:01 <tonytan4ever> I am just saying.
19:39:18 <obulpathi> if we have threading based workers (like greenlets), poppy will be able to handle much higher load (compared to process based)
19:39:45 <obulpathi> a significant portion of time, the workers spend in waiting for network io state
19:40:10 <obulpathi> this limits the number of messages that can get processed
19:40:23 <mpanetta> My main concern is that we can mange the processes with something like supervisord or circus.
19:40:41 <obulpathi> if we use process based workers, we can not have large number of workers (swapping, memory etc ..)
19:40:42 <miqui_> worker pool?
19:41:15 <obulpathi> for every network io, provider call , DNS call, they need to be swapped, which created lot of overhead
19:41:33 <amitgandhinz> it amy be worth looking into how designate is doing their workers (just for comparison)
19:41:38 <amitgandhinz> s/amy/may
19:41:41 <obulpathi> good idea
19:42:04 <miqui_> i like how nodejs model...
19:42:07 <obulpathi> does any other poppy project use queue/worker mechanism?
19:42:18 <miqui_> how it handles multiple load use cases..
19:42:34 <miqui_> not sure if any of you have read some about this javascript v8 engine...
19:42:44 <mpanetta> It is all event driven isn't it?
19:42:51 <miqui_> yes
19:42:57 <miqui_> it is very very fast,,,,
19:43:06 <amitgandhinz> miqui_: unfortunately nodejs isnt openstacky =/
19:43:13 <miqui_> bu i think we have stick to python
19:43:17 <amitgandhinz> but there are non blocking python libs
19:43:22 <miqui_> yup amitgandhinz...you are right...
19:43:23 <mpanetta> twisted
19:43:27 <amitgandhinz> like twisted
19:43:29 <amitgandhinz> =P
19:43:29 <mpanetta> asyncio
19:43:30 <mpanetta> etc
19:43:38 <amitgandhinz> asyncio is py3+
19:43:43 * mpanetta is probably still on an ansync kick...
19:43:57 <mpanetta> Twisted is kinda cool, but the docs are kinda lacking.
19:43:57 <amitgandhinz> nsync was so 90s !
19:44:05 <mpanetta> haha
19:44:14 <mpanetta> s/ansync/async/ :P
19:44:39 <amitgandhinz> im not sure if twisted is acceptable ....
19:44:49 <miqui> ..hehehe
19:44:52 <amitgandhinz> so lets research this this week also
19:44:59 <amitgandhinz> come up with ideas for thursday
19:45:20 <amitgandhinz> #action investigate how to do worker processes with non blocking io
19:45:28 <amitgandhinz> #action research oslo.messaging
19:45:40 <mpanetta> Are there any openstack preferred libs for this?
19:45:51 <amitgandhinz> idk
19:46:20 <obulpathi> got it
19:46:59 <amitgandhinz> ok....
19:47:03 <amitgandhinz> any thing else ?
19:48:05 <obulpathi> nothing from me
19:48:13 <malini> none from me too..
19:48:19 <miqui> none from me...
19:48:24 <tonytan4ever> One thing from me.
19:48:53 <amitgandhinz> go for it tonytan4ever
19:49:15 <tonytan4ever> A quick FYI, the domain field in Poppy service will have a "shared ssl" field, to support shared ssl domain
19:49:25 <tonytan4ever> `shared_ssl` - optional: whether to use shared ssl certificate on this domain, right now only support true. Defaults to false.
19:49:45 <tonytan4ever> This is what I will put on apiary doc.
19:50:18 <amitgandhinz> right.  its also worth noting that if shared_ssl = true, then the domain entered will be overwritten with a generated domain using the shared cert
19:50:46 <malini> overwritten in the service details?
19:50:47 <tonytan4ever> Yeah.
19:50:52 <amitgandhinz> yeah
19:51:05 <amitgandhinz> because a custom user domain doesnt make sense with a shared domain
19:51:17 <amitgandhinz> you cant cname your own domain to a shared domain with SSL
19:51:25 <amitgandhinz> the cert would be rejected in the browser
19:51:28 <malini> in tht case, does it make sense to still keep domain a required field?
19:51:43 <tonytan4ever> You are right, it does not.
19:51:48 <amitgandhinz> thinking about making it not required if shared_ssl = true
19:51:53 <malini> +1
19:52:51 <amitgandhinz> cool =)
19:53:02 <amitgandhinz> anyone else?
19:53:14 <amitgandhinz> i think everyone has asked a question =P
19:54:39 <malini> guess tht is it?
19:54:45 <obulpathi> I think so
19:54:48 <malini> 1 min never seemed longer :D
19:55:11 <amitgandhinz> hehe
19:55:15 <amitgandhinz> ok thanks everyone
19:55:17 <mpanetta> Sooo quiet
19:55:19 <amitgandhinz> #endmeeting