19:01:21 <amitgandhinz> #startmeeting Poppy Weekly Meeting
19:01:22 <openstack> Meeting started Thu Jan 22 19:01:21 2015 UTC and is due to finish in 60 minutes.  The chair is amitgandhinz. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:23 <tonytan4ever> o/
19:01:26 <openstack> The meeting name has been set to 'poppy_weekly_meeting'
19:01:26 <miqui> 0/
19:01:32 <amitgandhinz> #topic RollCall
19:01:41 <amitgandhinz> hi tonytan4ever, miqui !
19:01:43 <cpowell_> o/
19:01:44 <miqui> o/
19:01:47 <tonytan4ever> hi
19:01:53 <miqui> hi folks....
19:02:04 <obulpathi> o/
19:02:13 <catherineR> o/
19:02:22 <megan_w_> o/
19:02:37 <amitgandhinz> #link Agenda - https://wiki.openstack.org/wiki/Meetings/Poppy
19:02:50 <amitgandhinz> #topic Last Weeks Items
19:03:07 <amitgandhinz> #link http://eavesdrop.openstack.org/meetings/poppy_weekly_meeting/2015/poppy_weekly_meeting.2015-01-15-19.00.html
19:03:35 <amitgandhinz> tonytan4ever to reach out to the TaskFlow IRC channel to learn the differences between celery and taskflow
19:03:36 <wbrothers> o/
19:03:47 <amitgandhinz> we have an agenda later to talk about this in detail
19:03:54 <tonytan4ever> I did some research about TaskFlow
19:04:05 <tonytan4ever> and compared it with Celery.
19:04:09 <miqui> yeah..i did some pking at it..
19:04:18 <amitgandhinz> cool.  we can dig in a bit
19:04:30 <amitgandhinz> wbrothers to add doc to cdn vendor wiki page on how to add a mimic plugin for their api
19:04:46 <tonytan4ever> This is the advantage with TF: a ) Tasks are grouped into workflows that can define ordering and dependency
19:05:01 <amitgandhinz> tonytan4ever: save it for the agenda items in a little bit ;-)
19:05:02 <tonytan4ever> . which are compiled to a directed graph  and can be executed in parallel. These workflows can have portions that retry, and can revert their work on failure,.
19:05:15 <tonytan4ever> OK
19:05:53 <amitgandhinz> wbrothers: did you document how to build mimic plugin's for cdn providers?
19:06:53 <amitgandhinz> ok moving on then
19:07:09 <amitgandhinz> ALL - investigate worker management
19:07:17 <amitgandhinz> we'll discuss that shortly with the TF stuff
19:07:19 <wbrothers> No I haven't
19:07:30 <amitgandhinz> ok thanks wbrothers ,i'll push it forwrad
19:07:42 <amitgandhinz> #action wbrothers to add doc to cdn vendor wiki page on how to add a mimic plugin for their api
19:08:10 <amitgandhinz> #topic Updates on Blueprints and Bugs
19:08:13 <amitgandhinz> #link https://launchpad.net/poppy/+milestone/kilo-3
19:08:29 <amitgandhinz> yes you read that right! we are now in Kilo-3 cycle
19:08:52 <amitgandhinz> i released Kilo-2 yesterday which contained mostly bug fixes (https://launchpad.net/poppy/+milestone/kilo-2)
19:09:14 <amitgandhinz> i moved any remaining bugs forward into k3, and groomed the blueprints targeted
19:09:27 <miqui> thanks amitgandhinz
19:09:33 <amitgandhinz> the focus is now back on to features
19:09:59 <amitgandhinz> lets quickly go over the bugs that are being worked on....
19:10:12 <amitgandhinz> https://bugs.launchpad.net/bugs/1408296
19:10:39 <amitgandhinz> obulpathi: are you currrently working on this or can i unassign you
19:10:40 <obulpathi> I still have to start this
19:10:52 <amitgandhinz> same for #1408301 ?
19:11:11 <obulpathi> yes
19:11:27 <amitgandhinz> ok, removing your name
19:11:30 <obulpathi> ok
19:12:25 <amitgandhinz> miqui: https://bugs.launchpad.net/bugs/1381091
19:12:54 <amitgandhinz> are you working on this or should i remove your name also
19:13:10 <miqui> keep my name on it...
19:13:23 <amitgandhinz> ok
19:13:50 <amitgandhinz> then there are 3 bugs in progress that need reviews or are being worked on
19:14:06 <amitgandhinz> obulpathi: https://bugs.launchpad.net/bugs/1380679
19:14:13 <amitgandhinz> what is the status of this one, its been around for a while
19:14:38 <obulpathi> I will finish it up by this week
19:14:44 <amitgandhinz> i dont see a PR for it
19:14:49 <obulpathi> lets leave the status the same for now
19:14:54 <amitgandhinz> ok
19:15:11 <amitgandhinz> ok, moving on to blueprints
19:15:25 <amitgandhinz> https://blueprints.launchpad.net/poppy/+spec/queue-driver
19:15:34 <amitgandhinz> we are going to discuss this in a bit, so no update needed right now
19:15:54 <amitgandhinz> malini is out so cant get update on https://blueprints.launchpad.net/poppy/+spec/conformance-tests
19:16:10 <amitgandhinz> obulpathi: https://blueprints.launchpad.net/poppy/+spec/mimic-racksapce-dns
19:16:20 <obulpathi> I did some progreass on thsi
19:16:33 <obulpathi> good chunks of API are in working state: CREATE and search
19:16:48 <obulpathi> about 50% is done
19:16:53 <amitgandhinz> so change it to Good Progress then
19:16:56 <obulpathi> yep
19:17:18 <amitgandhinz> miqui: any update on https://blueprints.launchpad.net/poppy/+spec/home-doc
19:17:47 <miqui> started to look at this... did some one else in the team make a change?
19:17:55 <miqui> i notice some differences...
19:18:03 <amitgandhinz> i dont think so
19:18:21 <amitgandhinz> let me look...
19:18:51 <amitgandhinz> https://github.com/stackforge/poppy/commits/master/poppy/transport/pecan/controllers/v1/home.py
19:18:56 <amitgandhinz> no changes since dev 2
19:18:58 <amitgandhinz> dec 2
19:19:06 <miqui> hmm, ok... well.. i will dedicate more time to it.. sometimes i cannot correlate the  actual output with the verb/action
19:19:15 <miqui> ah k...
19:19:36 <miqui> well...i'll keep at it...
19:19:40 <amitgandhinz> ok
19:19:45 <miqui> probably get it wrong first couple of passes...
19:20:01 <miqui> so miost likely i'll be asking a review
19:20:26 <amitgandhinz> thats fine, all part of the process =)
19:21:00 <amitgandhinz> ok any other updates for any of the bp here https://launchpad.net/poppy/+milestone/kilo-3
19:21:11 <amitgandhinz> kilo 3 ends on March 5
19:21:35 <obulpathi> nop
19:21:39 <obulpathi> no updates
19:21:50 <amitgandhinz> most of these stories are related to adding some more basic features customers need, and cleaning up code to make it better or more maintainable
19:22:29 <amitgandhinz> ok moving on then...
19:22:39 <amitgandhinz> #topic New Items - Taskflow vs Celery Discussion
19:22:46 <tonytan4ever> OK
19:23:00 <amitgandhinz> from miqui: http://paste.forj.io/show/289/
19:23:14 <miqui> yeah that was my chat with tf folks...
19:23:22 <amitgandhinz> go for it tonytan4ever, tell us what you learned over the past week
19:23:29 <miqui> then started looking at their architecture...
19:24:12 <tonytan4ever> In short TaskFlow provides mechanism for workflow patterns, gives you ability to build task based on dependencies and retries, and things of that nature.
19:25:12 <amitgandhinz> so our tasks and workflow is basically 1. talk to cdn provider.  2. update storage.  3. update dns
19:25:13 <tonytan4ever> So the dependencies and retry mechanism are buzzword here
19:25:40 <tonytan4ever> Yes, execept in create we need to update storage first.
19:25:45 <amitgandhinz> ok
19:25:52 <cpowell_> from miqui's discussion, "celery is highly (imho) intrusive" stuck out to me alot
19:25:58 <amitgandhinz> and dependency is 2 only happens after 1 succeeds?
19:26:15 <tonytan4ever> Yeah
19:26:28 <tonytan4ever> But I argue since we only have three tasks at most
19:26:34 <amitgandhinz> and retry is retry on failure (eg rate limiting, server unavailable, etc)
19:26:47 <tonytan4ever> yeh
19:27:15 <miqui> true, workflow might be simple now...but could grow tomore use cases...
19:27:17 <tonytan4ever> Following what I want to argue, since we only have 3 tasks at most, their dependencies/retries are not that important.
19:27:43 <tonytan4ever> And TF does have a draw back that monitoring worker process is not as easy as in Celery.
19:27:47 <amitgandhinz> yeh, that could be added later if needed/wanted
19:27:55 <obulpathi> +1
19:28:04 <amitgandhinz> how do you monitor workers in TF?
19:28:11 <tonytan4ever> You can only read logs.
19:28:12 <miqui> good qurstion...
19:28:39 <tonytan4ever> Celery has flower which is favored by me and will be liked by Ops as well.
19:29:03 <amitgandhinz> harlowja: ^^
19:29:24 <harlowja> hallo
19:29:34 <obulpathi> hi
19:29:38 <miqui> ...hi
19:29:42 <amitgandhinz> harlowja: if you have a sec, the Poppy team is discussing between choosing TaskFlow or Celery for our tasks/workers
19:29:42 <tonytan4ever> H harlowja
19:29:47 <harlowja> hi there
19:30:02 <harlowja> well i'm still of the believe that celery is a full-app solution, u adopt it all or none of it :)
19:30:11 <harlowja> while taskflow i think isn't
19:30:16 <harlowja> *or wasn't intended to be
19:30:28 <amitgandhinz> how does monitoring workers work with TF?
19:30:52 <harlowja> so it doesn't have a flower like thing (yet) but u are free to make one
19:31:07 <harlowja> but depends on what u mean by monitoring workers
19:31:20 <miqui> up/down?
19:31:34 <amitgandhinz> up/down, do we need to scale them out more
19:31:38 <tonytan4ever> For example, we need to know if a worker has been successfully completed
19:31:39 <obulpathi> progress and failures
19:31:40 <miqui> reached a threshold (like max pool size)
19:31:43 <harlowja> don't there exist a ton of existing solutions to monitor this stuff? :)
19:32:15 <harlowja> well completed succesfully is different than up/down; up/down i'd hope can be monitored by the existing tools that monitor process no?
19:32:28 <miqui> yes
19:32:36 <amitgandhinz> cpowell_: thoughts on monitoring?
19:32:37 <harlowja> progress and failures, that would need to be added, but i don't suspect its hard
19:33:13 <cpowell_> normally just looking for process name
19:33:24 <cpowell_> count number of PIDs
19:34:17 <cpowell_> even with Celery, your celery worker would be managed by Supervisord or something
19:34:25 <miqui> sensuapp easily do this...
19:34:38 <cpowell_> <3 Sensu
19:34:44 <amitgandhinz> lol
19:34:44 <harlowja> ya, i think there is like 10 trillion things that do this no?
19:34:51 <miqui> yup
19:35:00 <miqui> nagios 4, etc...etc....
19:35:21 <amitgandhinz> tonytan4ever: so what does celery's flower give us that we need?
19:35:35 <tonytan4ever> It gives us all what we need.
19:35:55 <harlowja> pretty pictures :-P
19:36:00 <tonytan4ever> Up/down, scale situation, task success/failure.
19:36:07 <cpowell_> have we considered documentation?
19:36:15 <tonytan4ever> Yeah, and it's pretty.
19:36:31 <harlowja> if celery docs are much better than taskflow, i failed, ha
19:36:37 <harlowja> i hope its nearly the same :-P
19:36:46 <harlowja> *and/or better
19:37:06 <amitgandhinz> how widely used is taskflow currently (in production systems)?
19:37:08 <tonytan4ever> I would say I can read both celery and tf. Tf's docs are fine for me.
19:37:24 <harlowja> amitgandhinz so cinder has been using it (not in the same manner u have in mind) for a while
19:37:31 <harlowja> http://www.rackspace.com/cloud/big-data/ uses the workers
19:37:35 <harlowja> and i guess thats been going for a year
19:37:49 <harlowja> mirantis uses it in https://github.com/MirantisLabs/pumphouse
19:38:08 <harlowja> it though becomes a question of what funcitonality do u want to us
19:38:09 <harlowja> *use
19:38:22 <harlowja> since u don't have to use the workers (if u just want to run workflows in local threads/processes)
19:38:42 <harlowja> celery i think has bigger names, but thats imho partially because its older
19:38:55 <amitgandhinz> yeh we want it to perform async tasks that are initiated from the rest api, so we will be using the workers
19:39:34 <tonytan4ever> Yeah. To use worker/queue is our primary reason to have this discussion here in the first.
19:39:35 <harlowja> imho; i'd do an evaluation, decide on what u guys feel is appropriate and go with it (if its not taskflow, thats fine)
19:39:47 <harlowja> use the right tool that u guys feel is ok :)
19:40:06 <harlowja> celery hasn't been accepted to well into openstack (its been tried like 3+ times)
19:40:34 <harlowja> but maybe that situation is better now
19:40:52 <amitgandhinz> i saw barbican's issues with getting it accepted, but wasnt sure how wide spread its lack of acceptance was
19:41:17 <tonytan4ever> Yeah, plus celery is FreeBSD license.
19:41:46 <amitgandhinz> tonytan4ever: from your view, besides monitoring, what other benefits do you see with using celery?
19:41:52 <harlowja> ah the good ole http://lists.openstack.org/pipermail/openstack-tc/2013-December/000436.html :-/
19:42:02 <amitgandhinz> harlowja: lol ya
19:42:11 <harlowja> amitgandhinz i don't think its been attempted since
19:42:21 <harlowja> but its been a year, maybe time to try again :)
19:42:27 <harlowja> *seems to happen once per year, lol
19:42:37 <harlowja> *year+-
19:42:43 <tonytan4ever> Celery also has simple exception/failure handling mechanism, for us only with three tasks it should be good enough.
19:43:15 <harlowja> tonytan4ever what does freebsd license help?
19:43:17 <harlowja> apache not good?
19:43:31 * harlowja curious
19:43:44 <miqui> i thought apache was preferred...
19:43:53 <tonytan4ever> That's a drawback of celery comparing to TF, harlowja
19:44:00 <amitgandhinz> i think thats what tonytan4ever was implying =P
19:44:12 <harlowja> tonytan4ever oh, i'm not such a license expert, not really sure the diff, ha
19:44:33 <tonytan4ever> And we can adopt our current async workers for celery task functions pretty easily.
19:44:44 <tonytan4ever> Not too much code their.
19:44:56 <tonytan4ever> This is the benefit of celery ^^
19:45:40 <harlowja> meh, i'm meh meh on that, stuff is never as easy as that ;)
19:46:01 <tonytan4ever> With tf we'd have to break those function down into tasks and combine them into flows.
19:46:20 <obulpathi> The Apache 2.0 licenses contain a patent grant, which means that at least the authors of the code are giving you any rights that you need for the authors' patents that happen to be in the code that you are using.
19:46:34 <harlowja> tonytan4ever ya, the amount of refactoring either way is imho marginal
19:46:37 <obulpathi> FreeBSD license does not grant the use the rights on patents used by the code
19:46:40 <amitgandhinz> im ok with a bit more work if the code becomes more granular into units
19:46:46 <obulpathi> s othat the major upside of Apache license
19:46:53 <harlowja> obulpathi good to know
19:47:01 <obulpathi> :)
19:47:39 <amitgandhinz> ok do we have more questions on celery vs taskflow or are we ready to vote on it?
19:47:39 <miqui> amitgandhinz: +1, granular so it does one thing and one thing well..
19:47:56 <harlowja> it might be useful to prototype both ways, idk
19:47:57 <harlowja> just a thought
19:48:10 <harlowja> *up to u guys though
19:48:21 <miqui> /and let tf  handle the life cycle of it...
19:49:26 <amitgandhinz> #startvote Should we implement Taskflow or Celery?  Taskflow, Celery, Both
19:49:27 <openstack> Begin voting on: Should we implement Taskflow or Celery? Valid vote options are Taskflow, Celery, Both.
19:49:28 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
19:49:54 <cpowell_> well, judging from the tone of the email harlowja linked, I think that celery is a non-starter...
19:49:57 <tonytan4ever> #vote Celery
19:50:15 <harlowja> cpowell_ the tone might of calmed down over the past year (i'm must not sure), lol
19:50:20 <harlowja> *i'm just not sure
19:50:35 <amitgandhinz> we could always seed the thought again on the dev list
19:51:01 <harlowja> *bring your flak jackets (just incase)
19:51:05 <amitgandhinz> lol
19:51:24 <amitgandhinz> cpowell_: miqui: obulpathi: ready to vote yet, or do you have more questions?
19:51:34 <cpowell_> are there docs for prodution installation/configuration
19:51:51 <cpowell_> I see how to use it in the code...
19:51:59 <miqui> #vote TaskFlow
19:52:09 <harlowja> probably could be made better cpowell_  in that area; but i don't think its to much work there
19:52:26 <harlowja> be nice to have some examples/recommendations for that  i guess
19:52:57 <harlowja> the rackspace guys maybe can make that (they've been using this model/workers... for a whie)
19:52:58 <harlowja> *while
19:53:12 * harlowja will poke them on this, ha
19:53:21 <harlowja> and poke and poke :-P
19:53:21 <amitgandhinz> cpowell_: we should sync up with the big data folks
19:53:35 <cpowell_> ok
19:53:39 <harlowja> def amitgandhinz cpowell_ #openstack-state-management channel (dkrause or jimbohickville)
19:53:52 <harlowja> dkrause not on right now, but jimbo (greg) is
19:54:00 <amitgandhinz> we can do that offline
19:54:01 <harlowja> nm, he left to, ha
19:54:09 * amitgandhinz heard us and ran =P
19:54:11 <harlowja> :)
19:54:21 <amitgandhinz> ok only 6 minutes left
19:54:24 <amitgandhinz> can we vote?
19:55:21 <cpowell_> I reserve the right to complain about either choice
19:55:32 <harlowja> lol
19:55:35 <cpowell_> #vote TaskFlow
19:55:36 <tonytan4ever> You can vote both right, lol
19:55:37 <amitgandhinz> lol, no we hold you to it for ever and ever
19:55:40 <obulpathi> #vote None (Start with only queues and implement worker mechanism later, depending on the needs and our ability to handle TaskFlow. I prefer TaskFlow tough, if we can manage it)
19:55:41 <openstack> obulpathi: None (Start with only queues and implement worker mechanism later, depending on the needs and our ability to handle TaskFlow. I prefer TaskFlow tough, if we can manage it) is not a valid option. Valid options are Taskflow, Celery, Both.
19:55:54 <obulpathi> cpowell_: Same thoughts here
19:56:06 <amitgandhinz> #vote TaskFlow
19:56:22 <tonytan4ever> OK. I guess it's TaskFlow then...
19:56:23 <obulpathi> #vote TaskFlow
19:56:26 <cpowell_> I am insterested in TaskFlow, but from an Ops perspective I need to see more
19:56:47 <amitgandhinz> miqui: ???
19:57:42 <amitgandhinz> its going to have to start with a prototype either way
19:57:48 <miqui> voting..
19:57:52 <miqui> #vote TaskFlow
19:58:16 <amitgandhinz> ok i think thats everyone
19:58:21 <amitgandhinz> #endvote
19:58:22 <openstack> Voted on "Should we implement Taskflow or Celery?" Results are
19:58:23 <openstack> Celery (1): tonytan4ever
19:58:24 <openstack> Taskflow (4): miqui, obulpathi, cpowell_, amitgandhinz
19:58:44 <amitgandhinz> ok, so lets do a prototype of taskflow
19:59:03 <tonytan4ever> I am alway be minority... Lol.
19:59:03 <amitgandhinz> and we should build it in a way such that if in the future we want to build celery workers we arent locked into a corner
19:59:16 <amitgandhinz> tonytan4ever: even minority's have a voice =)
19:59:26 <tonytan4ever> OK.
19:59:31 <tonytan4ever> agreed.
19:59:47 <amitgandhinz> the kicker for me i feel is openstack acceptance of taskflow, and the monitoring is still possible with taskflow workers (with a little mrore work)
20:00:06 <amitgandhinz> and the nice to have of smaller units of tasks makes for better code
20:00:12 <amitgandhinz> at the expense of more time to build it
20:00:16 <amitgandhinz> anyway, time is up
20:00:23 <amitgandhinz> we should continue discussing in the channel
20:00:24 <amitgandhinz> thanks harlowja
20:00:25 <harlowja> cools, #openstack-state-management guys
20:00:28 <tonytan4ever> See you all.
20:00:36 <amitgandhinz> #endmeeting