20:01:14 <harlowja> #startmeeting state-management
20:01:15 <openstack> Meeting started Thu Aug  1 20:01:14 2013 UTC and is due to finish in 60 minutes.  The chair is harlowja. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:01:18 <openstack> The meeting name has been set to 'state_management'
20:01:57 <harlowja> howday all!
20:02:04 <changbl> hello there
20:02:13 <harlowja> hi hi
20:02:23 <akarpinska1> Hi
20:02:28 <harlowja> hi hi
20:02:51 * harlowja waits a few for others
20:03:26 <harlowja> jlucci u around, kevin?
20:04:08 <harlowja> ok well anyway, all fine
20:04:14 <harlowja> #topic status
20:04:35 <harlowja> so i've been trying to rebase the cinder patch and make sure thats all working, currently hitting some CI weirdness
20:04:43 <harlowja> #link https://review.openstack.org/#/c/29862/
20:04:55 <harlowja> working with the infra folks to try to figure out whats going on
20:05:15 <harlowja> also trying to help do reviews at the same time (and other internal y! stuff)
20:05:23 <kebray> harlowja  my apologies, I just got summoned to another meeting.
20:05:27 <harlowja> np
20:05:28 <jlucci> Yeah I'm here -sorry
20:05:35 <jlucci> Just got back from Dallas a couple minutes ago
20:05:38 <harlowja> ??
20:05:39 <harlowja> woah
20:05:41 <harlowja> ha
20:05:48 <jlucci> Yeah. I'm that good. haha
20:06:09 <harlowja> np, just going through status, not sure if u have anything to mention
20:06:30 <harlowja> kebray i think submitted the speaker session, so thx kebray  :-)
20:06:39 <jlucci> Um -just the distributed stuff online, and about to have a tasfklow video/tutorial thingy online
20:06:47 <jlucci> Was just going to e-mail it out to openstack-dev
20:06:49 <harlowja> about, wow
20:06:52 <harlowja> nice
20:07:01 <harlowja> very cool
20:07:09 <harlowja> speaker abstract btw
20:07:12 <harlowja> #link https://wiki.openstack.org/wiki/TaskFlow/HavanaSummitPresentationAbstract
20:07:18 <harlowja> (for HK summit)
20:07:31 <jlucci> Awesome kebray :D
20:07:40 <changbl> good job, like the abstract
20:07:45 <harlowja> yahoo will hopefully have a few others, time will tell :)
20:08:16 <harlowja> changbl thx for fixing a few issues u found :)
20:08:27 <changbl> np, the code is written well:)
20:08:37 <changbl> i looked through the code base
20:08:59 <harlowja> ha, its a continual WIP, thx
20:09:19 <changbl> sure, will keep watching the code :)
20:09:20 <harlowja> akarpinska1 anything for u to report, know its late for u so thx for showing up
20:09:32 <harlowja> don't give up on blocks, still think it has a place :)
20:10:15 <harlowja> thx changbl the more eyes the better of course :)
20:10:50 <akarpinska1> I'm a little bit confused about what I really need to do with my TaskFlow tasks and with my patch. Want to hear some comments about it.
20:11:59 <harlowja> sure, np
20:12:14 <jlucci> akarpinska1 - anastasia I'm guessing?
20:12:20 <akarpinska1> yes
20:12:32 <jlucci> haha Awesome. Yeah - I'm still looking through that doc you sent out
20:12:48 <harlowja> so akarpinska1 i put a few review comments up, nothing major, so that will be part 1 i think
20:13:05 <harlowja> kevin i think will land his db stuff soon to, so then there will be the question of how thats used there to
20:13:10 <jlucci> Awesome. I'll get to those/doc as soon as I post that video
20:13:22 <kchenweijie> yup. got about 2/3 of it commented today
20:13:26 <harlowja> sweet
20:13:30 <kchenweijie> taking a break from commenting for a little bit
20:14:00 <kchenweijie> but it will be up there by tomorrow morning at the latest
20:14:19 <harlowja> akarpinska1 so also once jlucci i think is ok with the block stuff (or other comments) then i think that will allow for a little more work there, then we can get that in, make more adjustments and such
20:14:24 <harlowja> kchenweijie sounds great
20:14:28 <harlowja> more comments the better :)
20:14:40 <harlowja> akarpinska1 sound good?
20:14:58 <harlowja> its a in progress library, so nothing will always be clear :)
20:15:03 <harlowja> although i'll try to help it be, ha
20:15:46 <akarpinska1> of course
20:15:55 <harlowja> sounds great
20:16:40 <harlowja> #topic projects_using
20:17:04 <harlowja> so if its fine just want to see where we are with other projects on getting usage of taskflow in
20:17:18 <harlowja> after i fix this cinder stuff, i'm gonna try to see about possible nova 'small' use-case soon
20:17:32 <harlowja> akarpinska1 might also be doing another cinder use-case, which is cool
20:17:38 <jlucci> Awesome - I'm getting ready to start doing some trove integration
20:17:39 <harlowja> and jlucci maybe reddwarf/trove
20:17:40 <lifeless> dkranz: mtreinish: there is regex exclusion
20:17:41 <harlowja> cool
20:17:50 <lifeless> dkranz: mtreinish: will find you in -qa to follow up.
20:18:09 <dkranz> lifeless: Thanks.
20:18:16 <harlowja> hmmm
20:18:19 <akarpinska1> I think, we need to choose one implementation and then apply it to Cinder
20:18:51 <jlucci> Implementation being patterns, or current task flow vs. new block approach?
20:18:57 <akarpinska1> Now we have two commands rewritten in different manner
20:19:13 <changbl> Can I ask what is "block"?
20:19:26 <jlucci> https://review.openstack.org/#/c/36472/
20:19:34 <jlucci> It's a new way of running tasks
20:19:47 <jlucci> I'm sure akarpinska1 can explain it much better than me though. (:
20:19:59 <changbl> what is different? any brief intro?
20:20:13 <harlowja> akarpinska1 i'm fine with picking one way, just unsure which way still, i think the current refactoring can be adjusted, but i don't want to miss H release for that
20:20:59 <akarpinska1> There is a documentation https://docs.google.com/document/d/1376-8vRgJRvbvQLY4ZBqmClJVa3HwCOaMoO2yRpOC1o/edit?usp=sharing
20:22:04 <harlowja> to me a block is similar to a way to structure a task right
20:22:15 <harlowja> *structure the way a task set is ran
20:22:41 <harlowja> which also can be composed of sub-blocks
20:23:22 <changbl> ok
20:23:30 <akarpinska1> harlowja: in your implementation there are three types of flows, blocks give us a possibility to combine different behavior in one flow
20:23:33 <changbl> will read the doc, thanks
20:23:54 <harlowja> akarpinska1 ya, which i think is good
20:24:03 <harlowja> changbl thx, hopefully it makes it more clear
20:24:13 <changbl> harlowja, np
20:24:34 <akarpinska1> also I tried to split it on small components to make it flexible to change
20:25:14 <harlowja> ya, which is good stuff, i'm still myself trying to figure out how it fits, thats all :)
20:25:56 <harlowja> the challenge becomes make something really easy for library users to use, without adding to much new syntax/structure that they need to learn
20:26:13 <changbl> agree, harlowja
20:26:22 <harlowja> blocks has alot of new structure right now that they must learn, thats my current worry about it
20:26:26 <akarpinska1> but user need to know only about the blocks
20:26:28 <changbl> as long as it can do the job, the simpler, the better
20:27:54 <akarpinska1> by default user should know nothing about the actions and builders until he wants to change internal implementation of the flow
20:28:40 <akarpinska1> User will describe his tasks and the flow structure with the blocks. That's all
20:29:13 <harlowja> sure, sure, not saying its bad or anything, just the block way does require a little bit more learning
20:30:26 <harlowja> i'm still working through my understanding of it also, so it might just be fine :)
20:30:46 <harlowja> its just the harder it is to compose these, the more pushback i think we will get
20:31:07 <harlowja> the nice thing about the linear_flow is that it likely solves most of the use-cases of openstack, ha
20:31:21 <harlowja> i can see the sequential block possibly being the same thing
20:31:54 <akarpinska1> Yes, like in Cinder create_snapshot, I used only the sequential block
20:32:09 <harlowja> ya, i'll try to find some time to mess around with the code also, that i think will help
20:32:30 <harlowja> maybe jlucci and changbl and others with free time (if any) can also
20:32:38 <harlowja> sound good akarpinska1 ?
20:32:56 <changbl> sure, will read the code and help with code review
20:33:02 <harlowja> thx
20:33:03 <jlucci> Sounds good
20:33:10 <akarpinska1> Will be excellent to get feedback
20:33:13 <harlowja> def
20:33:28 <harlowja> sounds great
20:34:05 <harlowja> i had another random idea also, but might not be needed if blocks works out
20:34:08 <harlowja> #link https://wiki.openstack.org/wiki/TaskMachine
20:34:20 <harlowja> something else to read over, ha
20:34:23 <harlowja> might be to much though
20:34:41 <harlowja> let me change topic to open discussion, since thats more releated to that :)
20:34:46 <harlowja> #topic open-discuss
20:35:32 <changbl> one question: shall we have more examples in docs/ ? Those examples are quite helpful
20:35:45 <harlowja> i think so :)
20:36:02 <changbl> harlowja, great
20:36:04 <harlowja> i'll try to make some of those up also, jlucci will your video introduce any?
20:36:25 <harlowja> akarpinska1 if u have any thoughts on https://wiki.openstack.org/wiki/TaskMachine let me know
20:36:35 <harlowja> it might be to similar to blocks that its not needed
20:36:40 <akarpinska1> Interesting, trying to understand
20:36:43 <harlowja> np
20:36:55 <jlucci> Yup. Although the video is geared towards a distributed backend, so the majority of the examples lean more towards that pattern
20:37:07 <harlowja> sure sure
20:37:24 <changbl> looking forward your video, jlucci
20:37:43 <harlowja> changbl i think after we get some of the DB stuff in it will become really neat with examples
20:37:55 <changbl> harlowja, yep!
20:38:03 <harlowja> like start a example process, have it do something, kill it, then see it resume after starting another process up
20:38:13 <harlowja> like magic, ha
20:38:37 <jlucci> hehe
20:38:58 <jlucci> But yeah - the DB stuff will provide for a ton of new functionality
20:39:17 <harlowja> akarpinska1 the idea was along the line that all these task things could be instrumented in a 'machine' that is devoted to running them with its own concept of 'memory'
20:39:24 <harlowja> then openstack is powered by a machine that runs tasks
20:39:25 <harlowja> ha
20:39:32 <harlowja> was just an interesting idea :-p
20:39:48 <harlowja> *not a physical machine, but a mini-one that runs in memory
20:40:04 <harlowja> and has enough control over what its running and its own 'memory' that we can just save/restore the machine
20:40:13 <akarpinska1> I'm trying to imagine how it can be implemented) sounds interesting, but it can be a lot of questions
20:40:27 <harlowja> well its similiar to the block stuff i think in a way
20:40:37 <harlowja> you have a concept of 'memory' there
20:40:44 <harlowja> and a runtime stack (block structure)
20:41:24 <akarpinska1> flow.kwargs is a memory )
20:41:27 <harlowja> ya
20:41:36 <harlowja> so they are similar in that way
20:41:48 <harlowja> if we had such machine it could be the base for all other patterns ontop
20:41:52 <harlowja> block pattern, linear, ...
20:41:59 <harlowja> *maybe*
20:41:59 <akarpinska1> I thought about some flows manager, maybe it can be a TaskMachine
20:42:11 <harlowja> possibly could be
20:42:36 <harlowja> the neat thing about the machine is that i think jlucci and all the dynamic stuff that is wanted there can also use the same machine
20:42:46 <harlowja> i'm not sure how blocks would fit into the dynamic modification model
20:42:56 <jlucci> Yeah, I'm still working through that
20:43:10 <jlucci> Need to get more familiar with the block code as well before I can really say something one way or the other
20:43:15 <harlowja> ya, same here :)
20:43:31 <changbl> me too ~
20:43:33 <akarpinska1> But one problem is that the service executes each command in the separate thread, as I know
20:44:00 <harlowja> what do u mean there
20:44:50 <akarpinska1> if we have a task machine, OpenStack services must be redesigned not to create a thread for the API request, but to redirect it to the TaskMachine
20:45:06 <harlowja> well depends on what level it is integrated with
20:45:22 <harlowja> an API request could just use a pool of machines
20:45:26 <harlowja> or make a new one
20:45:44 <harlowja> doesn't have to be 1:1 since the machine isn't phsyical
20:45:49 <akarpinska1> one machine for one request?
20:46:05 <harlowja> well the machine is just software, so it could be that
20:46:37 <harlowja> probably to complex for the foreseable future i think, but an interesting idea, ha
20:46:48 <harlowja> most of openstack i think will do with the basics for a long time, ha
20:47:03 <harlowja> *or be happy with the basics, even just the restructuring into 'tasks' is a big accomplishment
20:47:21 <harlowja> so maybe this is at stage ++++++
20:47:22 <harlowja> haha
20:47:43 <changbl> extremely like this is the case:)
20:47:49 <harlowja> ;)
20:47:52 <changbl> s/like/likely/
20:48:51 <harlowja> ya, interesting to think about, i'm pretty sure it will take a while to just do the task restructuring in all the projects
20:48:56 <akarpinska1> one machine for the whole service can easily lock resources
20:49:22 <harlowja> ya, akarpinska1 i don't think we would want 1 'task' machine
20:50:08 <harlowja> but if we can offer some neat possiblities, which i think we already are starting to, the more adoption we will get, which is great, but have to keep the complexity low, but features high, not an easy balance :)
20:50:24 <akarpinska1> 1 machine for one request is easily to implement =)
20:50:57 <harlowja> akarpinska1 ya, it likely is, it becomes an interstsing question if the 'machine' thing could be the base for all task patterns
20:51:11 <harlowja> be nice to have 1 lowest level component that everything is built on for said patterns
20:51:17 <harlowja> blocks, distributed...
20:51:52 <akarpinska1> In my implementation Flow class is like a flow controller, maybe it could be a machine later
20:51:53 <harlowja> the neat thing about the machine is that there can be conditional tasks that let u choose which task to put on the 'machine' stack next
20:52:10 <harlowja> akarpinska1 it might be able to
20:53:05 <harlowja> anyway, somethign to think about, think if we can get more project adoption that will be the key for the short-term
20:53:28 <harlowja> which means just beginning to use what we have and adjusting openstack code to use it, while at the same time learning from said adjustments and changing taskflow
20:53:43 <harlowja> then we'll have the best library and the usage in openstack in many places :)
20:53:45 <harlowja> a win for all
20:54:11 <jlucci> :D
20:54:18 <harlowja> sound good to everyone? :)
20:55:02 <jlucci> yup
20:55:15 <harlowja> sweet! and just in time
20:55:28 <harlowja> any further stuff, #openstack-state-management
20:55:32 <harlowja> keep up the good work all!
20:55:57 <jlucci> thanks
20:55:58 <changbl> harlowja, sounds good
20:56:01 <harlowja> :)
20:56:07 <akarpinska1> agree :)
20:56:24 <harlowja> woot
20:56:26 <harlowja> #endmeeting