16:00:09 <rakhmerov> #startmeeting Mistral
16:00:10 <openstack> Meeting started Mon Apr  7 16:00:09 2014 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:14 <openstack> The meeting name has been set to 'mistral'
16:00:18 <xazel> hey
16:00:27 <rakhmerov> hi
16:02:28 <rakhmerov> just to know, is there anyone else here with us?
16:02:53 <NikolayM> hi Renat!
16:02:59 <rakhmerov> hi!
16:03:04 <dzimine> hi all !
16:03:10 <rakhmerov> hi!
16:03:11 <NikolayM> hi!
16:04:14 <rakhmerov> ok, let's quickly go over AIs
16:04:21 <rakhmerov> #topic Action Items
16:04:37 <rakhmerov> 1. (all) review and discuss Kirill's stub on MistralTaskflow prototype https://github.com/enykeev/mistral/pull/1
16:05:00 <rakhmerov> I personally reviewed it with Kirill's help
16:05:36 <rakhmerov> we also had a meeting with Joshua last week where we discussed the further direction of this work
16:06:11 <rakhmerov> currently we're waiting for more detailed design on ideas provided by Ivan Melnikov
16:06:42 <rakhmerov> and enykeev keeps tracking it and working on it
16:06:42 <NikolayM> Is it any conclusion with intagration TaskFlow in Mistral?
16:07:29 <rakhmerov> enykeev is now writing sorta unit tests for functionality that we need in TaskFlow but doesn't exist yet
16:08:12 <rakhmerov> It's hard to say if there are certain conclusions at this point
16:08:37 <rakhmerov> TF team seems to agree with us almost on all points that we've been discussing
16:08:59 <rakhmerov> we're now trying to convince them to start working on required changes
16:09:30 <rakhmerov> anyway it's going to be tough and take a long time (i'm pretty sure)
16:09:44 <rakhmerov> ok, next AI
16:09:50 <rakhmerov> 2. (tnurlygayanov) plan to write new tempest tests with execution of the real workflows
16:10:27 <rakhmerov> it is in progress, Timur has not been effectively available for a while
16:10:33 <rakhmerov> I'll keep it
16:10:36 <rakhmerov> #action (tnurlygayanov) plan to write new tempest tests with execution of the real workflows
16:10:46 <rakhmerov> 3. Timur, make sure to look at running Mistral tests in devstack gate
16:10:56 <rakhmerov> the same as 2
16:11:43 <rakhmerov> actually on Friday I got all the information I needed on formal requirements for the project to get incubated
16:11:57 <rakhmerov> this item is just one part of it
16:12:34 <rakhmerov> we're planning to review all these requirements and submit all the BPs bases on this info so that we can assign them
16:12:40 <rakhmerov> #action Timur, make sure to look at running Mistral tests in devstack gate
16:12:54 <rakhmerov> 4. review https://review.openstack.org/#/c/81941/ ignoring -1 from murano-ci, change it once we give +
16:12:56 <rakhmerov> this is done
16:13:02 <rakhmerov> it got merged in
16:13:18 <dzimine> nice
16:13:43 <m4dcoder> thanks!
16:13:44 <rakhmerov> yes, it's a cool patch
16:13:53 <rakhmerov> thanks Winson
16:13:57 <rakhmerov> nice job
16:14:02 <m4dcoder> :D
16:14:40 <rakhmerov> btw, I asked a couple of question about your latest commits (and other folks too), please take a look
16:15:01 <rakhmerov> generally I'm ok with it, just two things related with project structure and naming
16:15:07 <rakhmerov> 5. Find a solution for the question asked in https://review.openstack.org/#/c/83404/2/mistral/engine/data_flow.py
16:15:26 <rakhmerov> nmakhotkin, did we decide something on it?
16:15:52 <rakhmerov> we had a couple of suggestions last week but I'm not sure if it's still relevant
16:16:03 <rakhmerov> if the problem still exists
16:17:29 <NikolayM> nobody gives me some advice regarding to it
16:17:36 <rakhmerov> oooh
16:17:52 <NikolayM> I think we'll do it in the future
16:18:05 <rakhmerov> yes, I agree with it
16:18:10 <NikolayM> just leave todo
16:18:20 <rakhmerov> I have a couple of thoughts, let's take it offline
16:18:34 <rakhmerov> #topic Current Status
16:18:46 <rakhmerov> let's quickly report everyone's status
16:19:14 <dzimine> I've spent most of my Mistral time reviewing TaskFlow and Mistral integration.
16:19:34 <dzimine> This week I plan to work with Kirill on this and do post teh  summary
16:19:47 <rakhmerov> ok
16:19:58 <rakhmerov> my status is: fully got TaskFlow based prototype (i was so unhappy with what Kirill had to do to make it all work)
16:20:41 <rakhmerov> Kirill did a good job but what I saw just scared me because I made sure again there's a lot of work to do in TF
16:21:04 <NikolayM> I've added SSH action to our action list and sent the commit - https://review.openstack.org/#/c/85733/
16:21:16 <rakhmerov> I also made a series of refinements in https://wiki.openstack.org/wiki/Mistral/Blueprints/ActionsDesign and today started working on it (one patch is on review)
16:21:38 <enykeev> Still working on prototype. Here is first take on Lazy Engine unit https://github.com/enykeev/lazyengine
16:21:44 <rakhmerov> I'm planning to spend around 10-12 hours on it
16:22:04 <m4dcoder> thanks for doing the TF integration guys!  i know it's hard work.  but i think it'll be better for mistral in the long run toward being an official openstack project.  great job!
16:22:32 <rakhmerov> m4dcoder, yeah, you're right
16:22:54 <rakhmerov> it's not easy but we made some progress on that
16:23:25 <rakhmerov> dzimine, do you know what Manas is doing?
16:23:39 <rakhmerov> I'm not sure he has something to do actually :)
16:23:51 <m4dcoder> I'm removing the local engine.  I will be working on moving the engine to a standalone process next.  I'll remove "scalable" from the namespace when I do that.
16:24:13 <dzimine> he's done with Repeater, and has no next tasks
16:25:20 <dzimine> @m4dcoder - are you using oslo messaging the same way as with executors? I would just like to have the engine configurable to run in and out-of-process.
16:25:23 <rakhmerov> m4dcoder, yes, great. Please leave your comments for https://review.openstack.org/#/c/85575/ if we misunderstood something or you are planning to fix these things moving forward
16:25:41 <m4dcoder> i'll fix.  it was a mistake.
16:25:51 <rakhmerov> alright
16:25:52 <m4dcoder> previous it was in conflict with from mistral.engine import engine.
16:26:02 <rakhmerov> ok
16:26:48 <rakhmerov> dzimine, I'll find a task for Manas.
16:27:42 <rakhmerov> ok, guys, any roadblocks in your current work that you'd want to discuss?
16:28:13 <dzimine> what is between us and POC?
16:28:34 <rakhmerov> what do you mean by "between us"?
16:28:47 <rakhmerov> ooh, I see
16:28:51 <dzimine> what is left to do to complete POC scope?
16:29:06 <rakhmerov> this was going to be our next topic
16:29:06 <dzimine> run 'realistic' workflow against Openstack cluster?
16:29:18 <rakhmerov> #topic POC demo scenario readiness and ways to improve it
16:29:23 <dzimine> ok, move on
16:29:33 <rakhmerov> so, the summary
16:30:24 <rakhmerov> I was planning to spend the whole meeting today discussing Demo scenario looking at real workflow example
16:31:09 <rakhmerov> nmakhotkin keeps working on it, he was able to make something run
16:32:00 <rakhmerov> so, generally it's not done yet, but we're close
16:32:29 <rakhmerov> afaik, nmakhotking joined VM creation and waiting till it's up, right Nikolay?
16:33:09 <rakhmerov> and during his work he fixed several bugs including one in repeater
16:33:11 <NikolayM> yes, we can create VM, wait and fetch its IP, wait till OS is up (by sending SSH cmd) and so on
16:33:25 <rakhmerov> so repeater now works fine in different combinations
16:33:38 <NikolayM> e.g. launch our example app and do http request to it
16:33:45 <rakhmerov> ok
16:34:00 <rakhmerov> so as far as what's left, I'd really like to finish https://wiki.openstack.org/wiki/Mistral/Blueprints/ActionsDesign
16:34:23 <rakhmerov> everything except plugin architecture
16:34:56 <dzimine> rakhmerov, do you plan to remove launchign engine instance on the executor?
16:34:56 <rakhmerov> otherwise we won't be able to demonstrate any consistent DSL, it's just ugly now in some places
16:35:49 <rakhmerov> dzimine, no, I wasn't planning to do it any time soon, I think it's more related with what m4dcoder is doing
16:35:56 <dzimine> ok.
16:36:04 <dzimine> yes
16:36:47 <rakhmerov> so, for now, I'd suggest we discuss the scenario in general
16:38:33 <rakhmerov> I'll just remind the idea of it: create VM -> wait till it's up (via SSH action) -> send an HTTP req to preinstalled web service to do some calculation -> destroy VM. In any error scenario send an email
16:39:16 <rakhmerov> I'm not 100% happy with it and there may be more interesting ideas to suggest
16:39:51 <rakhmerov> currently we're targeting to make this scenario work (I believe we're in 2-3 days from the moment it will start working)
16:40:08 <rakhmerov> dzimine, what do you think?
16:40:11 <dzimine> we have 4 'real life' scenarios from w4dcoder's team
16:40:29 <dzimine> I don't know if they are going to be too complex...
16:40:37 <dzimine> Renat did you have a look?
16:40:43 <rakhmerov> yes, aren't they too complicated for Demo scenario?
16:41:08 <rakhmerov> yes, I did but not very deeply yet
16:41:08 <dzimine> That's what I thought, but take a look may be we can simplify them somehow?
16:41:17 <rakhmerov> ok
16:41:32 <dzimine> check your mail, too, there was a description of 2 more which I didn't put into workflows yet/
16:41:46 <rakhmerov> yep, I saw it
16:41:57 <rakhmerov> nmakhotkin, can you please also take a look at them all?
16:41:59 <dzimine> The excercise of writing workflow in YAML is very educational
16:42:09 <rakhmerov> :)
16:42:20 <m4dcoder> any from murano?  for example like building out a cluster of mysql?
16:42:24 <dzimine> when you do it you really appreciate what's good and what's constrained in the syntax.
16:42:49 <rakhmerov> yeah
16:43:24 <NikolayM> #action (nmakhotkin) take a look at dzimine's and m4dcoder's workflows
16:43:31 <rakhmerov> ok, I think we should finish the current one anyway. Since even trying to make this simple thing work we're finding serious problems :)
16:43:57 <dzimine> Agree: let's stay on course, have this one complete, then add others.
16:43:59 <rakhmerov> and in parallel we need to carefully look at these examples and see if we can make a show out of them :)
16:44:08 <rakhmerov> ok
16:44:52 <rakhmerov> m4dcoder, as far as Murano, we don't have a complete example from them even though we understand the general idea of how they can use Mistral
16:44:57 <dzimine> And to move forward to complicated workflows, I suggest we first implement https://blueprints.launchpad.net/mistral/+spec/mistral-shorthand-action-in-dsl
16:45:10 <dzimine> else, writing workflow is a bitch
16:45:22 <rakhmerov> we planned to start a discussion with Murano team this week and see how we can integrate
16:46:03 <dzimine> and probably this, too https://blueprints.launchpad.net/mistral/+spec/mistral-pluggable-task-actions
16:46:08 <rakhmerov> dzimine, yeah-yeah, absolutely. It's basically now a part of BP I mentioned above
16:46:14 <rakhmerov> which I'm currently working on
16:47:18 <rakhmerov> I had doubts if we need to implement pluggable actions now but I just need to estimate it, I have zero experience with plugin architecture in python
16:47:31 <rakhmerov> belive it's not that hard to do but just want to make sure
16:49:05 <dzimine> no let's not do them now,
16:49:11 <rakhmerov> #action Finish current Demo scenario (make it work) and evaluate workflow examples from Winson's team to find better ideas for Demo
16:49:13 <dzimine> but after we reach the POC scope
16:49:24 <rakhmerov> ok, that's what I thought, right
16:49:36 <rakhmerov> ok, we had another topic
16:49:39 <rakhmerov> #topic TaskFlow integration status
16:50:10 <rakhmerov> enykeev, basically I already wrote about it but do you have anything to add on that?
16:50:40 <dzimine> rakhmerov what do you refer to by 'wrote'?
16:51:20 <enykeev> i have not. Actually, i'm interested, is there are anything about that in a formal requirements you received at friday?
16:51:21 <rakhmerov> I mean I already wrote briefly about TaskFlow integration status
16:51:57 <rakhmerov> :))
16:52:07 <rakhmerov> you remember the meeting we had, hah :))
16:54:01 <rakhmerov> the requirements are basically described in that pull request at github
16:54:03 <enykeev> So, basically, right now we are waiting TF team to respond to us with their view on the problem...
16:54:11 <rakhmerov> ok
16:54:44 <rakhmerov> so it's kind of blurry
16:55:02 <rakhmerov> #topic Open Discussion
16:55:02 <dzimine> and we need to finish our summary as discussed last week, and post it out.
16:55:14 <rakhmerov> yes
16:55:28 <rakhmerov> I guess Kirill is working on it now
16:55:45 <rakhmerov> we have about 4 mins
16:56:06 <rakhmerov> guys, there was an idea today on how we can interact with TF team
16:56:14 <rakhmerov> interested in your input
16:56:46 <rakhmerov> enykeev, dzimine, what do you think if we just start making changes in TF and sending patches to them with what we need?
16:57:08 <dzimine> I think we need to give them some time to digest our input
16:57:10 <rakhmerov> maybe it's going to be more effective if we both look at the same code in TF gerrit?
16:57:29 <dzimine> after which, may be!
16:57:56 <enykeev> i agree with dzimine, lets not rush with that one
16:58:01 <dzimine> other is I really want TaskFlow team to turn to look into making Workflow reach (conditional transtions, etc)
16:58:01 <rakhmerov> yeah, I just mean we could help them digest a little bit better (faster) :)
16:58:12 <dzimine> :)
16:58:48 <rakhmerov> yes, that one also scares me, we didn't even touch it so far (my gosh...)
16:58:49 <dzimine> m4dcoder - what do you think? you see this both ways.
16:59:01 <rakhmerov> 1 min left folks
16:59:39 <dzimine> call it done :)
16:59:44 <rakhmerov> ok, let's wrap up :)
16:59:47 <rakhmerov> thanks!
16:59:49 <rakhmerov> bye
16:59:53 <rakhmerov> #endmeeting