16:00:08 <rakhmerov> #startmeeting Mistral
16:00:09 <openstack> Meeting started Mon Nov 25 16:00:08 2013 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:13 <openstack> The meeting name has been set to 'mistral'
16:00:17 <rakhmerov> hi all
16:00:46 <akuznetsov> hi
16:02:28 <rakhmerov> ok, let's start our weekly meeting
16:03:05 <rakhmerov> #topic Review Previous Action Items
16:03:31 <rakhmerov> We didn't assign any major action items last week actually
16:04:01 <rakhmerov> we had 3 minor AIs and they're all done I believe (bluprints priorities and so on).
16:04:46 <rakhmerov> I'd like to provide just a quick update on the activities that have been going on
16:05:06 <rakhmerov> #topic Current activities
16:05:34 <rakhmerov> So basically, we started working on PoC last week
16:07:14 <rakhmerov> which is going to include very basic functionality: simple DSL (tasks, actions, events), simple REST API and simple non-HA Workflow Execution (the thing that's responsible for running workflows)
16:08:00 <rakhmerov> we came up with a simple design for POC at this point that we're going to start discussing with the community this week
16:08:52 <rakhmerov> We also shared the first version of Mistral API at https://etherpad.openstack.org/p/MistralAPISpecification
16:09:24 <rakhmerov> and we invite everyone to comment on this
16:10:00 <rakhmerov> I believe we'll publish our first DSL specification in 1-2 days too at https://etherpad.openstack.org/p/MistralDSLSpecification
16:10:09 <rakhmerov> it's now almost empty
16:10:47 <akuznetsov> I hope it will be tomorrow
16:10:52 <rakhmerov> yes
16:11:03 <rakhmerov> cool
16:12:18 <rakhmerov> So if anyone has any questions we can discuss it right now
16:12:54 <rakhmerov> basically, our agenda for today is to talk about API/DSL  and PoC design
16:13:25 <rakhmerov> btw, we're targeting to get PoC done around Dec 10th
16:14:49 <rakhmerov> #topic Discuss PoC design, API spec
16:16:07 <rakhmerov> #info you now can find all the Mistral etherpads on the wiki at https://wiki.openstack.org/wiki/Mistral#Etherpads
16:17:08 <rakhmerov> jaypipes, do you have some questions that you'd like to discuss?
16:20:28 <rakhmerov> ok, does anyone have anything to ask/discuss?
16:22:10 <akuznetsov> Do we need include a predefined list of action into our dsl like Nova instances creation?
16:22:56 <rakhmerov> that's a good question
16:24:30 <rakhmerov> initially we thought that actions (associated with tasks) should be as generic as possible
16:27:26 <rakhmerov> and only include things like "send HTTP request", "send AMQP message"
16:27:32 <rakhmerov> and things like that
16:28:39 <rakhmerov> and there was a concern about including anything like Nova and other more high-level things because it may become a pile of different specific things
16:28:58 <akuznetsov> adding such action allows better integration with OpenStack and people will not reinvent bicycles for such types of actions
16:29:24 <stanlagun> this depends on how exectly you gonna add them
16:29:35 <rakhmerov> options?
16:30:01 <rakhmerov> I personally tend to think we may want to add something like nova api out of the box
16:30:16 <rakhmerov> just to make the product more valueable for consumers
16:30:26 <rakhmerov> but I think we should understand the scope of that
16:30:35 <stanlagun> There is no "code" in Mistral Actions, it cannot execute arbgitrary Python code. So I don't see how Python client can be referenced from Mistral workflow
16:30:41 <rakhmerov> we should not try to include whatever may be included
16:31:30 <rakhmerov> well, for our actions we could, for example, write something like: nova.createVM(…)
16:32:08 <rakhmerov> the alternative is to manually send HTTP requests to nova by specifying URLs and their parameters
16:32:10 <stanlagun> For now it inough that clietn app can use it. In the future we may try implement more sophisticated DSL with universal REST API client or something like that
16:32:52 <rakhmerov> yes, that was one of the interesting ideas that we had but postponed for a while
16:32:53 <stanlagun> rakhmerov: How do you implement authentication, dynamic URI schema etc&
16:33:30 <rakhmerov> stanlagun, we could have everything we need as action properties
16:33:59 <rakhmerov> one way or another, it could be done in DSL
16:34:14 <rakhmerov> the question is "Do we really want it?"
16:34:44 <rakhmerov> we thought that additional things like that might be implemented as:
16:34:52 <rakhmerov> 1) Plugins
16:35:00 <stanlagun> I don't think we can. Suppose that URL for method 2 contains something that is obtained from a call to method 1. How do you describe it in DSL?
16:35:07 <rakhmerov> 2) Additional frameworks/toolsets on top of Mistral
16:35:39 <stanlagun> The root problem is how to reference that code (be it built-in or plugin) from DSL
16:36:07 <rakhmerov> some sort of a namespace inroduced by a plugin I think
16:36:17 <rakhmerov> i.e., nova:createVM()
16:36:28 <rakhmerov> where nova: namespace has been added by a plugin
16:36:51 <stanlagun> Looks like a suggestion to write another programming language
16:37:27 <akuznetsov> stanlagun do you really what that everyone how will use Mistral and invoke Nova should write his own an actions?
16:37:39 <stanlagun> you will also need parameter passing, data manipulation, branching, loops, variables etc.
16:39:05 <stanlagun> akuznetsov: I haven't seen a way to do otherwise. Prove me wrong
16:39:59 <rakhmerov> in some form we might need all these things moving forward :) But we need to keep in mind that for vast majority of use cases DSL should be very very simple
16:40:17 <rakhmerov> otherwise nobody will be willing to use it
16:41:38 <stanlagun> The whole idea was that actions are in user's app. Doing something with Nova is an action. Users don't have to reinvent it as nova-client does all the authomation
16:42:19 <stanlagun> Anyway if you have an idea how to describe such actions in DSL we can discuss it
16:43:55 <rakhmerov> yes, I'm kinda in the middle here.. On one hand I agree with you, stanlagun, because we can compicate DSL drastically if we start adding lots of additional things
16:44:05 <rakhmerov> on the other hand, for some reason, I think we won't be able to avoid adding additional APIs like Nova in our DSL in the future
16:44:20 <rakhmerov> some people have already asked us about it
16:44:43 <rakhmerov> "Will we be able to call some core openstack APIs directly from a workflow?"
16:45:19 <rakhmerov> just not to worry about applications handling these plain HTTP request actions
16:45:46 <rakhmerov> it's like an additional unnecesary link in a chain for them
16:46:45 <rakhmerov> I would suggest we approach this problem in an evolutionary manner
16:46:46 <akuznetsov> rakhmerov agree with you
16:47:03 <stanlagun> I don't see it as a big problem supposed that addition of another action is straightforward and everything is already written in a form of python-clients. This is definitely simpler than turning DSL into programming language
16:47:38 <rakhmerov> yes, right, 100% agree but
16:47:39 <stanlagun> Again, I'm not against it. It is just nobody sad how
16:48:13 <stanlagun> ...said how
16:48:31 <rakhmerov> but to give control to pieces of code that works with python client we need to expose an endpoint first or start listening rabbit or whatever
16:48:59 <rakhmerov> having these capabilities right in DSl we woudn't need to expose anything in many cases
16:49:25 <rakhmerov> stanlagun, as far as "how", I personally don't see the whole picture too right now
16:49:41 <rakhmerov> but we didn't really think much on that
16:50:00 <rakhmerov> at least it seems to me (I didn't for sure)
16:50:01 <stanlagun> Yes. But what is simper - one line call to Mistral client library that does all the plumbing to listen for actions or coding all the callis in another DSL programming language from scratch?
16:50:28 <rakhmerov> not sure what you meant here
16:50:36 <rakhmerov> client library?
16:50:48 <rakhmerov> ooh, got it, ok
16:51:06 <stanlagun> Client apps need to listen to RabbitMQ, accept action calls etc. It can be automated. By us
16:51:39 <rakhmerov> that sounds good but I'm just not sure it's going to be one line. Just would love to see what it will look like
16:51:52 <rakhmerov> yes, understood
16:52:45 <stanlagun> I suggest we return to this as soon as someone has a clear understanding on how to implement it on server-side
16:53:20 <rakhmerov> something like: mistral_client.register_action_handler("create_vm", do_create_vm), where "do_create_vm" is a function to call on that action
16:53:45 <rakhmerov> akuznetsov, what do you think?
16:54:16 <rakhmerov> Stan, I would agree with it for now
16:54:29 <rakhmerov> I would prefer if we get there first
16:54:48 <stanlagun> this is not enaugh. You need to say what VM, pass additional parameters, make additional calls. This is not one-liner
16:54:49 <rakhmerov> just to have something working
16:55:24 <rakhmerov> yes, it was just a rough example, but the whole idea is like that
16:55:36 <stanlagun> the devil in details
16:55:46 <rakhmerov> we pass a callback while registering an action handler
16:55:49 <rakhmerov> yep, right
16:56:27 <akuznetsov> I opion that is too complicate for enduser
16:56:50 <rakhmerov> maybe
16:57:09 <rakhmerov> ok, I think we need to try to design how it may look like
16:59:17 <rakhmerov> #action Design how additional APIs may can be injected into DSL (i.e. Nova)
16:59:39 <rakhmerov> #action Design how Mistral client library may look like
16:59:50 <rakhmerov> ok, we need to wrap up, let's continue next week
16:59:54 <rakhmerov> thanks!
16:59:58 <rakhmerov> #endmeeting