16:00:14 <rakhmerov> #startmeeting Mistral
16:00:14 <openstack> Meeting started Mon Dec  2 16:00:14 2013 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:17 <openstack> The meeting name has been set to 'mistral'
16:01:30 <rakhmerov> here's the link to the agenda: https://wiki.openstack.org/wiki/Meetings/MistralAgenda
16:02:39 <gokrokve> Hi
16:02:49 <rakhmerov> I wasn't planning anything special or new for today's meeting so let's quickly cover previous action items and talk about Mistral design, DSL and API in a free manner
16:02:56 <rakhmerov> hi gokrokve
16:02:58 <rakhmerov> haven
16:03:05 <rakhmerov> haven't seen you for a while
16:03:54 <rakhmerov> so, action items
16:04:05 <rakhmerov> #topic Previous action items
16:04:19 <rakhmerov> 1. Design how additional APIs may can be injected into DSL
16:05:46 <rakhmerov> this question itself raised from the hot discussion we had about whether we need to include more high-level things into Mistral DSL like calling Nova, Glance or other OpenStack services
16:06:19 <rakhmerov> and we wanted to draft first how it can look in DSL if we call Nova
16:06:46 <gokrokve> I think yes, but not in the first phase.
16:07:08 <gokrokve> In general this is good idea to call APIs directly. The use case is live migration.
16:07:19 <rakhmerov> so the AI itself is in progress, we're now actively working on PoC but we can discuss it again
16:07:38 <rakhmerov> ok, gokrokve, let me explain what the problem was
16:07:50 <gokrokve> You will need to call nova to migrate a VM.
16:07:51 <rakhmerov> well, not the problem but rather a point of view agains it
16:08:07 <rakhmerov> yes, just a second, I'll explain
16:08:11 <gokrokve> ok
16:08:23 <rakhmerov> there are a few things here
16:09:21 <rakhmerov> first, we planned to implement Mistral so that it is agnostic of any special things that are not something really primitive like sending an HTTP request or a message over AMQP
16:10:19 <rakhmerov> because if we start adding some additional things it may quickly become a pile of different things and it's not clear where to stop
16:10:30 <rakhmerov> but that's the most important point
16:11:20 <rakhmerov> stanlagun said that if we want to implement all the capabilities to call OpenStack services then our DSL may become complicated
16:11:47 <rakhmerov> for example, we can't just call a REST API method without authenticating first
16:11:49 <stanlagun> rakhmerov: not exactly
16:12:07 <rakhmerov> and in most cases we'll need several calls instead of one
16:12:18 <rakhmerov> stanlagun, ok go ahead and explain yourself
16:13:17 <stanlagun> I think having Mistral call OpenStack APIs is a good idea. But not the way it was proposed. When we add ability to call arbitrty REST APIs, sub-workflows with enough capabilities to express branching, dataflow and loops then we could do it
16:13:36 <rakhmerov> so, in my understanding, the point is that we don't want to convert our DSL into a serious language
16:13:40 <stanlagun> Hardcoding several trivial one-liner actions into Mistral is useless
16:14:19 <rakhmerov> Stan, I was talking only about DSL
16:14:36 <rakhmerov> I just didn't finish explaining that a little bit :)
16:15:18 <stanlagun> This way you would not have to complicate DSL
16:15:54 <rakhmerov> so what is your suggestion?
16:16:11 <stanlagun> Calling OpenStack REST API would be no different as any other API and averyone would be able to take advantage of it or use any other REST API as well
16:17:08 <nmakhotkin> maybe we should include and support at least one service
16:17:11 <stanlagun> My suggestion is not to hurry and not to make useless shortcuts but to think on more fundamental approach. I have several ideas on this just not for current PoX scope
16:17:17 <rakhmerov> but as far as I understand you were agains introducing any special means in DSL for, say, Nova, right?
16:17:58 <rakhmerov> what I mean is basically, you don't like having special kinds of actions
16:18:00 <rakhmerov> in DSL
16:18:04 <rakhmerov> like:
16:18:12 <rakhmerov> action:
16:18:18 <rakhmerov> type: Nova
16:18:35 <rakhmerov> method: createVM()
16:18:38 <stanlagun> I'm against any built-in syntax that is designed specially for several Nova actions and it becomes useless if we go one step beyound this
16:19:09 <stanlagun> createVM requires parameters. Some of them need to be calculated
16:19:19 <rakhmerov> yes
16:19:37 <stanlagun> And createVM is useless if it is all that you can call
16:20:06 <rakhmerov> last time you were talking about having a convenient Mistral library that would allow creating callbacks easily
16:20:22 <rakhmerov> so that we could handle Mistral actions in python code
16:20:52 <igormarnat_> Guys, I'm somewhat late and missed something: do we have any agenda or do we do some design holy war as a part of the agenda?
16:20:56 <rakhmerov> this library under the hood exposes REST endpoints and all necessary machinery
16:21:25 <stanlagun> yes. Client libabrary is more promissing approach for short-medium term
16:21:32 <rakhmerov> igormarnat, the agenda for this meeting is discussing design, DSL, API
16:21:37 <stanlagun> Universal REST API client is a long-term approach
16:21:53 <rakhmerov> I believe this is not really a holly war, it's an important thing that we should realize
16:22:37 <rakhmerov> stanlagun, why is it a long-term approach?
16:22:42 <rakhmerov> not sure I'm understanding you
16:23:23 <stanlagun> Client library is generally gives more value and simpe to use. Long-term approach requires some form of workflow catalog that is need to be feeled with workflows. So we will think on it but start with client library. It is much simpler and generaly universal approach
16:24:11 <rakhmerov> btw, the second AI we had is "Design how Mistral client library may look like", in progress. But personally would love to see that done as soon as possible.
16:24:21 <rakhmerov> ok
16:24:56 <rakhmerov> for me the idea of being able to create this kind of callbacks easily looks very attractive
16:25:26 <rakhmerov> it could simplify things significantly and address lots of doubts we have
16:25:56 <rakhmerov> #topic Discuss the design, DSL, API
16:26:16 <rakhmerov> gokrokve, is there anything you could suggest on that?
16:26:45 <rakhmerov> I'm just not sure if our explanations were clear enough
16:26:55 <gokrokve> Can we pass API calls to taskflow layer?
16:27:28 <gokrokve> Anyway we planned to have webhooks for DSL, so it should almost the same as API calls.
16:27:43 <gokrokve> But I see why API calls might be a problem.
16:27:46 <rakhmerov> yes, webhooks will work
16:27:58 <rakhmerov> and yes, it's what we planned initially
16:28:11 <gokrokve> Let's keep DSL simple at least on PoC level.
16:28:33 <rakhmerov> as for TaskFlow, in PoC most likely we won't be able to use TaskFlow much
16:28:34 <gokrokve> No API calls in PoC.
16:28:43 <gokrokve> Ok.
16:28:50 <rakhmerov> ok, I would agree with that
16:29:26 <gokrokve> Send a DSL proposal to openstack-dev list.
16:29:42 <rakhmerov> I did
16:30:01 <gokrokve> Do you have a feedback from heat team?
16:30:22 <gokrokve> If not, please go to their IRC channel and try to talk with them directly.
16:30:26 <rakhmerov> nmakhotking keeps working on Workflow Execution (aka engine) for Mistral PoC and he'll try to list out most of the things that TaskFlow lacks now
16:30:44 <gokrokve> okk
16:30:47 <gokrokve> ok
16:30:56 <rakhmerov> well, we didn't hear anything from heat by now
16:31:10 <rakhmerov> I think I'll try to talk to them directly, yes
16:31:27 <nmakhotkin> I did it on wiki page
16:31:48 <rakhmerov> hmm, not sure if it contains everything
16:31:56 <nmakhotkin> I try to summarise more things
16:32:02 <rakhmerov> but let me check again, things changed since you did it
16:32:10 <rakhmerov> ok
16:32:22 <nmakhotkin> Ok
16:33:07 <rakhmerov> #action nmakhotkin, rakhmerov Check once more what TaskFlow lacks to be able to use it in Mistral (with the current vision)
16:33:26 <rakhmerov> and we'll need to move it to the public wiki :)
16:34:41 <rakhmerov> as far as heat team and others, I have a feeling that most of the people at this point pretty much understand what Mistral is going to be and we discussed lots of aspects with them and agreed upon them. And now everyone is just willing to see something working
16:35:06 <rakhmerov> that's why I think it's ok that we decided to concentrate on delivering PoC asap
16:35:25 <gokrokve> Ok. Then create a PoC. How much will it take to implement?
16:35:34 <rakhmerov> we're just creating a point to start a productive conversation with the community
16:36:04 <rakhmerov> gokrokve, yes, we've been working on it over one week and we have some things done already
16:36:41 <rakhmerov> i.e. Rest API that we suggested on etherpad is mostly finished and just needs to be connected to other parts
16:36:59 <rakhmerov> as far as how much, we're now targeting to Dec 13th
16:37:07 <rakhmerov> so 2 weeks from now
16:37:24 <rakhmerov> and I'm pretty sure we're on target
16:37:57 <gokrokve> cool
16:38:01 <rakhmerov> yep
16:38:56 <igormarnat_> Do you have in mind some ref app to be ready together with Mistral PoC? What is the use case of the app?
16:39:17 <rakhmerov> yes, we do
16:39:58 <rakhmerov> for now I think this app won't be doing anything complicated
16:40:43 <rakhmerov> we were thinking about something like "Create 3 VMs", "Attach cinder volumes", "Format volumes"
16:41:26 <rakhmerov> and the app should create this workflow, upload it to Mistral and tell Mistral to run it
16:41:53 <rakhmerov> I would suggest we discuss the particular scenario of this app on the next IRC meeting, it's worth discussing
16:42:17 <rakhmerov> I'm just not sure we'll be ready to work with real OpenStack services in the scope of PoC
16:42:29 <rakhmerov> so the app actually may be something simpler for now
16:42:51 <igormarnat_> It's fine, the simpler is the better.
16:43:02 <rakhmerov> yeah
16:43:02 <igormarnat_> Ok, let's plan it by the next meeting
16:43:22 <rakhmerov> yes, let's have an item for that
16:44:10 <rakhmerov> #action akuznetsov, nmakhotkin, rakhmerov Create a detailed plan for the demo application for PoC
16:44:59 <rakhmerov> the thing is that in order to do things like "Create VMS" we need to be able to authenticate and may be do some additional things
16:46:02 <rakhmerov> it'd be cool to be able to demonstrate this but at this point I don't see exactly how it's going to look like
16:46:17 <rakhmerov> especially in DSL
16:47:35 <rakhmerov> guys, are there any other questions to discuss or we'd like to finish the conversation about additional DSL constructions like calling Nova?
16:47:44 <rakhmerov> stanlagun, akuznetsov?
16:47:55 <igormarnat_> ok, and?
16:48:11 <akuznetsov> from my side no
16:48:16 <rakhmerov> ok
16:48:35 <igormarnat_> We do have the list of actions to implement, right?
16:48:53 <rakhmerov> yes, REST and AMQP
16:49:42 <rakhmerov> so are we done then for today?
16:49:51 <rakhmerov> is there anything else to discuss?
16:51:05 <rakhmerov> ok, I'll wait a couple of more minutes and then wrap up the meeting
16:52:46 <rakhmerov> ok, thanks for joining today!
16:52:52 <rakhmerov> let's end the meeting
16:53:10 <rakhmerov> bye, will see you next time
16:53:17 <rakhmerov> #endmeeting