16:01:12 <rakhmerov> #startmeeting Mistral 16:01:12 <gokrokve> hi 16:01:13 <openstack> Meeting started Mon Nov 11 16:01:12 2013 UTC and is due to finish in 60 minutes. The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:01:16 <openstack> The meeting name has been set to 'mistral' 16:01:39 <igormarnat> Hey there 16:01:48 <akuznetsov> Hi 16:01:59 <rakhmerov> Let's discuss things happened within the last couple of weeks 16:02:19 <rakhmerov> so we didn't have a meeting last Monday due to summit activities 16:02:49 <rakhmerov> just a couple of words about the summit itself and how everything went regarding Mistral 16:03:06 <rakhmerov> I think it was actually pretty successful 16:03:25 <rakhmerov> we had a lightning talk and an Unconference track 16:04:00 <rakhmerov> and Mistral was also discussed at other design sessions 16:04:41 <rakhmerov> for example, we were at the design session of Heat where they were trying to decide how to switch to TaskFlow 16:05:00 <rakhmerov> and I had a couple minutes to tell briefly about what Mistral is going to be 16:05:24 <rakhmerov> they definitely liked the idea of the project itself and seem to have endorsed it completely 16:05:56 <rakhmerov> and they offered their help if we need 16:06:12 <rakhmerov> I think we will be in touch with them anyway 16:06:46 <gokrokve> Sure. 16:06:54 <gokrokve> We plan to work with Heat team. 16:07:07 <rakhmerov> yep 16:07:10 <gokrokve> Actually we already spoke with Steven about contribution to Heat. 16:07:45 <rakhmerov> there were also a number of people who approached me between the sessions and asked questions about Mistral 16:08:15 <rakhmerov> there was a guy who said "This is really what I need in my work, I'm going to use it when it's done" 16:08:44 <rakhmerov> so I think all went pretty well, I'm glad to realize that 16:09:19 <gokrokve> Cool. 16:09:21 <rakhmerov> Solum guys I believe will be using Mistral too when they get to some more complicated scenarios 16:09:38 <rakhmerov> they will definitely need it 16:09:48 <gokrokve> It actually gives some pressure on us too. We have to deliver something valuable during Icehouse release. 16:10:05 <rakhmerov> yes, I think we'll be able to 16:10:24 <rakhmerov> the game is becoming more interesting :) 16:10:28 <rakhmerov> and exciting 16:10:32 <gokrokve> Which means, that we have to plan Mistral very carefully, to cover important common parts. 16:10:46 <rakhmerov> yes, you're right 16:11:03 <rakhmerov> that's why we got some new folks in our team 16:11:11 <rakhmerov> as akuznetsov 16:11:12 <gokrokve> So, do you have a list of features people asked for? 16:11:29 <gokrokve> Great. Welcome Alex! 16:11:32 <rakhmerov> Alex, welcome to the team! 16:11:41 <rakhmerov> I hope we'll be cool things together 16:11:48 <rakhmerov> * we'll do 16:11:49 <akuznetsov> thank you 16:12:27 <rakhmerov> gokrove, yes, I have some notes after conversations with people, they will be reflected in the documenation 16:13:06 <gokrokve> In BPs? 16:13:14 <rakhmerov> mostly, they're related with some more advanced functionality that we haven't thought so far 16:13:39 <gokrokve> Let's follow OpenStack way. If we have a feature let's document it in BP on the launchpad. 16:13:41 <rakhmerov> yes, as far as BPs we need to start publishing all our ideas as BPs 16:13:48 <gokrokve> So everybody can see it and comment. 16:13:56 <rakhmerov> gokrokve, agree 16:14:00 <rakhmerov> right 16:14:15 <rakhmerov> ok, now let's review the previous action items 16:14:28 <gokrokve> After we have all BPs, we can go to IRC canells asking for review. 16:14:35 <rakhmerov> ok 16:14:38 <ruhe> hi folks. is mistral going to be a rest api service for taskflow? 16:14:48 <gokrokve> Probably we will be able to collect some feedback from other teams, and heat in particular 16:15:12 <rakhmerov> ruhe, not exactly for taskflow 16:15:19 <gokrokve> That not exactly this way. Mistral is an API for workflows orchestration. 16:15:26 <rakhmerov> yes 16:15:28 <gokrokve> It will use taskflow under the hood. 16:15:41 <gokrokve> But it will have more features. 16:15:46 <rakhmerov> much more 16:15:47 <igormarnat> But it gonna be more than rest api for task flow. 16:15:53 <akuznetsov> In OpenStack with Mistral and TaskFlow project will have two way for workflow creation, we should give recommendation in which case Mistral or TaskFlow should be used 16:16:06 <gokrokve> Like even management and scheduled tasks. 16:16:09 <rakhmerov> and address some completely different use cases that can't be addressed by taskflow 16:16:47 <rakhmerov> akuznetsof, good idea. We need to start writing on this 16:16:50 <gokrokve> Yes. We are targeting more high level use cases. Like backup execution by schedule, live migration on event and so on. 16:17:34 <ruhe> could mistral be used as underlying infrastructure for Heat orchestration engine? 16:17:51 <rakhmerov> ruhe, yes, it could 16:17:56 <gokrokve> Mistral will have its own DSL to express task flows. Soy you will not need to create a code, Mistral will generate the code for task flow. 16:18:21 <gokrokve> Actually this was discussed on the summit during Heat design session. 16:18:36 <rakhmerov> we discussed it with Heat team and they positively look at it 16:18:46 <gokrokve> So, yes in general, but when Mistral will have at least some maturity. 16:19:03 <rakhmerov> but they want to start using something pretty soon 16:19:04 <rakhmerov> yes 16:19:24 <gokrokve> That is why we plan to work with Heat team closely, to have some feedback from them. 16:20:01 <rakhmerov> probably they will start using TaskFlow and then they'll continue with the engine for TaskFlow that schedules tasks across Mistral 16:20:25 <gokrokve> They will not probably be able to help writing the code but at least they may help us to make a list of required features. 16:20:26 <rakhmerov> that's going to be another point of integration between TaskFlow and Mistral 16:20:46 <rakhmerov> yes, that's what we need from them mostly 16:21:05 <rakhmerov> those guys I think are too busy to be able to write some code for us 16:21:07 <katyafervent> So the project design will be be as it is now, and there is no changes made after summit. Is it correct? 16:21:15 <rakhmerov> at least that's what I heard from them 16:21:43 <rakhmerov> katyafervent, yes, it's 99% true 16:22:06 <rakhmerov> people accepted our ideas just as they are 16:22:29 <rakhmerov> with some slight changes that are mostly related with some additional advanced features 16:23:17 <rakhmerov> I'll share all of this pretty soon 16:23:25 <katyafervent> rakhmerov, that's great! So now we need to read it deeply 16:23:27 <ruhe> i wonder, what tools are you going to use to store distributed state? zookeeper/sql/smth else? 16:24:01 <gokrokve> That is a good question. 16:24:18 <rakhmerov> there's a bunch of options, at this point it's not decided yet. You can contribute your thoughts if you want :) 16:24:38 <gokrokve> Right now I don't see any intentions to use external tools for that. 16:25:08 <gokrokve> We don't have a line of code, so we can properly design this from initial stage. 16:25:09 <akuznetsov> possibly we should have a plugble system for state storage 16:25:20 <rakhmerov> I think we first need to start diving deeper into design 16:25:33 <rakhmerov> yes, I would love to see a way to abstract it out 16:26:10 <rakhmerov> guys, I would also ask all of you to use openstak-dev mailing list more actively 16:26:24 <rakhmerov> you got an idea - you share it via the mailing list 16:26:35 <rakhmerov> and it could be absolutely anything 16:26:52 <rakhmerov> like "Where do I need to put a comma in this line?" 16:27:05 <rakhmerov> so that people could participate 16:27:41 <gokrokve> Agree. 16:27:45 <rakhmerov> I think we hasitate to do it frequently but we need to change our mindset here 16:27:57 <gokrokve> Yep. 16:28:03 <rakhmerov> communication is everything 16:28:18 <gokrokve> So, Renat will you take an AI to create a list of BP for the features? 16:28:49 <rakhmerov> #action rakhmerov, create a list of Blueprints for Mistral features 16:28:50 <gokrokve> Publish them on Launchpad and them start discussion in mailing list. 16:29:00 <rakhmerov> ok 16:29:23 <rakhmerov> so, now let's get back to some previous action items, just to be consistent 16:29:32 <gokrokve> We also need to strat conversation about design in general. 16:29:37 <igormarnat> We can shortly discuss list of potential topics here. For now there was a question about the way to keep track of state 16:29:41 <rakhmerov> #topic Previous Action Items 16:29:42 <gokrokve> ok 16:30:12 <rakhmerov> 1. rakhmerov, stanlagun, tsufiev, keep working on DSL/API Mistral specification 16:30:16 <igormarnat> Anyway, let's stick to the agenda now and return back to potential questions later 16:30:57 <rakhmerov> this is still in progress and we now have a target date Nov 20 to complete this 16:31:12 <rakhmerov> after the summit we have some new things to include into the specs 16:31:22 <rakhmerov> so, I'll transfer it on 16:31:37 <igormarnat> Let's add one more owner for this AI 16:31:44 <igormarnat> akuznetsov 16:32:09 <rakhmerov> #action rakhmerov, akuznetsov, stanlagun, tsufiev, keep working on DSL/API Mistral specification and include all features discussed at the summit 16:32:12 <rakhmerov> ok 16:32:29 <rakhmerov> 2. rakhmerov, finish detailed description of "Long-running Business Process" use case 16:32:46 <rakhmerov> so regarding this one, I've done it 16:33:31 <rakhmerov> however, we decided not to publish it on the wiki for a while because it's not what we're going to target within the next 6 months 16:34:07 <rakhmerov> I really think, because of the structure of the wiki we have here it could be published actually 16:34:16 <rakhmerov> and accessible via "Read more" link 16:34:33 <rakhmerov> I personally don't see any harm in doing that 16:34:44 <rakhmerov> what do you think? 16:35:12 <rakhmerov> I'm talking about that use case where we calculate salary using several enterprise systems 16:35:59 <rakhmerov> anyway, this documentation will be evolving all the time 16:36:26 <rakhmerov> if you agree, I'll create an AI to publish it 16:37:11 <rakhmerov> gokrokve, igormarnat? 16:37:27 <gokrokve> Yes. publish it. But add a note that this is not a goal. It is just an example how to use it in general as a generic engine. 16:37:40 <rakhmerov> sure, ok 16:37:57 <igormarnat> We can publish it, no issue, though I don't see a lot of value in that for now:) 16:38:10 <igormarnat> Anyway, let's do it 16:38:14 <rakhmerov> #action rakhmerov, publish detailed description of "Long-running business process" use case on wiki 16:38:23 <rakhmerov> yes, it's a 10 min thing to do :) 16:38:27 <rakhmerov> ok 16:38:31 <rakhmerov> the next one 16:38:46 <rakhmerov> 3. stanlagun, finish detailed description of "Post Deployment Configuration" use case 16:38:49 <rakhmerov> done 16:39:02 <rakhmerov> but under a different name 16:39:40 <rakhmerov> #info "Cloud Environment Deployment" use case, https://wiki.openstack.org/wiki/Mistral/Cloud_Environment_Deployment_details 16:40:19 <rakhmerov> 4. ativelkov, collect the info about use case related to setting up a network (not a high priority) 16:40:32 <rakhmerov> in progress, not a high priority 16:40:48 <rakhmerov> #action ativelkov, collect the info about use case related to setting up a network (not a high priority) 16:41:04 <rakhmerov> 5. rakhmerov, prepare the first version of Roadmap and the development plan 16:41:29 <rakhmerov> the first version of Roadmap is done and event shared at the summit 16:41:43 <rakhmerov> but I still didn't share it anywhere else 16:42:00 <igormarnat> So here comes the next AI for us 16:42:18 <rakhmerov> what ways of sharing it would you recommend guys? Except wiki, of course 16:42:23 <igormarnat> To put together AIs and agreements achieved in HK and publish them in mailing list 16:42:35 <rakhmerov> yes 16:43:17 <rakhmerov> #action akhmerov, put together AIs and agreements achieved in HK and publish them in mailing list 16:44:08 <rakhmerov> the Roadmap we have now is kind of a draft, how can we discuss it with the community in the best way? 16:44:12 <rakhmerov> mailing list? 16:44:22 <rakhmerov> or some sort of a Blueprint? 16:44:25 <igormarnat> Let's discuss them prior to publishing though in order to see if me and Gosha and Alexander can add anything 16:44:38 <rakhmerov> yes, ok 16:45:12 <rakhmerov> #action gokrokve, igormarnat, akuznetsom review Roadmap and add whatever you think needs to be added in there 16:45:45 <rakhmerov> #action rakhmerov, share Roadmap with the community to start discussing it 16:46:38 <rakhmerov> #topic Free discussion 16:46:53 <rakhmerov> anything else guys that you'd like to discuss? 16:46:59 <igormarnat> Great. Let's return back to the list of questions we had 16:47:13 <igormarnat> The first one was the way to track states 16:47:20 <rakhmerov> ok 16:47:45 <igormarnat> ruhe, akuznetsov, what else was there? 16:48:01 <ruhe> the only thing i wanted to mention... 16:48:15 <ruhe> there is a project https://github.com/airbnb/chronos which might be a good source of inspiration :) 16:48:21 <rakhmerov> so I see it could be encapsulated in a subsystem of Mistral from the very beginning so that we may vary the actual storage whenever we want 16:48:46 <rakhmerov> #info https://github.com/airbnb/chronos 16:48:51 <rakhmerov> thanks! 16:49:41 <rakhmerov> you mean in the context of storing a distributed state? 16:49:52 <ruhe> storing and executing 16:49:57 <rakhmerov> ok 16:50:18 <ruhe> chronos uses http://mesos.apache.org/ which itself might be a good engine for distributed task execution 16:50:23 <akuznetsov> we should define what part of workflow functionality in Mistral and which in TaskFlow 16:51:28 <rakhmerov> #action rakhmerov, akuznetsov, NikolayM review https://github.com/airbnb/chronos and http://mesos.apache.org/ with regard to storing and executing distributed tasks 16:52:04 <rakhmerov> akuznetsov, yes, I believe TaskFlow is currently missing some stuff required for Mistral 16:52:28 <rakhmerov> I think we need to identify that as soon as possible 16:52:56 <igormarnat> rakhmerov, not only that, but we might want to describe explicitly what's the difference between them and when one use Taskflow, when does he use Mistral 16:53:11 <igormarnat> Description of what's need to be added to Taskflow is a separate topic 16:53:22 <rakhmerov> and start discussing it with TaskFlow folks so that they either implement that themselves or let us contribute, or both 16:53:43 <rakhmerov> ooh, apologize. I may have misunderstood that 16:53:58 <rakhmerov> yes, got it now 16:54:35 <igormarnat> Are there any news from our big and powerful prototyping team? What did we prototype already? :) 16:55:28 <rakhmerov> #action rakhmerov, kuznetsov, harlowja clearly describe the high-level separation of functionality between TaskFlow and Mistral in terms of addressed use cases 16:55:35 <rakhmerov> sounds good enough? 16:56:06 <rakhmerov> ok 16:56:10 <rakhmerov> 4 mins left 16:56:15 <igormarnat> LGTM 16:56:23 <rakhmerov> ok 16:57:29 <rakhmerov> looks like we have a bunch of AIs this time :) 16:57:33 <rakhmerov> a lot of work 16:58:06 <rakhmerov> anything else? 16:58:45 <rakhmerov> ok, they we'll end for today 16:59:04 <rakhmerov> thanks to everyone! I'll see you all soon 16:59:05 <igormarnat> Ok, thanks for coming, everyone. TTYL, BFN:) 16:59:13 <rakhmerov> #endmeeting