16:01:06 <rakhmerov> #startmeeting Mistral
16:01:07 <openstack> Meeting started Mon Nov  7 16:01:06 2016 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:10 <openstack> The meeting name has been set to 'mistral'
16:01:16 <rakhmerov> hi
16:01:21 <d0ugal> Hey!
16:01:23 <rakhmerov> name yourself :)
16:01:27 <rakhmerov> yep, hi )
16:02:17 <rakhmerov> d0ugal: it'll be cool to have just us two )
16:02:25 <trinaths> hi
16:02:31 <rakhmerov> trinaths: hi
16:03:21 <d0ugal> rakhmerov: but more people is better
16:03:24 <d0ugal> I think rbrady is around
16:03:25 <rakhmerov> yes
16:03:29 <rakhmerov> ok
16:03:33 * d0ugal just pinged him
16:03:38 <rbrady> o/
16:03:39 <rakhmerov> anyway, let's start
16:03:42 <d0ugal> sure
16:03:48 <rakhmerov> rbrady: hi Ryan, how have you been?
16:04:09 <rbrady> hey!  doing well.  glad to have moved from newton to ocata :)
16:04:23 <rakhmerov> yeah
16:04:39 <rakhmerov> ok, I assume we have no actions itmes
16:05:02 <d0ugal> I would like to discuss the custom actions spec at some point, but any time is fine.
16:06:07 <rakhmerov> yeah, sure
16:06:22 <rakhmerov> #topic Current status (progress, issues, roadblocks, further plans)
16:07:48 <rakhmerov> my status: doing house keeping in our Launchpad project, reviewing a bunch of patches, sorting out all the notes after the summit
16:08:21 <rakhmerov> going to implement a couple of small blueprints within the next week
16:08:32 <rakhmerov> if you have any updates please share
16:08:42 <d0ugal> I have done a bunch of Mistral reviews, but otherwise I have been working on TripleO Newton bugs and backports.
16:08:52 <rakhmerov> ok
16:08:57 <d0ugal> I do hope and plan to have more time for Mistral itself soon.
16:09:08 <rakhmerov> d0ugal: I hope too )
16:10:00 <rakhmerov> ok, if nobody else has updates we can go on
16:10:10 <rakhmerov> rbrady: maybe something from your side?
16:10:42 <rbrady> I haven't had much time for mistral in the last part of the newton cycle
16:10:49 <rakhmerov> ok
16:11:18 <ddeja> o/ Sorry for being late
16:11:19 <rakhmerov> d0ugal: did have a chance to look at the time slots?
16:11:25 <rakhmerov> ddeja: heey, glad you're here
16:11:26 <rakhmerov> welcome
16:11:28 <d0ugal> rakhmerov: Yeah, so I am writing the email to openstack-dev now
16:11:30 <d0ugal> ddeja: Hey
16:11:40 <rakhmerov> d0ugal: cool, looking forward
16:11:46 <rakhmerov> we need to solve this issue
16:12:01 <rakhmerov> ddeja: Dawid, any updates from you?
16:12:01 <d0ugal> rakhmerov: I was just going to do something like this: https://etherpad.openstack.org/p/mistral-meeting-time
16:12:05 <d0ugal> Ideas are welcome :)
16:12:15 <rakhmerov> yes, great
16:12:44 <ddeja> not much. Last week some reviews, probably would be the same this week. I'm watitng for reviews on the workflow-precondition spec :)
16:12:52 <ddeja> so I can move this work forward
16:13:48 <d0ugal> ddeja: we did discuss that a bit at summit, I am not sure if we have good notes from the discussion or not :/
16:14:01 <d0ugal> I can maybe try and write on the spec everything I remember.
16:14:18 <rakhmerov> d0ugal: yes, please do
16:14:32 <rakhmerov> ddeja: the outcome is that many people didn't really like the idea itself
16:14:46 <rakhmerov> although I don't quite agree personally
16:15:03 <d0ugal> Yeah, I don't think I fully understood why they didn't like it
16:15:09 <rakhmerov> ddeja: the reasoning they provide is that it should not be a burden of Mistral
16:15:15 <ddeja> rakhmerov: yes, Roman told me
16:15:17 <d0ugal> right
16:15:36 <rakhmerov> ddeja: but I'm going to review it anyway and move this forward
16:15:47 <rakhmerov> I really believe that some things should be in Mistral
16:15:53 <rakhmerov> maybe not all you're proposing but some
16:16:02 <ddeja> ok, thanks. I'd like to get +2 or -2 on this eventually :)
16:16:47 <rakhmerov> because
16:16:51 <rakhmerov> 1) it may be not easy to implement them in the upper level system (that uses Mistral) and people have to do a lot of coding themselves
16:17:23 <rakhmerov> 2) some of those things may be nearly impossible to implement outside of Mistral (atomicity, concurrency etc.)
16:17:34 <rakhmerov> ddeja: yes, we'll take care of it
16:18:02 <ddeja> ok, thanks :)
16:18:02 <rakhmerov> ddeja, d0ugal: btw, an interesting notice that mostly StackStorm folks were against it
16:18:03 <rakhmerov> :)
16:18:14 <rakhmerov> and I know why
16:18:24 <d0ugal> rakhmerov: oh, do they do something similar?
16:18:24 <rakhmerov> because it's their bread and butter
16:18:26 <d0ugal> hah
16:18:28 <rakhmerov> yes
16:18:33 <ddeja> oh
16:18:39 <d0ugal> so, can we just copy it in? :P
16:18:42 <rakhmerov> they sometimes call themselves "before" and "after" Mistral
16:18:56 <rakhmerov> d0ugal: that's the whole point :)
16:19:04 <rakhmerov> do you think they want to? :)
16:19:11 <d0ugal> Good question.
16:19:16 <rakhmerov> well, it maybe just one side of it
16:19:25 <rakhmerov> political/business side
16:19:30 <ddeja> well if there is something that can be used to provide such functionality, it may be ok to use it
16:19:48 <rakhmerov> technically, Dmitri is always for good decomposition
16:20:00 <ddeja> but I'm not related with their product, is it opensource also?
16:20:11 <d0ugal> ddeja: Yeah
16:20:17 <rakhmerov> yeah, in such cases we usually have different opinions with them
16:20:20 <rakhmerov> I mean me and them
16:20:23 <d0ugal> ddeja: https://github.com/StackStorm
16:20:29 <ddeja> d0ugal: thanks
16:20:39 <rakhmerov> because I want Mistral to be useful and successful also w/o StackStorm
16:20:54 <d0ugal> ddeja: (I have no experience with it, I have just looked at a little of the code)
16:21:02 <d0ugal> rakhmerov: indeed, we need to think about Mistral primarily
16:21:28 <ddeja> (offtop - Stackstorm uses mistral and Netflix runs on stackstorm so we are basically make Netflix works? Awesome!)
16:21:30 <rakhmerov> by the way, we can look at how StackStorm implements their actions
16:21:45 <rakhmerov> they had to do a similar thing to our Custom Actions API
16:22:23 <ddeja> rakhmerov: mistral is useful, in triple0 for example
16:22:35 <rakhmerov> ddeja: as far as I know, Netflix specifically doesn't use Mistral yet (StackStorm can work w/o workflows also for some use cases)
16:22:50 <rakhmerov> but what I know is that folks from Netflix really like Mistral provides
16:22:54 <rakhmerov> language, capabilities
16:23:05 <rakhmerov> they are just not sure if it's stable enough etc.
16:23:20 <rakhmerov> so they are waiting
16:23:21 <rakhmerov> kind of
16:23:32 <ddeja> oh, that's cool
16:23:49 <d0ugal> If they start to contribute that'd be nice :)
16:23:56 <ddeja> that would be nice to tell my wife while launching next movie 'see, I did it!' ;)
16:24:11 <d0ugal> lol
16:24:13 <rakhmerov> yeah, it's a kind of a side note but I'd like to repeat again: I think the #1 thing is to make Mistral really stable, reliable and useable
16:24:22 <rakhmerov> lots of people will come and start using it
16:24:26 <rakhmerov> I just know it
16:24:32 <ddeja> yes
16:24:35 <rakhmerov> I talked to so many people about it
16:24:50 <rakhmerov> ddeja: haha :)) yeah
16:24:59 <rakhmerov> and we can make it happen! absolutely
16:25:27 <rakhmerov> ok, let's go back to our stuff
16:25:32 <ddeja> sure
16:25:47 <rakhmerov> #topic Summit summary
16:26:09 <rakhmerov> well, on this one
16:26:27 <rakhmerov> I just wanted to provide some comments on the summit
16:26:46 <rakhmerov> these are the notes from our design sessions: https://etherpad.openstack.org/p/mistral-barcelona-summit-topics-2016
16:27:00 <rakhmerov> I'm now sorting them out in order to file all necessary blueprints
16:27:25 <rakhmerov> d0ugal: I won't take much time talking about it so that we can discuss actions
16:27:47 <d0ugal> Sure.
16:28:00 <rakhmerov> so, the biggest thigns we need to fix:
16:28:41 <rakhmerov> * HA (doesn't work if we use 'join' tasks)
16:28:44 <rakhmerov> * broken 'concurrency' property
16:29:11 <rakhmerov> * Documentation (a lot of work but people have hard time with it, it's been proven)
16:29:17 <rakhmerov> * Actions
16:29:46 <rakhmerov> by the last one I mean exactly Custom Actions API and how we integrate/install them
16:29:52 <rakhmerov> it's a big thing
16:30:12 <rakhmerov> it's also about how we implement OpenStack actions and maintain and test them automatically
16:30:48 <rakhmerov> because huge number of people (believe me) start getting familiar with Mistral exactly with launching some workflows that use OpenStack actions
16:30:56 <rakhmerov> and they often simply don't work
16:31:05 <rakhmerov> because they go out of sync with APIs
16:31:32 <rakhmerov> or some of them have been out of sync since almost the very beginning
16:31:46 <rakhmerov> we don't know because we don't have good function tests for them
16:32:02 <rakhmerov> so I see these items as the most important ones
16:32:55 <rakhmerov> HA and 'concurrency' are probably on me, at least HA as it was before my summer refactorings
16:33:12 <rakhmerov> overall, the summit for Mistral was quite good
16:33:32 <d0ugal> +1
16:33:49 <rakhmerov> I really didn't have time to attend presentations within the first 3 days because I was talking to various people about Mistral all the time
16:33:49 <d0ugal> I found it very useful
16:33:52 <rakhmerov> yeah
16:34:08 <rakhmerov> btw, Tacker project is also going to use Mistral
16:34:22 <rakhmerov> for NFV orchestration (kind of what we're doing in CloudBand)
16:34:22 <d0ugal> I think the actions (openstack and custom) are probably of most interest to me
16:34:27 <rakhmerov> yes
16:34:30 <rbrady> me too
16:34:37 <rakhmerov> so that's all that I wanted to share
16:34:40 <d0ugal> because we do  use both, and will do so more over time :)
16:34:51 <rakhmerov> #topic Open Discussion
16:34:58 <rakhmerov> d0ugal: let's discuss them
16:35:01 <d0ugal> Okay!
16:35:19 <d0ugal> So, at summit this came up, but I think it had been months since any of us had really looked at it, so we didn't get far
16:35:26 <rakhmerov> I have to admit that I may have forgotten some of the details that we discussed before
16:35:39 <d0ugal> Yeah, I think I am in a similar situation
16:35:40 <rakhmerov> yeah
16:35:45 <rakhmerov> that's fine
16:35:58 <d0ugal> What I really want to is start moving the spec forward again, it stalled before.
16:36:05 <rakhmerov> just repeating: I think it makes huge sense to look at StackStorm's actions
16:36:18 <d0ugal> Agreed, that would be a good idea.
16:36:34 <d0ugal> and it would make sense for us to be compatible with their actions - something that interested them I think.
16:36:46 <d0ugal> but I am not sure that is a requirement for the first pass of this
16:36:47 <rakhmerov> yeah, and I think we may want to throw some of the things away from the spec
16:36:53 <d0ugal> Yup
16:36:57 <rakhmerov> like reusing existing actions within new ones
16:37:18 <d0ugal> I think we probably need to revisit the spec, go through it again and submit a changed for review and go from there.
16:37:26 <d0ugal> rakhmerov: Yeah, I got really confused by that bit :)
16:37:38 <d0ugal> rbrady: Do we have a need to call existing actions from within an action?
16:37:40 <rakhmerov> d0ugal: we will need to talk to them more specifically about APIs etc after we research what they have
16:37:48 <d0ugal> Yup
16:37:54 <rakhmerov> yes
16:37:56 <rbrady> d0ugal: we do in many places right now
16:38:05 <d0ugal> rakhmerov: for example?
16:38:26 <d0ugal> rbrady: we use other openstack clients, but we don't call mistral actions, do we?
16:39:22 <rakhmerov> d0ugal: what I mean is that after we revisit their implementation and ideas we will know more about what to discuss with them
16:39:26 <rbrady> d0ugal: ahh, now I understand - we don't call mistral actions from our actions
16:39:29 <rakhmerov> and how to make our actions compatible
16:39:50 <d0ugal> rbrady: okay, cool :)
16:40:19 <rakhmerov> d0ugal: so what you do is just calling OpenStack clients?
16:40:21 <rakhmerov> not actions?
16:40:25 <rakhmerov> I just want to confirm
16:40:26 <d0ugal> rakhmerov: correct
16:40:28 <rakhmerov> ok
16:41:00 <d0ugal> rakhmerov: The strange situation we have is that we have two different clients - i.e. we create an ironic client in our actions and mistral creates an ironic client in the openstack actions
16:41:00 <rakhmerov> d0ugal: did you understand their reasoning against this feature?
16:41:25 <rakhmerov> d0ugal: yeah, it is strange
16:41:26 <rakhmerov> I agree
16:41:28 <d0ugal> rakhmerov: The reasoning against calling openstack clients?
16:41:32 <rakhmerov> I think we can solve it
16:42:19 <rakhmerov> d0ugal: no, remember Winson and Dmitri (mostly Winson) said that they are against of calling existing actions from other actions because this is dangerous
16:42:30 <d0ugal> rakhmerov: Yeah, I think I understand.
16:42:36 <rakhmerov> but why dangerous, I didn't quite understand honestly
16:42:44 <d0ugal> I don't know why it is dangerous.
16:42:46 <rakhmerov> d0ugal: ok, can you try to explain?
16:42:49 <rakhmerov> :)
16:42:51 <d0ugal> but I think it is wrong because it breaks the workflow model
16:43:07 <rakhmerov> hm.. why?
16:43:10 <d0ugal> workflows are a series of tasks, but we have tasks calling other tasks we hide that
16:43:25 <d0ugal> so from the outside it only looks like one task, but really it is more
16:43:31 <rakhmerov> well, my understanding of that is slightly different
16:43:35 <d0ugal> oh :)
16:43:39 <rakhmerov> so
16:44:04 <d0ugal> but to be clear, we don't want to do this, and I don't know anyone that does, so it doesn't matter all that much if we don't fully understand :)
16:44:14 <rakhmerov> to me: action is equivalent to function
16:44:26 <rakhmerov> yes, sure
16:44:33 <rakhmerov> so action is simply a function
16:44:52 <d0ugal> (it would be nice if actions could just be functions and not classes, but that is another subject! :) )
16:45:28 <rakhmerov> it doesn't do anything with with workflow context data or other stuff that determines how workflow is proceeding (policies, routes etc.)
16:45:39 <rakhmerov> d0ugal: yeah, I mean semantically
16:45:43 <rakhmerov> action is a function
16:45:53 <rakhmerov> not speaking about how they are represented in code
16:45:57 <d0ugal> Sure
16:46:00 <rakhmerov> so
16:46:25 <rakhmerov> from that perspective I don't really understand how it is harmful if one function can call another function
16:46:46 <rakhmerov> could you be just using existing Mistral actions when implementing new ones?
16:46:52 <rakhmerov> instead of OpenStack clients
16:47:25 <rakhmerov> to me it simply sounds like having an ability to do composition
16:47:29 <d0ugal> rakhmerov: sometimes, maybe. However, sometimes we are calling python functions that expect full clients to be passed in
16:47:45 <rakhmerov> so that we can compose complex actions out of more primitive
16:47:54 <d0ugal> Yeah
16:47:58 <rakhmerov> d0ugal: why is that?
16:48:01 <ddeja> rakhmerov: it is dangerous on the implementation level
16:48:08 <rakhmerov> why do these functions require full clients?
16:48:13 <rakhmerov> ddeja: why?
16:48:17 <ddeja> since we would be calling the engine from the executor
16:48:26 <ddeja> and if we do it again and again and again
16:48:31 <d0ugal> rakhmerov: just because they are legacy code that use the openstack clients and do many things with them
16:48:42 <ddeja> we can end up with no more free threads = a dead lock
16:48:44 <d0ugal> rakhmerov: we could re-write that code to call many mistral actions, but I guess it will get slow.
16:48:45 <rakhmerov> d0ugal: ok, I see
16:49:03 <rakhmerov> ddeja: yes, accepted, this is a concern
16:49:13 <rakhmerov> although maybe it is solvable
16:49:16 <d0ugal> rakhmerov: and re-writing them like that would take quite a bit of effort, so I think that we would be better breaking this code up slowly overtime into workflows.
16:49:42 <rakhmerov> ok
16:49:53 <rakhmerov> hm.. ok
16:50:19 <rakhmerov> d0ugal: what else on that do we need to discuss now?
16:50:22 <rakhmerov> 9 mins left
16:50:29 <d0ugal> Yeah, lets put that bit aside for now
16:51:00 <rakhmerov> you mean reusing existing actions?
16:51:01 <rakhmerov> yes
16:51:05 <d0ugal> Yeah :0
16:51:08 <d0ugal> :)
16:51:11 <d0ugal> for action items...
16:51:26 <d0ugal> I'd like to try and do an update to the spec with rbrady - see if we can simplify it and find a way for us to move forward.
16:52:13 <rakhmerov> #action rakhmerov, rbrady, d0ugal: look at the StackStorm's implementation of actions to get more ideas and understanding of how it should be done
16:52:28 <d0ugal> that is also a good idea :)
16:52:35 <rakhmerov> d0ugal, rbrady: update the Custom Actions API spec
16:52:46 <rbrady> ack
16:52:48 <rakhmerov> I can actually help with this too
16:52:50 <d0ugal> Did you mean to add #action for that second one?
16:53:00 <rakhmerov> yes!
16:53:08 <rakhmerov> #action d0ugal, rbrady: update the Custom Actions API spec
16:53:21 <d0ugal> I will try and start doing this at the start of next week. I don't think I will have time this week.
16:53:30 <rakhmerov> ok, np
16:53:51 <d0ugal> but I shall try
16:53:55 <d0ugal> That is all I have :)
16:54:06 <rakhmerov> d0ugal, rbrady: summaries the challenges we have with the current approach (decomposition into different repos etc.)
16:54:12 <rakhmerov> yes
16:54:22 <rakhmerov> ooh, shoot...
16:54:23 <rakhmerov> again
16:54:27 <d0ugal> :)
16:54:40 <rakhmerov> #action d0ugal, rbrady: summarize the challenges we have with the current approach (decomposition into different repos etc.)
16:55:03 <rakhmerov> d0ugal: I actually believe that some diagram would help us all a lot
16:55:18 <d0ugal> Yup, good idea.
16:55:24 <rakhmerov> that explains the whole idea of how our repos and modules should be decomposed
16:55:32 <rakhmerov> yep, ok
16:55:54 <rakhmerov> I don't have anything else for now too
16:56:32 <rakhmerov> just FYI: we keep testing Mistral in various use cases in CloudBand and keep finding some reliability bugs but less and less
16:56:40 <rakhmerov> which is a good thing
16:56:47 <d0ugal> Great
16:57:21 <rakhmerov> in that regard I believe our new RPC will also help
16:57:27 <rakhmerov> once we start using it actively
16:57:50 <rakhmerov> ok, anything else guys?
16:57:59 <rbrady> nothing from me at this time
16:58:11 <ddeja> nothing from my side
16:58:17 <d0ugal> Not from me, thanks
16:58:40 <rakhmerov> ok
16:58:42 <rakhmerov> bye
16:58:45 <rakhmerov> thanks for coming
16:58:57 <ddeja> bye, thanks :)
16:59:10 * rakhmerov hoping to fix time slot issue so that we have more people
16:59:16 <rakhmerov> :)
16:59:22 <rakhmerov> #endmeeting