16:01:11 <rakhmerov> #startmeeting Mistral
16:01:13 <openstack> Meeting started Mon Mar 14 16:01:11 2016 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:17 <openstack> The meeting name has been set to 'mistral'
16:01:20 <rbrady> o/
16:01:27 <rakhmerov> hi
16:02:27 <akuznetsova> hi there!
16:02:34 <rakhmerov> hi, how are you?
16:02:40 <akuznetsova> great)
16:02:46 <NikolayM> hi everyone!
16:02:47 <akuznetsova> work hard
16:02:49 <rakhmerov> okay
16:03:00 <rakhmerov> I'm going to wait for a couple of minutes
16:04:21 <d0ugal> o/
16:04:33 <rakhmerov> let's start
16:04:41 <rakhmerov> thanks for joining
16:05:16 <rakhmerov> I don't have any special agenda for today, just want to quickly go over our usual stuff
16:05:36 <rakhmerov> and also talk about ha gate
16:05:53 <rakhmerov> #topic Review action items
16:06:20 <rakhmerov> 1. akhmerov: propose time slots for 2 different meetings (for 2 groups of people from completely different time zones)
16:07:20 <akuznetsova> sorry, I missed previous meeting, what meetings do you mean?
16:07:21 <rakhmerov> didn't have a chance actually to discuss it with all folks, I guess it won't be solved till the summit
16:07:49 <rakhmerov> akuznetsova: for our team meeting
16:08:06 <rakhmerov> there's a suggestion to split it into two meetings
16:08:35 <rakhmerov> one for Europe/Asia and another one for North America
16:08:49 <rakhmerov> because now a lot of people just can't join
16:09:03 <rakhmerov> from US or NZ, for example
16:09:16 <akuznetsova> we can try to make it floating
16:09:18 <rakhmerov> even for folks in Israel it's not too convenient too
16:09:46 <rakhmerov> hm... yes, but it'll be challenging because we have to reserve time slots anyway
16:09:54 <rakhmerov> the number of channels is limited
16:10:21 <akuznetsova> yes, I see
16:10:43 <rakhmerov> we also need to understand how it's going to work
16:11:10 <rakhmerov> for example, who should be on both meetings and how to transfer information between locations
16:11:22 <rakhmerov> let me keep this AI for records
16:11:24 <NikolayM> what if we hold the meeting at openstack-mistral sometimes for that?
16:11:27 <rakhmerov> #action 1. akhmerov: propose time slots for 2 different meetings (for 2 groups of people from completely different time zones)
16:11:37 <rakhmerov> NikolayM: yes, we can
16:11:49 <rakhmerov> the channel is now set up to record meetings
16:12:14 <rakhmerov> I'm going to start a public discussion for that with my ideas on that
16:12:31 <rakhmerov> it's my fault that I haven't done it so far
16:12:50 <akuznetsova> yes, need to find the best approach
16:12:55 <rakhmerov> aha
16:13:07 <rakhmerov> 2. rakhmerov: ask ALU folks to rebase patches
16:13:08 <rakhmerov> done
16:13:21 <rakhmerov> #topic Current status (progress, issues, roadblocks, further plans)
16:14:16 <rakhmerov> my status: reviews, one simple bug fix, added documentation for asynchronous actions
16:14:47 <rakhmerov> hoping to fix https://bugs.launchpad.net/mistral/+bug/1524477 in the next couple of days
16:14:47 <openstack> Launchpad bug 1524477 in Mistral "After-task logic runs multiple times if tasks run in parallel" [High,In progress] - Assigned to Renat Akhmerov (rakhmerov)
16:15:00 <NikolayM> my status: tried to write a unit-test for my patch related to oslo.messaging but failed. Keep it as is, a test (functional) will be in the future when HA-gate will be available
16:15:12 <rakhmerov> I've done some investigation and understand the general reason but am not sure yet how to fix it properly
16:15:33 <rakhmerov> NikolayM: I left my +2 for it
16:15:39 <rakhmerov> need one more +2
16:15:44 <NikolayM> that's about the 'processed' field?
16:15:44 <akuznetsova> my status: started work on ha gate, prepared some base test code (it isn't on review) which will allow to manage mistral services, like to scale up number of engine or kill them all. Faced with a problem that it is not as easy as I thought to change our devstack script to have a ability to run services in a different processes
16:15:52 <rakhmerov> NikolayM: yes
16:16:33 <rakhmerov> akuznetsova: what is that challenage?
16:16:40 <rakhmerov> can you tell quickly?
16:17:53 <akuznetsova> rakhmerov, as I understand we can run only one enabled service in one screen,  now we have one enabled service (from devstack side) "mistral"
16:18:09 <akuznetsova> and we run it in separate screen
16:18:46 <rakhmerov> can we have multiple screens?
16:19:12 <NikolayM> can we have multiple services in settings?
16:19:12 <akuznetsova> we use one command  "screen_it mistral mistral-server --server all ...." - it runs command in "mistral" screen
16:20:00 <akuznetsova> we can't use "screen_it mistral" 3 times for --server engine, --server api and --server executor
16:20:23 <akuznetsova> I propose to make a change and make it similar to sahara project
16:21:01 <rakhmerov> hm.. can we just call screens differently for different components?
16:21:20 <rakhmerov> not all of them just "mistral"
16:21:45 <akuznetsova> as far as I understand there is common global value "USE_SCREEN=True"
16:21:50 <rakhmerov> but "mistral_eng", "mistral_exc" etc
16:22:04 <rakhmerov> ok
16:22:19 <NikolayM> I thought we can add 'mistral-api' (engine/executor) to enabled_service
16:22:25 <akuznetsova> I like how it works in sahara https://github.com/openstack/sahara/blob/master/setup.cfg#L31-L33
16:22:39 <akuznetsova> NikolayM, yes
16:23:25 <rakhmerov> ok, they just have multiple start scripts
16:23:33 <rakhmerov> it's fine too
16:23:36 <rakhmerov> to me
16:23:38 <akuznetsova> after that we can run mistral-api / mistral-engine / mistral-executor
16:24:02 <akuznetsova> and don't do it via mistral-server --server *
16:24:09 <NikolayM> what is the main difference?
16:24:11 <rakhmerov> akuznetsova: will it allow us to run, for example, two executors? or engines?
16:24:39 <akuznetsova> rakhmerov, why not?
16:24:39 <rakhmerov> NikolayM, akuznetsova: yes, I don't actually see a serious difference
16:25:26 <rakhmerov> akuznetsova: I mean will we be able to run 2 executors? In what screens will they be running?
16:25:26 <akuznetsova> rakhmerov, NikolayM the main difference between what? I don't get it
16:25:58 <NikolayM> difference between mistral-api /engine/executor and mistral-server --server api/engine/executor
16:26:13 <NikolayM> they are just different commands
16:26:16 <rakhmerov> yes
16:26:18 <rakhmerov> agree
16:26:31 <rakhmerov> I don't understand how it is different too
16:26:32 <akuznetsova> rakhmerov, we will run only 1 executor in the gate by default
16:26:50 <akuznetsova> another executors will be run not in a screen
16:26:53 <rakhmerov> yes, but we need to have a possibility to run more
16:27:01 <rakhmerov> akuznetsova: ok
16:27:16 <rakhmerov> why are we worried about screens then?
16:27:26 <akuznetsova> problem now that I can't split executor, engine and api
16:27:39 <akuznetsova> in the gate
16:27:48 <rakhmerov> because they would have to run in the same screen, right?
16:28:24 <NikolayM> why they can't be run in separate screens?
16:28:53 <akuznetsova> need to make it possible)
16:28:59 <rakhmerov> guys, let me clarify pls
16:29:02 <NikolayM> I thought we can add in the devstack/settings - enabled_service mistral-api, mistral-engine and so on
16:29:05 <rakhmerov> there's a command
16:29:06 <rakhmerov> screen_it mistral mistral-server --server all ....
16:29:29 <rakhmerov> in this command, "mistral" is the name of the screen, right?
16:29:30 <NikolayM> and then screen_it mistral-api/mistral-engine mistral-server
16:29:40 <NikolayM> rakhmerov, yes
16:29:44 <akuznetsova> rakhmerov, yes
16:29:46 <rakhmerov> if so, we can just use different names for these screens
16:29:54 <rakhmerov> what's the issue then?
16:29:56 <akuznetsova> NikolayM, I didn't try it
16:30:13 <NikolayM> rakhmerov, the name of this service should be enabled first
16:30:13 <rakhmerov> ok, I think we're now on the same page
16:30:28 <rakhmerov> enabled where?
16:30:34 <NikolayM> for mistral-api or mistral-engine
16:30:38 <NikolayM> in devstack/settings
16:30:49 <rakhmerov> is there a problem to enable it?
16:30:51 <rakhmerov> them
16:30:58 <NikolayM> there is a script in mistral/devstack/
16:31:07 <NikolayM> no, I don't think it is a problem
16:31:07 <akuznetsova> I thought that it should be registered somewhere else
16:31:31 <NikolayM> where?
16:31:38 <rakhmerov> ok, let's just agree for now that it needs more experiments
16:31:51 <NikolayM> I think it is the certain place for registering
16:31:59 <akuznetsova> ok, I will check your idea
16:31:59 <NikolayM> ok
16:32:18 <akuznetsova> and update patch
16:33:03 <akuznetsova> after that will push prepared code for ha tests
16:33:10 <rakhmerov> another question here though is, we need to think about the default set of services that we need to run
16:33:31 <rakhmerov> and how other component instances will be running if we need them
16:33:35 <rakhmerov> during our tests
16:34:12 <rbrady> I'm interested in discussing the blueprint for the custom actions. https://blueprints.launchpad.net/mistral/+spec/mistral-custom-actions-api
16:34:35 <rakhmerov> rbrady: sure, let's now just go to open discussion
16:34:40 <rakhmerov> #topic Open Discussion
16:34:49 <akuznetsova> I guess that we can have 1 engine, 1 executor and 1 api by default, all other needed instances can be run as a processes
16:35:09 <rakhmerov> akuznetsova: agree
16:35:13 <rakhmerov> most likely yes
16:35:34 <rakhmerov> rbrady: yes, I created this BP based on the patch I recently reviewed in TripleO
16:35:58 <rbrady> I implemented new auth based on your feedback
16:35:59 <rakhmerov> I just realized that it's not clear enough in Mistral what should be treated a public API
16:36:09 <rakhmerov> rbrady: ok
16:36:11 <rakhmerov> same patch?
16:36:31 <rbrady> but as I continue down the path of creating more custom actions, I'm still trying to figure out what is public and what should be considered private
16:36:39 <rbrady> rakhmerov: yes
16:36:45 <rakhmerov> ok
16:37:30 <rbrady> rakhmerov: in a new patch, I'm looking at "from mistralclient.api.v2 import environments"
16:37:32 <rakhmerov> well, effectively all public API now for creating actions is mistral.actions.Action
16:37:41 <rakhmerov> the base Action interface
16:38:28 <rakhmerov> ok, there's two parts here, just to be clear
16:38:33 <rakhmerov> even 3 parts I guess
16:38:37 <rbrady> I know tripleo might be an interesting case as we're trying to integrate several openstack services
16:39:06 <rakhmerov> 1. All that's exposed in REST API is, of course, a public API
16:39:17 <rakhmerov> and it's expressed in the client too
16:39:33 <rbrady> but specifically we have some requirements for accessing the mistral-environment from an action execution and for now I think using the EnvironmentManager would work
16:39:39 <rakhmerov> 2. mistral.actions.base.Action, should be used for subclassing when we need to implement a new action
16:40:16 <rakhmerov> ok, just to understand on 100%, what do you mean by mistral-environment?
16:40:49 <rbrady> mistral environment, the place for saving state
16:41:12 <rbrady> http://docs.openstack.org/developer/mistral/guides/mistralclient_guide.html#environments
16:41:39 <rakhmerov> oh, ok
16:41:54 <rakhmerov> I thought you were mostly talking about security context
16:42:01 <akuznetsova> rakhmerov, we can create env with some default values for params and use it in wf
16:42:02 <rbrady> I'm looking at treating mistral just like we treat all of the other openstack services
16:42:04 <rakhmerov> at least I saw that in the previous patchset
16:42:32 <rakhmerov> rbrady: in what way?
16:42:42 <rbrady> the new patchset follows the guidance in the review and no longer attempts to use mistral's keystone auth or mistral.context
16:43:25 <rbrady> rakhmerov: similar to the default openstack actions in mistral
16:43:27 <rakhmerov> yes, that's ok. But do you still need to use that info? I mean mistral.context?
16:43:43 <rakhmerov> rbrady: ooh, ok. I got it
16:43:48 <rbrady> rakhmerov: I think I can access the data I need to through mistral client
16:44:11 <rakhmerov> so, let me try to summarize
16:44:44 <rakhmerov> now, in Mistral we can use Nova, Glance etc. with corresponding actions w/o having to pass security specific info
16:44:48 <rakhmerov> like auth token
16:45:13 <rakhmerov> but we can't do it with custom actions easily
16:45:24 <rbrady> rakhmerov: yes, I'm now requiring auth_token passed to an execution via param
16:45:42 <rakhmerov> yes
16:46:26 <rakhmerov> so what's the solution?
16:46:44 <rbrady> and in follow on patches when adding more custom actions, I want to be able to use the mistralclient in the same way I'd use nova or heat clients
16:47:29 <rbrady> so using mistralclient in a custom action to get access to read and write to mistral environment
16:47:44 <rakhmerov> ok
16:47:46 <rbrady> in the 2 cases where a workflow would not fit our use case
16:48:30 <rbrady> I plan to have a patch up today or tomorrow to illustrate all of this
16:48:43 <rakhmerov> rbrady: can you please leave your comment in the BP too, please?
16:48:59 <rakhmerov> rbrady: yep, the patch would be great
16:49:14 <rakhmerov> I actually now see even better the source of confusion
16:49:20 <rbrady> rakhmerov: yes, but I don't have access to leave a comment on the blueprint
16:49:38 <rakhmerov> rbrady: hm, why?
16:49:58 <rbrady> rakhmerov: nm, I guess it's through whiteboard
16:50:15 <rakhmerov> I guess I need to add you to the team or something
16:50:19 <rbrady> I've been using specs for so long I seem to have forgotten the blueprint process
16:50:29 <rakhmerov> #action add rbrady to Mistral team
16:50:37 <rbrady> o/
16:50:40 <rakhmerov> np, I'll do it
16:50:44 <rbrady> thanks!
16:50:49 <d0ugal> I don't think blueprints support comments.
16:50:55 <rakhmerov> so, just to clarify...
16:51:13 <rakhmerov> right, there's whiteboard
16:51:16 <rakhmerov> and work items
16:51:20 <d0ugal> Yup
16:51:30 <rakhmerov> anyway, it could be added somewhere )
16:51:36 <d0ugal> I was going to comment on it too :) Maybe we can just add to the whiteboard.
16:51:48 <rakhmerov> I'd be also ok to add it into description with keeping authority
16:52:09 <rakhmerov> sure, all options are fine to me
16:52:20 <rakhmerov> so, here's the thing on actions
16:52:24 <rakhmerov> the confusion is
16:53:04 <rakhmerov> on one hand, if we look at our OpenStack specific actions then we'll see that their implementations use mistra.context and other kind of private modules
16:53:07 <rakhmerov> :)
16:53:16 <rakhmerov> but we told you not to use them
16:53:30 <rakhmerov> it is confusing, yes
16:54:29 <rakhmerov> and actually, we need to check if that all works fine even with standard openstack actions
16:54:46 <rakhmerov> because I now have some bad thoughts on this
16:54:58 <d0ugal> hah :)
16:55:06 <rakhmerov> so that's why we need to clearly define the contract for writing new actions
16:55:35 <rakhmerov> that security and environment info must always be accessible for actions
16:55:38 <rakhmerov> for all of them
16:55:48 <rakhmerov> and that must be treated a public API
16:56:04 <rakhmerov> along with what's exposed in the client
16:56:16 <rbrady> rakhmerov: +1
16:56:26 <rakhmerov> ok
16:56:46 <rakhmerov> thanks for bringing this up
16:56:51 <rakhmerov> it is very important
16:57:44 <rbrady> rakhmerov: as I'm going to building on these custom actions in tripleo for the next few weeks leading up to the summit, I'd like to help in any way I can
16:58:07 <rakhmerov> ok, thanks
16:58:12 <rakhmerov> we'll use you )
16:58:16 <rbrady> :)
16:58:44 <rakhmerov> I see it one of them most important issues that we need to solve in the near future
16:58:56 <rakhmerov> because it's related to programming interface
16:59:23 <rakhmerov> and I'm almost sure that our OpenStack actions won't work in case if engine and executor will be separate processes )
16:59:32 <rakhmerov> it might be a bug
16:59:45 <rakhmerov> ok, we're running out of time
16:59:52 <akuznetsova> will see in the nearest future))
16:59:58 <rbrady> do you have a way to set up the processes separately in CI?
17:00:01 <rakhmerov> thanks again for bringing this up and for joining us today
17:00:11 <rbrady> np.  glad to be here
17:00:24 <akuznetsova> rbrady, we are working on it
17:00:27 <d0ugal> Thanks!
17:00:27 <rakhmerov> rbrady: that's what we were discussing today too ) I mean what we call HA gate
17:00:33 <rbrady> ahh
17:00:45 <rakhmerov> once it's done we'll be able to test all those corner cases
17:00:54 <rakhmerov> and distributed setups in general
17:01:14 <rakhmerov> so it all fits into one picture )
17:01:20 <akuznetsova> it is time to go
17:01:21 <rakhmerov> thanks, bye all
17:01:24 <NikolayM> bye!
17:01:25 <akuznetsova> bye
17:01:25 <rbrady> thanks everyone
17:01:31 <rakhmerov> #endmeeting