15:00:14 <d0ugal> #startmeeting mistral
15:00:19 <openstack> Meeting started Mon Apr 30 15:00:14 2018 UTC and is due to finish in 60 minutes.  The chair is d0ugal. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:23 <openstack> The meeting name has been set to 'mistral'
15:01:07 <d0ugal> rakhmerov, apetrich, bobh, mcdoker181818: https://etherpad.openstack.org/p/mistral-office-hours
15:01:13 <d0ugal> Office hour time :)
15:01:24 <apetrich> o/
15:01:33 <d0ugal> Reminder to anyone else, if you want a ping like that at the start of the meeting, add yourself to the etherpad ^
15:04:00 <apetrich> so I was reading this and was wondering if I can pick your minds?
15:04:03 <apetrich> https://bugs.launchpad.net/tripleo/+bug/1757430
15:04:04 <openstack> Launchpad bug 1757430 in tripleo "The ssh_keys and tripleo.undercloud-config Mistral environments should be move to swift" [High,Triaged]
15:04:39 <apetrich> it is a bit tripleo-ish but my questions is more about the mistral side of it
15:04:43 <d0ugal> apetrich: That is a tripleo bug... but maybe it is relevant here? :)
15:04:46 <d0ugal> oh, okay
15:04:48 <d0ugal> Go for it
15:05:18 <apetrich> The bug is store some env keys in swift. but those are done with the environment-get
15:06:25 <apetrich> I was wondering if there's a way to do that in a way that is better for the community
15:06:59 <apetrich> like maybe a setting for a backend storage for environment vars
15:07:00 <d0ugal> "do so that" - do what exactly?
15:07:05 <d0ugal> right
15:07:49 <apetrich> because the best way that I think is having that keep the same api that we have today
15:08:25 <d0ugal> apetrich: I'd argue the Swift API is better than the Mistral Environments API - for tripleo's use anyway :)
15:08:54 <d0ugal> but having a pluggable environment backend would be interesting - it would make it easier to secure things with say a barbican backend
15:09:00 <d0ugal> or the secure swift containers.
15:09:09 <bobh> o/
15:09:10 <apetrich> or memcache if that is needed a lot
15:09:44 <d0ugal> Aye
15:09:52 <d0ugal> bobh: Hey!
15:10:04 <d0ugal> apetrich: sounds like it would be worth exploring anyway - I am not sure how easy it would be
15:10:19 <d0ugal> I don't know much much of the implementation is tied to the environment database code.
15:10:47 <apetrich> me neither but either doing a all in approach like a setting in the mistral.conf or a per env var
15:11:21 <apetrich> from my quick lazy ass research seems that we store that in the database and that's it
15:11:51 <d0ugal> apetrich: yeah, there is the weird mechanism for loading an environment into an execution - but I forget how that works exactly
15:11:56 <d0ugal> It isn't something we use in tripleo
15:12:12 <apetrich> oh that's that
15:12:19 <apetrich> yeah I forgot that
15:12:30 <d0ugal> but that should be easy to make it lookup which backend to use
15:13:14 <apetrich> a setting to change the backend for it would be the best in case of a memcache/elasticsomething storage is welcome
15:13:25 <apetrich> because not hitting the database to get that is important
15:13:37 <d0ugal> yeah
15:13:58 <apetrich> on the other hand an extra optional json field in the database poiting to where it is is an other idea to store that
15:14:15 <d0ugal> lol
15:14:17 <d0ugal> Indeed
15:14:25 <d0ugal> I'd probably prefer a global setting, just for simplicity
15:14:27 <apetrich> if that is empty just return that otherwise use the values that to get the needed value
15:14:42 <d0ugal> I'm not sure I can think of a good use-case for having multiple environment backenvs.
15:14:45 <d0ugal> backends
15:14:48 <apetrich> +1 for the global setting
15:15:45 <apetrich> the use case I think is you just want to store the ssh_keys var into a secured swift that might be extra slow the rest is just trivial
15:15:53 <apetrich> and can be stored in the database
15:16:02 <d0ugal> True
15:16:47 <d0ugal> For tripleo, part of the desire is to have everything stored in one place - so we want everything in Swift
15:17:03 <d0ugal> bobh: Feel free to jump in if you have anything to discuss
15:17:08 <d0ugal> We are just chatting in general
15:17:14 <d0ugal> The same goes to anyone else ^
15:17:51 <bobh> d0ugal: one question came up last week - would there be any value in allowing a "publish-on-error" setting in task-defaults?
15:18:23 <d0ugal> bobh: yes!
15:18:30 <d0ugal> I would like this :)
15:18:43 <bobh> d0ugal: ok, I'll put it on my list of things to do
15:18:56 <bobh> I thought it would be useful too, and was surprised it wasn't supported
15:19:10 <d0ugal> Indeed - I think it makes lots of sense.
15:20:09 <d0ugal> bobh: oh, actually - I don't think there is even "publish" on task-defaults?
15:20:15 <bobh> d0ugal: nope
15:20:40 <bobh> d0ugal: I think they both make sense, if you are doing any sort of structured workflow
15:20:51 <d0ugal> Yeah - I think if we add one we need both.
15:21:06 <d0ugal> I wonder if there is a blueprint for this already... /me searches
15:23:37 <d0ugal> bobh: I didn't find a blueprint for this, but I found this: https://blueprints.launchpad.net/mistral/+spec/mistral-task-default-context
15:24:06 <d0ugal> I wonder if not having the context there might make it hard to support?
15:24:12 <d0ugal> Otherwise it sounds like it should be easy.
15:24:40 <bobh> I wondered the same thing, but theoretically the expressions should not be evaluated until the task context it available - never on their own
15:24:57 <d0ugal> Yup, good point
15:25:02 <bobh> assuming it was implemented properly :-)
15:25:41 <d0ugal> haha, I'm sure it was ;)
15:26:34 <bobh> one other question - I ran into a problem with DB updates failing that was caused by this line:
15:26:35 <bobh> https://github.com/openstack/mistral/blob/master/mistral/engine/tasks.py#L379
15:27:20 <d0ugal> What error did you get?
15:27:22 <bobh> I thought it was interesting that there is an assumption of a specific field name in an output of an action
15:27:56 <bobh> The DB update failed because it was trying to set state_info to a null dict
15:27:58 <d0ugal> bobh: the output from actions is always under the result key.
15:28:06 <bobh> ah - makes sense
15:28:11 <d0ugal> sort-of :)
15:28:21 <bobh> so in this case it was a null dict
15:28:33 <d0ugal> Yeah, that is strange. I am not sure what would cause that.
15:28:41 <bobh> I had to add "or None" after the action output reference to fix the problem
15:29:02 <bobh> that works fine, but I'm trying to come up with a test to reproduce the problem and verify the fix
15:29:13 <d0ugal> That would be good
15:29:15 <d0ugal> Was it a custom action?
15:29:33 <bobh> I'll have to go back and look -could be, we do a lot of that
15:29:46 <bobh> and we don't always do it well...
15:30:28 <bobh> I think adding the or None is probably a good idea in general, just to be safe
15:30:44 <bobh> but I do want to test it
15:31:06 <d0ugal> https://github.com/openstack/mistral-lib/blob/master/mistral_lib/actions/types.py#L58-L60
15:31:20 <d0ugal> I think that is where/why there should always be a result key
15:31:36 <d0ugal> (that code was moved from the mistral repo to mistral-lib, so I'm not sure why it was done like that originally)
15:32:29 <bobh> makes sense
15:38:00 <d0ugal> We had a new bug this week! https://bugs.launchpad.net/mistral/+bug/1767830
15:38:01 <openstack> Launchpad bug 1767830 in Mistral "Task and execution description" [Medium,Confirmed]
15:38:25 <d0ugal> Which seems valid - we don't expost the descriptions for tasks and executions.
15:39:05 <d0ugal> I am a little concerned about how much more database space this would take... since it would need to be duplicated for each exectuion.
15:39:13 <d0ugal> But maybe we store it all anyway? I'd need to check.
15:42:11 <bobh> The execution description is set when you execution-create or execution-update -d
15:42:24 <bobh> it doesn't get copied from the workflow description
15:44:01 <d0ugal> bobh: Good point. There are two different descriptions we need to make sure we don't confuse them :)
15:44:24 <d0ugal> There is the execution description, and the workflow description for that execution
15:45:33 <bobh> seems like copying the workflow description into every execution would not make a lot of sense
15:45:42 <bobh> the workflow_id is there, so you can go get it if you need it
15:45:55 <d0ugal> bobh: but it can change, I think that is the point
15:46:07 <bobh> the workflow description can change?  at runtime?
15:46:16 <d0ugal> Not quite.
15:46:18 <bobh> or over time
15:46:28 <d0ugal> You can upload a workflow, start an execution, update a workflow and start a new execution
15:46:46 <bobh> right - but how much of the previous version of the workflow do we need to preserve?
15:46:49 <d0ugal> The two executions could have different desriptions
15:47:07 <d0ugal> bobh: I think we preserve almost all of it.
15:47:09 <bobh> do we have the spec that was executed?  The description would be in there
15:47:21 <d0ugal> That is what I am not sure about.
15:47:32 <d0ugal> I believe we do have it all.
15:48:01 <d0ugal> bobh: https://github.com/openstack/mistral/blob/master/mistral/db/v2/sqlalchemy/models.py#L173
15:48:38 <bobh> which should have the description, I think
15:49:32 <d0ugal> yup, I think so.
15:49:48 <bobh> though its not exposed via the API for retrieval maybe?
15:49:52 <bobh> spec field
15:49:54 <d0ugal> I don't think so
15:50:09 <d0ugal> so, then I guess the question is - do we want to expost the full spec via the API?
15:50:48 <bobh> probably makes sense to be able to tell what exactly was executed
15:50:55 <d0ugal> True.
15:51:42 <bobh> output will be a huge
15:51:50 <d0ugal> Yeah, that is my concern :)
15:52:38 <d0ugal> we don't want to add that to every request - so maybe we need to add an option or a new endpoint.
15:53:03 <bobh> similar to get-definition
15:53:07 <d0ugal> Yup
15:53:30 <d0ugal> I'm going to copy some of this into the bug ^
15:57:19 <d0ugal> I am going to end the meeting slightly early - I gotta run and do something
15:57:23 <d0ugal> Thanks everyone!
15:57:25 <d0ugal> #endmeeting