15:00:14 #startmeeting mistral 15:00:19 Meeting started Mon Apr 30 15:00:14 2018 UTC and is due to finish in 60 minutes. The chair is d0ugal. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:23 The meeting name has been set to 'mistral' 15:01:07 rakhmerov, apetrich, bobh, mcdoker181818: https://etherpad.openstack.org/p/mistral-office-hours 15:01:13 Office hour time :) 15:01:24 o/ 15:01:33 Reminder to anyone else, if you want a ping like that at the start of the meeting, add yourself to the etherpad ^ 15:04:00 so I was reading this and was wondering if I can pick your minds? 15:04:03 https://bugs.launchpad.net/tripleo/+bug/1757430 15:04:04 Launchpad bug 1757430 in tripleo "The ssh_keys and tripleo.undercloud-config Mistral environments should be move to swift" [High,Triaged] 15:04:39 it is a bit tripleo-ish but my questions is more about the mistral side of it 15:04:43 apetrich: That is a tripleo bug... but maybe it is relevant here? :) 15:04:46 oh, okay 15:04:48 Go for it 15:05:18 The bug is store some env keys in swift. but those are done with the environment-get 15:06:25 I was wondering if there's a way to do that in a way that is better for the community 15:06:59 like maybe a setting for a backend storage for environment vars 15:07:00 "do so that" - do what exactly? 15:07:05 right 15:07:49 because the best way that I think is having that keep the same api that we have today 15:08:25 apetrich: I'd argue the Swift API is better than the Mistral Environments API - for tripleo's use anyway :) 15:08:54 but having a pluggable environment backend would be interesting - it would make it easier to secure things with say a barbican backend 15:09:00 or the secure swift containers. 15:09:09 o/ 15:09:10 or memcache if that is needed a lot 15:09:44 Aye 15:09:52 bobh: Hey! 15:10:04 apetrich: sounds like it would be worth exploring anyway - I am not sure how easy it would be 15:10:19 I don't know much much of the implementation is tied to the environment database code. 15:10:47 me neither but either doing a all in approach like a setting in the mistral.conf or a per env var 15:11:21 from my quick lazy ass research seems that we store that in the database and that's it 15:11:51 apetrich: yeah, there is the weird mechanism for loading an environment into an execution - but I forget how that works exactly 15:11:56 It isn't something we use in tripleo 15:12:12 oh that's that 15:12:19 yeah I forgot that 15:12:30 but that should be easy to make it lookup which backend to use 15:13:14 a setting to change the backend for it would be the best in case of a memcache/elasticsomething storage is welcome 15:13:25 because not hitting the database to get that is important 15:13:37 yeah 15:13:58 on the other hand an extra optional json field in the database poiting to where it is is an other idea to store that 15:14:15 lol 15:14:17 Indeed 15:14:25 I'd probably prefer a global setting, just for simplicity 15:14:27 if that is empty just return that otherwise use the values that to get the needed value 15:14:42 I'm not sure I can think of a good use-case for having multiple environment backenvs. 15:14:45 backends 15:14:48 +1 for the global setting 15:15:45 the use case I think is you just want to store the ssh_keys var into a secured swift that might be extra slow the rest is just trivial 15:15:53 and can be stored in the database 15:16:02 True 15:16:47 For tripleo, part of the desire is to have everything stored in one place - so we want everything in Swift 15:17:03 bobh: Feel free to jump in if you have anything to discuss 15:17:08 We are just chatting in general 15:17:14 The same goes to anyone else ^ 15:17:51 d0ugal: one question came up last week - would there be any value in allowing a "publish-on-error" setting in task-defaults? 15:18:23 bobh: yes! 15:18:30 I would like this :) 15:18:43 d0ugal: ok, I'll put it on my list of things to do 15:18:56 I thought it would be useful too, and was surprised it wasn't supported 15:19:10 Indeed - I think it makes lots of sense. 15:20:09 bobh: oh, actually - I don't think there is even "publish" on task-defaults? 15:20:15 d0ugal: nope 15:20:40 d0ugal: I think they both make sense, if you are doing any sort of structured workflow 15:20:51 Yeah - I think if we add one we need both. 15:21:06 I wonder if there is a blueprint for this already... /me searches 15:23:37 bobh: I didn't find a blueprint for this, but I found this: https://blueprints.launchpad.net/mistral/+spec/mistral-task-default-context 15:24:06 I wonder if not having the context there might make it hard to support? 15:24:12 Otherwise it sounds like it should be easy. 15:24:40 I wondered the same thing, but theoretically the expressions should not be evaluated until the task context it available - never on their own 15:24:57 Yup, good point 15:25:02 assuming it was implemented properly :-) 15:25:41 haha, I'm sure it was ;) 15:26:34 one other question - I ran into a problem with DB updates failing that was caused by this line: 15:26:35 https://github.com/openstack/mistral/blob/master/mistral/engine/tasks.py#L379 15:27:20 What error did you get? 15:27:22 I thought it was interesting that there is an assumption of a specific field name in an output of an action 15:27:56 The DB update failed because it was trying to set state_info to a null dict 15:27:58 bobh: the output from actions is always under the result key. 15:28:06 ah - makes sense 15:28:11 sort-of :) 15:28:21 so in this case it was a null dict 15:28:33 Yeah, that is strange. I am not sure what would cause that. 15:28:41 I had to add "or None" after the action output reference to fix the problem 15:29:02 that works fine, but I'm trying to come up with a test to reproduce the problem and verify the fix 15:29:13 That would be good 15:29:15 Was it a custom action? 15:29:33 I'll have to go back and look -could be, we do a lot of that 15:29:46 and we don't always do it well... 15:30:28 I think adding the or None is probably a good idea in general, just to be safe 15:30:44 but I do want to test it 15:31:06 https://github.com/openstack/mistral-lib/blob/master/mistral_lib/actions/types.py#L58-L60 15:31:20 I think that is where/why there should always be a result key 15:31:36 (that code was moved from the mistral repo to mistral-lib, so I'm not sure why it was done like that originally) 15:32:29 makes sense 15:38:00 We had a new bug this week! https://bugs.launchpad.net/mistral/+bug/1767830 15:38:01 Launchpad bug 1767830 in Mistral "Task and execution description" [Medium,Confirmed] 15:38:25 Which seems valid - we don't expost the descriptions for tasks and executions. 15:39:05 I am a little concerned about how much more database space this would take... since it would need to be duplicated for each exectuion. 15:39:13 But maybe we store it all anyway? I'd need to check. 15:42:11 The execution description is set when you execution-create or execution-update -d 15:42:24 it doesn't get copied from the workflow description 15:44:01 bobh: Good point. There are two different descriptions we need to make sure we don't confuse them :) 15:44:24 There is the execution description, and the workflow description for that execution 15:45:33 seems like copying the workflow description into every execution would not make a lot of sense 15:45:42 the workflow_id is there, so you can go get it if you need it 15:45:55 bobh: but it can change, I think that is the point 15:46:07 the workflow description can change? at runtime? 15:46:16 Not quite. 15:46:18 or over time 15:46:28 You can upload a workflow, start an execution, update a workflow and start a new execution 15:46:46 right - but how much of the previous version of the workflow do we need to preserve? 15:46:49 The two executions could have different desriptions 15:47:07 bobh: I think we preserve almost all of it. 15:47:09 do we have the spec that was executed? The description would be in there 15:47:21 That is what I am not sure about. 15:47:32 I believe we do have it all. 15:48:01 bobh: https://github.com/openstack/mistral/blob/master/mistral/db/v2/sqlalchemy/models.py#L173 15:48:38 which should have the description, I think 15:49:32 yup, I think so. 15:49:48 though its not exposed via the API for retrieval maybe? 15:49:52 spec field 15:49:54 I don't think so 15:50:09 so, then I guess the question is - do we want to expost the full spec via the API? 15:50:48 probably makes sense to be able to tell what exactly was executed 15:50:55 True. 15:51:42 output will be a huge 15:51:50 Yeah, that is my concern :) 15:52:38 we don't want to add that to every request - so maybe we need to add an option or a new endpoint. 15:53:03 similar to get-definition 15:53:07 Yup 15:53:30 I'm going to copy some of this into the bug ^ 15:57:19 I am going to end the meeting slightly early - I gotta run and do something 15:57:23 Thanks everyone! 15:57:25 #endmeeting