| *** thrash is now known as thrash|g0ne | 00:18 | |
| *** bobh has quit IRC | 01:35 | |
| *** DaveTurner has quit IRC | 02:10 | |
| *** bobh has joined #openstack-mistral | 02:13 | |
| *** jamielennox is now known as jamielennox|away | 02:30 | |
| *** bobh has quit IRC | 02:49 | |
| *** bobh has joined #openstack-mistral | 02:57 | |
| *** jamielennox|away is now known as jamielennox | 02:59 | |
| *** bobh has quit IRC | 03:04 | |
| *** bobh has joined #openstack-mistral | 03:16 | |
| *** sharatss has quit IRC | 03:18 | |
| *** sharatss has joined #openstack-mistral | 03:19 | |
| *** catintheroof has joined #openstack-mistral | 03:27 | |
| *** catintheroof has quit IRC | 03:40 | |
| *** bobh has quit IRC | 04:02 | |
| rakhmerov | dtantsur|afk: no support for microversions | 04:53 |
|---|---|---|
| rakhmerov | dtantsur|afk: yes, you need to update mapping.json and update the database (mistral-db-manage --config-file <file> populate) | 04:58 |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Add timestamp at the bottom of every page https://review.openstack.org/397761 | 06:22 |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Add timestamp at the bottom of every page https://review.openstack.org/397761 | 06:23 |
| *** ist has joined #openstack-mistral | 06:24 | |
| *** janki has joined #openstack-mistral | 06:26 | |
| *** ist has quit IRC | 06:34 | |
| d0ugal | dtantsur|afk: so, we could write simple tripleo actions that call ironic client initially. Then migrate over once either it is enabled by default or this is resolved in Mistral | 08:18 |
| *** dtantsur|afk is now known as dtantsur | 08:27 | |
| dtantsur | d0ugal, right, this was my thought too... I just didn't like the idea of such wrapper | 08:29 |
| *** jaosorior has joined #openstack-mistral | 08:29 | |
| *** shardy has joined #openstack-mistral | 08:30 | |
| *** jpich has joined #openstack-mistral | 08:30 | |
| d0ugal | dtantsur: agreed | 08:32 |
| d0ugal | dtantsur: I suspect it wont be trivial to solve in Mistral - but it has been on my list, so this helps move it up in priority :) | 08:33 |
| dtantsur | cool :) | 08:33 |
| d0ugal | I guess a simple solution would be to make it globally configurable | 08:34 |
| d0ugal | Which wouldn't be ideal, but it would be easy | 08:34 |
| d0ugal | and it would be cleaner than having to specify the client version in every action call. | 08:35 |
| *** ist has joined #openstack-mistral | 08:35 | |
| dtantsur | yep | 08:37 |
| d0ugal | dtantsur: so it is just a case of passing in this? https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/base.py#L59 | 08:38 |
| d0ugal | rakhmerov: ping | 08:40 |
| rakhmerov | yes | 08:40 |
| rakhmerov | here | 08:40 |
| rakhmerov | hi, what's up? | 08:41 |
| d0ugal | rakhmerov: How would you feel about additional config settings to pass in extra parameters to the openstack clients? | 08:41 |
| d0ugal | rakhmerov: more context just above in my conversation with dtantsur | 08:41 |
| rakhmerov | Istvan Imre (ist) is going to work on that | 08:42 |
| rakhmerov | soon | 08:42 |
| rakhmerov | he needs it too | 08:42 |
| d0ugal | ah, I was thinking about working on it now :) | 08:42 |
| rakhmerov | :) | 08:42 |
| rakhmerov | ist: hi Istvan, you here? | 08:42 |
| rakhmerov | would you please be able to share the details on those additional params passed to clients? | 08:42 |
| rakhmerov | so that we see if that correlates to what we're discussing | 08:43 |
| d0ugal | so, we want to pass this: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/base.py#L59 | 08:43 |
| d0ugal | because some ironic features are not enabled by default | 08:43 |
| d0ugal | this version is newer than the default | 08:43 |
| rakhmerov | ooh, this.. | 08:44 |
| rakhmerov | I think it's somewhat different actually | 08:44 |
| d0ugal | oh, what are you thinking of? | 08:44 |
| rakhmerov | Istvan's idea was to be able to pass any context parameters from the client side | 08:44 |
| rakhmerov | so that actions could use them | 08:45 |
| d0ugal | Can you give an example? | 08:45 |
| dtantsur | if I get it right, that would probably work too.. it's less convenient, but more in line with microversioning idea | 08:46 |
| rakhmerov | any environmental stuff that is meaningful for a particular client. Actually those TARGET_* params could be considered just arbitrary dynamic params that certain actions use (in our case all OpenStack actions) | 08:46 |
| rakhmerov | it's basically to give a client to influence more on what they are doing with Mistral | 08:47 |
| rakhmerov | to make Mistral less connected to a certain OpenStack | 08:47 |
| d0ugal | Right, I see | 08:48 |
| rakhmerov | Istvan has some other use cases, I'll ask him to share that | 08:48 |
| d0ugal | so openstack actions could then be used against other openstacks. | 08:48 |
| ist | hello | 08:48 |
| rakhmerov | d0ugal: yes | 08:48 |
| rakhmerov | ist: hey ) | 08:48 |
| d0ugal | so it is esentially the same problem, we just have different motivation for it :) | 08:48 |
| rakhmerov | can you please share your idea with dynamic params | 08:48 |
| rakhmerov | ? | 08:48 |
| rakhmerov | d0ugal: in my understanding, yes | 08:49 |
| ist | We just passing some hidden parameters to our actions, which we would like to hide from workflow write | 08:49 |
| ist | writer | 08:49 |
| rakhmerov | author, yep | 08:49 |
| rakhmerov | ist: what's your use case? | 08:49 |
| rakhmerov | for example | 08:49 |
| rakhmerov | we already discussed it but it slipped away from my head | 08:50 |
| d0ugal | :) | 08:50 |
| ist | Basically we would like to pass some read-only and hidden data to our custom actions | 08:50 |
| rakhmerov | :) yeah | 08:50 |
| d0ugal | ist: the auth token for example? | 08:50 |
| rakhmerov | so the idea is to be able to customize workflow behavior but w/o rewriting workflow itself | 08:51 |
| rakhmerov | from the client | 08:51 |
| rakhmerov | via having more flexible actions | 08:51 |
| d0ugal | right | 08:51 |
| rakhmerov | taking some additional params | 08:51 |
| ist | for example if we pass them to workflow as explicit parameter our 3rd party workflow developer could easily mix these parameters in their workflow, and may it could do harm on our system | 08:52 |
| rakhmerov | ok | 08:52 |
| ist | that's why we need some hidden/read-only parameters to protect ourself from 3rd party WF developers | 08:53 |
| rakhmerov | :) | 08:53 |
| rakhmerov | I see | 08:53 |
| rakhmerov | it's because we let them write some custom workflows | 08:54 |
| rakhmerov | as part of using our product | 08:54 |
| d0ugal | right | 08:54 |
| rakhmerov | hm.. | 08:54 |
| d0ugal | so it this a bit like sandboxing a little bit? | 08:54 |
| rakhmerov | a little bit tricky but I get it | 08:54 |
| rakhmerov | kind of, yes | 08:54 |
| d0ugal | okay | 08:54 |
| rakhmerov | dtantsur: is this any helpful to you? | 08:55 |
| d0ugal | ist: Had you thought about how you would do this in Mistral? | 08:55 |
| d0ugal | rakhmerov: if it allows passing arbitrary parameters to the openstack client initialization then I think it would help dtantsur too. | 08:56 |
| rakhmerov | ist: is it ok if we share that design document that you sent me some time ago? In a little bit different form | 08:56 |
| dtantsur | rakhmerov, looks so that we can pass TARGET_OS_BAREMETAL_API_VERSION=.. or something | 08:56 |
| rakhmerov | it had all enough info | 08:56 |
| dtantsur | ist, my only problem: will these parameters be visible to client initialization code? | 08:56 |
| rakhmerov | dtantsur: yes, and our actions would be able to extract this param from the context | 08:56 |
| dtantsur | rakhmerov, from this object? https://github.com/openstack/mistral/blob/30d05ab7d2c47f7e77ab333392c619aeda1f9f68/mistral/actions/openstack/actions.py#L367 | 08:57 |
| d0ugal | rakhmerov: we need to be careful with client caching if we re-enable it, as this will make the number of unique clients far greater. | 08:57 |
| d0ugal | (just a side point) | 08:57 |
| d0ugal | dtantsur: Yeah, I think so. | 08:58 |
| rakhmerov | dtantsur: it's a tricky thing. We have this problem now I believe. We need to do it more smartly. In fact, we'd like to not create clients all the time (cache them) but we need to see what we pass and cache based on some critical init params | 08:58 |
| rakhmerov | yes, right | 08:58 |
| dtantsur | right, so API version would be such critical parameter | 08:58 |
| rakhmerov | d0ugal: may be we shouldn't be caching at all or only subset of clients | 08:58 |
| rakhmerov | agnostic to any param changes | 08:59 |
| rakhmerov | right | 08:59 |
| d0ugal | rakhmerov: I think an LRU cache is fine, we pick a sensible default max cache size and make it configurable. | 08:59 |
| rakhmerov | affirmative | 09:00 |
| d0ugal | we probably need to add a cache key function to each client action | 09:00 |
| rakhmerov | +1 | 09:00 |
| d0ugal | so it does get a little more complicated, but I don't think it will be that bad. | 09:00 |
| dtantsur | is all this planned for the near future? | 09:01 |
| d0ugal | rakhmerov, ist: How would we pass this context to the workflow? | 09:01 |
| dtantsur | if so, I won't end up doing hacks in TripleO | 09:01 |
| d0ugal | dtantsur: I'll see if I can do it, I want to avoid the hacks too :) | 09:01 |
| d0ugal | or see if I can help anyway | 09:02 |
| dtantsur | cool! ping me as well for testing/reviews/discussions (I'll stay on this channel for a while then) | 09:02 |
| rakhmerov | dtantsur: this is one of the important things that we agreed to address within the team. d0ugal and rbrady (and myself) are assigned to do it in the current cycle | 09:02 |
| dtantsur | great | 09:02 |
| d0ugal | Yeah, it is part of a larger problem. | 09:03 |
| rakhmerov | d0ugal: so, it's just supposed to be info stored in our context transparently. So for example, we passed something from the client, json = {"my_custom_params": {"key1": ..., "key2": ...}} | 09:04 |
| rakhmerov | it gets stored in WF context in some special section "custom_client_params" or similar | 09:04 |
| d0ugal | rakhmerov: right, that's fine - I just don't understand where it comes from originally? | 09:04 |
| rakhmerov | and also becomes a part of our security context | 09:04 |
| rakhmerov | which is available to all actions | 09:05 |
| rakhmerov | so they can extract that info too | 09:05 |
| rakhmerov | it comes from the client | 09:05 |
| d0ugal | rakhmerov: so it is added as a new parameter in the API? | 09:05 |
| rakhmerov | that's the whole point, we need to make them dynamic | 09:06 |
| ist | yes new parameter to client is needed | 09:06 |
| rakhmerov | so that we can pass virtually anything we want | 09:06 |
| d0ugal | so, for example, something like this... | 09:06 |
| rakhmerov | yes, in the client it's a new param but with a flexible structure | 09:07 |
| d0ugal | mistral execution-create my_workflow $WORKFLOW_INPUT $WORKFLOW_CONTEXT | 09:07 |
| rakhmerov | hm... kind of | 09:07 |
| rakhmerov | on the second thought, how is that different from the currently existing environment? :) | 09:08 |
| rakhmerov | env | 09:08 |
| d0ugal | heh, good question. | 09:08 |
| * rakhmerov renat is slightly confused | 09:08 | |
| * d0ugal too | 09:08 | |
| rakhmerov | ist: help us :) | 09:08 |
| d0ugal | but environments confuse me, because I don't understand how anybody else uses them lol | 09:08 |
| d0ugal | (and I know our use-case is a bit weird) | 09:08 |
| rakhmerov | d0ugal: the idea of environments is very simple IMO. It's the 100% analogy to regular OS environments | 09:09 |
| ist | but if we would like to go even further, we could consider refactoring of security / OS context, and create some structure like action_context / <any key> / ... where <any key> could be OS for example for OpenStack security context | 09:10 |
| rakhmerov | i.e. we have a bash script which uses some environment variables | 09:10 |
| rakhmerov | and they can be different for different users | 09:10 |
| rakhmerov | ist: yes, right | 09:10 |
| d0ugal | rakhmerov: right, I understand that - but is there a way to read and write to an environment from the workflow? or do you only do that from custom actions? | 09:11 |
| rakhmerov | d0ugal: yes, indeed | 09:11 |
| ist | Curerently we use only from custom actions | 09:11 |
| rakhmerov | <% env().my_env_var %> | 09:11 |
| rakhmerov | where env() points to the environment passed alongside with WF input (either just a name of the stored env or the entire json structure) | 09:12 |
| d0ugal | rakhmerov: does that environment only exist for the duration of the workflow? | 09:12 |
| rakhmerov | d0ugal: so one difference from env that I see is that like Istvan said those params must be hidden from workflow | 09:12 |
| rakhmerov | d0ugal: yes | 09:12 |
| rakhmerov | ooh, no!! | 09:12 |
| rakhmerov | d0ugal: depends | 09:12 |
| d0ugal | hah | 09:12 |
| rakhmerov | there's two cases: 1) named environment 2) explicit JSON | 09:13 |
| rakhmerov | in #1 it's just a DB persistent object | 09:13 |
| d0ugal | Right, we only use named environments. | 09:13 |
| rakhmerov | and when we start a WF we just specify its name | 09:13 |
| rakhmerov | for #2 it's a transient structure existing only for this WF execution | 09:13 |
| d0ugal | okay, thans | 09:14 |
| d0ugal | thanks | 09:14 |
| d0ugal | I guess I might try and write a bit more here: http://docs.openstack.org/developer/mistral/dsl/dsl_v2.html#environment | 09:14 |
| d0ugal | :) | 09:14 |
| rakhmerov | very simple example | 09:14 |
| rakhmerov | In my workflows I need to be sending emails | 09:14 |
| rakhmerov | whereas I use the same sent of workflows different users need to have different SMTP settings | 09:15 |
| rakhmerov | SMTP details could be passed in the environment | 09:15 |
| rakhmerov | so that WF don't have to have them hardcoded (which doesn't work for many users) | 09:16 |
| rakhmerov | d0ugal: yeah, I believe we have some bug to write more about it. It's very poorely documented, I admit | 09:16 |
| d0ugal | okay, that makes sense. thanks | 09:16 |
| rakhmerov | .. set of workflows .. | 09:16 |
| rakhmerov | ok | 09:17 |
| d0ugal | I'll see if I can improve the documentation a bit, and you can tell me where I get it wrong :) | 09:17 |
| d0ugal | Back to the action/context discussion. So it seems environment isn't the right place for it | 09:17 |
| rakhmerov | so, on Istvan's idea, we'll write a spec first anyway | 09:17 |
| d0ugal | Okay | 09:17 |
| rakhmerov | d0ugal: sure | 09:17 |
| rakhmerov | d0ugal: yeah, seems like it's not | 09:17 |
| d0ugal | Let me know how I can help, or I will just review/test. | 09:18 |
| rakhmerov | so Istvan's idea is more about "Mistral client -> actions" relationship | 09:20 |
| rakhmerov | ok | 09:20 |
| d0ugal | Yeah, I think it is essentially the same for us | 09:21 |
| dtantsur | please add me to the spec review as well once it's up | 09:21 |
| dtantsur | completely different thought: "OpenStack context is available by $.openstack" I wonder if we should pass API versions there after them being passed from standard versioning headers OS-API-Version (or whatever it is called now) | 09:23 |
| * d0ugal reads http://developer.openstack.org/api-guide/compute/microversions.html | 09:24 | |
| d0ugal | dtantsur: so mistral would look at the API version for each individual project it creates clients for? | 09:25 |
| d0ugal | It seems both like a nice idea but also an abuse of the system. I'm not sure. | 09:26 |
| dtantsur | d0ugal, actually I think this is how microversions are intended to be used.. but I may be wrong | 09:29 |
| d0ugal | dtantsur: You are usually correct tho' :) | 09:36 |
| dtantsur | lol, not always definitely :) | 09:36 |
| *** dtantsur is now known as dtantsur|bbl | 10:00 | |
| *** jamielennox is now known as jamielennox|away | 10:21 | |
| openstackgerrit | Merged openstack/mistral: Updated from global requirements https://review.openstack.org/400904 | 10:24 |
| *** pkoniszewski has quit IRC | 10:35 | |
| openstackgerrit | Gal Margalit proposed openstack/mistral-dashboard: mistral-dashboard: added action executions screens https://review.openstack.org/401188 | 10:35 |
| *** ddeja has quit IRC | 10:36 | |
| *** ddeja has joined #openstack-mistral | 10:44 | |
| *** pkoniszewski has joined #openstack-mistral | 10:44 | |
| *** igormarnat has quit IRC | 11:00 | |
| *** rakhmerov has quit IRC | 11:01 | |
| *** akuznetsova has quit IRC | 11:02 | |
| *** akuznetsova has joined #openstack-mistral | 11:02 | |
| *** ist has quit IRC | 11:03 | |
| *** rakhmerov has joined #openstack-mistral | 11:05 | |
| *** igormarnat has joined #openstack-mistral | 11:07 | |
| *** ist_ has joined #openstack-mistral | 11:19 | |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Updated the retries_remain statement https://review.openstack.org/383617 | 11:21 |
| *** dtantsur|bbl is now known as dtantsur | 11:36 | |
| openstackgerrit | fengchaoyang proposed openstack/mistral: Fix configuration_guide docs not very good configuration. https://review.openstack.org/401216 | 11:43 |
| *** openstackgerrit has quit IRC | 12:03 | |
| *** openstackgerrit has joined #openstack-mistral | 12:03 | |
| *** catintheroof has joined #openstack-mistral | 12:09 | |
| *** catintheroof has quit IRC | 12:15 | |
| *** catintheroof has joined #openstack-mistral | 12:20 | |
| openstackgerrit | fengchaoyang proposed openstack/mistral: Fix configuration_guide docs not very good configuration. https://review.openstack.org/401216 | 12:31 |
| *** thrash|g0ne is now known as thrash | 12:32 | |
| *** shardy is now known as shardy_lunch | 12:39 | |
| *** catintheroof has quit IRC | 12:44 | |
| *** dprince has joined #openstack-mistral | 12:56 | |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Fix devstack plugin compatibility https://review.openstack.org/401249 | 13:09 |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Fix devstack plugin compatibility https://review.openstack.org/401249 | 13:12 |
| *** shardy_lunch is now known as shardy | 13:28 | |
| openstackgerrit | Sharat Sharma proposed openstack/python-mistralclient: Wrap the contents of the input and description fields https://review.openstack.org/401275 | 14:08 |
| sharatss | rakhmerov, d0ugal ddeja https://review.openstack.org/#/c/401249/ this fixes gate-mistral-devstack-dsvm job. Please have a look | 14:12 |
| d0ugal | sharatss: will do! | 14:31 |
| ddeja | sharatss: +2 from me | 14:33 |
| d0ugal | and me :) | 14:43 |
| *** bobh has joined #openstack-mistral | 14:49 | |
| *** bobh has quit IRC | 14:49 | |
| *** bobh has joined #openstack-mistral | 14:50 | |
| *** jaosorior has quit IRC | 14:53 | |
| *** jaosorior has joined #openstack-mistral | 14:54 | |
| *** jaosorior has quit IRC | 15:09 | |
| *** jaosorior has joined #openstack-mistral | 15:10 | |
| *** chlong has joined #openstack-mistral | 15:25 | |
| *** chlong has quit IRC | 15:34 | |
| *** janki has quit IRC | 15:38 | |
| *** ist_ has quit IRC | 15:43 | |
| *** chlong has joined #openstack-mistral | 15:49 | |
| openstackgerrit | Merged openstack/mistral: Fix devstack plugin compatibility https://review.openstack.org/401249 | 15:57 |
| *** DaveTurner has joined #openstack-mistral | 16:14 | |
| *** dprince has quit IRC | 16:33 | |
| *** weshay is now known as weshay_lunch | 16:37 | |
| *** Kiall has joined #openstack-mistral | 16:54 | |
| *** dprince has joined #openstack-mistral | 16:58 | |
| *** bobh has quit IRC | 17:01 | |
| *** bobh has joined #openstack-mistral | 17:04 | |
| *** jaosorior has quit IRC | 17:15 | |
| openstackgerrit | Michal Gershenzon proposed openstack/mistral: Yaql Tasks Function https://review.openstack.org/401360 | 17:23 |
| *** weshay_lunch is now known as weshay | 17:42 | |
| *** shardy has quit IRC | 17:43 | |
| *** chlong has quit IRC | 17:50 | |
| *** dtantsur is now known as dtantsur|afk | 18:24 | |
| *** DaveTurner has quit IRC | 18:57 | |
| *** catintheroof has joined #openstack-mistral | 19:27 | |
| *** catintheroof has quit IRC | 19:34 | |
| *** jpich has quit IRC | 20:03 | |
| *** dprince has quit IRC | 20:46 | |
| *** catintheroof has joined #openstack-mistral | 21:08 | |
| *** jamielennox|away is now known as jamielennox | 21:22 | |
| *** catintheroof has quit IRC | 21:40 | |
| *** chlong has joined #openstack-mistral | 22:19 | |
| *** chlong has quit IRC | 22:30 | |
| *** catintheroof has joined #openstack-mistral | 22:48 | |
| *** catintheroof has quit IRC | 22:52 | |
| *** catintheroof has joined #openstack-mistral | 23:00 | |
| *** thrash is now known as thrash|g0bble | 23:07 | |
| *** catinthe_ has joined #openstack-mistral | 23:23 | |
| *** bobh has quit IRC | 23:24 | |
| *** catintheroof has quit IRC | 23:26 | |
| *** bobh has joined #openstack-mistral | 23:33 | |
| *** bobh has quit IRC | 23:38 | |
| *** catinthe_ has quit IRC | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!