| *** catintheroof has quit IRC | 00:29 | |
| *** catintheroof has joined #openstack-mistral | 00:30 | |
| *** catintheroof has quit IRC | 00:30 | |
| *** rbrady has quit IRC | 00:30 | |
| *** jamielennox is now known as jamielennox|away | 00:36 | |
| *** jamielennox|away is now known as jamielennox | 00:37 | |
| *** catintheroof has joined #openstack-mistral | 00:42 | |
| *** zhurong has joined #openstack-mistral | 00:56 | |
| *** gongysh has joined #openstack-mistral | 02:16 | |
| *** catintheroof has quit IRC | 03:57 | |
| *** catintheroof has joined #openstack-mistral | 03:58 | |
| *** catintheroof has quit IRC | 03:59 | |
| openstackgerrit | Merged openstack/mistral-specs master: Force the docs to be built with Python 2 https://review.openstack.org/443468 | 05:24 |
|---|---|---|
| openstackgerrit | Merged openstack/mistral-specs master: Add AUTHORS to the .gitignore https://review.openstack.org/443467 | 05:24 |
| *** jaosorior has joined #openstack-mistral | 06:09 | |
| *** shardy has joined #openstack-mistral | 07:58 | |
| *** jaosorior is now known as jaosorior_breakf | 08:33 | |
| *** fultonj has joined #openstack-mistral | 08:41 | |
| *** jaosorior_breakf is now known as jaosorior | 08:48 | |
| *** fultonj has quit IRC | 08:49 | |
| *** amoralej|off is now known as amoralej | 08:51 | |
| *** jpich has joined #openstack-mistral | 08:54 | |
| *** zhurong has quit IRC | 10:10 | |
| *** gongysh has quit IRC | 10:26 | |
| rakhmerov | d0ugal: hi, did you see my comment in https://review.openstack.org/#/c/443477/ ? | 10:29 |
| d0ugal | rakhmerov: oh, cool - I hadn't seen yours | 10:29 |
| *** openstackgerrit has quit IRC | 10:33 | |
| rakhmerov | np :) | 10:33 |
| *** openstackgerrit has joined #openstack-mistral | 10:33 | |
| openstackgerrit | Merged openstack/mistral-specs master: Add Pike specs into index https://review.openstack.org/441089 | 10:33 |
| rakhmerov | ddeja, d0ugal, breton: I've just read your conversation. 1) Like breton said it's pretty easy to DOS any non- real-time system if there's such a goal in mind 2) Is it really a strict requirement to have seconds precision? Based on #1 I have serious doubts that available existing billing systems support seconds precision | 10:37 |
| rakhmerov | 3) Such a system can use Mistral exclusively (w/o any concurrent activities) | 10:37 |
| rakhmerov | if #3 is possible then precision will be close to seconds | 10:38 |
| d0ugal | thanks! | 10:40 |
| rakhmerov | just my humble opinion ) | 10:53 |
| * rakhmerov is hoping that core reviewers will look at his patches soon ;) | 10:54 | |
| ddeja | rakhmerov: I can take my time todays evening and have fun reviewing patches ;) | 11:13 |
| rakhmerov | ok, thanks! | 11:14 |
| rakhmerov | kong: here? | 11:27 |
| openstackgerrit | Merged openstack/mistral master: Deleting the expired execution with batch size https://review.openstack.org/424621 | 11:31 |
| *** gongysh has joined #openstack-mistral | 11:41 | |
| d0ugal | rakhmerov: hey, is there a way to have on-error called for each item in with-items? | 11:45 |
| rakhmerov | d0ugal: no, on- clauses work based on task result which can be a collection of action results | 11:46 |
| d0ugal | right | 11:46 |
| rakhmerov | so it's a task level, not action level | 11:46 |
| shardy | http://paste.openstack.org/show/602238/ | 11:47 |
| rakhmerov | currently there's nothing available | 11:47 |
| shardy | hey rakhmerov, d0ugal - the use case is I want to fail if an object is missing in swift | 11:47 |
| shardy | and it'd be super-helpful to tell the user which one :) | 11:47 |
| rakhmerov | let me see | 11:47 |
| rakhmerov | hi shardy | 11:47 |
| shardy | hey rakhmerov, and thanks for the help! :) | 11:48 |
| rakhmerov | np :) | 11:48 |
| d0ugal | It would be nice if the task result was {"succeeded": [output, output, ...], "failed": [...]} | 11:49 |
| rakhmerov | so | 11:49 |
| d0ugal | I'm not sure what the task output looks like now actually | 11:49 |
| rakhmerov | I'm not sure 100% if it's going to work in case if the whole task fails but essentially a task always has a result accessible as <% task(task_name).result %> which in case of "with-items" returns a collection of corresponding action results | 11:51 |
| rakhmerov | so you can run YAQL/Jinja against this collection and select only those that failed | 11:51 |
| d0ugal | shardy: ^ | 11:51 |
| rakhmerov | that's how I would do it | 11:51 |
| shardy | rakhmerov: Ok, I can probably work around it with yaql, I was just wondering if there was a cleaner way | 11:51 |
| shardy | thanks :) | 11:51 |
| d0ugal | it does sound like something we could improve | 11:52 |
| rakhmerov | another option: if you need to react immediately upon an action failure you can create a small subworkflow calling swift action | 11:52 |
| d0ugal | yeah, I almost suggested that approach | 11:52 |
| rakhmerov | from a task, and this task's on-error will have a handler | 11:52 |
| rakhmerov | yep | 11:53 |
| shardy | Any examples of that? I'm not quite clear how that would work | 11:53 |
| rakhmerov | d0ugal: however, I'm not really sure if task().result will return a task result correctly in case of task failure, we need to see if this is even covered. If it doesn't work that's a bug | 11:53 |
| rakhmerov | shardy: you mean the second option? | 11:54 |
| d0ugal | shardy: example incoming, but I think it is messier :) | 11:54 |
| rakhmerov | ok ) | 11:54 |
| shardy | rakhmerov: yes the subworkflow suggestion isn't clear to me atm | 11:55 |
| d0ugal | shardy: http://paste.openstack.org/show/602243/ | 11:55 |
| rakhmerov | ook, d0ugal you said you're coming up with an example, right? | 11:55 |
| rakhmerov | ok ) | 11:55 |
| d0ugal | rakhmerov: do you mean this? | 11:55 |
| d0ugal | http://paste.openstack.org/show/602243/ | 11:55 |
| rakhmerov | yep, exactly | 11:55 |
| d0ugal | shardy: The advantage is that the on-errir is called when the head_object fails - otherwise on-error is called at the end of all the with-items | 11:56 |
| shardy | d0ugal: aha! | 11:56 |
| shardy | I get it now, thanks! :) | 11:56 |
| rakhmerov | so basically we just converted actions into tasks so that we can apply individual handlers | 11:56 |
| shardy | that will do what I want I think | 11:56 |
| d0ugal | but you have the complexity of having two workflows :) the sub-workflow could do a zaqar.queue_post | 11:56 |
| shardy | d0ugal: if I do it that way, will the with-items stop when check_swift fails, or keep going so we have to handle potentially multiple error messages from zaqar? | 11:57 |
| shardy | the docs suggest with-items iterates all items regardless of failure? | 11:57 |
| d0ugal | shardy: yeah, it will keep going | 11:58 |
| rakhmerov | the only thing that's missing in the example is workflow actual parameters, should be "workflow: check_swift container=... path=..." | 11:58 |
| rakhmerov | d0ugal, shardy: I need to clarify this | 11:58 |
| d0ugal | rakhmerov: oh yeah | 11:58 |
| d0ugal | shardy: I would probably go with the YAQL approach. | 11:59 |
| shardy | a stop_on_error option would be cool | 11:59 |
| d0ugal | +1 | 11:59 |
| shardy | then you just grab the last task output from the list | 11:59 |
| rakhmerov | with-tems itself will fail upon the first failure but 1) keep in mind that with-items actions can be running in parallel 2) we can design a subworkflow in a way that it doesn't fail even if swift action fails so that the enclosing with-items continues normally | 12:00 |
| shardy | rakhmerov: can the run in parallel even without specifying concurrency? | 12:00 |
| rakhmerov | by default is all parallel | 12:00 |
| rakhmerov | as far as I remember | 12:00 |
| rakhmerov | let me check... | 12:01 |
| shardy | I was thinking the stop on error thing would only work for the non-concurrent case (or, at least the list lookup would only work in that case) | 12:01 |
| shardy | I thought I had to specify concurrency: $n | 12:01 |
| breton | rakhmerov: meh, i just dropped seconds precision | 12:02 |
| rakhmerov | shardy: yes, by default it schedules them all immediately | 12:02 |
| rakhmerov | if you want to limit parallelism you need to define concurrency explicitly | 12:03 |
| rakhmerov | breton: ok :) | 12:03 |
| rakhmerov | need to step away.. | 12:03 |
| shardy | rakhmerov: ack, thanks for clarifying | 12:04 |
| d0ugal | rakhmerov: is there a default overall concurrency? i.e. number of actions/workflows running at the same time. | 12:09 |
| d0ugal | rakhmerov: all I can see is an rpc thread pool of 100 | 12:09 |
| d0ugal | oh and an executor_thread_pool_size | 12:09 |
| d0ugal | I guess that is it :) | 12:10 |
| rakhmerov | d0ugal: yes, I think this is the only thing limiting a number of simultaneous actions | 12:15 |
| rakhmerov | but engine by default schedules them all at once | 12:15 |
| rakhmerov | if "concurrency" is not set | 12:16 |
| d0ugal | rakhmerov: yup, makes sense | 12:16 |
| rakhmerov | btw, not sure what our doc says about it ) | 12:16 |
| d0ugal | rakhmerov: we have a couple of with-items in tripleo that I realise could be quite large - so I am wondering how many will actually run at the same time :) | 12:16 |
| d0ugal | and it sounds like it might be 100 | 12:17 |
| rakhmerov | well, yeah, "at the same time" is tricky thing actually | 12:17 |
| d0ugal | heh | 12:17 |
| rakhmerov | "close to the same time" I guess ) | 12:17 |
| d0ugal | true | 12:17 |
| rakhmerov | d0ugal: yeah, our doc doesn't tell anything about default concurrency | 12:17 |
| rakhmerov | https://bugs.launchpad.net/mistral/+bug/1671803 | 12:19 |
| openstack | Launchpad bug 1671803 in Mistral "with-items default concurrency is not explained in docs" [Medium,Confirmed] | 12:19 |
| *** jkilpatr has joined #openstack-mistral | 12:21 | |
| openstackgerrit | Steven Hardy proposed openstack/mistral master: Add head_object action mapping for swift https://review.openstack.org/444261 | 12:21 |
| *** shardy is now known as shardy_lunch | 12:26 | |
| *** catintheroof has joined #openstack-mistral | 12:34 | |
| *** rbrady has joined #openstack-mistral | 13:09 | |
| *** rbrady has quit IRC | 13:09 | |
| *** rbrady has joined #openstack-mistral | 13:09 | |
| *** gongysh has quit IRC | 13:32 | |
| *** shardy_lunch is now known as shardy | 13:33 | |
| shardy | d0ugal: hey can you see where I'm going wrong here? http://paste.openstack.org/show/602266/ | 14:17 |
| d0ugal | looking | 14:17 |
| shardy | wherever I publish the task output I seem to get nothing | 14:17 |
| shardy | I'm probably doing something stupid but I can't see what :) | 14:18 |
| d0ugal | shardy: I'll try running it and see | 14:19 |
| shardy | d0ugal: ack, thanks - I locally applied the head_object patch, do I need to do anything more than sudo mistral-db-manage populate and restart the services? | 14:19 |
| d0ugal | no, that should do it | 14:20 |
| shardy | it seems to be working as I got an error due to a typo in the action arguments | 14:20 |
| d0ugal | oh, interesting. | 14:20 |
| shardy | (fixed in the version in the paste) | 14:20 |
| *** amoralej is now known as amoralej|lunch | 14:21 | |
| *** rbrady has quit IRC | 14:23 | |
| d0ugal | shardy: the result of the workflow is the result of the final tasks | 14:27 |
| d0ugal | shardy: so in your case that is always task_fail | 14:27 |
| shardy | d0ugal: aha, so I have to publish there, and not in the task which fails? | 14:28 |
| d0ugal | Yeah, if you want that to be the workflow result | 14:28 |
| d0ugal | the output is still there tho' - you can get it with "mistral task-list" and "mistral task-get-result ID" | 14:28 |
| shardy | d0ugal: ah, I see - I misunderstood and thought the execution-output showed the final context with all the published things | 14:29 |
| shardy | let me try, thanks for the help! | 14:29 |
| d0ugal | shardy: and you can see the individual action calls with mistral action-execution-list - which is quite handy, I can see one passes and fails :) | 14:29 |
| shardy | woohoo, that's what I was aiming for :) | 14:29 |
| shardy | d0ugal: Ok I see the task output, which is what I want to access in the workflow, but I'm still doing something wrong: | 14:34 |
| shardy | http://paste.openstack.org/show/602268/ | 14:34 |
| shardy | https://docs.openstack.org/developer/mistral/main_features.html#processing-collections-with-items | 14:35 |
| shardy | that says the task result (accessible in workflow context as <% $.task_name %>) will be a list | 14:35 |
| shardy | but it appears in the failure path, swift_verify_output: <% $.verify_environment_path_exists %>) fails | 14:35 |
| * shardy tries not using publish | 14:36 | |
| d0ugal | shardy: sorry, in a call - will be with you shortly | 14:37 |
| shardy | d0ugal: no worries | 14:37 |
| *** rbrady has joined #openstack-mistral | 14:39 | |
| *** rbrady has quit IRC | 14:39 | |
| *** rbrady has joined #openstack-mistral | 14:39 | |
| shardy | d0ugal: nvm, I think I've got it now | 14:50 |
| shardy | thanks for the help | 14:50 |
| d0ugal | shardy: cool :) | 14:50 |
| d0ugal | np | 14:50 |
| openstackgerrit | Merged openstack/mistral master: Add head_object action mapping for swift https://review.openstack.org/444261 | 14:58 |
| *** catintheroof has quit IRC | 15:00 | |
| *** catintheroof has joined #openstack-mistral | 15:01 | |
| *** thrash is now known as thrash|vet | 15:12 | |
| *** chlong_ has joined #openstack-mistral | 15:15 | |
| *** shardy has quit IRC | 15:17 | |
| *** jaosorior has quit IRC | 15:24 | |
| *** toure|gone is now known as toue | 15:25 | |
| *** toue is now known as toure | 15:25 | |
| *** amoralej|lunch is now known as amoralej | 15:39 | |
| *** chlong_ has quit IRC | 16:19 | |
| *** thrash|vet is now known as thrash | 16:28 | |
| *** jaosorior has joined #openstack-mistral | 16:34 | |
| *** jaosorior has quit IRC | 16:42 | |
| *** catintheroof has quit IRC | 17:07 | |
| *** catintheroof has joined #openstack-mistral | 17:08 | |
| *** jpich has quit IRC | 17:16 | |
| *** vishwanathj has quit IRC | 17:37 | |
| *** vishwanathj has joined #openstack-mistral | 17:37 | |
| *** jkilpatr has quit IRC | 18:03 | |
| *** jkilpatr has joined #openstack-mistral | 18:06 | |
| *** jkilpatr has quit IRC | 19:33 | |
| *** jkilpatr has joined #openstack-mistral | 19:36 | |
| *** chlong_ has joined #openstack-mistral | 19:42 | |
| *** jkilpatr has quit IRC | 19:44 | |
| *** rbrady is now known as rbrady-afk | 19:48 | |
| *** jkilpatr has joined #openstack-mistral | 19:58 | |
| *** vishwanathj has quit IRC | 20:25 | |
| *** vishwanathj has joined #openstack-mistral | 20:26 | |
| *** amoralej is now known as amoralej|off | 20:44 | |
| *** jtomasek has quit IRC | 21:30 | |
| *** vishwanathj has quit IRC | 21:49 | |
| *** vishwanathj has joined #openstack-mistral | 21:50 | |
| *** chlong_ has quit IRC | 22:21 | |
| *** catintheroof has quit IRC | 22:23 | |
| *** Kevin_Zheng has quit IRC | 22:53 | |
| *** jkilpatr has quit IRC | 23:34 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!