| *** ricolin has joined #openstack-mistral | 00:59 | |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672390 | 01:02 |
|---|---|---|
| *** gkadam has joined #openstack-mistral | 03:50 | |
| *** gkadam has quit IRC | 03:50 | |
| *** apetrich has quit IRC | 04:20 | |
| *** eyalb1 has joined #openstack-mistral | 04:45 | |
| *** pgaxatte has joined #openstack-mistral | 06:01 | |
| *** pgaxatte has quit IRC | 06:05 | |
| *** pgaxatte has joined #openstack-mistral | 06:05 | |
| *** pgaxatte has quit IRC | 06:39 | |
| *** pgaxatte has joined #openstack-mistral | 06:41 | |
| openstackgerrit | Eyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672390 | 07:38 |
| openstackgerrit | Eyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672390 | 07:41 |
| openstackgerrit | Eyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672390 | 07:49 |
| *** vgvoleg has joined #openstack-mistral | 07:56 | |
| rakhmerov | hi all | 08:01 |
| eyalb1 | \o | 08:01 |
| vgvoleg | o/ | 08:01 |
| rakhmerov | vgvoleg, eyalb1, d0ugal: hi | 08:01 |
| d0ugal | Hi | 08:01 |
| rakhmerov | #startmeeting Mistral | 08:01 |
| openstack | Meeting started Wed Jul 24 08:01:47 2019 UTC and is due to finish in 60 minutes. The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:01 |
| *** openstack changes topic to " (Meeting topic: Mistral)" | 08:01 | |
| openstack | The meeting name has been set to 'mistral' | 08:01 |
| vgvoleg | I'd like to discuss this https://review.opendev.org/#/c/670211/ | 08:02 |
| rakhmerov | folks, eyalb1 is my colleague from Nokia who's now the Vitrage PTL and he's also going to help with Mistral | 08:02 |
| rakhmerov | just so you know | 08:02 |
| rakhmerov | vgvoleg: sure, go ahead | 08:02 |
| vgvoleg | I don't understand why Andras didn't like this :D | 08:03 |
| rakhmerov | let me see | 08:04 |
| vgvoleg | This is just an opportunity to implement actions and to write a wf which could be run as usual or in dry-run mode | 08:04 |
| vgvoleg | This feature could be compared with ansible's check mode | 08:05 |
| *** apetrich has joined #openstack-mistral | 08:05 | |
| rakhmerov | well, I've not reviewed the patch itself so it's a bit difficult to discuss it | 08:06 |
| rakhmerov | looking.. | 08:06 |
| vgvoleg | So if you want to run the flow with dry-run mode, you should be sure that actions from this flow support this | 08:07 |
| rakhmerov | hm.. ok | 08:10 |
| rakhmerov | so | 08:10 |
| rakhmerov | I kind of understand his point | 08:10 |
| rakhmerov | because yes, in a real workflow data is critically important for all the transitions etc. | 08:11 |
| rakhmerov | but at the same time I see no harm in having this feature merged | 08:11 |
| vgvoleg | Yes, I understand this | 08:11 |
| rakhmerov | at least we could see if it works in some very simple configuration | 08:12 |
| rakhmerov | its simplest paths | 08:12 |
| vgvoleg | It would be great to adapt std actions for this | 08:12 |
| rakhmerov | so what your patch does is it makes it possible to run "test" in actions instead of "run"? | 08:12 |
| rakhmerov | is this it? | 08:12 |
| vgvoleg | yes | 08:13 |
| vgvoleg | and maybe openstack actions | 08:14 |
| vgvoleg | to return stubs in test run | 08:14 |
| rakhmerov | ok | 08:14 |
| vgvoleg | but first of all, this feature is for custom actions | 08:15 |
| rakhmerov | yeah | 08:15 |
| rakhmerov | so, I'm not against it | 08:15 |
| rakhmerov | the only thing I'm concerned with is maybe we need to consider a more sophisticated approach to testing workflows | 08:16 |
| rakhmerov | some kind of framework for mocking workflow steps | 08:16 |
| rakhmerov | but I'm not sure.. because it would involve serious time investements | 08:16 |
| rakhmerov | it's basically "How can we make sure that the workflow works correctly in any possible situation" | 08:17 |
| rakhmerov | sort of unit testing for workflows | 08:17 |
| rakhmerov | but that's complex | 08:17 |
| rakhmerov | we've discussed that idea in the past but realized it'd be very difficult to make it usable enough | 08:18 |
| rakhmerov | not overcomplicated | 08:18 |
| apetrich | Morning rakhmerov | 08:18 |
| rakhmerov | apetrich: hey Adriano | 08:19 |
| apetrich | How was the vacations? | 08:19 |
| rakhmerov | long time no see | 08:19 |
| rakhmerov | ho's it going? | 08:19 |
| rakhmerov | my vacation was good ) I had good rest | 08:19 |
| rakhmerov | and almost forgot about the work (which is very good) :) | 08:19 |
| apetrich | :) | 08:19 |
| rakhmerov | vgvoleg: so Oleg, let us review your patch first | 08:20 |
| rakhmerov | I've already glanced quickly and it looks ok but I still have some questions | 08:20 |
| rakhmerov | vgvoleg: but generally the idea is OK to me | 08:21 |
| rakhmerov | apetrich: btw, I remember you helped investigating a problem with the requirements | 08:22 |
| apetrich | aye | 08:22 |
| rakhmerov | if you have a few min could you please look at http://logs.openstack.org/90/672390/4/check/requirements-check/00c6495/job-output.txt.gz ? | 08:22 |
| rakhmerov | we're having hard time to see why the requirement check keeps failing | 08:23 |
| rakhmerov | in stable/stein branch | 08:23 |
| apetrich | sure can do right now.. I'm trying a deploynment | 08:23 |
| rakhmerov | in master this patch is OK | 08:23 |
| rakhmerov | apetrich: thanks a lot | 08:23 |
| apetrich | rakhmerov, can I have a link to the patch? | 08:24 |
| rakhmerov | yes, sure | 08:24 |
| rakhmerov | https://review.opendev.org/#/c/672390 | 08:24 |
| *** jtomasek has joined #openstack-mistral | 08:25 | |
| apetrich | cheers | 08:25 |
| rakhmerov | :) | 08:25 |
| vgvoleg | I have a question: is it possible to "bind" execution to one engine? | 08:26 |
| vgvoleg | so if I had a small workflow and a lot of engine replicas, it would be great not to have a huge overhead to load this definition to the cache of every engine | 08:28 |
| rakhmerov | vgvoleg: currently no | 08:28 |
| rakhmerov | well, if the workflow is small then there won't be a huge overhead ) | 08:29 |
| rakhmerov | it'll be huge if the workflow is big | 08:29 |
| rakhmerov | I think we've already discussed this with you | 08:30 |
| rakhmerov | it'd be good to see some BP about that | 08:30 |
| vgvoleg | I perform a small svt, results are tangible | 08:30 |
| rakhmerov | svt? | 08:30 |
| vgvoleg | stress test | 08:30 |
| rakhmerov | then share this with us somehow | 08:31 |
| rakhmerov | comparison, what you did etc | 08:31 |
| rakhmerov | I remember we talked about the opportunity to have kind of "ad-hoc" workflows | 08:31 |
| vgvoleg | it's not done, but I can share what's done | 08:32 |
| rakhmerov | that is, we create a workflow and immediately run it | 08:32 |
| rakhmerov | ok | 08:32 |
| vgvoleg | oh what an amazing english skills I have | 08:32 |
| vgvoleg | sorry :D | 08:32 |
| rakhmerov | is that the same thing? | 08:32 |
| vgvoleg | so | 08:32 |
| rakhmerov | vgvoleg: no worries, your english is fine ) | 08:32 |
| vgvoleg | no, but it's all about one case | 08:32 |
| vgvoleg | so the case is | 08:32 |
| rakhmerov | and will become even better over time ;) | 08:32 |
| rakhmerov | ok | 08:32 |
| rakhmerov | what I've just described I believe is doable | 08:33 |
| vgvoleg | we run in the parallel way next scenario: create workflow -> execute workflow -> wait till it's done -> repeat | 08:33 |
| rakhmerov | but that's mostly about workflow definition life-time | 08:33 |
| rakhmerov | not about how we distribute workflow processing | 08:33 |
| rakhmerov | yes, ok | 08:33 |
| vgvoleg | the workflow is small - 8-10 http tasks | 08:34 |
| rakhmerov | ok | 08:34 |
| vgvoleg | I scale users, that performs this scenario | 08:34 |
| rakhmerov | and you want to bind it to one engine | 08:34 |
| rakhmerov | I see | 08:34 |
| vgvoleg | https://docs.google.com/spreadsheets/d/1S26ImGdb3V6TnbkIVyDA85OHGx0Dvbt7o0kX-rWfmDE/edit#gid=0 | 08:35 |
| rakhmerov | vgvoleg: the thing is that binding to one engine conflicts with the fundamental idea behind Mistral | 08:35 |
| rakhmerov | that every workflow step must be stored in DB | 08:35 |
| vgvoleg | and the current results (it's not done) shows that scaling engine makes performance worse | 08:36 |
| rakhmerov | so that any other processing unit (engine) could continue with the processing | 08:36 |
| rakhmerov | vgvoleg: of course, it makes it worse | 08:36 |
| rakhmerov | but it makes it more reliable | 08:36 |
| rakhmerov | which is a critically important property of any workflow engine | 08:36 |
| rakhmerov | it's all about scaling | 08:36 |
| vgvoleg | value in google doc describes how many flows could be success i one minute | 08:36 |
| vgvoleg | in one* | 08:37 |
| rakhmerov | I sent a permission request | 08:37 |
| rakhmerov | vgvoleg: that's all understandable, of course :) | 08:37 |
| vgvoleg | done | 08:37 |
| rakhmerov | If I just write a program in Python it'll be even faster | 08:38 |
| rakhmerov | and we won't need any YAML, YAQL and scale ) | 08:38 |
| vgvoleg | :D | 08:38 |
| vgvoleg | yes | 08:38 |
| rakhmerov | but we'll lose the most important property of the system: reliability and scalability | 08:39 |
| vgvoleg | what do you mean "ad-hoc workflow"? | 08:39 |
| rakhmerov | if we talk about just routing messages related to some workflow to one specific engine then it's doable I think, yes | 08:39 |
| rakhmerov | but what if the engine crashes? | 08:39 |
| vgvoleg | yes I understand you | 08:40 |
| rakhmerov | by "ad-hoc" workflow I mean: we do something like "mistral execution-create --definition my_wf.yaml" and Mistral would create an execution based on my_wf.yaml | 08:40 |
| vgvoleg | the main challenge is to adapt mistral to work with one-off workflows as good as with others | 08:40 |
| vgvoleg | yes, that's it | 08:41 |
| rakhmerov | but without creating an object in DB that we could access with "mistral workflow-get" | 08:41 |
| rakhmerov | yep | 08:41 |
| rakhmerov | vgvoleg: that feature is totally OK to me | 08:41 |
| rakhmerov | I see no issues and I think it's not so difficult to implement | 08:41 |
| vgvoleg | ok I'll try and maybe there would be something to discuss in the next meeting :) | 08:42 |
| rakhmerov | but as far as binding a workflow to one engine, well, I have serious doubts. I'll look at this Google Doc though | 08:42 |
| rakhmerov | yes | 08:42 |
| rakhmerov | sure | 08:42 |
| rakhmerov | let's keep discussing it | 08:43 |
| vgvoleg | I understand that binding is a bad idea | 08:44 |
| vgvoleg | probably the "ad-hoc workflow feature" will solve this problem | 08:45 |
| rakhmerov | ok | 08:46 |
| rakhmerov | guys, do you have anything else to discuss? | 08:46 |
| vgvoleg | nope | 08:47 |
| rakhmerov | eyalb1, apetrich: btw, I didn't release T-1 and T-2 because I kind of didn't see a big point | 08:47 |
| vgvoleg | wait | 08:47 |
| vgvoleg | one question | 08:47 |
| rakhmerov | what do you think we need to do moving further? | 08:47 |
| rakhmerov | apetrich, eyalb1: I thought about just releasing T-1 instead of T-3 when the T-3 time comes and then go to RCs | 08:48 |
| vgvoleg | why don't we build and deploy docker image in CI steps? | 08:48 |
| rakhmerov | vgvoleg: aah, Andras knows better but he's on vacation now | 08:48 |
| rakhmerov | I think he'll be here next week | 08:49 |
| rakhmerov | there was a reason for it but I honestly don't remember | 08:49 |
| vgvoleg | nice | 08:49 |
| rakhmerov | we used to have this job but then it was removed | 08:49 |
| rakhmerov | let's ask Andras when he's back | 08:50 |
| vgvoleg | ok | 08:51 |
| *** altlogbot_1 has quit IRC | 08:51 | |
| *** irclogbot_1 has quit IRC | 08:51 | |
| rakhmerov | ok, so let's finish the meeting now | 08:52 |
| rakhmerov | apetrich, eyalb1: please reply to the release question when you have a chance | 08:52 |
| rakhmerov | #endmeeting | 08:52 |
| *** openstack changes topic to "Mistral the Workflow Service for OpenStack. https://docs.openstack.org/mistral/latest/" | 08:52 | |
| openstack | Meeting ended Wed Jul 24 08:52:32 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 08:52 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-07-24-08.01.html | 08:52 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-07-24-08.01.txt | 08:52 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-07-24-08.01.log.html | 08:52 |
| *** altlogbot_2 has joined #openstack-mistral | 08:52 | |
| *** irclogbot_1 has joined #openstack-mistral | 08:52 | |
| *** vgvoleg has quit IRC | 09:05 | |
| openstackgerrit | Eyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672390 | 09:38 |
| rakhmerov | apetrich: eyalb1 managed to fix the requirements problem | 09:55 |
| rakhmerov | apetrich: can you please review https://review.opendev.org/#/c/672245/ ? | 09:55 |
| rakhmerov | it's small | 09:55 |
| rakhmerov | we need it desperately ) | 09:55 |
| rakhmerov | d0ugal: ^ | 09:55 |
| *** apetrich has quit IRC | 09:56 | |
| d0ugal | rakhmerov: Seems like a hack, but I guess it is fine. | 09:57 |
| *** quenti[m] has quit IRC | 10:08 | |
| rakhmerov | d0ugal: yeah, it is a hack actually ) | 10:10 |
| rakhmerov | but I've no idea how to do it better | 10:10 |
| d0ugal | I'm not sure either :) | 10:12 |
| *** pgaxatte has quit IRC | 10:24 | |
| *** quenti[m] has joined #openstack-mistral | 10:47 | |
| *** irclogbot_1 has quit IRC | 11:33 | |
| *** irclogbot_0 has joined #openstack-mistral | 11:35 | |
| *** apetrich has joined #openstack-mistral | 11:49 | |
| *** pgaxatte has joined #openstack-mistral | 12:03 | |
| *** vgvoleg has joined #openstack-mistral | 12:04 | |
| openstackgerrit | Merged openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672390 | 12:46 |
| *** apetrich has quit IRC | 13:47 | |
| *** apetrich has joined #openstack-mistral | 13:56 | |
| *** eyalb1 has quit IRC | 14:21 | |
| *** ricolin_ has joined #openstack-mistral | 14:42 | |
| *** ricolin has quit IRC | 14:45 | |
| *** bobh has joined #openstack-mistral | 14:49 | |
| openstackgerrit | Elod Illes proposed openstack/mistral stable/rocky: Add bindep.txt file for binary dependencies used in unit tests https://review.opendev.org/672543 | 14:51 |
| *** ricolin_ is now known as ricolin | 15:02 | |
| *** pgaxatte has quit IRC | 15:07 | |
| *** ricolin has quit IRC | 16:23 | |
| *** mmethot_ is now known as mmethot | 17:02 | |
| openstackgerrit | Merged openstack/mistral master: Retry a DB transaction on "Too many connections" error https://review.opendev.org/672245 | 17:19 |
| *** bobh has quit IRC | 17:42 | |
| *** vgvoleg has quit IRC | 19:00 | |
| *** bobh has joined #openstack-mistral | 19:07 | |
| *** jtomasek has quit IRC | 19:39 | |
| *** bobh has quit IRC | 20:28 | |
| *** bobh has joined #openstack-mistral | 20:30 | |
| *** bobh has quit IRC | 20:39 | |
| openstackgerrit | guang-yee proposed openstack/mistral stable/rocky: disable triggering version discovery in test_generator unit tests https://review.opendev.org/672185 | 21:03 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!