*** bobh has joined #openstack-mistral | 02:07 | |
*** bobh has quit IRC | 02:17 | |
*** gkadam has joined #openstack-mistral | 02:48 | |
*** gkadam has quit IRC | 02:50 | |
*** openstackgerrit has joined #openstack-mistral | 04:54 | |
openstackgerrit | Renat Akhmerov proposed openstack/mistral master: Add "convert_input_data" config property for YAQL expressions https://review.openstack.org/648316 | 04:54 |
---|---|---|
*** openstackgerrit has quit IRC | 06:09 | |
*** jtomasek has joined #openstack-mistral | 06:29 | |
*** nguyenhai has joined #openstack-mistral | 06:46 | |
*** nguyenhai_ has quit IRC | 06:49 | |
*** pgaxatte has joined #openstack-mistral | 06:52 | |
vgvoleg | omg, we tried to start about 100 engines for big process and there was a lot of | 07:43 |
vgvoleg | oslo_db.exception.DBDeadlock: (psycopg2.extensions.TransactionRollbackError) deadlock detected | 07:43 |
vgvoleg | in scheduler logic | 07:44 |
vgvoleg | after process was stucked we have seen a lot of idle connections with db | 07:50 |
vgvoleg | Have anyone ever hit this behavior? | 07:52 |
vgvoleg | Was `random_delay` add because of this? | 07:53 |
*** akovi has joined #openstack-mistral | 08:00 | |
*** gkadam has joined #openstack-mistral | 08:10 | |
*** akovi has quit IRC | 09:25 | |
*** akovi has joined #openstack-mistral | 09:26 | |
*** akovi has quit IRC | 09:29 | |
*** akovi has joined #openstack-mistral | 09:29 | |
*** zigo has joined #openstack-mistral | 10:00 | |
*** openstackgerrit has joined #openstack-mistral | 10:01 | |
openstackgerrit | Renat Akhmerov proposed openstack/mistral master: Set the lower version of sphinxcontrib-pecanwsme to 0.10.0 https://review.openstack.org/649270 | 10:01 |
*** vgvoleg has quit IRC | 10:04 | |
*** vgvoleg has joined #openstack-mistral | 10:14 | |
*** akovi has quit IRC | 10:14 | |
*** akovi has joined #openstack-mistral | 10:15 | |
rakhmerov | vgvoleg: please talk to me before doing something like this :) | 11:04 |
rakhmerov | 100 engines is a way beyond Mistral's current scalability | 11:04 |
rakhmerov | 10 is about the max number for something big | 11:04 |
rakhmerov | as far as scheduler, yes, it's a narrow place now, it's being fixed by adding a new implementation of scheduler (the prototype is already in the code) that doesn't hit DB so badly | 11:05 |
*** akovi has quit IRC | 11:50 | |
*** akovi has joined #openstack-mistral | 11:51 | |
*** nguyenhai has quit IRC | 12:32 | |
vgvoleg | guys, are there any known memory leaks in engine and any parameters to handle it? Seems like engine don't free it's memory at all | 13:49 |
vgvoleg | test case: big workflow with huge context | 13:50 |
vgvoleg | I'm going to retry this with 10 engines and I'll share grafana graphics | 13:51 |
*** pgaxatte has quit IRC | 15:44 | |
*** akovi has quit IRC | 16:06 | |
*** gkadam has quit IRC | 17:02 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!