Wednesday, 2018-03-21

*** caoyuan has joined #openstack-qinling00:36
*** caoyuan has quit IRC00:44
*** caoyuan has joined #openstack-qinling00:51
*** huntxu has joined #openstack-qinling00:56
*** caoyuan has quit IRC01:00
*** openstackgerrit has joined #openstack-qinling01:22
openstackgerritXiaohan Bai proposed openstack/qinling master: Add a basic feature description about Sync/Async  https://review.openstack.org/55475201:22
openstackgerritXiaohan Bai proposed openstack/qinling master: Add a basic feature description about Sync/Async  https://review.openstack.org/55475201:25
openstackgerritXiaohan Bai proposed openstack/qinling master: Add a basic feature description about Sync/Async  https://review.openstack.org/55475201:37
openstackgerritXiaohan Bai proposed openstack/qinling master: Test  https://review.openstack.org/55475801:46
*** jiangyuan has joined #openstack-qinling01:52
*** caoyuan has joined #openstack-qinling01:58
*** caoyuan has quit IRC01:59
*** caoyuan has joined #openstack-qinling02:00
openstackgerritMerged openstack/qinling master: Add a basic feature description about Sync/Async  https://review.openstack.org/55475202:01
*** caoyuan_ has joined #openstack-qinling02:06
*** caoyuan has quit IRC02:06
openstackgerritMerged openstack/qinling master: zuul: create Neutron config dir for the devstack job  https://review.openstack.org/55441003:35
openstackgerritJiangyuan proposed openstack/qinling master: add customized mem/cpu for image type function  https://review.openstack.org/55394704:55
openstackgerritHunt Xu proposed openstack/qinling master: Exclude tests and db migrations from coverage report  https://review.openstack.org/55480206:25
jiangyuanhi, huntxu ! What do you think of using resource.RLIMIT_CPU and resource.RLIMIT_AS to limit resource taken by file/package function in runtime server subprocess?06:42
huntxujiangyuan: I don't have very good idea either. For me, RLIMIT_CPU is about time limitation, so it seems better to be taken care of by the timeout param07:08
huntxujiangyuan: and take the python2 runtime as the example, if you use resource.set_rlimit, it will limit the flask app's resource usage, but not the actual function which we would like to limit07:11
huntxujiangyuan: they can be close if the overhead of the flask app is considered as a constant and users run the function synchronously.07:12
huntxujiangyuan: But for async executions, one flask app can run serveral copies of the function at the same time, so we still have to find a way to limit the function itself07:13
jiangyuaneach function will be run in a different subprocess, and what if  we set limit in the individual subprocess07:18
huntxujiangyuan: then it is fine for the memory limitation07:23
jiangyuanDo you think that the limitation in  subprocess will affact it's parent process if we use  resource.set_rlimit in subprocess?07:26
jiangyuanAnd for RLIMIT_CPU, I want to use cpu param and timeout param to generate the value07:27
huntxujiangyuan: oh, current python2-runtime uses multiprocessing lib, that means it runs the functions as its childprocesses, so I think they will be limited seperately if you set the limit in the parent process(the flask app)07:32
huntxujiangyuan: but for nodejs runtime, I wonder there would be a way to achieve this07:32
jiangyuanyou said,'set the limit in the parent process'? not  in the individual subprocess ?07:35
jiangyuanI'm not familiar with nodejs, sorry07:37
huntxujiangyuan: yes a child process created via fork will inherit its parent's limitation, and the python2 runtime uses multiprocessing and on linux it should by default using for to create the child processes07:38
huntxujiangyuan: I'm not meaning we can always set the limit on the parent process. Just for the current python2 runtime implemetation, we can get what we want in this way07:42
jiangyuanIf you set the limit on the parent process, then all the executions(all subprocess) will share the same limitation, but we need to Re limit each subprocess every time of execution.07:47
jiangyuanthe parent process should be unlimited due to the pod is unlimited07:52
huntxujiangyuan: if the child process is created via fork, then the resource limitation is inherited, not shared07:57
huntxujiangyuan: but setting limitations for the child process is the right way to go, as it always works07:58
jiangyuanOh, I see. Thank you. I am going to do more tests on it, if it's ok , then I will commit a patch08:17
*** caoyuan_ has quit IRC10:13
*** huntxu has quit IRC11:55
*** jiangyuan has quit IRC12:41
*** caoyuan has joined #openstack-qinling14:19
*** caoyuan has quit IRC15:49
*** caoyuan has joined #openstack-qinling15:58
*** caoyuan_ has joined #openstack-qinling16:19
*** caoyuan has quit IRC16:20
*** caoyuan_ has quit IRC17:09
*** rbrady is now known as rbrady-afk21:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!