| *** hongbin has joined #heat | 00:40 | |
| *** openstack has joined #heat | 01:21 | |
| *** ChanServ sets mode: +o openstack | 01:21 | |
| *** yangyapeng has joined #heat | 01:29 | |
| *** ramishra has joined #heat | 01:54 | |
| *** neatherweb has joined #heat | 01:58 | |
| *** Guest86210 has quit IRC | 02:22 | |
| *** ricolin has joined #heat | 02:30 | |
| *** Guest86210 has joined #heat | 02:38 | |
| *** skramaja has joined #heat | 02:57 | |
| *** jerrygb has joined #heat | 03:00 | |
| *** jerrygb has quit IRC | 03:05 | |
| *** jerrygb has joined #heat | 03:52 | |
| *** jerrygb has quit IRC | 04:05 | |
| *** d0ugal_ has joined #heat | 04:14 | |
| *** d0ugal has quit IRC | 04:15 | |
| *** hongbin has quit IRC | 04:22 | |
| *** ysandeep has joined #heat | 04:30 | |
| *** ysandeep has quit IRC | 04:34 | |
| *** nicolasbock has joined #heat | 05:21 | |
| *** armaan has joined #heat | 05:29 | |
| *** yangyapeng has quit IRC | 05:33 | |
| *** yangyapeng has joined #heat | 05:57 | |
| *** yangyapeng has quit IRC | 06:02 | |
| *** yangyapeng has joined #heat | 06:03 | |
| *** jcoufal has joined #heat | 06:16 | |
| *** jcoufal has quit IRC | 06:16 | |
| *** jcoufal has joined #heat | 06:17 | |
| *** hoonetorg has quit IRC | 06:20 | |
| openstackgerrit | Xinni Ge proposed openstack/heat-dashboard master: Group resource icons by project https://review.openstack.org/577732 | 06:27 |
|---|---|---|
| *** kazsh_ has quit IRC | 06:28 | |
| *** kazsh has joined #heat | 06:29 | |
| *** hoonetorg has joined #heat | 06:32 | |
| *** jtomasek has joined #heat | 06:43 | |
| *** lifeless has quit IRC | 06:49 | |
| *** ysandeep has joined #heat | 06:56 | |
| *** ysandeep has quit IRC | 07:02 | |
| *** armaan has quit IRC | 07:05 | |
| *** armaan has joined #heat | 07:06 | |
| *** armaan has quit IRC | 07:12 | |
| *** ysandeep has joined #heat | 07:32 | |
| *** hjensas has quit IRC | 07:37 | |
| *** armaan has joined #heat | 07:51 | |
| strigazi | ricolin: ramishra are you here? | 08:08 |
| *** serlex has joined #heat | 08:09 | |
| *** jistr|off is now known as jistr | 08:14 | |
| ricolin | strigazi, o/ | 08:14 |
| *** d0ugal_ has quit IRC | 08:16 | |
| *** dbecker has joined #heat | 08:16 | |
| *** d0ugal has joined #heat | 08:17 | |
| *** d0ugal has quit IRC | 08:17 | |
| *** d0ugal has joined #heat | 08:17 | |
| *** armaan has quit IRC | 08:21 | |
| strigazi | ricolin: I still have problems with db connections | 08:23 |
| strigazi | ricolin: question | 08:23 |
| *** shardy has joined #heat | 08:24 | |
| strigazi | ricolin: is max_overflow and max_pool_size defined per process? | 08:24 |
| strigazi | ricolin: there is a possibility that this fix is wrong: | 08:24 |
| strigazi | ricolin: https://git.openstack.org/cgit/openstack/heat/commit/?id=d7c2de60cb9ed0d3ddb077d603b7028f8d91d238 | 08:25 |
| * ricolin reading sqlalchemy doc | 08:27 | |
| strigazi | ricolin: this patch sets max_overflow = num_threads maybe we need max_overflow = (num_threads * num_engines) | 08:28 |
| *** derekh has joined #heat | 08:30 | |
| therve | strigazi: engines are different processes? | 08:32 |
| *** sshnaidm|off is now known as sshnaidm | 08:39 | |
| strigazi | therve: child processes, but still processes | 08:41 |
| therve | I would think that config is per process | 08:42 |
| strigazi | therve: I got the error that the patch is fixing again :( imeoutError: QueuePool limit of size 5 overflow 10 reached, connection | 08:42 |
| strigazi | timed out, timeout 30 | 08:42 |
| *** namnh has joined #heat | 08:43 | |
| therve | strigazi: Did you set max_overflow in the config? | 08:43 |
| strigazi | therve: yes | 08:44 |
| therve | strigazi: So that code path is ignored I would say | 08:44 |
| therve | Hum no it depends on the value I gues | 08:45 |
| ricolin | strigazi, how about config executor_thread_pool_size | 08:45 |
| strigazi | therve: max_overflow = 18 executor_thread_pool_size = 18 num_engine_workers = 4 | 08:45 |
| strigazi | ricolin: ^^ | 08:45 |
| strigazi | therve: ricolin max_overflow = 18 executor_thread_pool_size = 18 num_engine_workers = 4 max_pool_size = 4 | 08:46 |
| therve | So it's possible that code only works when you don't tinker with other values | 08:46 |
| therve | IE we're not testing the correct values | 08:47 |
| therve | strigazi: It says overflow 10 reached and you pass 18? | 08:48 |
| ricolin | strigazi, in this case _configure_db_conn_pool_size function actually done nothing (I mean from https://git.openstack.org/cgit/openstack/heat/commit/?id=d7c2de60cb9ed0d3ddb077d603b7028f8d91d238) | 08:49 |
| strigazi | therve: this is the error from the bug, mine said 4 and 18 | 08:49 |
| therve | Ah | 08:49 |
| therve | So yeah that code path is not triggered | 08:52 |
| therve | Try to find the right numbers that work for you | 08:52 |
| strigazi | therve: ricolin Maybe, I'm not sure, we need max_overflow = (num_threads * num_engines) | 08:53 |
| ricolin | therve, does engines share pool? | 08:54 |
| strigazi | therve: ricolin maybe max_overflow is set per parent process and executor_thread_pool_size per child process? | 08:54 |
| therve | Maybe executor_thread_pool_size is unused now | 08:54 |
| therve | It's deprecated | 08:54 |
| therve | Ah no it's a new name | 08:54 |
| *** gfidente has joined #heat | 08:59 | |
| *** gfidente has quit IRC | 08:59 | |
| *** gfidente has joined #heat | 08:59 | |
| strigazi | therve: does my assumption make sense? | 08:59 |
| therve | strigazi: No | 08:59 |
| strigazi | therve: so whet I have 4 worker engines i have the overflow and executor threads per engine? | 09:01 |
| therve | That's my understanding | 09:01 |
| strigazi | therve: maybe it is better one engine and more executors? | 09:01 |
| therve | It's a convergence issue, it's the number of worker the problem | 09:01 |
| therve | strigazi: AFAIU it works when you don't change executor_thread_pool_size | 09:02 |
| therve | Just you have more connection than expected? | 09:02 |
| strigazi | therve with executor_thread_pool_size unchanged I hit the limit of the db server | 09:03 |
| therve | OK | 09:03 |
| *** r-mibu has joined #heat | 09:04 | |
| strigazi | I could scale down the number of engines and not set executor_thread_pool_size, that would work | 09:04 |
| strigazi | therve: I hope it would work | 09:05 |
| therve | I guess yeah | 09:06 |
| strigazi | I could have 1 engine per node or 2 | 09:06 |
| *** jcoufal has quit IRC | 09:10 | |
| *** neatherweb has quit IRC | 09:10 | |
| *** jcoufal has joined #heat | 09:13 | |
| *** jcoufal has quit IRC | 09:13 | |
| *** jcoufal has joined #heat | 09:14 | |
| ricolin | strigazi, just curious, what will happen when you set max_overflow to 19(executor_thread_pool_size+1) | 09:20 |
| ramishra | strigazi: Did you try with a greater max_oveflow than thread_pool_size (may be -1: unlimited), may be connections are not returned to pool/closed quickly enough? But porbably this may lead to hit the db limit. | 09:26 |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Option for retrieving child templates and env files from swift https://review.openstack.org/551871 | 09:30 |
| ramishra | therve: What do you think would be good default? https://review.openstack.org/#/c/551871/6/heat/common/config.py | 09:31 |
| *** hjensas has joined #heat | 09:33 | |
| *** hjensas has joined #heat | 09:33 | |
| *** namnh has quit IRC | 09:36 | |
| strigazi | ramishra: ricolin therve now I have another problem, I have stack in check_in_progress for 30 mins and I don't know what to do with it | 09:40 |
| strigazi | ramishra: ricolin therve I restarted the engines and no engines are doing anything on it | 09:40 |
| *** namnh has joined #heat | 09:41 | |
| strigazi | ramishra: ricolin therve should I do stack cancel? | 09:43 |
| ramishra | strigazi: If you're using convergence, so you can try update it with the same templates | 09:43 |
| strigazi | ramishra: openstack stack update --existing <stack> | 09:43 |
| ramishra | if not convergence then the stack should be marked CHECK_FAILED when you restart the engines | 09:43 |
| ramishra | strigazi: yep | 09:43 |
| strigazi | I use convergence | 09:43 |
| ricolin | strigazi, ramishra we can also send another stack check right? it's not block to check on a check inprogress stack | 09:46 |
| ramishra | ricolin: I think stack-check uses non-convergence path, not sure if it works on IN_PROGRESS stacks, Also, I'm not sure why the stack should be stuck IN_PROGRESS for 30 mins, so better try and update.. | 09:48 |
| ricolin | ramishra, it works. and yeah, use update is better.:) | 09:49 |
| *** hjensas has quit IRC | 09:51 | |
| strigazi | ramishra I think i have a timeout set a bit high and there is a lock somewhere | 09:54 |
| strigazi | engine_life_check_timeout = 600 | 09:56 |
| strigazi | rpc_poll_timeout = 600 | 09:56 |
| strigazi | rpc_response_timeout = 600 | 09:56 |
| strigazi | these three | 09:56 |
| strigazi | ramishra: will it help if lower them and restart the engines? | 09:57 |
| *** hjensas has joined #heat | 09:59 | |
| *** hjensas has joined #heat | 09:59 | |
| *** peereb has joined #heat | 09:59 | |
| *** peereb has quit IRC | 10:01 | |
| *** peereb has joined #heat | 10:01 | |
| *** peereb has quit IRC | 10:02 | |
| *** peereb has joined #heat | 10:03 | |
| *** peereb has quit IRC | 10:04 | |
| *** peereb has joined #heat | 10:04 | |
| *** peereb has quit IRC | 10:05 | |
| *** namnh_ has joined #heat | 10:21 | |
| ricolin | strigazi, Back to previous question, do you think to set max_overflow to worker_pool_size+1 will help in your case? | 10:21 |
| *** namnh_ has quit IRC | 10:21 | |
| *** gfidente has quit IRC | 10:23 | |
| *** namnh has quit IRC | 10:24 | |
| strigazi | ricolin: I will try | 10:33 |
| *** jerrygb has joined #heat | 10:33 | |
| openstackgerrit | Merged openstack/heat master: Improve error reporting for missing nested template https://review.openstack.org/537994 | 10:38 |
| *** shardy has quit IRC | 10:44 | |
| *** edmondsw has joined #heat | 10:45 | |
| *** edmondsw_ has joined #heat | 10:48 | |
| *** edmondsw has quit IRC | 10:49 | |
| *** gfidente has joined #heat | 10:57 | |
| *** jcoufal has quit IRC | 11:08 | |
| openstackgerrit | Merged openstack/heat master: Import Zuul setting from project-config https://review.openstack.org/576193 | 11:13 |
| therve | ramishra: Could you max_template_size instead? | 11:15 |
| ramishra | therve: I thought about that in the beginning. IMO, checking for 'a' template_size would not possibly add any value as file list would have number of templates and that list can kill the engine | 11:17 |
| ramishra | if we really want to controll it then checking on the total size of objects to be fetched would be useful | 11:17 |
| therve | ramishra: How does the current limit works when you have multiple files? | 11:18 |
| ramishra | we add up the sizes from the file list | 11:19 |
| therve | So that the total is less than max_template_size? | 11:20 |
| ramishra | IMO, that would be misleading | 11:20 |
| therve | Why? | 11:21 |
| ramishra | The way it's used elsewhere is different(per file), What's your concern about adding a swift download specific config? | 11:22 |
| therve | That people will just duplicate the value in the other config | 11:23 |
| therve | It serves the same purpose | 11:23 |
| ramishra | therve: I think it would be confusing as it means size of one template aggregate of all needed for a stack. But if there is a strong opinion about not adding another config..then | 11:25 |
| ramishra | *on template and not aggregate | 11:25 |
| therve | ramishra: We should make it not confusing and the same thing then | 11:28 |
| therve | Why wouldn't it be the aggregate as well? | 11:28 |
| ramishra | btw, default for max_template_szie is just 0.5 MB and we've another one https://github.com/openstack/heat/blob/master/heat/common/wsgi.py#L212 | 11:30 |
| ramishra | which is 1 MB | 11:30 |
| therve | Ah yes that's the one I was thinking about | 11:31 |
| therve | Use this one? | 11:31 |
| ramishra | therve: probably we can, if people think it's not going to be confusing;) | 11:33 |
| therve | I don't know it makes some sense to me | 11:34 |
| therve | ramishra: At least, it should default to it | 11:34 |
| therve | But I'm generally not a big fan of adding options | 11:35 |
| ramishra | therve: by coincidence, I've used the same default:) | 11:35 |
| therve | Right :) | 11:35 |
| therve | ramishra: It's easy to add an option later | 11:36 |
| therve | It's hard to remove it though | 11:36 |
| therve | bbiab | 11:36 |
| strigazi | folks I have one more question, different subject still for heat :) | 11:39 |
| strigazi | userA in projectA creates stackA which includes serverA with keypairA | 11:39 |
| strigazi | How can userB in projectA do a stack update on stackA with existing parameters? (by default userB can not see the keys of userA, this what I try to workaround) | 11:41 |
| strigazi | thoughts? ricolin ramishra therve ^^ | 11:41 |
| *** neatherweb has joined #heat | 11:43 | |
| *** edmondsw_ has quit IRC | 11:53 | |
| *** jerrygb has quit IRC | 12:02 | |
| ricolin | strigazi, interesting you mention, I'm facing on same issue just now (for different reason of course) | 12:02 |
| ricolin | for now, to create same key for different user will help I think | 12:03 |
| ricolin | Thinking about ways to fix this, maybe best way is to ask keypair service to allow keypair with project scope | 12:05 |
| *** shardy has joined #heat | 12:12 | |
| *** r-mibu has quit IRC | 12:19 | |
| strigazi | this might work: | 12:20 |
| strigazi | http://paste.openstack.org/show/724219/ | 12:21 |
| strigazi | ricolin: ^^ | 12:22 |
| strigazi | in our env it works | 12:22 |
| strigazi | not sure why :_ | 12:22 |
| strigazi | not sure why :) | 12:22 |
| *** weshay_ is now known as weshay|ruck | 12:22 | |
| strigazi | I mean, not sure why it doesn trigger a rebuild | 12:23 |
| *** edmondsw has joined #heat | 12:30 | |
| ricolin | strigazi, are you changing your stack name? | 12:40 |
| *** yangyapeng has quit IRC | 12:41 | |
| strigazi | no? the stack name is there for the example | 12:46 |
| *** jerrygb has joined #heat | 12:47 | |
| *** ramishra has quit IRC | 13:04 | |
| *** ramishra has joined #heat | 13:06 | |
| *** Guest71763 is now known as honza | 13:07 | |
| openstackgerrit | Thomas Herve proposed openstack/heat master: Handle invalid depends field https://review.openstack.org/577813 | 13:09 |
| *** yangyapeng has joined #heat | 13:21 | |
| *** skramaja has quit IRC | 13:32 | |
| *** neatherweb has quit IRC | 13:48 | |
| *** jcoufal has joined #heat | 14:00 | |
| *** felipemonteiro__ has joined #heat | 14:07 | |
| *** hjensas has quit IRC | 14:08 | |
| *** felipemonteiro_ has joined #heat | 14:10 | |
| *** felipemonteiro__ has quit IRC | 14:14 | |
| *** jerrygb has quit IRC | 14:19 | |
| *** hongbin has joined #heat | 14:30 | |
| *** yangyapeng has quit IRC | 14:34 | |
| *** felipemonteiro_ has quit IRC | 14:39 | |
| *** felipemonteiro__ has joined #heat | 14:39 | |
| *** hjensas has joined #heat | 14:42 | |
| *** felipemonteiro__ has quit IRC | 14:59 | |
| *** ShilpaSD has joined #heat | 15:32 | |
| *** pbourke has quit IRC | 15:45 | |
| *** ramishra has quit IRC | 15:47 | |
| *** pbourke has joined #heat | 15:47 | |
| *** hjensas is now known as hjensas|afk | 16:00 | |
| *** gfidente has quit IRC | 16:08 | |
| *** derekh has quit IRC | 16:24 | |
| *** shardy has quit IRC | 16:25 | |
| *** AlexeyAbashkin has joined #heat | 16:28 | |
| *** ysandeep has quit IRC | 16:50 | |
| *** jmlowe has quit IRC | 16:59 | |
| *** Alexey_Abashkin has joined #heat | 17:04 | |
| *** AlexeyAbashkin has quit IRC | 17:07 | |
| *** AlexeyAbashkin has joined #heat | 17:07 | |
| *** Alexey_Abashkin has quit IRC | 17:09 | |
| *** AlexeyAbashkin has quit IRC | 17:49 | |
| *** ndahiwade has joined #heat | 17:53 | |
| *** felipemonteiro has joined #heat | 17:54 | |
| *** harlowja has joined #heat | 17:55 | |
| *** felipemonteiro_ has joined #heat | 17:56 | |
| *** felipemonteiro has quit IRC | 18:00 | |
| *** jmlowe has joined #heat | 18:01 | |
| *** felipemonteiro_ has quit IRC | 18:06 | |
| *** felipemonteiro_ has joined #heat | 18:06 | |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Ignore errors in purging events https://review.openstack.org/577308 | 18:07 |
| openstackgerrit | Zane Bitter proposed openstack/heat master: DNM: Test purging events https://review.openstack.org/577432 | 18:07 |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Purge events that cross-reference with backup stack https://review.openstack.org/577891 | 18:07 |
| *** ricolin has quit IRC | 18:10 | |
| *** hjensas|afk is now known as hjensas | 18:19 | |
| *** jmlowe has quit IRC | 18:36 | |
| *** sshnaidm is now known as sshnaidm|afk | 18:41 | |
| *** ndahiwade has quit IRC | 18:41 | |
| *** jcoufal has quit IRC | 18:43 | |
| *** felipemonteiro_ has quit IRC | 18:52 | |
| *** felipemonteiro_ has joined #heat | 18:53 | |
| *** jcoufal has joined #heat | 18:53 | |
| *** jcoufal has quit IRC | 18:54 | |
| *** Guest86210 has quit IRC | 19:06 | |
| *** felipemonteiro_ has quit IRC | 20:15 | |
| *** felipemonteiro_ has joined #heat | 20:15 | |
| *** serlex has left #heat | 20:19 | |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Purge events that cross-reference with backup stack https://review.openstack.org/577891 | 20:36 |
| openstackgerrit | Zane Bitter proposed openstack/heat master: DNM: Test purging events https://review.openstack.org/577432 | 20:36 |
| *** felipemonteiro__ has joined #heat | 20:45 | |
| *** felipemonteiro_ has quit IRC | 20:48 | |
| *** EmilienM has joined #heat | 21:02 | |
| *** felipemonteiro__ has quit IRC | 21:03 | |
| *** felipemonteiro__ has joined #heat | 21:03 | |
| *** EmilienM_PTO has quit IRC | 21:04 | |
| *** harlowja has quit IRC | 21:04 | |
| *** EmilienM has quit IRC | 21:04 | |
| *** EmilienM has joined #heat | 21:04 | |
| *** felipemonteiro_ has joined #heat | 21:05 | |
| *** flwang has quit IRC | 21:06 | |
| *** flwang has joined #heat | 21:08 | |
| *** felipemonteiro__ has quit IRC | 21:08 | |
| *** nicolasbock has quit IRC | 21:37 | |
| *** felipemonteiro_ has quit IRC | 21:42 | |
| *** jmlowe has joined #heat | 21:44 | |
| *** rcernin has joined #heat | 21:45 | |
| *** jmlowe has quit IRC | 21:56 | |
| *** edmondsw has quit IRC | 21:58 | |
| *** Guest86210 has joined #heat | 22:06 | |
| *** harlowja has joined #heat | 22:14 | |
| *** jmlowe has joined #heat | 22:15 | |
| *** edmondsw has joined #heat | 22:57 | |
| *** Guest86210 has quit IRC | 23:14 | |
| *** yassine has joined #heat | 23:15 | |
| *** yassine is now known as Guest92236 | 23:15 | |
| *** hongbin has quit IRC | 23:17 | |
| *** jmlowe has quit IRC | 23:19 | |
| *** jmlowe has joined #heat | 23:41 | |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Purge events that cross-reference with backup stack https://review.openstack.org/577891 | 23:51 |
| openstackgerrit | Zane Bitter proposed openstack/heat master: DNM: Test purging events https://review.openstack.org/577432 | 23:51 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!