| *** spotz is now known as spotz_zzz | 00:28 | |
| *** tkarke has quit IRC | 00:30 | |
| ramishra | stevebaker: hi, around? | 00:42 |
|---|---|---|
| *** sdake has quit IRC | 00:43 | |
| *** sdake has joined #heat | 00:45 | |
| ramishra | stevebaker: If you can help review https://review.openstack.org/#/c/400946/, We created a mess with 1.6.0(backward incompatible change) and it was blacklisted. | 00:47 |
| *** ctayal has joined #heat | 00:54 | |
| stevebaker | ramishra: ok | 01:04 |
| *** sdake has quit IRC | 01:05 | |
| *** sdake has joined #heat | 01:06 | |
| ramishra | stevebaker: thanks! | 01:10 |
| ramishra | stevebaker: btw, I'm making good progess in adding more gabbi tests on top of your patch. I'll have them up for review in a day or two. | 01:14 |
| stevebaker | ramishra: cool, how are you finding the gabbi experience? | 01:14 |
| ramishra | stevebaker: it's good, but we would be the first project to add these I think? | 01:15 |
| stevebaker | ramishra: in tempest we will. ceilometer were the first to have them in tree | 01:15 |
| ramishra | stevebaker: oh, I did not know that ceilometer have them. | 01:16 |
| stevebaker | gabbi started when cdent worked on ceilometer | 01:16 |
| ramishra | stevebaker: yeah, is it actively maintained by him? To start with I'm looking at the tempest tests and converting them. | 01:17 |
| stevebaker | ramishra: possibly he is doing other things now | 01:18 |
| stevebaker | ramishra: git log says its fairly active, and he is still doing a fair amount | 01:18 |
| *** ctayal has quit IRC | 01:19 | |
| *** ctayal has joined #heat | 01:19 | |
| ramishra | stevebaker: yeah, I've not faced any road block yet. learning as I'm doing things:) | 01:19 |
| *** zhurong has joined #heat | 01:19 | |
| *** ccard has joined #heat | 01:20 | |
| *** ccard__ has quit IRC | 01:20 | |
| stevebaker | some kind of variable stored state would be handy for more complex tests | 01:21 |
| *** wangbo has quit IRC | 01:23 | |
| ramishra | I found the substitutions very useful http://gabbi.readthedocs.io/en/latest/format.html#substitution | 01:23 |
| *** wangbo has joined #heat | 01:26 | |
| *** sdake has quit IRC | 01:27 | |
| *** ctayal has quit IRC | 01:28 | |
| *** yanyanhu has joined #heat | 01:29 | |
| *** sdake has joined #heat | 01:29 | |
| *** steveg_afk has joined #heat | 01:40 | |
| *** sdake has quit IRC | 01:50 | |
| *** sdake has joined #heat | 01:53 | |
| *** elynn has joined #heat | 01:55 | |
| *** ctayal has joined #heat | 01:59 | |
| *** elynn_ has joined #heat | 02:02 | |
| *** cwolferh has quit IRC | 02:04 | |
| *** elynn has quit IRC | 02:05 | |
| *** EricGonczer_ has quit IRC | 02:09 | |
| *** wangbo has quit IRC | 02:10 | |
| *** EricGonczer_ has joined #heat | 02:11 | |
| openstackgerrit | Merged openstack/heat: Use both endpoint and endpoint_override https://review.openstack.org/400946 | 02:11 |
| *** sdake has quit IRC | 02:14 | |
| *** steveg_afk has quit IRC | 02:14 | |
| *** sdake has joined #heat | 02:15 | |
| *** gongysh has joined #heat | 02:24 | |
| *** ctayal has quit IRC | 02:24 | |
| *** bana_k has quit IRC | 02:25 | |
| *** elynn__ has joined #heat | 02:25 | |
| *** elynn_ has quit IRC | 02:26 | |
| *** wangbo has joined #heat | 02:28 | |
| *** jamielennox is now known as jamielennox|away | 02:30 | |
| *** sdake has quit IRC | 02:36 | |
| *** sdake has joined #heat | 02:37 | |
| *** cwolferh has joined #heat | 02:42 | |
| *** fragatina has quit IRC | 02:53 | |
| *** sdake has quit IRC | 02:57 | |
| *** jamielennox|away is now known as jamielennox | 02:59 | |
| *** sdake has joined #heat | 02:59 | |
| *** zhurong has quit IRC | 03:04 | |
| *** zhurong has joined #heat | 03:07 | |
| *** sdake has quit IRC | 03:19 | |
| *** sdake has joined #heat | 03:21 | |
| *** catintheroof has joined #heat | 03:27 | |
| *** cwolferh has quit IRC | 03:29 | |
| *** Drago_ has joined #heat | 03:33 | |
| *** liusheng has quit IRC | 03:33 | |
| *** zhurong has quit IRC | 03:35 | |
| *** zhurong has joined #heat | 03:36 | |
| *** catintheroof has quit IRC | 03:40 | |
| *** sdake has quit IRC | 03:41 | |
| *** links has joined #heat | 03:42 | |
| *** sdake has joined #heat | 03:43 | |
| *** wangbo has quit IRC | 03:45 | |
| *** EricGonczer_ has quit IRC | 03:45 | |
| *** fragatina has joined #heat | 03:51 | |
| *** fragatina has quit IRC | 03:52 | |
| *** fragatina has joined #heat | 03:52 | |
| *** Drago_ has quit IRC | 03:54 | |
| *** wangbo has joined #heat | 03:57 | |
| *** ricolin has joined #heat | 04:00 | |
| *** sdake has quit IRC | 04:03 | |
| *** sdake has joined #heat | 04:05 | |
| *** flwang1 has quit IRC | 04:08 | |
| *** misty-cloud has joined #heat | 04:16 | |
| *** sdake has quit IRC | 04:25 | |
| *** sdake has joined #heat | 04:28 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/heat: Updated from global requirements https://review.openstack.org/400899 | 04:30 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-heatclient: Updated from global requirements https://review.openstack.org/398617 | 04:38 |
| *** elynn__ has quit IRC | 04:41 | |
| *** sdake has quit IRC | 04:49 | |
| *** EricGonczer_ has joined #heat | 04:51 | |
| *** sdake has joined #heat | 04:52 | |
| *** EricGonczer_ has quit IRC | 04:58 | |
| *** sdake_ has joined #heat | 05:31 | |
| *** steveg_afk has joined #heat | 05:32 | |
| *** sdake has quit IRC | 05:35 | |
| *** steveg_afk has quit IRC | 05:38 | |
| *** gongysh has quit IRC | 05:49 | |
| *** elynn__ has joined #heat | 05:53 | |
| *** elynn__ has quit IRC | 05:57 | |
| *** elynn__ has joined #heat | 05:58 | |
| *** yprokule has joined #heat | 06:01 | |
| *** ctayal has joined #heat | 06:03 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat: DNM Test Zaqar devstack plugin fix https://review.openstack.org/401071 | 06:06 |
| *** liusheng has joined #heat | 06:07 | |
| *** ricolin has quit IRC | 06:17 | |
| *** bana_k has joined #heat | 06:20 | |
| *** bana_k has quit IRC | 06:27 | |
| *** bana_k has joined #heat | 06:33 | |
| *** ricolin has joined #heat | 06:34 | |
| *** ricolin has quit IRC | 06:46 | |
| *** bana_k has quit IRC | 06:49 | |
| *** willise has joined #heat | 06:58 | |
| *** willise has left #heat | 06:59 | |
| *** hjensas has quit IRC | 07:11 | |
| *** ccamacho has joined #heat | 07:17 | |
| *** jprovazn has joined #heat | 07:23 | |
| *** ctayal_ has joined #heat | 07:40 | |
| *** ctayal has quit IRC | 07:44 | |
| *** ctayal_ has quit IRC | 07:53 | |
| ramishra | skraynev: stable/newton gate is broken, we need https://review.openstack.org/#/c/400149/(neutron-lbaas change) and https://review.openstack.org/#/c/400689/ in that order to land | 07:56 |
| ramishra | master is broken too:( I think https://review.openstack.org/#/c/401069/ would fix it. | 07:59 |
| *** hjensas has joined #heat | 08:00 | |
| *** dimtruck is now known as zz_dimtruck | 08:11 | |
| *** Drago_ has joined #heat | 08:12 | |
| *** Drago_ has quit IRC | 08:12 | |
| *** yprokule_ has joined #heat | 08:20 | |
| *** derekjhyang has quit IRC | 08:21 | |
| *** yprokule has quit IRC | 08:22 | |
| *** yprokule_ is now known as yprokule | 08:22 | |
| *** jaosorior has joined #heat | 08:29 | |
| *** shardy has joined #heat | 08:30 | |
| *** jpich has joined #heat | 08:30 | |
| *** amoralej|off is now known as amoralej | 08:43 | |
| openstackgerrit | huangtianhua proposed openstack/heat-templates: Add example to configure ca certificates with userdata https://review.openstack.org/401128 | 08:49 |
| *** yprokule_ has joined #heat | 08:52 | |
| *** yprokule has quit IRC | 08:53 | |
| *** yprokule_ is now known as yprokule | 08:53 | |
| openstackgerrit | huangtianhua proposed openstack/heat-templates: Add example to configure ca certificates with userdata https://review.openstack.org/401128 | 08:56 |
| *** zzzeek has quit IRC | 09:00 | |
| *** gongysh has joined #heat | 09:01 | |
| *** zzzeek has joined #heat | 09:02 | |
| *** willise has joined #heat | 09:02 | |
| *** willise has left #heat | 09:02 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/heat: Updated from global requirements https://review.openstack.org/400899 | 09:15 |
| *** wangbo has quit IRC | 09:17 | |
| *** fragatina has quit IRC | 09:20 | |
| *** fragatina has joined #heat | 09:20 | |
| *** sshnaidm|away is now known as sshnaidm | 09:26 | |
| *** wangbo has joined #heat | 09:29 | |
| *** shinobu_ has joined #heat | 09:37 | |
| *** strigazi_AFK is now known as strigazi | 09:39 | |
| *** derekh has joined #heat | 09:41 | |
| *** dbecker has joined #heat | 09:42 | |
| *** yassine has joined #heat | 09:46 | |
| *** yassine is now known as Guest58074 | 09:46 | |
| *** ccamacho is now known as ccamacho|afk | 09:47 | |
| *** zhurong has quit IRC | 10:01 | |
| openstackgerrit | Thomas Herve proposed openstack/heat: Sleep between check_resource calls https://review.openstack.org/400155 | 10:07 |
| *** yanyanhu has quit IRC | 10:11 | |
| *** gongysh has quit IRC | 10:15 | |
| *** elynn__ has quit IRC | 10:18 | |
| *** jamielennox is now known as jamielennox|away | 10:21 | |
| *** tkarke has joined #heat | 10:21 | |
| *** tkarke has quit IRC | 10:22 | |
| *** tkarke has joined #heat | 10:37 | |
| *** tkarke has quit IRC | 10:38 | |
| *** ccamacho|afk is now known as ccamacho | 10:50 | |
| *** misty-cloud has quit IRC | 10:59 | |
| *** prazumovsky has joined #heat | 11:00 | |
| shardy | ramishra: Hi! | 11:23 |
| shardy | ramishra: If you're around I have a question re parameter merge strategies | 11:23 |
| shardy | ramishra: was there some validation changes to enable catching the case where multiple environments are specified, but not all specify a strategy, or the strategies are conflicting? | 11:24 |
| shardy | IIRC it was discussed | 11:24 |
| shardy | I have a problem because if not all of the env files contain parameter_merge_strategies all of those that do are silently ignored | 11:24 |
| *** ccamacho is now known as ccamacho|lunch | 11:34 | |
| ramishra | shardy: hi | 11:36 |
| shardy | ramishra: Hi! | 11:37 |
| ramishra | shardy: no we've not added the validation for conflicting strategies yet. | 11:37 |
| shardy | ramishra: Ok, np - I'm currently integrating merge_strategies with tripleo and I think that would be very useful | 11:38 |
| ramishra | yeah, but if you specificy the parameters in any environment file without a strategy then it would overwrite. | 11:39 |
| shardy | another option would be to only require it to be specified in *any* environment file I guess | 11:39 |
| ramishra | but not otherwise | 11:39 |
| shardy | ramishra: yeah, that's what I'm seeing | 11:39 |
| ramishra | if that makes sense. | 11:39 |
| shardy | it's kind of annoying as you don't discover it until you've waitied for the deployment to finish ;) | 11:39 |
| ramishra | yeah, we agreed that we'll add the valiudation | 11:40 |
| shardy | cool, thanks for the update | 11:40 |
| *** Guest58074 has quit IRC | 11:41 | |
| ramishra | shardy: thanks! | 11:42 |
| openstackgerrit | Merged openstack/heat: Move call trove cluster delete to check complete https://review.openstack.org/390828 | 12:02 |
| *** openstackgerrit has quit IRC | 12:03 | |
| *** openstackgerrit has joined #heat | 12:04 | |
| *** yassine has joined #heat | 12:08 | |
| *** catintheroof has joined #heat | 12:09 | |
| *** yassine is now known as Guest22022 | 12:09 | |
| *** jcoufal has joined #heat | 12:09 | |
| *** catintheroof has quit IRC | 12:15 | |
| *** gongysh has joined #heat | 12:15 | |
| *** gongysh has quit IRC | 12:17 | |
| *** catintheroof has joined #heat | 12:20 | |
| therve | ramishra, Urm, grenade failure? | 12:28 |
| therve | Can we catch a break already | 12:28 |
| ramishra | which one? | 12:29 |
| therve | ramishra, http://logs.openstack.org/55/400155/2/check/gate-grenade-dsvm-heat-ubuntu-xenial/205d74d/ | 12:29 |
| ramishra | therve: There is a patch that merged after that. no? | 12:30 |
| ramishra | This one https://review.openstack.org/390828 | 12:30 |
| therve | ramishra, Oh I didn't say it was there all the time | 12:30 |
| therve | I'd like the gate to be green 50%, that'd be nice | 12:31 |
| ramishra | therve: yeah atleast 50%:) | 12:31 |
| ramishra | http://logs.openstack.org/55/400155/2/check/gate-grenade-dsvm-heat-ubuntu-xenial/205d74d/logs/grenade.sh.txt.gz#_2016-11-23_10_43_30_237 | 12:32 |
| ramishra | therve: ^^ | 12:32 |
| *** hjensas has quit IRC | 12:32 | |
| therve | ramishra, Yeah it looks like some kind of race condition | 12:32 |
| *** shardy is now known as shardy_lunch | 12:39 | |
| *** ccard__ has joined #heat | 12:41 | |
| *** ccard has quit IRC | 12:43 | |
| *** catintheroof has quit IRC | 12:44 | |
| ramishra | shardy_lunch: raised a bug to add the validation https://bugs.launchpad.net/heat/+bug/1644217. | 12:45 |
| openstack | Launchpad bug 1644217 in heat "Add validation for conflicting merge strategies" [Medium,New] | 12:45 |
| ramishra | shardy_lunch: though it would not be easy;) | 12:45 |
| *** hjensas has joined #heat | 12:46 | |
| *** steveg_afk has joined #heat | 12:53 | |
| *** links has quit IRC | 12:54 | |
| *** dprince has joined #heat | 12:56 | |
| ramishra | therve: It seems there are no 'Too many connections' failures on stable/newton without changing 'executor_thread_pool_size'. | 12:59 |
| ramishra | newton jobs also use xenial nodes. | 13:00 |
| *** ccamacho|lunch is now known as ccamacho | 13:08 | |
| therve | ramishra, Interesting | 13:09 |
| *** ccard has joined #heat | 13:12 | |
| *** ccard__ has quit IRC | 13:14 | |
| therve | ramishra, It's possible the underlying issue has been fixed? | 13:20 |
| therve | Maybe we can try in master again | 13:20 |
| *** amoralej is now known as amoralej|lunch | 13:20 | |
| ramishra | I tried a test patch but it failed yesterday with one error. | 13:20 |
| therve | The fix I have is still relevant for another problem | 13:20 |
| ramishra | http://logs.openstack.org/07/400507/1/check/gate-heat-dsvm-functional-convg-mysql-lbaasv2-ubuntu-xenial/9218961/console.html#_2016-11-22_04_57_04_245363 | 13:22 |
| ramishra | https://review.openstack.org/#/c/400507/ | 13:22 |
| ramishra | let me try it one more time. | 13:24 |
| *** shardy_lunch is now known as shardy | 13:28 | |
| shardy | ramishra: ack, thanks - I'll see if I can take a look unless anyone gets to it first | 13:30 |
| ramishra | shardy: may be I'll get to it before you:) | 13:31 |
| shardy | ramishra: sure, even better! :) | 13:33 |
| shardy | if you have time it would be a great help | 13:33 |
| shardy | if not I'll try to take a look next week | 13:33 |
| ramishra | shardy: sure | 13:33 |
| *** lamt has joined #heat | 13:39 | |
| *** derekh is now known as derekh_afk | 13:51 | |
| *** ramishra has quit IRC | 13:59 | |
| *** ramishra has joined #heat | 14:01 | |
| *** prazumovsky has quit IRC | 14:02 | |
| *** ccard__ has joined #heat | 14:13 | |
| *** ccard has quit IRC | 14:16 | |
| *** annegentle has joined #heat | 14:19 | |
| *** code-R has joined #heat | 14:20 | |
| *** amoralej|lunch is now known as amoralej | 14:22 | |
| *** EricGonczer_ has joined #heat | 14:25 | |
| *** yprokule_ has joined #heat | 14:29 | |
| *** randallburt has joined #heat | 14:29 | |
| *** yprokule has quit IRC | 14:30 | |
| *** yprokule_ is now known as yprokule | 14:30 | |
| *** randallburt has quit IRC | 14:34 | |
| *** jprovazn has quit IRC | 14:34 | |
| *** rajpatel has joined #heat | 14:40 | |
| *** randallburt has joined #heat | 14:42 | |
| *** derekh_afk is now known as derekh | 14:47 | |
| *** rajpatel1 has joined #heat | 14:48 | |
| *** rajpatel has quit IRC | 14:49 | |
| *** ricolin has joined #heat | 14:53 | |
| *** ccard_ has joined #heat | 14:53 | |
| *** jaosorior has quit IRC | 14:53 | |
| *** jaosorior has joined #heat | 14:54 | |
| ramishra | hi all, irc meeting in 6 mins, please feel free to add to the agenda https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda_.282016-11-23_1500_UTC.29 | 14:54 |
| ramishra | zaneb, therve, ricolin, Qiming, skraynev, shardy meeting? | 15:01 |
| *** cwolferh has joined #heat | 15:02 | |
| *** jaosorior has quit IRC | 15:09 | |
| *** jaosorior has joined #heat | 15:10 | |
| *** chris_hultin|AWA is now known as chris_hultin | 15:15 | |
| *** spotz_zzz is now known as spotz | 15:19 | |
| *** zz_dimtruck is now known as dimtruck | 15:19 | |
| *** chlong has joined #heat | 15:25 | |
| *** EricGonc_ has joined #heat | 15:34 | |
| *** chlong has quit IRC | 15:34 | |
| *** yprokule_ has joined #heat | 15:35 | |
| *** EricGonczer_ has quit IRC | 15:36 | |
| *** yprokule has quit IRC | 15:36 | |
| *** yprokule_ is now known as yprokule | 15:36 | |
| *** jprovazn has joined #heat | 15:39 | |
| openstackgerrit | Merged openstack/heat: Avoid redundant validations https://review.openstack.org/400961 | 15:43 |
| *** chlong has joined #heat | 15:50 | |
| *** EricGonc_ has quit IRC | 15:52 | |
| *** EricGonczer_ has joined #heat | 15:53 | |
| *** tkarke has joined #heat | 15:56 | |
| *** tkarke has quit IRC | 15:57 | |
| ramishra | zaneb: http://docs.openstack.org/developer/oslo.db/api/options.html | 16:02 |
| zaneb | ramishra: so max_pool_size connections are always kept around + open | 16:02 |
| ramishra | yep | 16:02 |
| zaneb | and we can have up to max_pool_size + max_overflow connections in total | 16:02 |
| ramishra | yeah | 16:02 |
| ramishra | thats per pool | 16:04 |
| *** strigazi is now known as strigazi_AFK | 16:04 | |
| zaneb | and we were running out of connections on the heat side? or too many things were trying to connect to the DB? | 16:04 |
| zaneb | (from multiple engines) | 16:04 |
| ramishra | too many connections open | 16:05 |
| ramishra | from heat. I assume they were not finishing fast. | 16:05 |
| ramishra | and it used to exceed the max_connections i.e 1200 | 16:05 |
| ramishra | but it seems to work fine now on stable | 16:06 |
| ramishra | we don't have that change in stable/newton and it's using the xenial nodes. | 16:06 |
| ramishra | my guess is, the nodes were slow, hence the issue | 16:06 |
| ramishra | yeah, but we should throttle it somehow | 16:07 |
| *** EricGonc_ has joined #heat | 16:07 | |
| *** code-R has quit IRC | 16:07 | |
| ramishra | zaneb: does that make sense? | 16:07 |
| *** EricGonczer_ has quit IRC | 16:08 | |
| ramishra | therve may explain in better:) | 16:10 |
| *** bana_k has joined #heat | 16:10 | |
| *** crose has joined #heat | 16:16 | |
| *** bana_k has quit IRC | 16:25 | |
| *** dprince has quit IRC | 16:33 | |
| *** ramishra has quit IRC | 16:34 | |
| *** ccamacho has quit IRC | 16:41 | |
| *** annegentle has quit IRC | 16:41 | |
| *** ramishra has joined #heat | 16:43 | |
| *** yprokule has quit IRC | 16:48 | |
| *** ccard__ has quit IRC | 16:53 | |
| *** jdandrea has quit IRC | 16:57 | |
| *** dprince has joined #heat | 16:58 | |
| *** ctayal has joined #heat | 16:58 | |
| *** ccamacho has joined #heat | 16:59 | |
| *** beagles is now known as beagles-brb | 17:08 | |
| *** sdake_ is now known as sdake | 17:09 | |
| *** annegentle has joined #heat | 17:12 | |
| *** dhubbard_ has joined #heat | 17:13 | |
| *** dhubbard has quit IRC | 17:13 | |
| *** jaosorior has quit IRC | 17:15 | |
| *** ricolin has quit IRC | 17:15 | |
| *** shardy has quit IRC | 17:43 | |
| *** chlong has quit IRC | 17:50 | |
| *** derekh has quit IRC | 17:52 | |
| *** sdake has quit IRC | 17:54 | |
| *** crose has quit IRC | 17:55 | |
| *** beagles-brb is now known as beagles | 18:02 | |
| *** amoralej is now known as amoralej|off | 18:02 | |
| *** crose has joined #heat | 18:02 | |
| *** crose has quit IRC | 18:08 | |
| openstackgerrit | Crag Wolfe proposed openstack/heat: Do not count events for every event created https://review.openstack.org/400388 | 18:21 |
| *** rajpatel1 has quit IRC | 18:30 | |
| *** annegentle has quit IRC | 18:31 | |
| *** flwang1 has joined #heat | 18:41 | |
| *** ctayal has quit IRC | 18:56 | |
| *** rajpatel has joined #heat | 18:59 | |
| *** crose has joined #heat | 19:05 | |
| *** rajpatel has quit IRC | 19:10 | |
| *** bana_k has joined #heat | 19:10 | |
| *** jdandrea has joined #heat | 19:12 | |
| *** beagles is now known as beagles_brb | 19:21 | |
| *** fragatina has quit IRC | 19:22 | |
| *** catintheroof has joined #heat | 19:27 | |
| *** ctayal has joined #heat | 19:32 | |
| *** flwang1 has quit IRC | 19:34 | |
| *** catintheroof has quit IRC | 19:34 | |
| *** beagles_brb is now known as beagles | 19:55 | |
| *** fragatina has joined #heat | 19:56 | |
| *** jpich has quit IRC | 20:03 | |
| *** ctayal has quit IRC | 20:03 | |
| *** crose has quit IRC | 20:08 | |
| *** dbecker has quit IRC | 20:11 | |
| *** fragatina has quit IRC | 20:30 | |
| therve | zaneb, We were running out of connections on the mysql side | 20:37 |
| therve | so 1024 opened AFAIU | 20:38 |
| *** shinobu_ has quit IRC | 20:38 | |
| zaneb | therve: ok, thanks | 20:38 |
| therve | Considering we can have 512 worker threads just in heat, that can be it | 20:38 |
| zaneb | and we're competing with other services for those connections too, presumably | 20:38 |
| therve | Yes | 20:38 |
| zaneb | that's a hard problem to solve then :( | 20:39 |
| *** flwang1 has joined #heat | 20:39 | |
| therve | I mean, yes and no | 20:39 |
| zaneb | ideally we need to detect the connection failure, handle it gracefully and apply backpressure there | 20:40 |
| therve | Ah yeah, if you want to do that sure :) | 20:40 |
| zaneb | like, if we made sure we had an *open* connection available before we took a message out of the queue to process... that would do it | 20:40 |
| therve | zaneb, I'm tempted to revert https://review.openstack.org/#/c/262374/ though | 20:41 |
| zaneb | I bet oslo_messaging doesn't make that easy for us though :( | 20:41 |
| therve | I think setting max_overflow is a bad idea in general | 20:41 |
| therve | Using overflow is a bad idea, as we'll keeping opening and closing connections | 20:42 |
| therve | Instead of using the pool | 20:42 |
| zaneb | therve: I don't really agree. every one of those threads needs to be able to get a db connection. we shouldn't be taking messages out of the queue if we know we're going to fail to process them | 20:42 |
| zaneb | that's true, but it's a tradeoff v. how many we keep open all the time even when Heat is not busy | 20:43 |
| therve | zaneb, We need to have less threads then :) | 20:43 |
| therve | It's not like we benefit from having so much pseudo concurrency | 20:43 |
| zaneb | well, this is the thing I wanted to get at with the backpressure: sometimes we do and sometime we don't | 20:44 |
| therve | I don't think this 64 number is a reasonable default | 20:45 |
| zaneb | agree, that's too high | 20:45 |
| zaneb | it's really hard to base it on the right thing | 20:46 |
| *** dprince has quit IRC | 20:46 | |
| zaneb | even the CPU is typically shared with other services | 20:46 |
| *** rajpatel has joined #heat | 20:46 | |
| zaneb | so even if we are spending most of our time sleeping, that might be enabling the API queries we've sent out to actually be processed | 20:47 |
| zaneb | otoh, we could be just sitting idle when there's more messages there we could be processing | 20:47 |
| therve | Well from what I've seen with https://review.openstack.org/400155 , even internally that's enough | 20:48 |
| zaneb | maybe we should dynamically vary the thread pool size in inverse proportion to the CPU load average? | 20:50 |
| therve | Suuuurreee | 20:51 |
| *** ccard has joined #heat | 20:53 | |
| zaneb | lol | 20:53 |
| *** sdake has joined #heat | 20:54 | |
| *** fragatina has joined #heat | 21:01 | |
| *** jprovazn has quit IRC | 21:03 | |
| -openstackstatus- NOTICE: Due to a problem with the cinder volume backing the log server, jobs are failing with POST_FAILURE. Please avoid issuing 'recheck' commands until the issue is resolved. | 21:05 | |
| *** ChanServ changes topic to "Due to a problem with the cinder volume backing the log server, jobs are failing with POST_FAILURE. Please avoid issuing 'recheck' commands until the issue is resolved." | 21:05 | |
| *** catintheroof has joined #heat | 21:08 | |
| *** spotz is now known as spotz_zzz | 21:10 | |
| *** randallburt has quit IRC | 21:18 | |
| *** jamielennox|away is now known as jamielennox | 21:22 | |
| *** ccard__ has joined #heat | 21:31 | |
| *** ccard has quit IRC | 21:32 | |
| *** jdandrea has quit IRC | 21:36 | |
| *** catintheroof has quit IRC | 21:40 | |
| *** pradk has quit IRC | 21:43 | |
| *** rajpatel has quit IRC | 21:45 | |
| *** rajpatel has joined #heat | 21:47 | |
| *** ctayal has joined #heat | 21:50 | |
| *** jdandrea has joined #heat | 21:51 | |
| *** jcoufal has quit IRC | 21:55 | |
| *** ccamacho has quit IRC | 22:11 | |
| *** bana_k has quit IRC | 22:16 | |
| *** slagle has quit IRC | 22:18 | |
| *** rajpatel has quit IRC | 22:18 | |
| *** chlong has joined #heat | 22:20 | |
| *** chlong has quit IRC | 22:30 | |
| *** chris_hultin is now known as chris_hultin|AWA | 22:37 | |
| *** beagles is now known as beagles_brb | 22:43 | |
| *** catintheroof has joined #heat | 22:48 | |
| *** jdandrea has quit IRC | 22:49 | |
| *** ChanServ changes topic to "support @ https://ask.openstack.org | developer wiki @ https://wiki.openstack.org/wiki/Heat | development @ https://launchpad.net/heat | logged @ http://eavesdrop.openstack.org/irclogs/%23heat/" | 22:50 | |
| -openstackstatus- NOTICE: The affected filesystems on the log server are repaired. Please leave 'recheck' comments on any changes which failed with POST_FAILURE. | 22:50 | |
| *** catintheroof has quit IRC | 22:52 | |
| *** dimtruck is now known as zz_dimtruck | 22:59 | |
| *** catintheroof has joined #heat | 23:00 | |
| *** annegentle has joined #heat | 23:03 | |
| *** beagles_brb is now known as beagles | 23:14 | |
| *** catinthe_ has joined #heat | 23:23 | |
| *** catintheroof has quit IRC | 23:26 | |
| *** sdake has quit IRC | 23:37 | |
| *** sdake has joined #heat | 23:40 | |
| *** sdake has quit IRC | 23:40 | |
| *** sdake has joined #heat | 23:40 | |
| *** lamt has quit IRC | 23:43 | |
| *** catinthe_ has quit IRC | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!