| *** yangyapeng has quit IRC | 00:12 | |
| *** rwsu has joined #heat | 00:13 | |
| *** yangyapeng has joined #heat | 00:13 | |
| *** yangyapeng has quit IRC | 00:17 | |
| *** fragatina has quit IRC | 00:31 | |
| *** fragatina has joined #heat | 00:31 | |
| *** fragatina has quit IRC | 00:32 | |
| *** fragatina has joined #heat | 00:32 | |
| *** Shunli has joined #heat | 00:56 | |
| *** kiennt26 has joined #heat | 01:02 | |
| *** fragatina has quit IRC | 01:05 | |
| *** fragatina has joined #heat | 01:06 | |
| *** fragatina has quit IRC | 01:10 | |
| *** yangyapeng has joined #heat | 01:12 | |
| *** yangyapeng has quit IRC | 01:12 | |
| *** yangyapeng has joined #heat | 01:13 | |
| *** namnh has joined #heat | 01:31 | |
| *** Qiming has quit IRC | 01:41 | |
| *** wangbo has joined #heat | 01:44 | |
| *** wangbo has quit IRC | 01:45 | |
| *** wangbo has joined #heat | 01:46 | |
| *** hongbin has joined #heat | 01:56 | |
| *** Qiming has joined #heat | 02:01 | |
| *** yangyape_ has joined #heat | 02:07 | |
| *** yangyapeng has quit IRC | 02:10 | |
| *** Qiming has quit IRC | 02:19 | |
| *** gouthamr has quit IRC | 02:33 | |
| *** ricolin has joined #heat | 02:38 | |
| *** wangbo has quit IRC | 02:46 | |
| *** wangbo has joined #heat | 02:47 | |
| *** fragatina has joined #heat | 02:52 | |
| *** AlexeyAbashkin has joined #heat | 02:57 | |
| *** AlexeyAbashkin has quit IRC | 03:02 | |
| *** yangyape_ has quit IRC | 03:08 | |
| *** yangyapeng has joined #heat | 03:09 | |
| *** yangyape_ has joined #heat | 03:10 | |
| openstackgerrit | Rico Lin proposed openstack/heat master: Remove .testrepository/times.dbm in tox.ini https://review.openstack.org/513636 | 03:14 |
|---|---|---|
| *** fragatina has quit IRC | 03:14 | |
| *** fragatina has joined #heat | 03:15 | |
| *** ricolin has quit IRC | 03:15 | |
| *** ricolin has joined #heat | 03:16 | |
| *** yangyapeng has quit IRC | 03:16 | |
| *** fragatina has quit IRC | 03:19 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Don't create a replacement resource if the traversal has changed https://review.openstack.org/501682 | 03:27 |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Set stack.thread_group_mgr for cancel_update https://review.openstack.org/499088 | 03:27 |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Add REST api support for cancel without rollback https://review.openstack.org/491467 | 03:27 |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Add functional test for cancel without rollback https://review.openstack.org/499550 | 03:27 |
| *** mdnadeem has joined #heat | 03:27 | |
| *** Qiming has joined #heat | 03:29 | |
| ramishra | ricolin: Hi, Is the Queens-1 milestone tagging delayed? | 03:33 |
| *** fragatina has joined #heat | 03:33 | |
| *** armaan has quit IRC | 03:33 | |
| *** Qiming has quit IRC | 03:33 | |
| ramishra | I would have liked the cancel-update stuff ^^^ to go in before that. been in the review queue for more than 2 months | 03:35 |
| ricolin | ramishra, yes, I delay it for a week, to make sure patches can make in, there been a long block time, so I thinking we can hold till this Wed.(maybe not a good idea consider consist deliver) | 03:37 |
| ramishra | zaneb: Hey! Now that the your patches for convergence tests have gone in, would you able to review https://review.openstack.org/#/q/topic:bug/1709041 again? | 03:38 |
| ramishra | ricolin: OK, we can probably target to land some bp kind of patches before going for the tagging | 03:41 |
| ricolin | ramishra, yeah, that's what I'm hopping for:) | 03:41 |
| *** hongbin has quit IRC | 03:47 | |
| *** gouthamr has joined #heat | 03:49 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Check for existing interfaces before adding implicit one https://review.openstack.org/513611 | 03:51 |
| *** gkadam_ has joined #heat | 03:56 | |
| *** gouthamr has quit IRC | 04:04 | |
| *** armaan has joined #heat | 04:14 | |
| *** yangyape_ has quit IRC | 04:24 | |
| *** yangyapeng has joined #heat | 04:25 | |
| *** armaan has quit IRC | 04:29 | |
| *** wangbo has quit IRC | 04:30 | |
| *** Qiming has joined #heat | 04:33 | |
| ramishra | ricolin: around? | 04:33 |
| ricolin | ramishra, yes | 04:33 |
| ramishra | why are you still using a legacy job as parent here https://review.openstack.org/#/c/509098/12/.zuul.yaml ? | 04:34 |
| ramishra | Can we not have a base job in our repo whose parent is 'devstack'? | 04:34 |
| ramishra | like in https://review.openstack.org/#/c/500365/34/.zuul.yaml | 04:35 |
| ricolin | ramishra, actually I think we can. Let we change it now | 04:36 |
| ramishra | We probably have to get rid of the 'legacy' word in these jobs? ex. "Ensure legacy workspace directory" | 04:36 |
| ricolin | ramishra, yep, that's final goal to migration | 04:37 |
| openstackgerrit | Merged openstack/heat master: Do not validate property network of sahara cluster https://review.openstack.org/514207 | 04:39 |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate functional jobs to Zuul v3 https://review.openstack.org/509098 | 04:43 |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate grenade jobs to zuul v3 https://review.openstack.org/509140 | 04:43 |
| *** skramaja has joined #heat | 04:47 | |
| *** kiennt26 has quit IRC | 04:51 | |
| *** maestropandy has joined #heat | 05:01 | |
| *** Qiming has quit IRC | 05:02 | |
| *** maestropandy has quit IRC | 05:02 | |
| *** Qiming has joined #heat | 05:03 | |
| *** omie888777 has joined #heat | 05:06 | |
| *** Qiming has quit IRC | 05:07 | |
| *** Qiming has joined #heat | 05:12 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Remove cloudwatch service from devstack plugin https://review.openstack.org/514515 | 05:14 |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate grenade jobs to zuul v3 https://review.openstack.org/509140 | 05:15 |
| *** Qiming has quit IRC | 05:20 | |
| *** tbh has joined #heat | 05:21 | |
| *** Qiming has joined #heat | 05:21 | |
| tbh | hey, do we still support download the image and uploading to glance throught HOT? | 05:21 |
| *** zzzeek has quit IRC | 05:24 | |
| *** Qiming_ has joined #heat | 05:24 | |
| *** omie888777 has quit IRC | 05:24 | |
| *** zzzeek has joined #heat | 05:26 | |
| *** Qiming has quit IRC | 05:27 | |
| *** jtomasek has quit IRC | 05:27 | |
| *** wangbo has joined #heat | 05:31 | |
| ramishra | tbh: not with glance v2, https://github.com/openstack/heat/blob/master/heat/engine/resources/openstack/glance/image.py#L32-L33 | 05:37 |
| ramishra | if you're using glance v1, probably it still works | 05:38 |
| ramishra | grenade job has been made non-voting since migration to zuulv3:/ | 05:53 |
| *** hoonetorg has quit IRC | 06:02 | |
| *** wangbo has quit IRC | 06:03 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Remove cloudwatch service from devstack plugin https://review.openstack.org/514515 | 06:05 |
| *** wangbo has joined #heat | 06:05 | |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: initial-setup/devstack_requirements https://review.openstack.org/514521 | 06:09 |
| *** Qiming_ has quit IRC | 06:10 | |
| *** wangbo has quit IRC | 06:11 | |
| *** wangbo has joined #heat | 06:13 | |
| *** jprovazn has joined #heat | 06:14 | |
| *** hoonetorg has joined #heat | 06:16 | |
| *** wangbo has quit IRC | 06:16 | |
| *** wangbo has joined #heat | 06:18 | |
| *** wangbo has quit IRC | 06:21 | |
| *** cmyster_ has joined #heat | 06:21 | |
| *** wangbo has joined #heat | 06:23 | |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate grenade jobs to zuul v3 https://review.openstack.org/509140 | 06:23 |
| *** cmyster_ has quit IRC | 06:24 | |
| *** wangbo has quit IRC | 06:25 | |
| *** wangbo has joined #heat | 06:28 | |
| *** wangbo_ has joined #heat | 06:37 | |
| *** wangbo has quit IRC | 06:39 | |
| *** omie888777 has joined #heat | 06:39 | |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: add additional files. https://review.openstack.org/514530 | 06:44 |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add files for initialization of this repository. https://review.openstack.org/514532 | 06:50 |
| *** gfidente has joined #heat | 06:55 | |
| *** gfidente has joined #heat | 06:55 | |
| *** ioggstream has joined #heat | 06:56 | |
| ioggstream | hi@heat | 06:58 |
| *** maestropandy has joined #heat | 07:02 | |
| *** kazsh has left #heat | 07:09 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Remove cloudwatch service from devstack plugin https://review.openstack.org/514515 | 07:11 |
| *** kiennt26 has joined #heat | 07:12 | |
| *** Guest89283 has quit IRC | 07:13 | |
| *** yassine has joined #heat | 07:13 | |
| *** yassine is now known as Guest62896 | 07:14 | |
| *** yangyape_ has joined #heat | 07:14 | |
| *** yangyapeng has quit IRC | 07:16 | |
| openstackgerrit | Xinni Ge proposed openstack/heat-dashboard master: Add unittests for modules, services and several controllers. https://review.openstack.org/514546 | 07:19 |
| *** Qiming has joined #heat | 07:20 | |
| *** wangbo__ has joined #heat | 07:21 | |
| *** wangbo_ has quit IRC | 07:22 | |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add files for initialization of this repository. https://review.openstack.org/514532 | 07:23 |
| *** Qiming has quit IRC | 07:25 | |
| *** ccamacho has joined #heat | 07:25 | |
| *** yangyape_ has quit IRC | 07:27 | |
| *** yangyapeng has joined #heat | 07:27 | |
| *** yangyape_ has joined #heat | 07:30 | |
| *** AlexeyAbashkin has joined #heat | 07:32 | |
| *** yangyapeng has quit IRC | 07:32 | |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add files for initialization of this repository. https://review.openstack.org/514550 | 07:35 |
| *** maestropandy has quit IRC | 07:42 | |
| *** amoralej|off is now known as amoralej | 07:42 | |
| *** SmearedBeard has joined #heat | 07:50 | |
| *** xinni9e has joined #heat | 07:52 | |
| *** xinni9e has quit IRC | 07:52 | |
| *** maestropandy has joined #heat | 07:53 | |
| *** xinni9e has joined #heat | 07:53 | |
| *** xinni9e has quit IRC | 07:55 | |
| tbh | ramishra, will it work in v1 too? | 07:59 |
| *** dbecker has joined #heat | 07:59 | |
| *** xinni9e has joined #heat | 07:59 | |
| ramishra | tbh: AFAICT, image resource would 'only' work with glance v1 and not v2, that's why it's deprecated. | 08:00 |
| tbh | ramishra, can we specifically use v1 and not v2? | 08:02 |
| tbh | I mean how to change v2 to v1? | 08:02 |
| *** ccard has quit IRC | 08:03 | |
| *** tbh has quit IRC | 08:06 | |
| ramishra | tbh: I don't understand what you mean? does it not work when 'enable_v1_api¶=True' in glance.api? | 08:06 |
| *** SmearedBeard is now known as SmearedBeard-AFK | 08:08 | |
| *** SmearedBeard-AFK is now known as SmearedBeard | 08:08 | |
| *** maestropandy has quit IRC | 08:10 | |
| *** SmearedBeard is now known as SmearedBeard-AFK | 08:10 | |
| *** SmearedBeard-AFK is now known as SmearedBeard | 08:13 | |
| *** wangbo_ has joined #heat | 08:20 | |
| *** namnh has quit IRC | 08:21 | |
| *** wangbo__ has quit IRC | 08:21 | |
| ricolin | ramishra, yep, I will change it back, seems that part still not resolved, I will ask monty's suggestion about it | 08:23 |
| *** zsli_ has joined #heat | 08:23 | |
| ramishra | ricolin: I probably know what's to be changed, may be I'll change it once your patches land | 08:24 |
| ricolin | ramishra, sure, or you can update my patches if needs | 08:25 |
| *** Shunli has quit IRC | 08:26 | |
| ramishra | seems infra team is not accepting any job patches ex. https://review.openstack.org/#/c/514519/, so we should get the job definitions in-tree quickly | 08:27 |
| *** maestropandy has joined #heat | 08:27 | |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate functional jobs to Zuul v3 https://review.openstack.org/509098 | 08:28 |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate grenade jobs to zuul v3 https://review.openstack.org/509140 | 08:28 |
| ricolin | ramishra, we also got two tripleO jobs should goes to tripleO`s repo(tripleO-ci maybe) and rally job to rally repo depends on when those two project decide to start migrate to zuul v3 | 08:30 |
| ramishra | yeah those are experimental jobs, probably can worry abt it later;) | 08:31 |
| ricolin | also a legacy-check-osc-plugins under python heatclient repo | 08:32 |
| ricolin | but, yeah, not quick worry about it. Just wait for those project to migrate by them self | 08:34 |
| *** derekh has joined #heat | 08:37 | |
| ramishra | ricolin: legagcy job defns are not going away unless someone removes them (of course after migrating them in specific project tree). Though it seems we can't make any changes to the legagcy defns inspite of some them being incorrect i.e our grenade job is now non-voting:/ | 08:38 |
| ramishra | Doh! I don't know how to spell 'legacy' | 08:40 |
| ricolin | ramishra, yeah, right now I change the grenade zuul to voting in our .zuul.yaml, so merge that patch should be able to fix that. | 08:40 |
| *** jtomasek has joined #heat | 08:44 | |
| *** gmoro has quit IRC | 08:46 | |
| *** gmoro has joined #heat | 08:48 | |
| *** jtomasek has quit IRC | 08:50 | |
| *** jtomasek has joined #heat | 08:51 | |
| *** armaan has joined #heat | 08:52 | |
| *** maestropandy has quit IRC | 08:54 | |
| *** yangyapeng has joined #heat | 08:57 | |
| *** yangyape_ has quit IRC | 09:01 | |
| openstackgerrit | Xinni Ge proposed openstack/heat-dashboard master: Add unittests for modules, services and several controllers. https://review.openstack.org/514572 | 09:03 |
| *** gkadam_ has quit IRC | 09:11 | |
| *** wangbo_ has quit IRC | 09:13 | |
| *** gkadam has joined #heat | 09:13 | |
| *** maestropandy has joined #heat | 09:13 | |
| *** wangbo_ has joined #heat | 09:16 | |
| *** Qiming has joined #heat | 09:17 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Remove cloudwatch service from devstack plugin https://review.openstack.org/514515 | 09:27 |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: Remove cloudwatch service from devstack plugin https://review.openstack.org/514515 | 09:30 |
| *** zzzeek has quit IRC | 09:30 | |
| *** tbh has joined #heat | 09:31 | |
| *** zsli_ has quit IRC | 09:32 | |
| *** zzzeek has joined #heat | 09:32 | |
| tbh | ramishra, sorry I was disconnected from irc, did I miss anything on api versioning? | 09:33 |
| *** Qiming has quit IRC | 09:33 | |
| ramishra | tbh: np:) I think it should work if v1 api is enabled in glance_api.conf | 09:36 |
| *** yangyape_ has joined #heat | 09:40 | |
| *** yangyapeng has quit IRC | 09:44 | |
| *** Qiming has joined #heat | 09:46 | |
| tbh | ramishra, enable_v1_api is False | 09:48 |
| ramishra | tbh: so you don't have v1 api enabled, hence that resource would not work. If you enable that and other related config options it would work | 09:49 |
| *** Qiming has quit IRC | 09:50 | |
| tbh | ramishra, what are other config options | 09:51 |
| tbh | ramishra, I made enable_v1_api to True and restarted glance api , but it didn't work | 09:52 |
| *** maestropandy has quit IRC | 09:53 | |
| ramishra | enable_v1_registry should enabled as well | 09:54 |
| ramishra | First check if you can do v1 api calls to glance and then try using heat. image resource is deprecated, so you should not possibly use it though. | 09:55 |
| *** armaan has quit IRC | 09:58 | |
| *** yangyapeng has joined #heat | 10:02 | |
| *** yangyapeng has quit IRC | 10:03 | |
| *** yangyapeng has joined #heat | 10:03 | |
| *** yangyape_ has quit IRC | 10:06 | |
| *** yangyapeng has quit IRC | 10:08 | |
| *** wangbo_ has quit IRC | 10:10 | |
| *** Qiming has joined #heat | 10:11 | |
| openstackgerrit | Xinni Ge proposed openstack/heat-dashboard master: Add unittests for modules, services and several controllers. https://review.openstack.org/514572 | 10:13 |
| *** wangbo_ has joined #heat | 10:19 | |
| *** wangbo_ has quit IRC | 10:29 | |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add installation guide, contributor guide and configuration guide. https://review.openstack.org/514594 | 10:32 |
| *** wangbo_ has joined #heat | 10:34 | |
| *** pbourke has quit IRC | 10:38 | |
| *** pbourke has joined #heat | 10:40 | |
| *** jprovazn has quit IRC | 10:41 | |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add installation guide, contributor guide and configuration guide. https://review.openstack.org/514597 | 10:49 |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add installation guide, contributor guide and configuration guide. https://review.openstack.org/514598 | 10:51 |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add installation guide, contributor guide and configuration guide. https://review.openstack.org/514599 | 10:52 |
| openstackgerrit | Keiichi Hikita proposed openstack/heat-dashboard master: Add installation guide, contributor guide and configuration guide. https://review.openstack.org/514600 | 10:53 |
| *** jiye_ has joined #heat | 10:56 | |
| *** jiye_ is now known as jiye | 10:58 | |
| *** jprovazn has joined #heat | 11:01 | |
| *** jprovazn has joined #heat | 11:01 | |
| *** Qiming has quit IRC | 11:02 | |
| *** wangbo_ has quit IRC | 11:03 | |
| *** serlex has joined #heat | 11:07 | |
| *** jiye has quit IRC | 11:14 | |
| *** kazsh has joined #heat | 11:20 | |
| *** wangbo_ has joined #heat | 11:24 | |
| *** dbecker has quit IRC | 11:30 | |
| *** Qiming has joined #heat | 11:30 | |
| *** yangyapeng has joined #heat | 11:32 | |
| *** yangyapeng has quit IRC | 11:33 | |
| *** yangyapeng has joined #heat | 11:33 | |
| *** Qiming has quit IRC | 11:34 | |
| *** fragatina has quit IRC | 11:35 | |
| *** fragatina has joined #heat | 11:36 | |
| *** Qiming has joined #heat | 11:39 | |
| *** omie888777 has quit IRC | 11:42 | |
| *** Qiming has quit IRC | 11:43 | |
| *** amoralej is now known as amoralej|lunch | 12:01 | |
| *** Qiming has joined #heat | 12:04 | |
| *** wangbo_ has quit IRC | 12:05 | |
| *** hoonetorg has quit IRC | 12:06 | |
| *** wangbo has joined #heat | 12:08 | |
| *** Qiming has quit IRC | 12:09 | |
| *** hoonetorg has joined #heat | 12:09 | |
| *** Qiming has joined #heat | 12:13 | |
| *** dprince has joined #heat | 12:16 | |
| *** Qiming has quit IRC | 12:20 | |
| *** Qiming has joined #heat | 12:30 | |
| *** dbecker has joined #heat | 12:34 | |
| *** wangbo has quit IRC | 12:41 | |
| *** wangbo has joined #heat | 12:44 | |
| *** tbh has quit IRC | 12:52 | |
| *** wangbo has quit IRC | 12:59 | |
| *** wangbo has joined #heat | 13:01 | |
| *** fragatina has quit IRC | 13:03 | |
| *** fragatina has joined #heat | 13:03 | |
| *** amoralej|lunch is now known as amoralej|mtg | 13:03 | |
| *** armaan has joined #heat | 13:04 | |
| *** jcoufal has joined #heat | 13:11 | |
| *** wangbo has quit IRC | 13:12 | |
| *** jmlowe has quit IRC | 13:14 | |
| *** SmearedBeard has quit IRC | 13:15 | |
| *** kiennt26_ has joined #heat | 13:19 | |
| *** kei-ichi has quit IRC | 13:21 | |
| *** skramaja has quit IRC | 13:21 | |
| *** kei-ichi has joined #heat | 13:23 | |
| *** kei-ichi has quit IRC | 13:27 | |
| *** mchlumsky has joined #heat | 13:27 | |
| *** kei-ichi has joined #heat | 13:28 | |
| *** blomquisg_gone is now known as blomquisg | 13:29 | |
| *** felipemonteiro_ has joined #heat | 13:33 | |
| *** felipemonteiro__ has joined #heat | 13:38 | |
| *** felipemonteiro_ has quit IRC | 13:42 | |
| therve | zaneb, Seeing this on one of my deployment: http://paste.openstack.org/show/624477/ | 13:44 |
| therve | Is it something fixed by you recent patch(es) ? | 13:45 |
| *** AlexeyAbashkin has quit IRC | 13:48 | |
| *** SmearedBeard has joined #heat | 13:49 | |
| *** AlexeyAbashkin has joined #heat | 13:50 | |
| *** jmlowe has joined #heat | 13:52 | |
| openstackgerrit | Dan Prince proposed openstack/heat master: Move FakeKeystoneClient to engine.clients https://review.openstack.org/512035 | 13:52 |
| *** mdnadeem has quit IRC | 14:07 | |
| zaneb | therve: looks like https://review.openstack.org/#/c/510674/ | 14:12 |
| therve | Yeah I thought so | 14:12 |
| *** kiennt26_ has quit IRC | 14:13 | |
| *** kiennt26 has quit IRC | 14:14 | |
| *** shardy has joined #heat | 14:15 | |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate functional jobs to Zuul v3 https://review.openstack.org/509098 | 14:18 |
| openstackgerrit | Rico Lin proposed openstack/heat master: Migrate grenade jobs to zuul v3 https://review.openstack.org/509140 | 14:18 |
| *** hongbin has joined #heat | 14:18 | |
| therve | Hum | 14:25 |
| therve | zaneb, My stack update is still stuck, any idea | 14:25 |
| zaneb | therve: if you're getting that error, then there must be a previous update that's locked the resource | 14:29 |
| therve | zaneb, Yeah. It timed out, and I'm retrying | 14:30 |
| zaneb | therve: did the previous update time out? if so it may be fixed by https://review.openstack.org/#/c/509921/ | 14:30 |
| therve | I updated heat but so I don't have the error anymore | 14:30 |
| therve | Looking | 14:30 |
| therve | zaneb, I do have that one already | 14:31 |
| zaneb | therve: so your first update timed out, you saw that log trying to do a second stack update before the first completed, updated heat & restarted, tried another stack update & it still gets stuck? | 14:38 |
| therve | zaneb, Yeah almost, except I retried a few times before updating heat | 14:38 |
| zaneb | that's weird. I'm not aware of a bug that should bork it permanently like that | 14:39 |
| zaneb | once the first update times out you should be good to go | 14:39 |
| therve | zaneb, Do you want to check the env? | 14:41 |
| zaneb | sure | 14:41 |
| *** gouthamr has joined #heat | 14:42 | |
| *** omie888777 has joined #heat | 14:45 | |
| *** AlexeyAbashkin has quit IRC | 14:51 | |
| *** armaan has quit IRC | 14:55 | |
| *** AlexeyAbashkin has joined #heat | 14:59 | |
| *** ccamacho has quit IRC | 15:02 | |
| sai | zaneb: hey, trying to delet eovercloud stack and i see DELETE_FAILED Resource DELETE failed: IndexError: resources.Controller.resources[2].resources.Controller: list index out of range | 15:02 |
| sai | seeing this in heat logs on undercloud https://gist.githubusercontent.com/smalleni/3d9f2fb55c94533dbee460a32fca3384/raw/ab0e92de91423f1939ce7f0083c3a6cb065723bc/gistfile1.txt | 15:03 |
| zaneb | sai: that's what I was just about to ask for :) | 15:03 |
| * sai knew it | 15:04 | |
| sai | :P | 15:04 |
| zaneb | strangely enough that's an exception in the code that's trying to print a warning | 15:05 |
| * zaneb has flashbacks to last week | 15:05 | |
| sai | if it helps, some of the hosts had double entries like overcloud-controller-1 for example which was in ERROR(never deployed), there was an active overcloud-controller-1 too | 15:06 |
| sai | maybe that could be causing this incosistency | 15:06 |
| sai | ? | 15:06 |
| zaneb | nope | 15:06 |
| zaneb | this is the code that is failing: http://git.openstack.org/cgit/openstack/python-novaclient/tree/novaclient/v2/contrib/__init__.py#n35 | 15:07 |
| zaneb | that's 100% python-novaclient's problem :) | 15:07 |
| *** dbecker has quit IRC | 15:09 | |
| *** AlexeyAbashkin has quit IRC | 15:10 | |
| sai | zaneb: thank you! just trying to understand, this is a bug right(index out of range), or could this have been caused due to something i did | 15:14 |
| sai | want to see next steps | 15:14 |
| *** crose has joined #heat | 15:15 | |
| sai | because i didnt always see this | 15:15 |
| zaneb | sai: it appears to be a bug in the python stdlib, because all python-novaclient is doing here is calling inspect.stack() and it's barfing withing the inspect module | 15:16 |
| zaneb | sai: what version of python do you have? | 15:16 |
| sai | zaneb: Python 2.7.5 | 15:16 |
| zaneb | hmm, that's kinda old | 15:17 |
| zaneb | iPython has a monkey patch to fix it: https://github.com/ipython/ipython/pull/2232/commits/298fdab5025745cd25f7f48147d8bc4c65be9d4a I wonder if they ever reported upstream... | 15:21 |
| *** ccamacho has joined #heat | 15:23 | |
| zaneb | if they did it still isn't fixed :( | 15:23 |
| *** serlex has quit IRC | 15:25 | |
| sai | thanks for diggin in zaneb ...looks like 2.7.5 is what we get by default in the rhel image | 15:25 |
| zaneb | I'm looking at the lastest source and that patch from iPython still isn't applied, so there's a good chance the bug is still present in 2.7.13 (unless they fixed it somewhere else) | 15:26 |
| zaneb | sai: https://bugs.python.org/issue1628987 | 15:34 |
| zaneb | Resolution: wont fix | 15:34 |
| zaneb | reproducer still fails in python 2.7.13/3.5.4 | 15:35 |
| *** ricolin has quit IRC | 15:35 | |
| sai | weird | 15:36 |
| zaneb | sai: are you installed from RPMs? | 15:36 |
| sai | not eve sure where that sequence of steps of opening the module, changing source and then calling inspect is happening | 15:36 |
| sai | zaneb: yes | 15:36 |
| sai | rhel7.4 | 15:37 |
| sai | python-2.7.5-58.el7.x86_64 | 15:37 |
| zaneb | the reason I ask is because if .py and .pyc files were out of sync then that could potentially explain it | 15:38 |
| zaneb | but that shouldn't be possible with RPMs | 15:38 |
| sai | is deleting all pyc files on the system a bad idea at this point :P | 15:39 |
| zaneb | is this reproducible? | 15:42 |
| zaneb | we really need to know *which* frame is causing the problem | 15:43 |
| openstackgerrit | Dan Prince proposed openstack/heat master: Move FakeKeystoneClient to engine.clients https://review.openstack.org/512035 | 15:45 |
| zaneb | that'd tell us where we need to fix | 15:47 |
| sai | zaneb: its failing 10/10 times ive tried now, lemme know if u wanna take a peek | 15:47 |
| zaneb | sai: can you restart heat-engine and see if it still fails? If it does we'll hack some debugging into the stdlib to help us track it down | 15:49 |
| *** felipemonteiro__ has quit IRC | 15:49 | |
| *** ccamacho has quit IRC | 15:51 | |
| *** Roedy has quit IRC | 15:59 | |
| sai | zaneb: restarted heat-engine and tried 2 times | 16:01 |
| sai | same thing | 16:01 |
| sai | however, 3rd time it worked | 16:01 |
| sai | magic | 16:01 |
| sai | lol | 16:01 |
| zaneb | wtf | 16:01 |
| sai | yeh i wish it didnt work | 16:02 |
| sai | :/ | 16:02 |
| *** amoralej|mtg is now known as amoralej | 16:05 | |
| *** Roedy has joined #heat | 16:05 | |
| *** omie888777 has quit IRC | 16:15 | |
| *** crose has quit IRC | 16:17 | |
| ioggstream | bye@heat | 16:20 |
| *** AlexeyAbashkin has joined #heat | 16:22 | |
| *** fragatina has quit IRC | 16:25 | |
| *** AlexeyAbashkin has quit IRC | 16:26 | |
| *** shardy has quit IRC | 16:33 | |
| zaneb | sai: so I'd suggest raising a bug against novaclient in ocata, include the stack trace and the link to the python bug, and ask them to catch & work around the exception | 16:36 |
| sai | zaneb: ack, thanks for all ur help! | 16:40 |
| *** ramishra has quit IRC | 16:41 | |
| *** derekh has quit IRC | 16:52 | |
| openstackgerrit | Matt Riedemann proposed openstack/heat master: docs: fix user-data end user guide link https://review.openstack.org/514761 | 16:52 |
| *** AlexeyAbashkin has joined #heat | 16:55 | |
| *** jtomasek has quit IRC | 16:56 | |
| *** AlexeyAbashkin has quit IRC | 16:59 | |
| *** felipemonteiro_ has joined #heat | 17:01 | |
| *** felipemonteiro__ has joined #heat | 17:02 | |
| *** felipemonteiro_ has quit IRC | 17:06 | |
| *** fragatina has joined #heat | 17:11 | |
| *** gkadam has quit IRC | 17:30 | |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Don't try resource-level locks when stack locked https://review.openstack.org/514476 | 17:34 |
| *** blomquisg is now known as blomquisg_gone | 17:36 | |
| *** dyasny__ has quit IRC | 17:50 | |
| *** dyasny__ has joined #heat | 17:51 | |
| *** amoralej is now known as amoralej|off | 18:14 | |
| *** SmearedBeard is now known as SmearedBeard-AFK | 18:29 | |
| *** SmearedBeard-AFK has quit IRC | 18:33 | |
| *** ioggstream has quit IRC | 18:34 | |
| *** SmearedBeard has joined #heat | 18:35 | |
| *** AlexeyAbashkin has joined #heat | 18:55 | |
| *** AlexeyAbashkin has quit IRC | 19:00 | |
| mwynne | `openstack stack list` is hanging for me, and timing out with this error: Unexpected error occurred serving API: Timed out waiting for a reply to message ID | 19:03 |
| mwynne | No other errors in any of the heat logs | 19:03 |
| mwynne | Any suggestions for place to look for errors? | 19:04 |
| mwynne | I can see the request in the api-log, but it just hangs after that. | 19:07 |
| *** jmlowe has quit IRC | 19:33 | |
| zaneb | mwynne: is heat-engine still running? | 19:33 |
| *** gfidente has quit IRC | 19:35 | |
| mwynne | zaneb: Yeah, systemctl status heat-engine looks good | 19:39 |
| zaneb | mwynne: could rabbitmq be not working? | 19:40 |
| mwynne | zaneb: The service is up and running | 19:44 |
| zaneb | so how it works is, heat-api gets a request, sends a message to heat-engine, and waits for a response | 19:45 |
| zaneb | so the possibilities are either (a) one of the messages is not getting through, or (b) computing the response takes so long that the api times out while waiting | 19:46 |
| mwynne | Is there a good way for me to test each of these? | 19:46 |
| *** felipemonteiro__ has quit IRC | 19:47 | |
| zaneb | you can bump the rpc_response_timeout in heat.conf, restart heat-engine & heat-api and see if that helps | 19:47 |
| zaneb | it also wouldn't hurt to restart all the services + rabbit to see if they've failed somehow | 19:48 |
| mwynne | Yeah restarting all the services was where I started unfortunately | 19:48 |
| zaneb | if you're not seeing anything at all in the heat-engine log pertaining to the request then there's a good chance it's the messaging | 19:49 |
| mwynne | when i run `openstack stack list` should I see a message in the engine log? | 19:51 |
| *** jmlowe has joined #heat | 20:00 | |
| *** ayoung has quit IRC | 20:02 | |
| *** Smeared_Beard has joined #heat | 20:03 | |
| *** SmearedBeard has quit IRC | 20:03 | |
| zaneb | mwynne: yes, but probably only if debug logs are enabled | 20:11 |
| *** Guest62896 has quit IRC | 20:17 | |
| *** Guest62896 has joined #heat | 20:21 | |
| *** jcoufal has quit IRC | 20:29 | |
| mwynne | zaneb: debug is enabled | 20:32 |
| zaneb | mwynne: then you should see debug messages showing RPC messages arriving, if they are arriving | 20:32 |
| *** jprovazn has quit IRC | 20:33 | |
| *** felipemonteiro_ has joined #heat | 20:34 | |
| *** felipemonteiro__ has joined #heat | 20:35 | |
| mwynne | zaneb: DEBUG heat.engine.service [req-08af6824-62cf-4395-b81d-09b6f7f4d6e2 - - - - -] Service 400c4210-d972-4c00-bb59-029c1ffc203f is updated service_manage_report /usr/lib/python2.7/dist-packages/heat/engine/service.py:2384 | 20:37 |
| mwynne | this is all I'm getting in heat-engine.log | 20:37 |
| zaneb | that's just the heartbeat thing | 20:37 |
| zaneb | so it appears your RPC is dead, somehow | 20:37 |
| mwynne | zaneb: how can I debug this? | 20:38 |
| mwynne | any idea? | 20:38 |
| *** felipemonteiro_ has quit IRC | 20:39 | |
| *** jmlowe has quit IRC | 20:39 | |
| *** felipemonteiro__ has quit IRC | 20:42 | |
| *** felipemonteiro__ has joined #heat | 20:42 | |
| *** dprince has quit IRC | 20:45 | |
| mwynne | /usr/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py:185: SAWarning: Unicode type received non-unicode bind param value 'data-processing (util.ellipses_string(value),)) | 20:45 |
| mwynne | zaneb: I get that in the heat-engine systemctl status | 20:46 |
| zaneb | I would concentrate on rabbit | 20:47 |
| mwynne | If it's running and I haven't changed anything, any suggestions for where to look? | 20:51 |
| mwynne | zaneb: Other services should be failing if rabbit's down, right? | 21:01 |
| zaneb | you'd imagine so | 21:02 |
| mwynne | I would have assumed nova list and neutron net-list to fail as well | 21:03 |
| mwynne | but they're not | 21:03 |
| *** jmlowe has joined #heat | 21:08 | |
| mwynne | zaneb: 60 [Note] Aborted connection 60 to db: 'heat' user: 'heat' host: 'compute-2' (Got an error reading communication packets) | 21:17 |
| mwynne | That's what I'm getting in the mysql error log | 21:17 |
| *** jmlowe has quit IRC | 21:18 | |
| zaneb | mwynne: nova & neutron have a completely different architecture, where the api processes talk directly to the DB, not over RPC | 21:18 |
| *** felipemonteiro_ has joined #heat | 21:23 | |
| *** omie888777 has joined #heat | 21:24 | |
| mwynne | zaneb: yeah, something doesn't seem right here. | 21:26 |
| *** felipemonteiro__ has quit IRC | 21:26 | |
| mwynne | zaneb: heat-engine[3544]: /usr/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py:185: SAWarning: Unicode type received non-unicode bind param value 'data-processing-cluster'. (this warning may be suppressed after 10 occurrences) | 21:32 |
| mwynne | this seems to be the issue I think | 21:32 |
| mwynne | zaneb: I get one of those for each worker it seems | 21:33 |
| *** ayoung has joined #heat | 21:34 | |
| *** dyasny__ has quit IRC | 21:38 | |
| zaneb | looks like just a warning, but anything is possible I guess | 21:51 |
| *** AlexeyAbashkin has joined #heat | 21:56 | |
| *** AlexeyAbashkin has quit IRC | 22:01 | |
| *** mchlumsky has quit IRC | 22:14 | |
| *** mchlumsky has joined #heat | 22:16 | |
| mwynne | zaneb: well i'm not getting anywhere | 22:21 |
| *** jmlowe has joined #heat | 22:34 | |
| *** felipemonteiro_ has quit IRC | 22:43 | |
| *** felipemonteiro_ has joined #heat | 22:44 | |
| *** felipemonteiro_ has quit IRC | 23:01 | |
| *** askb has quit IRC | 23:12 | |
| *** jmlowe has quit IRC | 23:12 | |
| *** hongbin has quit IRC | 23:14 | |
| *** Smeared_Beard has quit IRC | 23:15 | |
| *** weshay|ruck is now known as weshay|PTO | 23:38 | |
| *** mchlumsky has quit IRC | 23:51 | |
| *** jmlowe has joined #heat | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!