| weshay|ruck | stevebaker, EmilienM mwhahaha be aware of the gating.repo name change https://review.openstack.org/#/c/572736/ | 00:00 |
|---|---|---|
| EmilienM | :-o | 00:01 |
| *** slagle has quit IRC | 00:05 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Use the in-built image prepare for undercloud install https://review.openstack.org/571613 | 00:05 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Use role tripleo-modify-image to update containers https://review.openstack.org/571631 | 00:05 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Switch to workflow driven image prepare https://review.openstack.org/573476 | 00:05 |
| *** akrivoka has quit IRC | 00:06 | |
| *** dhill_ has quit IRC | 00:07 | |
| *** jlibosva has joined #tripleo | 00:08 | |
| weshay|ruck | https://review.openstack.org/571613 | 00:09 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1736950 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776301 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776503 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 00:10 |
| openstack | Launchpad bug 1736950 in tripleo "CI: mistral testmistral_tempest_tests.tests.api.v2.test_actions.ActionTestsV2.test_get_list_actions_not_in_list_filter fails in gate scenario003 containers" [Critical,Triaged] | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1776301 in tripleo "[master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 00:10 |
| openstack | Launchpad bug 1776503 in tripleo "rdocloud outage recovery - contention for resources and jobs showing long wait times" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 00:10 |
| *** jlibosva has quit IRC | 00:12 | |
| *** rlandy is now known as rlandy|afk | 00:13 | |
| *** liverpooler has joined #tripleo | 00:14 | |
| trozet | dsneddon: i would but i dont have +2 in this repo | 00:14 |
| *** jlibosva has joined #tripleo | 00:15 | |
| *** jlibosva has quit IRC | 00:19 | |
| *** jlibosva has joined #tripleo | 00:22 | |
| *** gkadam has quit IRC | 00:23 | |
| *** social has quit IRC | 00:23 | |
| *** jlibosva has quit IRC | 00:26 | |
| *** jlibosva has joined #tripleo | 00:31 | |
| *** pmannidi has joined #tripleo | 00:31 | |
| *** jlibosva has quit IRC | 00:35 | |
| *** mhenkel_ has joined #tripleo | 00:36 | |
| *** dmacpher has joined #tripleo | 00:39 | |
| *** psahoo has joined #tripleo | 00:39 | |
| *** mhenkel_ has quit IRC | 00:41 | |
| beagles | dsneddon: the unit tests need updating - can the patch wait until Bob can update it or would you rather fix it now | 00:43 |
| openstackgerrit | Merged openstack/diskimage-builder master: Add log directory option to functional tests https://review.openstack.org/570095 | 00:43 |
| beagles | dsneddon: re: https://review.openstack.org/#/c/575220/ that is | 00:44 |
| openstackgerrit | Bob Fournier proposed openstack/os-net-config master: Restore the order of params in ifcfg file that was inadvertently changed https://review.openstack.org/575220 | 00:44 |
| beagles | lol never mind bfournie got to it ! | 00:46 |
| bfournie | beagles: all set, thanks for finding that! | 00:46 |
| beagles | bfournie: np | 00:46 |
| EmilienM | weshay|ruck, stevebaker: email sent, feel free to comment or add anything I missed | 00:50 |
| EmilienM | weshay|ruck: if you can review Steve's patches in the meantime it would help | 00:51 |
| EmilienM | weshay|ruck: https://review.openstack.org/#/c/571613/, https://review.openstack.org/#/c/571631 and https://review.openstack.org/#/c/573476 | 00:51 |
| *** limao has joined #tripleo | 00:52 | |
| *** liverpooler has quit IRC | 01:04 | |
| *** jlibosva has joined #tripleo | 01:05 | |
| *** jlibosva has quit IRC | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1736950 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776301 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776503 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 01:10 |
| openstack | Launchpad bug 1736950 in tripleo "CI: mistral testmistral_tempest_tests.tests.api.v2.test_actions.ActionTestsV2.test_get_list_actions_not_in_list_filter fails in gate scenario003 containers" [Critical,Triaged] | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1776301 in tripleo "[master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 01:10 |
| openstack | Launchpad bug 1776503 in tripleo "rdocloud outage recovery - contention for resources and jobs showing long wait times" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 01:10 |
| *** rlandy|afk is now known as rlandy | 01:10 | |
| *** pliu_ has joined #tripleo | 01:11 | |
| *** ohochman has joined #tripleo | 01:13 | |
| *** mdnadeem has joined #tripleo | 01:14 | |
| *** Petersingh has joined #tripleo | 01:18 | |
| *** dims has joined #tripleo | 01:20 | |
| dims | EmilienM : yo! | 01:20 |
| dims | EmilienM : where do you see timeouts .. docker daemon logs? | 01:21 |
| larsks | I'm seen failures for tripleo::profile::base::cinder::volume::dellemc_vnx, which doesn't have anything to do with my reviews. Has anyone else seen this? | 01:23 |
| larsks | (e.g., https://review.openstack.org/#/c/574851/) | 01:23 |
| *** mrsoul has joined #tripleo | 01:25 | |
| *** rpioso is now known as rpioso|afk | 01:26 | |
| EmilienM | larsks: patch for that in progress | 01:27 |
| larsks | Awesome. Thanks! | 01:27 |
| EmilienM | larsks: but stuck because our gate is broken | 01:27 |
| EmilienM | larsks: check on my Gerrit I'm on my phone | 01:27 |
| larsks | Bummer. Thanks! | 01:28 |
| dims | EmilienM : if you see docker daemon mis-behaving there are a couple of things that can be bumped up "max-concurrent-downloads" and "max-concurrent-uploads" the downloads is especially low (default is 3) | 01:28 |
| EmilienM | dims: on my phone but weshay|ruck can help | 01:28 |
| EmilienM | dims: its not really docker timeout | 01:29 |
| *** mschuppert has quit IRC | 01:29 | |
| EmilienM | It's more how we prepared the host | 01:29 |
| EmilienM | The fact we pull same containers multiple times | 01:29 |
| *** cdearborn has joined #tripleo | 01:30 | |
| weshay|ruck | larsks, /me looks | 01:30 |
| *** jlibosva has joined #tripleo | 01:31 | |
| dims | EmilienM : ack, just in case you have to tweak at the docker daemon, here's the info - https://stackoverflow.com/questions/43479614/docker-parallel-operations-limit/43485688 | 01:31 |
| EmilienM | larsks: https://review.openstack.org/#/c/574873/ | 01:31 |
| EmilienM | dims: will check thx | 01:31 |
| dims | "The default limits are well suited for a development environment" (quote from there) | 01:31 |
| weshay|ruck | meh.. larsks sorry I'm not familiar w/ the puppet unit test jobs | 01:33 |
| EmilienM | weshay|ruck: see my link | 01:33 |
| larsks | weshay|ruck: no worries, it sounds as if EmilienM is already aware of it. That last post from him looks like a fix. | 01:33 |
| weshay|ruck | rockin | 01:34 |
| *** rlandy has quit IRC | 01:35 | |
| *** jlibosva has quit IRC | 01:35 | |
| weshay|ruck | toure|brb, FYI.. transient error on mistral, it worked this time | 01:36 |
| *** psahoo has quit IRC | 01:36 | |
| *** dims has left #tripleo | 01:36 | |
| *** psahoo has joined #tripleo | 01:38 | |
| EmilienM | larsks: rebase your stuff on top of it | 01:43 |
| larsks | Will do. | 01:43 |
| *** limao has quit IRC | 01:44 | |
| *** psahoo has quit IRC | 01:45 | |
| *** limao has joined #tripleo | 01:45 | |
| weshay|ruck | EmilienM, just noting that the rdo 3rd party jobs have recently been finishing more quickly than the multinode jobs.. ( weird ) | 01:48 |
| weshay|ruck | EmilienM, https://bugs.launchpad.net/tripleo/+bug/1776796 | 01:51 |
| openstack | Launchpad bug 1776796 in tripleo "tripleo gate jobs timing out, duplicate containers pulls a possible cause" [High,Triaged] | 01:51 |
| *** Petersingh is now known as Petersingh|afk | 01:53 | |
| *** psahoo has joined #tripleo | 01:55 | |
| *** Petersingh|afk is now known as Petersingh | 01:56 | |
| EmilienM | weshay|ruck: :( | 01:58 |
| weshay|ruck | wha? | 01:58 |
| EmilienM | weshay|ruck: ovb faster | 01:58 |
| weshay|ruck | oh | 01:58 |
| weshay|ruck | btw.. fs20 is not in check or gate.. just full tempest used for production pipeline bs.. | 01:59 |
| weshay|ruck | ack to promote master if that fails again on storage? | 01:59 |
| weshay|ruck | https://review.rdoproject.org/zuul/ | 01:59 |
| *** jlibosva has joined #tripleo | 02:02 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Allow a containerized mistral-executor to access docker https://review.openstack.org/574905 | 02:02 |
| *** yamahata has quit IRC | 02:03 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo master: enable support for openidc federation in keystone https://review.openstack.org/574851 | 02:05 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776301 | 02:10 |
| openstack | Launchpad bug 1776301 in tripleo "[master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776503 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1776503 in tripleo "rdocloud outage recovery - contention for resources and jobs showing long wait times" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 02:10 |
| *** ohochman has quit IRC | 02:16 | |
| EmilienM | weshay|ruck: yes? | 02:16 |
| weshay|ruck | good | 02:16 |
| EmilienM | weshay|ruck: if you're sure 😂 | 02:17 |
| weshay|ruck | https://review.openstack.org/#/c/574876/ | 02:19 |
| *** ramishra has joined #tripleo | 02:22 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Rename output log files https://review.openstack.org/570096 | 02:23 |
| openstackgerrit | Merged openstack/diskimage-builder master: Don't install zypper on bionic https://review.openstack.org/570500 | 02:23 |
| *** limao has quit IRC | 02:30 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: fix typo in chart https://review.openstack.org/575167 | 02:32 |
| *** brault has joined #tripleo | 02:36 | |
| *** itlinux has joined #tripleo | 02:36 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates master: [WIP] enable support for openidc federation in keystone https://review.openstack.org/574855 | 02:38 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo master: [WIP] enable support for openidc federation in keystone https://review.openstack.org/574851 | 02:39 |
| *** brault has quit IRC | 02:41 | |
| *** jlibosva has quit IRC | 02:42 | |
| openstackgerrit | Nguyen Van Trung proposed openstack/diskimage-builder master: Fix 'Operation not supported' issue for setfiles https://review.openstack.org/575315 | 02:43 |
| *** ayoung has quit IRC | 02:50 | |
| *** radeks has joined #tripleo | 02:56 | |
| openstackgerrit | Nguyen Van Trung proposed openstack/diskimage-builder master: Add iscsi-boot element https://review.openstack.org/511494 | 02:58 |
| openstackgerrit | Nguyen Van Trung proposed openstack/diskimage-builder master: Add iscsi-boot element for CentOS images https://review.openstack.org/542708 | 02:58 |
| *** ayoung has joined #tripleo | 03:00 | |
| *** cdearborn has quit IRC | 03:02 | |
| EmilienM | weshay|ruck: you need +A on this one? | 03:03 |
| *** jlibosva has joined #tripleo | 03:07 | |
| EmilienM | stevebaker: 571613 is failing in check now, I'm looking why | 03:08 |
| EmilienM | stevebaker: http://logs.openstack.org/13/571613/16/check/tripleo-ci-centos-7-containers-multinode/6e6f69c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-06-14_02_37_59 | 03:08 |
| stevebaker | EmilienM: missing BUILD_ID in the prep script, I'm looking | 03:09 |
| EmilienM | BUILD_ID=${BUILD_ID:-"{{ container_build_id }}"} is in the else | 03:09 |
| EmilienM | I think that's why? | 03:09 |
| stevebaker | EmilienM: thats it | 03:10 |
| stevebaker | fixing | 03:10 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 03:10 |
| *** jlibosva has quit IRC | 03:12 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Use the in-built image prepare for undercloud install https://review.openstack.org/571613 | 03:12 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Use role tripleo-modify-image to update containers https://review.openstack.org/571631 | 03:12 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Switch to workflow driven image prepare https://review.openstack.org/573476 | 03:12 |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates master: Set ODL API restart policy to unless-stopped https://review.openstack.org/575111 | 03:13 |
| *** dxiri has quit IRC | 03:13 | |
| *** dxiri has joined #tripleo | 03:15 | |
| EmilienM | stevebaker: thx! | 03:15 |
| EmilienM | the revert is probably going to make it tonight | 03:16 |
| *** Petersingh is now known as Petersingh|afk | 03:22 | |
| *** Petersingh|afk is now known as Petersingh | 03:31 | |
| *** yamahata has joined #tripleo | 03:35 | |
| *** Petersingh is now known as Petersingh|afk | 03:38 | |
| *** ayoung has quit IRC | 03:40 | |
| *** skramaja has joined #tripleo | 03:43 | |
| *** yamahata has quit IRC | 03:43 | |
| *** jlibosva has joined #tripleo | 03:44 | |
| *** yamahata has joined #tripleo | 03:51 | |
| *** udesale has joined #tripleo | 03:56 | |
| *** Petersingh|afk is now known as Petersingh | 04:07 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 04:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| *** ykarel|away has joined #tripleo | 04:16 | |
| *** ykarel|away is now known as ykarel | 04:16 | |
| *** shrjoshi has joined #tripleo | 04:17 | |
| *** jlibosva has quit IRC | 04:17 | |
| *** lifeless_ has quit IRC | 04:22 | |
| *** ksambor has joined #tripleo | 04:24 | |
| *** jlibosva has joined #tripleo | 04:28 | |
| *** agopi has quit IRC | 04:31 | |
| *** jlibosva has quit IRC | 04:32 | |
| *** saneax has joined #tripleo | 04:35 | |
| *** ksambor has quit IRC | 04:38 | |
| *** ksambor has joined #tripleo | 04:38 | |
| *** ksambor has quit IRC | 04:41 | |
| *** threestrands has quit IRC | 04:47 | |
| *** Petersingh is now known as Petersingh|afk | 04:47 | |
| EmilienM | stevebaker: :( http://logs.openstack.org/13/571613/17/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/08d783f/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-06-14_04_35_34 | 04:50 |
| EmilienM | wondering if it's because we need https://review.openstack.org/#/c/574485 merged and containerized | 04:51 |
| *** radeks has quit IRC | 04:51 | |
| *** janki has joined #tripleo | 04:57 | |
| *** links has joined #tripleo | 05:00 | |
| *** yprokule has joined #tripleo | 05:04 | |
| *** jlibosva has joined #tripleo | 05:04 | |
| *** jlibosva has quit IRC | 05:08 | |
| *** pcaruana has quit IRC | 05:09 | |
| *** Petersingh|afk is now known as Petersingh | 05:10 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 05:10 |
| *** psahoo has quit IRC | 05:11 | |
| *** pdeore has joined #tripleo | 05:12 | |
| *** psahoo has joined #tripleo | 05:13 | |
| *** lifeless has joined #tripleo | 05:17 | |
| *** marios has quit IRC | 05:25 | |
| *** marios has joined #tripleo | 05:25 | |
| *** leifmadsen has quit IRC | 05:27 | |
| *** leifmadsen has joined #tripleo | 05:28 | |
| *** moshele has joined #tripleo | 05:30 | |
| EmilienM | https://review.openstack.org/575264 is about to land, most probably | 05:30 |
| EmilienM | i'm going to restore some patches that were in the gate and see how much can land | 05:30 |
| EmilienM | please do not recheck, do not +A any patches fo rnow | 05:30 |
| *** radeks has joined #tripleo | 05:32 | |
| *** dxiri has quit IRC | 05:33 | |
| *** ratailor has joined #tripleo | 05:36 | |
| *** mdnadeem has quit IRC | 05:37 | |
| *** mdnadeem has joined #tripleo | 05:37 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Revert "Switch scenario00{1,2,3,4,7} to containerized undercloud" https://review.openstack.org/575264 | 05:38 |
| *** jlibosva has joined #tripleo | 05:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Switch scenario00{1,2,3,4,7} to containerized undercloud"" https://review.openstack.org/575330 | 05:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Switch scenario00{1,2,3,4,7} to containerized undercloud"" https://review.openstack.org/575330 | 05:39 |
| *** mdnadeem has quit IRC | 05:40 | |
| *** jaosorior has quit IRC | 05:41 | |
| *** jlibosva has quit IRC | 05:43 | |
| *** quiquell|off is now known as quiquell|rover | 05:45 | |
| *** moshele has quit IRC | 05:46 | |
| *** psahoo_ has joined #tripleo | 05:46 | |
| *** psahoo_ has quit IRC | 05:46 | |
| *** psahoo has quit IRC | 05:47 | |
| *** psahoo has joined #tripleo | 05:49 | |
| *** slaweq has quit IRC | 05:53 | |
| *** cshastri has joined #tripleo | 05:55 | |
| *** yprokule has quit IRC | 05:58 | |
| *** jlibosva has joined #tripleo | 06:00 | |
| *** yprokule has joined #tripleo | 06:01 | |
| *** jlibosva has quit IRC | 06:04 | |
| *** hjensas has quit IRC | 06:05 | |
| *** jlibosva has joined #tripleo | 06:07 | |
| *** anande has joined #tripleo | 06:07 | |
| *** mhenkel_ has joined #tripleo | 06:07 | |
| *** mhenkel_ has joined #tripleo | 06:07 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 06:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| *** jlibosva has quit IRC | 06:11 | |
| *** jaganathan has joined #tripleo | 06:11 | |
| *** mrsoul has quit IRC | 06:16 | |
| *** mrsoul_ has joined #tripleo | 06:16 | |
| *** mrsoul_ is now known as mschuppert | 06:16 | |
| *** itlinux has quit IRC | 06:17 | |
| *** pcaruana has joined #tripleo | 06:17 | |
| *** kopecmartin has joined #tripleo | 06:20 | |
| *** jlibosva has joined #tripleo | 06:21 | |
| *** rajinir has quit IRC | 06:22 | |
| *** mschuppert has quit IRC | 06:22 | |
| *** mrsoul has joined #tripleo | 06:23 | |
| *** mrsoul is now known as mschuppert | 06:23 | |
| *** bogdando has joined #tripleo | 06:24 | |
| *** mschuppert has quit IRC | 06:24 | |
| *** mrsoul_ has joined #tripleo | 06:24 | |
| *** mrsoul_ is now known as mschuppert | 06:24 | |
| *** saneax has quit IRC | 06:27 | |
| *** sanjayu_ has joined #tripleo | 06:27 | |
| *** threestrands has joined #tripleo | 06:27 | |
| *** ksambor has joined #tripleo | 06:29 | |
| *** ksambor has quit IRC | 06:29 | |
| *** ksambor has joined #tripleo | 06:30 | |
| *** sanjayu__ has joined #tripleo | 06:30 | |
| *** yprokule has quit IRC | 06:30 | |
| *** yprokule has joined #tripleo | 06:31 | |
| *** sanjayu_ has quit IRC | 06:33 | |
| *** sanjayu_ has joined #tripleo | 06:33 | |
| *** sanjayu__ has quit IRC | 06:34 | |
| *** sanjayu__ has joined #tripleo | 06:34 | |
| *** sanjayu_ has quit IRC | 06:34 | |
| *** sanjayu_ has joined #tripleo | 06:35 | |
| *** jlibosva has quit IRC | 06:36 | |
| *** moshele has joined #tripleo | 06:37 | |
| *** shardy has joined #tripleo | 06:38 | |
| *** sanjayu__ has quit IRC | 06:39 | |
| *** Petersingh is now known as Petersingh|afk | 06:42 | |
| *** pmannidi has quit IRC | 06:42 | |
| *** brault has joined #tripleo | 06:44 | |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart master: Define a 'baremetal' file for nodes config https://review.openstack.org/553465 | 06:44 |
| openstackgerrit | Merged openstack/tripleo-common master: Use correct modify_append_tag key (again) https://review.openstack.org/574485 | 06:44 |
| bandini | any takers for https://review.openstack.org/#/c/569578/ ? | 06:45 |
| *** quiquell|rover is now known as quiquell|rover|a | 06:45 | |
| *** quiquell|rover|a is now known as quique|rover|afk | 06:46 | |
| *** dparkes has joined #tripleo | 06:46 | |
| *** Petersingh|afk is now known as Petersingh | 06:46 | |
| shardy | bandini: looking | 06:49 |
| bandini | merci! | 06:50 |
| shardy | bandini: so we'll try for up to 30 minutes for the remotes to come up? | 06:52 |
| *** pmannidi has joined #tripleo | 06:52 | |
| shardy | that seems pretty long, just wondering how you decided on the tries value | 06:53 |
| flaper87 | shardy: hey, you may know how to help me here: I've requested infra to hold the nodes that were used for the last run of tripleo-ci-centos-7-scenario009-multinode-oooq in https://review.openstack.org/#/c/574233/ I just noticed that the node doesn't have the /etc/os-net-config/ files | 06:53 |
| flaper87 | although, it feels like it failed before running the os-net-config tasks... need to dig more | 06:53 |
| shardy | flaper87: most likely reason is the network connection to the undercloud was broken, as we get the os-net-config stuff on the first poll(s) to swift for metadata | 06:54 |
| shardy | flaper87: will take a look, sec | 06:54 |
| *** ccamacho has joined #tripleo | 06:54 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/575339 | 06:55 |
| bandini | shardy: yeah, I mimicked similar timeouts we have for the cluster to come up. (it is kind of high as I assumed in a largish IHA setup we might want to be extra tolerant of things to come up) | 06:55 |
| shardy | bandini: ack np - I was just thinking about it in the context of the default deploy timeout but we can always add a parameter for it later if it's a problem I guess | 06:56 |
| *** slaweq has joined #tripleo | 06:56 | |
| shardy | at least this way it'll hang for long enough to investigate what's up ;) | 06:57 |
| shardy | and make a coffee :D | 06:57 |
| bandini | shardy: I think you're right I should aat least add parameters :) | 06:57 |
| * bandini amends | 06:57 | |
| shardy | bandini: we can do it as a follow up I think | 06:57 |
| bandini | ack, will do one on top | 06:57 |
| shardy | thanks! | 06:57 |
| bandini | thank you! | 06:58 |
| shardy | http://logs.openstack.org/33/574233/7/experimental/tripleo-ci-centos-7-scenario009-multinode-oooq/426af74/logs/undercloud/var/lib/mistral/ff4bbb33-05dc-45f9-a01e-1128e50ba2ee/openshift/playbook.log.txt.gz | 07:01 |
| shardy | flaper87: so it's failed pulling a container from the fedora registryu | 07:01 |
| shardy | so one problem is the prepare isn't putting that on the undercloud I guess | 07:01 |
| shardy | http://logs.openstack.org/33/574233/7/experimental/tripleo-ci-centos-7-scenario009-multinode-oooq/426af74/logs/undercloud/var/lib/mistral/config-download-latest/Controller/centos-7-ovh-bhs1-0000095044/NetworkDeployment.gz | 07:02 |
| shardy | that looks weird though - the script seems wrong | 07:02 |
| *** Petersingh is now known as Petersingh|afk | 07:02 | |
| flaper87 | wonder if this is just a spurious failure, as it didn't do it in a previous recheck | 07:03 |
| shardy | flaper87: do we override the nic config for this job? | 07:04 |
| flaper87 | shardy: that's what I was trying to figure out | 07:04 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/net-config-multinode.yaml | 07:05 |
| shardy | Ok it's using that, which doesn't actually run os-net-config or write the data | 07:05 |
| mandre | shardy: we do not yet use the local registry | 07:05 |
| *** quique|rover|afk is now known as quiquell|rover | 07:05 | |
| mandre | we pull the needed containers, but the deployed openshift doesn't yet know how to use them | 07:06 |
| shardy | Ok so this seems like a possibly spurious outage trying to pull the container then I guess | 07:06 |
| shardy | I need to grab coffee before any further debugging, brb :) | 07:07 |
| *** jcoufal has joined #tripleo | 07:07 | |
| flaper87 | shardy: hahaha, go and get your coffee | 07:07 |
| flaper87 | if we don't use os-net-config in this job, then we need to revisit the nm dispatch script I wrote | 07:07 |
| flaper87 | as it depends on the os-net-config data being there | 07:07 |
| flaper87 | I'm not a huge fan of that script in general | 07:08 |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 07:10 |
| *** jlibosva has joined #tripleo | 07:12 | |
| *** amoralej|off is now known as amoralej | 07:12 | |
| *** pblaho has quit IRC | 07:13 | |
| *** tesseract has joined #tripleo | 07:16 | |
| *** jlibosva has quit IRC | 07:17 | |
| *** jlibosva has joined #tripleo | 07:18 | |
| *** dmacpher has quit IRC | 07:20 | |
| shardy | flaper87: there is a multinode-os-net-config version here: | 07:21 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/net-config-multinode-os-net-config.yaml#L60 | 07:21 |
| shardy | not sure which jobs use it but perhaps that's an option | 07:21 |
| *** gkadam has joined #tripleo | 07:22 | |
| flaper87 | I'll sync with EmilienM and weshay|ruck on this | 07:23 |
| *** pblaho has joined #tripleo | 07:23 | |
| *** rcernin has quit IRC | 07:27 | |
| *** jlibosva has quit IRC | 07:27 | |
| *** lifeless has quit IRC | 07:28 | |
| sri_ | shardy: Hi, quick question in overcloud can we put the provisioning nic in a bond | 07:29 |
| *** gfidente has joined #tripleo | 07:29 | |
| *** gfidente has quit IRC | 07:29 | |
| *** gfidente has joined #tripleo | 07:29 | |
| *** jpena|off is now known as jpena | 07:30 | |
| shardy | sri_: you can but you risk losing connectivity to the undercloud if the configuration goes wrong or takes too long | 07:30 |
| *** ffiore has joined #tripleo | 07:30 | |
| shardy | so it's not really reccomended | 07:30 |
| *** kopecmartin has quit IRC | 07:30 | |
| sri_ | shardy, got it, thanks :) | 07:31 |
| flaper87 | shardy: do you think it's ok for that script to depend on the os-net-config configs? Or should we attempt to have a fallback when those are not around? | 07:31 |
| flaper87 | I almost want to change the script to scan all the devices when os-net-config data is not present | 07:32 |
| flaper87 | get the list of unmanaged devices, and scan them. The risk is that we might end up generating a resolv.conf that doesn't use the right nic | 07:33 |
| *** lifeless has joined #tripleo | 07:35 | |
| *** cylopez has joined #tripleo | 07:36 | |
| *** cylopez has quit IRC | 07:36 | |
| *** nguyenhai has quit IRC | 07:37 | |
| *** nguyenhai has joined #tripleo | 07:37 | |
| *** cylopez has joined #tripleo | 07:37 | |
| *** tosky has joined #tripleo | 07:37 | |
| flaper87 | shardy: mandre running the job again | 07:37 |
| *** kopecmartin has joined #tripleo | 07:41 | |
| *** zoli is now known as zoli|wfh | 07:43 | |
| *** zoli|wfh is now known as zoli | 07:43 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Add overcloud-admin-user option to the cli https://review.openstack.org/573310 | 07:45 |
| *** flaper87 has quit IRC | 07:49 | |
| *** ksambor has quit IRC | 07:49 | |
| *** janki has quit IRC | 07:52 | |
| *** jpich has joined #tripleo | 07:57 | |
| openstackgerrit | Sylvain Afchain proposed openstack/tripleo-heat-templates master: skydive external_deploy_task migration https://review.openstack.org/561274 | 07:58 |
| *** flaper87 has joined #tripleo | 07:58 | |
| *** ksambor has joined #tripleo | 07:59 | |
| *** holser__ has joined #tripleo | 07:59 | |
| shardy | flaper87: Hey re os-net-config I think it's OK to depend on the data as in most non-CI cases I think it's expected to be there, but I do wonder if we could/should make it more explicit, e.g move creating the dispatcher script to the same place we run os-net-config in the nic config template or something | 07:59 |
| *** florianf has joined #tripleo | 08:00 | |
| shardy | obviously if there's another way to do it by inspecting the nics directly then that would be fine too | 08:00 |
| flaper87 | shardy: thing is, that dispatcher script is only useful in the OpenShift + OpenStack case | 08:00 |
| flaper87 | I'll put some thoughts on it while I wait for the new results to show up | 08:00 |
| shardy | flaper87: yes but we could have a custom addition to the nic config, e.g https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/net-config-multinode-os-net-config.yaml#L60 | 08:01 |
| shardy | we could have a special network_config_hook that's only defined for openshift or something | 08:02 |
| *** anande has quit IRC | 08:02 | |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/queens: Set docker environment files during ffu prepare and converge https://review.openstack.org/575232 | 08:02 |
| shardy | actually we might need a new post version of that hook but you get the idea :) | 08:02 |
| flaper87 | shardy: oh, ok, I see what you mean | 08:02 |
| *** athomas has joined #tripleo | 08:02 | |
| flaper87 | not super familiar with this code but I'll take a look | 08:03 |
| shardy | if we can just use net-config-multinode-os-net-config.yaml for this scenario then I guess we can just leave the existing approach in place | 08:03 |
| shardy | so maybe worth grepping to see if/where that's used first | 08:03 |
| *** yprokule_ has joined #tripleo | 08:03 | |
| *** yprokule has quit IRC | 08:04 | |
| *** yprokule_ is now known as yprokule | 08:04 | |
| flaper87 | shardy: I'm on it | 08:04 |
| *** janki has joined #tripleo | 08:09 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 08:10 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: [DNM] Add fs050 job to test from-release-upgrades without role changes https://review.openstack.org/575360 | 08:10 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: Update for openshift 3.9 https://review.openstack.org/574233 | 08:11 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: Use the multinode-os-net-config configs https://review.openstack.org/575361 | 08:11 |
| flaper87 | shardy: ^ let's see | 08:11 |
| flaper87 | mandre: ^ | 08:11 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-common stable/queens: DNM: test queens patch https://review.openstack.org/575244 | 08:15 |
| sshnaidm | quiquell|rover, is it right? ^^ | 08:15 |
| *** hamzy_ has joined #tripleo | 08:18 | |
| openstackgerrit | Sylvain Afchain proposed openstack/tripleo-heat-templates master: skydive external_deploy_task migration https://review.openstack.org/561274 | 08:18 |
| quiquell|rover | sshnaidm: Yep, let's wait for it. | 08:18 |
| *** ykarel is now known as ykarel|lunch | 08:19 | |
| *** hamzy has quit IRC | 08:20 | |
| *** owalsh has joined #tripleo | 08:21 | |
| *** anande has joined #tripleo | 08:26 | |
| *** jlibosva has joined #tripleo | 08:27 | |
| *** bdodd_ has joined #tripleo | 08:28 | |
| *** owalsh has quit IRC | 08:28 | |
| *** bdodd has quit IRC | 08:30 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Enable decoupled deployment plans e.g for OpenShift, DPDK etc https://review.openstack.org/574753 | 08:33 |
| *** pmannidi has quit IRC | 08:33 | |
| openstackgerrit | chen jiao proposed openstack/instack-undercloud master: incompatability spelling error https://review.openstack.org/575371 | 08:39 |
| *** mbu has joined #tripleo | 08:39 | |
| *** owalsh has joined #tripleo | 08:40 | |
| *** Distel has joined #tripleo | 08:43 | |
| bogdando | shardy, stevebaker, mwhahaha: PTAL https://review.openstack.org/#/c/572404/ | 08:43 |
| *** derekh has joined #tripleo | 08:45 | |
| *** ukalifon has joined #tripleo | 08:53 | |
| *** yamahata has quit IRC | 08:54 | |
| Distel | Hi all, I am getting an error when trying to install OSP13-beta(powered by director) with Tungsten fabric Contrail(version 5.0). i'm facing issues during the spawning of the node by nova, all the nodes end up with the failures message: "ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: Build of instance 0b81e619-2047-4a4a-8cb2-23c620d8a7ca aborted: Failure prepping block device., Code: 500" . | 08:56 |
| Distel | The same error appeas for role controller too.any helps would be appreciate. Thanks | 08:56 |
| *** radeks has quit IRC | 09:05 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-docs master: Add Fast Forward Upgrade docs https://review.openstack.org/549892 | 09:06 |
| *** salmankhan has joined #tripleo | 09:07 | |
| *** ykarel|lunch is now known as ykarel | 09:09 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 09:10 |
| *** jaosorior has joined #tripleo | 09:15 | |
| bogdando | EmilienM: rechecked https://review.openstack.org/#/c/571631/ as it is in the list of emergency fixes, I hope that's ok | 09:20 |
| *** anande has quit IRC | 09:21 | |
| shardy | bogdando: https://review.openstack.org/#/c/572404/8/releasenotes/notes/roles_file-8adea5990682504d.yaml says absolute or path relative to the t-h-t dir, but does this patch fix the issue where other relative paths don't work? | 09:23 |
| jistr | jlibosva: FYI regardless if the host NTP is the root cause or not, we should validate for it because we've seen the other issues caused by host desync earlier. So i at least reported a bug for now :D https://bugs.launchpad.net/tripleo/+bug/1776869 | 09:23 |
| openstack | Launchpad bug 1776869 in tripleo "[RFE] quickstart should validate that host machine is NTP synced" [Medium,Triaged] | 09:23 |
| shardy | I'm about to test it but wanted to check if the release note needs a clarification | 09:24 |
| shardy | since IMO the current behavior is a bug | 09:24 |
| *** anande has joined #tripleo | 09:24 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Set tuned profile for compute roles https://review.openstack.org/575380 | 09:24 |
| jlibosva | jistr: thanks, I didn't report a bug cause I suspected it's gonna be a pebkac | 09:24 |
| bogdando | shardy: no, it's limited to roles file only | 09:25 |
| bogdando | just aligns | 09:25 |
| *** pdeore_ has joined #tripleo | 09:30 | |
| *** Petersingh_ has joined #tripleo | 09:30 | |
| *** udesale_ has joined #tripleo | 09:30 | |
| *** ratailor_ has joined #tripleo | 09:30 | |
| *** gkadam_ has joined #tripleo | 09:30 | |
| *** shreshtha_ has joined #tripleo | 09:30 | |
| *** psahoo_ has joined #tripleo | 09:30 | |
| *** chkumar246 has joined #tripleo | 09:30 | |
| *** dtantsur|afk is now known as dtantsur | 09:31 | |
| *** Petersingh|afk has quit IRC | 09:32 | |
| *** cshastri_ has joined #tripleo | 09:32 | |
| *** links has quit IRC | 09:33 | |
| shardy | bogdando: that's what I mean -r ./myroles.yaml doesn't work because we don't do abspath - I'll rebase my fix on this if this doesn't handle that | 09:33 |
| *** shrjoshi has quit IRC | 09:33 | |
| *** ratailor has quit IRC | 09:33 | |
| *** links has joined #tripleo | 09:33 | |
| *** pdeore__ has joined #tripleo | 09:33 | |
| *** gkadam__ has joined #tripleo | 09:33 | |
| *** chandankumar has quit IRC | 09:33 | |
| *** Petersingh__ has joined #tripleo | 09:33 | |
| shardy | also we now have two utils functions that normalize the path, oops | 09:33 |
| *** udesale__ has joined #tripleo | 09:33 | |
| *** ratailor__ has joined #tripleo | 09:33 | |
| *** pdeore has quit IRC | 09:33 | |
| shardy | I'll clean that up in my rebase | 09:33 |
| *** psahoo has quit IRC | 09:33 | |
| *** shrjoshi has joined #tripleo | 09:34 | |
| *** gkadam has quit IRC | 09:34 | |
| *** udesale has quit IRC | 09:34 | |
| *** cshastri has quit IRC | 09:34 | |
| *** chkumar246 has quit IRC | 09:35 | |
| *** psahoo_ has quit IRC | 09:36 | |
| *** Petersingh_ has quit IRC | 09:36 | |
| *** shreshtha_ has quit IRC | 09:36 | |
| *** ratailor_ has quit IRC | 09:36 | |
| *** pdeore_ has quit IRC | 09:36 | |
| *** gkadam_ has quit IRC | 09:36 | |
| *** udesale_ has quit IRC | 09:36 | |
| *** Petersingh__ is now known as Petersingh | 09:37 | |
| *** Petersingh is now known as Petersingh|afk | 09:38 | |
| *** chandankumar has joined #tripleo | 09:39 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient master: Pass the correct client to delete_deployment_plan https://review.openstack.org/575384 | 09:46 |
| bogdando | shardy: ok, thanks! | 09:47 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient master: Pass the correct client to delete_deployment_plan https://review.openstack.org/575384 | 09:49 |
| d0ugal | jtomasek: ^ if you were curious. | 09:49 |
| *** psahoo_ has joined #tripleo | 09:49 | |
| openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient stable/pike: Add --stack to update, upgrade and ffwd-upgrade 'run' CLI. https://review.openstack.org/568273 | 09:49 |
| *** mkovacik__ has joined #tripleo | 09:51 | |
| *** mkovacik__ has left #tripleo | 09:51 | |
| *** bogdando has quit IRC | 09:52 | |
| *** threestrands has quit IRC | 09:54 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient master: Debug log the raw Zaqar message https://review.openstack.org/575385 | 09:57 |
| openstackgerrit | Daniel Alvarez proposed openstack/puppet-tripleo master: Adding wrapper script for haproxy in OVN metadata agent https://review.openstack.org/570942 | 10:00 |
| *** salmankhan has quit IRC | 10:07 | |
| *** psahoo_ has quit IRC | 10:08 | |
| *** Petersingh|afk is now known as Petersingh | 10:10 | |
| *** salmankhan has joined #tripleo | 10:10 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 10:10 |
| *** skramaja_ has joined #tripleo | 10:11 | |
| *** skramaja has quit IRC | 10:12 | |
| *** salmankhan has quit IRC | 10:13 | |
| *** salmankhan has joined #tripleo | 10:13 | |
| janki | jistr, jhey. you there? | 10:15 |
| anande | ping shardy, bnemec - while deploying overcloud,any idea I am seeing a "error parsing template while scanning for the next token found character '%' that cannot start any token" Openstack-Newton release | 10:20 |
| *** hjensas has joined #tripleo | 10:24 | |
| *** radeks has joined #tripleo | 10:26 | |
| *** Petersingh has quit IRC | 10:27 | |
| shardy | anande: Hi, sounds like you have an unquoted '%' character in your yaml somewhere? | 10:32 |
| shardy | anande: any clues e.g did it just start after creating some new templates or configuration? | 10:32 |
| shardy | anande: also worth checking the heat-engine logs on the undercloud to see what resource this relates to | 10:33 |
| shardy | (admittedly the error message should tell you that...) | 10:33 |
| anande | shardy, I have a j2.yaml that has the % declarations | 10:34 |
| Distel | Hi all, I am getting an error when trying to install OSP13-beta(powered by director) with Tungsten fabric Contrail(version 5.0). i'm facing issues during the spawning of the node by nova, all the nodes end up with the failures message: "ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: Build of instance 0b81e619-2047-4a4a-8cb2-23c620d8a7ca aborted: Failure prepping block device., Code: 500" | 10:34 |
| Distel | The same error appeas for role controller too.any idea? | 10:34 |
| shardy | anande: Ok can you check the rendered version, e.g mkdir tmp; cd tmp; openstack container save overcloud | 10:34 |
| shardy | then check the rendered .yaml file - sounds like a rendering mistake created some invalid yaml to me | 10:35 |
| anande | shardy, like this - https://paste.fedoraproject.org/paste/nOc2bsZDitpQtIi4t8~4JQ | 10:35 |
| anande | shardy, so I see a mixed usage of '{%' and '{%-' | 10:35 |
| anande | is that correct way of using it ? | 10:36 |
| shardy | anande: that just controls the newlines, check the j2 docs to see how it works and/or check the rendered template | 10:36 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: QDR for metrics collection purposes https://review.openstack.org/572312 | 10:36 |
| shardy | can you share the rendered template so we can ensure the j2 rendering is OK? | 10:36 |
| anande | shardy - in my case the overcloud container itself has not been created - so cant get the rendered template | 10:37 |
| shardy | anande: that sounds wrong, it should create the plan before attempting to validate anything | 10:37 |
| shardy | anande: you can also try using tools/process-templates.py | 10:37 |
| jistr | janki: hi, yes | 10:38 |
| * jistr on review spree | 10:38 | |
| janki | jistr, that restart patch worked!! | 10:39 |
| jistr | janki: amazing!!! | 10:39 |
| anande | shardy, I think those containers have been named according to the custom --stack name I have passed | 10:39 |
| jistr | folks can we please get reviews on ODL update patch https://review.openstack.org/#/c/575111/ | 10:39 |
| jistr | tested ^^^ | 10:39 |
| shardy | anande: yes that's expected | 10:39 |
| anande | downloading them now | 10:39 |
| shardy | I was assuming the default | 10:40 |
| anande | to share with you the rendered template | 10:40 |
| shardy | anande: in the resource_registry you are referencing the rendered .yaml right? | 10:40 |
| shardy | not the ,j2.yaml | 10:40 |
| shardy | I think I missed a question from you about that yesterday | 10:40 |
| shardy | you were gone when I saw it | 10:41 |
| *** suuuper has joined #tripleo | 10:47 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Enable decoupled deployment plans e.g for OpenShift, DPDK etc https://review.openstack.org/574753 | 10:49 |
| *** dtantsur is now known as dtantsur|brb | 10:50 | |
| anande | shardy, the original j2.yaml : https://gist.github.com/anande/2b55dc7c65428f2e5bbc63ce5488a876 | 10:53 |
| anande | the rendered version : https://gist.github.com/anande/f005ecde5fcce40e4d903e0b8bcc8a0f | 10:53 |
| shardy | anande: ack, looks fine, can you share the resource_registry you're using to include the template please? | 10:54 |
| *** anande has quit IRC | 10:55 | |
| *** anande has joined #tripleo | 10:56 | |
| anande | shardy, | 10:56 |
| anande | resource_registry: | 10:56 |
| anande | OS::TripleO::AllNodesExtraConfig: /home/stack/custom-templates/Wrk_neutron-ml2-cisco-nexus-ucsm.j2.yaml | 10:56 |
| *** jpena is now known as jpena|lunch | 10:57 | |
| shardy | anande: it should be OS::TripleO::AllNodesExtraConfig: /home/stack/custom-templates/Wrk_neutron-ml2-cisco-nexus-ucsm.yaml | 10:58 |
| shardy | the j2.yaml is rendered before we pass the environment to heat | 10:58 |
| anande | shardy, you mean the rendered one ? | 10:59 |
| shardy | anande: yes | 10:59 |
| shardy | heat doesn't know what to do with j2 templates | 10:59 |
| anande | so never am I to use the j2.yaml anywhere | 10:59 |
| shardy | anande: exactly | 10:59 |
| shardy | it's a pre-processing step done by tripleo | 10:59 |
| shardy | probably there's an opportunity to add a validation which fails more clearly | 11:00 |
| *** chem has quit IRC | 11:00 | |
| *** pdeore__ has quit IRC | 11:00 | |
| anande | shardy, ok so the takeaway is - I shall have the j2.yaml just in my /usr/share... location and point my resource_registry to the rendered one. The rendering will always take care that the name will remain same except that j2 will be removd from the filename | 11:01 |
| shardy | anande: yes that's right | 11:01 |
| shardy | anande: also you cannot render j2 templates outside of the --templates path | 11:01 |
| shardy | so that can be /usr/share/openstack-tripleo-heat-templates, or a copy of that trree | 11:02 |
| shardy | tree | 11:02 |
| shardy | but you can't copy one file and modify it | 11:02 |
| shardy | if it's a j2.yaml | 11:02 |
| anande | shardy, awesome - thanks , trying that out now! yea I believe a better failure message like 'j2.yaml cannot be parsed by tripleO' or some better text there | 11:04 |
| *** pkovar has joined #tripleo | 11:04 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use UpgradeInitCommonCommand with deployed-server too https://review.openstack.org/574709 | 11:05 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-upgrade stable/queens: Add timestamps to logs of upgrade tasks https://review.openstack.org/575402 | 11:05 |
| *** chem has joined #tripleo | 11:05 | |
| anande | shardy, shall I also have '/home/stack/custom-templates/neutron-ml2-cisco-nexus-ucsm.yaml' whose resource registry points to the rendered file in the default /usr/share location? placed in same folder as the rendered file | 11:07 |
| shardy | anande: is your neutron-ml2-cisco-nexus-ucsm.j2.yaml different to the one in /usr/share? | 11:08 |
| anande | shardy, no - its the same | 11:08 |
| *** zoli is now known as zoli|lunch | 11:09 | |
| shardy | anande: ack, so unless you're copying the tree you could probably do -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ml2-cisco-nexus-ucsm.yaml | 11:10 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 11:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| *** chem` has joined #tripleo | 11:13 | |
| *** chem has quit IRC | 11:13 | |
| *** chem` is now known as chem | 11:13 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Handle renaming of Cinder VNX backend parameter https://review.openstack.org/574873 | 11:14 |
| openstackgerrit | Merged openstack/puppet-tripleo master: HA overcloud: wait for db before running grant commands https://review.openstack.org/574732 | 11:14 |
| *** pradk has joined #tripleo | 11:17 | |
| anande | shardy, stack create is progressing now :) thanks | 11:19 |
| *** anande is now known as anande|brb | 11:20 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-upgrade stable/pike: Add log artifacts for upgrade steps https://review.openstack.org/574969 | 11:21 |
| *** chem` has joined #tripleo | 11:22 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Handle renaming of Cinder VNX backend parameter https://review.openstack.org/575114 | 11:24 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Run ovb (fs035) with a containerized undercloud https://review.openstack.org/566916 | 11:24 |
| *** chem has quit IRC | 11:26 | |
| *** ansmith has quit IRC | 11:26 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Fix ansible inventory clean up https://review.openstack.org/575407 | 11:28 |
| *** anande|brb has quit IRC | 11:28 | |
| *** sanjayu__ has joined #tripleo | 11:29 | |
| *** ykarel_ has joined #tripleo | 11:31 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Upgrades: Refactor playbooks to set facts https://review.openstack.org/562294 | 11:31 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Fix docker images validation script https://review.openstack.org/574345 | 11:31 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Updates: run host_prep_tasks between update tasks and deployment tasks https://review.openstack.org/574236 | 11:32 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Fix overcloud delete command https://review.openstack.org/574623 | 11:32 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix error in the role-specific parameter merging in rhsm service https://review.openstack.org/574987 | 11:32 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Add timestamps to update log files https://review.openstack.org/574886 | 11:32 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Add timestamps to logs of upgrade tasks https://review.openstack.org/574892 | 11:32 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Enable secure TUNNELLED mode for NFS https://review.openstack.org/574675 | 11:32 |
| *** sanjayu_ has quit IRC | 11:32 | |
| *** ykarel has quit IRC | 11:34 | |
| *** ykarel_ is now known as ykarel | 11:34 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Make sure memcached log dir permission survive reboot. https://review.openstack.org/575409 | 11:34 |
| chem` | jistr: so for the memcached issue I endup doing this ^ ... but didn't get to the bottom of it really | 11:35 |
| *** chem` is now known as chem | 11:35 | |
| chem | jistr: would need some serious docker specialist here I guess | 11:36 |
| jistr | chem: did we manage to reproduce it? | 11:36 |
| chem | jistr: I have a failed env running | 11:37 |
| chem | jistr: only one memcached failed there | 11:37 |
| chem | jistr: in the lp you have a complete description of what I'm seeing ... it's hairy | 11:37 |
| jistr | chem: i'm afraid the patch will not work as intended -- memcached writes to /var/log/memcached.log AFAIK, so if we mount /var/log/memcached instead of /var/log, the log wouldn't be in the mount | 11:38 |
| chem | jistr: explained why it should in the comment | 11:38 |
| jistr | ack sorry reading more carefully now | 11:38 |
| chem | jistr: and it's currently working in this configuration (even though it's writting in ".log" not in "log.1" or whatever) | 11:39 |
| chem | jistr: anyway ci will tell and that may unlock one of the issue of the bz | 11:39 |
| *** udesale_ has joined #tripleo | 11:42 | |
| *** udesale__ has quit IRC | 11:45 | |
| *** dtantsur|brb is now known as dtantsur | 11:46 | |
| *** anande|brb has joined #tripleo | 11:47 | |
| *** shrjoshi has quit IRC | 11:47 | |
| *** morazi has joined #tripleo | 11:48 | |
| *** udesale_ has quit IRC | 11:48 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Set tuned profile for compute roles https://review.openstack.org/575380 | 11:49 |
| *** morazi_ has joined #tripleo | 11:52 | |
| *** gfidente has quit IRC | 11:53 | |
| *** mbu has quit IRC | 11:54 | |
| *** morazi has quit IRC | 11:55 | |
| *** gfidente has joined #tripleo | 11:56 | |
| *** rlandy has joined #tripleo | 12:01 | |
| * jistr moving discussion here | 12:02 | |
| jistr | the patch looks valid, i'm only concerned by the 2nd redirect in $OPTIONS | 12:02 |
| jistr | and still looking at what could possibly be doing the rotation | 12:02 |
| *** jpena|lunch is now known as jpena | 12:03 | |
| jistr | jaosorior: hi, can you please look at this patch? https://review.openstack.org/#/c/575409 | 12:03 |
| *** amoralej is now known as amoralej|lunch | 12:03 | |
| *** atoth has joined #tripleo | 12:04 | |
| jistr | jaosorior: there seems to be log rotation happening for the memcached log (moving it, not copytruncate). If we restart the memcached container after that, if fails to come back up because it doesn't have permissions to create a fresh log file. | 12:04 |
| *** anande|brb is now known as anande | 12:04 | |
| anande | shardy thanks a tonne - overcloud is now deployed - great learning !! | 12:04 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Store ODL logs to file https://review.openstack.org/575416 | 12:05 |
| *** raildo has joined #tripleo | 12:05 | |
| *** ratailor__ has quit IRC | 12:05 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Set tuned profile for compute roles https://review.openstack.org/575380 | 12:08 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 12:10 |
| *** Haresh has joined #tripleo | 12:10 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Create an additional network interface on subnode https://review.openstack.org/572732 | 12:12 |
| *** toure|brb is now known as toure | 12:16 | |
| *** anande has quit IRC | 12:18 | |
| *** anande has joined #tripleo | 12:18 | |
| beagles | is anyone else seeing messages like "dockerd-current: time="2018-06-14T12:17:53.548576910Z" level=warning msg="Health check for container f0004e2e46193f5a9d7fcaabda6cdf04174438d92faea156280a6d6c1ab0ad01 error: No such exec instance '6791aa50d4db7a23fdfd78fd7b153d41aec29f5c0c747b186483b2c9224c2616' found in daemon" | 12:20 |
| beagles | bascially spamming /var/log/messages | 12:20 |
| weshay|ruck | tdasilva, arxcruz hola :) | 12:22 |
| tdasilva | arxcruz: does tempest use overcloudrc? | 12:24 |
| arxcruz | tdasilva: yup | 12:24 |
| tosky | arxcruz, tdasilva: tempest or tempestconf? | 12:24 |
| arxcruz | tosky: tempest in /etc/tempest.conf | 12:25 |
| arxcruz | tosky: tdasilva sorry, so, python-tempestconf uses overcloudrc to generate the tempest.conf who has the info required by tempest to run | 12:25 |
| tosky | yep | 12:25 |
| *** quiquell|rover is now known as quiquell|lunch | 12:26 | |
| *** tzumainn has joined #tripleo | 12:26 | |
| shardy | anande: good news! :) | 12:26 |
| *** trown|outtypewww is now known as trown | 12:27 | |
| tdasilva | arxcruz: i'm curious about this: operator_role = Member | 12:27 |
| *** abishop has joined #tripleo | 12:28 | |
| tdasilva | arxcruz: what's curious is that from cli if I source overcloudrc I can create containers and upload objects just fine | 12:28 |
| tdasilva | but tempest fails exactly on creating containers | 12:28 |
| arxcruz | tdasilva: with permission problem | 12:29 |
| tdasilva | right | 12:30 |
| *** ksambor has quit IRC | 12:31 | |
| *** lblanchard has joined #tripleo | 12:31 | |
| *** leanderthal has joined #tripleo | 12:32 | |
| *** ksambor has joined #tripleo | 12:32 | |
| *** zoli|lunch is now known as zoli|wfh | 12:35 | |
| *** zoli|wfh is now known as zoli | 12:35 | |
| *** myoung|off is now known as myoung | 12:36 | |
| tdasilva | arxcruz: could we try changing that operator_role to admin? | 12:37 |
| arxcruz | tdasilva: sure | 12:37 |
| *** liverpooler has joined #tripleo | 12:37 | |
| *** aufi has joined #tripleo | 12:41 | |
| *** paramite has quit IRC | 12:43 | |
| openstackgerrit | Merged openstack/tripleo-ha-utils master: Split heat stack creation in modules https://review.openstack.org/574791 | 12:43 |
| *** leanderthal has quit IRC | 12:48 | |
| *** lifeless has quit IRC | 12:48 | |
| *** lifeless has joined #tripleo | 12:49 | |
| *** gfidente has quit IRC | 12:49 | |
| *** gfidente has joined #tripleo | 12:51 | |
| *** gfidente has quit IRC | 12:51 | |
| *** gfidente has joined #tripleo | 12:51 | |
| *** quiquell|lunch is now known as quiquell | 12:52 | |
| *** quiquell is now known as quiquell|rover | 12:52 | |
| *** cshastri_ has quit IRC | 12:53 | |
| bfournie | hi beagles: could you +w https://review.openstack.org/#/c/575220/? | 12:53 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Fix ansible inventory clean up https://review.openstack.org/575407 | 12:54 |
| *** edmondsw has joined #tripleo | 12:54 | |
| *** florianf has quit IRC | 12:55 | |
| *** lblanchard has quit IRC | 12:55 | |
| *** tahini has quit IRC | 12:56 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates stable/queens: Use UpgradeInitCommonCommand with deployed-server too https://review.openstack.org/575424 | 12:57 |
| *** tahini has joined #tripleo | 12:57 | |
| *** florianf has joined #tripleo | 12:58 | |
| *** ansmith has joined #tripleo | 12:59 | |
| *** leanderthal has joined #tripleo | 13:00 | |
| *** sanjayu_ has joined #tripleo | 13:01 | |
| *** brault has quit IRC | 13:02 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: [DNM] Fix ansible inventory clean up https://review.openstack.org/575407 | 13:03 |
| *** brault has joined #tripleo | 13:04 | |
| *** sanjayu__ has quit IRC | 13:04 | |
| flaper87 | shardy: http://logs.openstack.org/33/574233/8/experimental/tripleo-ci-centos-7-scenario009-multinode-oooq/22f2687/logs/undercloud/var/lib/mistral/config-download-latest/openshift/playbook.log.txt.gz | 13:07 |
| flaper87 | :/ | 13:07 |
| shardy | flaper87: urgh, does it work locally? | 13:09 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 13:10 |
| flaper87 | shardy: it does | 13:10 |
| flaper87 | shardy: os-net-config is there now, I switched to the multinode-os-net-config env http://logs.openstack.org/33/574233/8/experimental/tripleo-ci-centos-7-scenario009-multinode-oooq/22f2687/logs/subnode-2/etc/os-net-config/config.json.txt.gz | 13:12 |
| *** cdearborn has joined #tripleo | 13:13 | |
| *** holser__ has quit IRC | 13:18 | |
| *** holser___ has joined #tripleo | 13:18 | |
| *** holser___ has quit IRC | 13:19 | |
| *** holser__ has joined #tripleo | 13:20 | |
| *** Goneri has joined #tripleo | 13:21 | |
| *** dhill_ has joined #tripleo | 13:22 | |
| alee_ | d0ugal, rbrady ping | 13:22 |
| alee_ | toure, ping | 13:23 |
| rbrady | alee_: hey | 13:23 |
| alee_ | hey :) | 13:23 |
| alee_ | so looking again at https://review.openstack.org/#/c/574782/2 | 13:23 |
| * toure looks | 13:24 | |
| alee_ | I rebuilt my machine and it works | 13:24 |
| toure | alee_ that is good | 13:24 |
| alee_ | that is the passwords are in fact persisted | 13:24 |
| toure | did you see my comments | 13:24 |
| *** matbu has quit IRC | 13:24 | |
| alee_ | I did the following 1) pip install the new tripleo-common 2) mistral-dbmanage populate 3) restart mistral | 13:25 |
| alee_ | even then I could not see my new workflows and actions though | 13:25 |
| alee_ | so I needed to update the workbook as follows -- openstack workbook update <path to plan workbook> | 13:26 |
| alee_ | that gave me the new workflows there , but I still dont see the new action ReplacePasswords | 13:26 |
| toure | ah yes you have to provide absolute path | 13:26 |
| alee_ | how do I get that? | 13:26 |
| alee_ | toure, I responded to your comment | 13:27 |
| toure | yeah reading it now | 13:27 |
| toure | alee_ what do you mean you don't see the new action | 13:27 |
| *** agopi has joined #tripleo | 13:27 | |
| abishop | alee_: I'll PM you the commands I run to install/update mistral actions | 13:28 |
| alee_ | rbrady, d0ugal , toure I added a new action ReplacePasswordAction | 13:28 |
| alee_ | abishop, thanks! | 13:28 |
| alee_ | I think I'm missing one command .. | 13:28 |
| abishop | alee_: posted here: http://paste.openstack.org/show/723476/ | 13:29 |
| alee_ | abishop, looking | 13:29 |
| *** amoralej|lunch is now known as amoralej | 13:29 | |
| *** salmankhan1 has joined #tripleo | 13:29 | |
| *** salmankhan has quit IRC | 13:30 | |
| *** salmankhan1 is now known as salmankhan | 13:30 | |
| alee_ | abishop, interesting -- I think I did most of that , but let me give it a shot in the order you did it | 13:30 |
| alee_ | abishop, maybe completely removing the /usr/lib/python2.7/site-packages/tripleo_common* triggers something | 13:31 |
| *** matbu has joined #tripleo | 13:31 | |
| EmilienM | hi all | 13:31 |
| EmilienM | sounds like things are merging again | 13:31 |
| EmilienM | the tripleo gate is empty | 13:32 |
| EmilienM | so I'm going ahead and restoring/approving the things that moved out from the gate | 13:32 |
| ykarel | nice | 13:33 |
| *** moshele has quit IRC | 13:33 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Use the in-built image prepare for undercloud install https://review.openstack.org/571613 | 13:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Use role tripleo-modify-image to update containers https://review.openstack.org/571631 | 13:35 |
| alee_ | abishop, yay - that worked - thanks! | 13:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Switch to workflow driven image prepare https://review.openstack.org/573476 | 13:35 |
| * alee_ filing away for later use | 13:35 | |
| *** EmilienM changes topic to "elcome to Rocky. CI status: GREENish - recheck/+A ok now | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest" | 13:35 | |
| EmilienM | @all recheck / +A is ok now, thanks everyone | 13:36 |
| *** paramite has joined #tripleo | 13:36 | |
| EmilienM | flaper87: anytime, let me know what's up | 13:36 |
| *** moshele has joined #tripleo | 13:36 | |
| *** mwhahaha changes topic to "Welcome to Rocky. CI status: GREENish - recheck/+A ok now | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest" | 13:38 | |
| abishop | alee_: yup, it's from own previously files notes ;-) | 13:38 |
| bfournie | EmilienM: while you are restoring can you +A https://review.openstack.org/#/c/575220/, even though you didn't move this one? its a blocker… thanks | 13:40 |
| EmilienM | bfournie: ok | 13:40 |
| *** florianf has quit IRC | 13:40 | |
| bfournie | EmilienM: thank you! | 13:41 |
| *** florianf has joined #tripleo | 13:42 | |
| openstackgerrit | Bob Fournier proposed openstack/os-net-config stable/queens: Restore the order of params in ifcfg file that was inadvertently changed https://review.openstack.org/575432 | 13:42 |
| *** toure is now known as toure|biab | 13:43 | |
| alee_ | rbrady, abishop , d0ugal , toure|biab - ok - so all my new actions work :) unfortunately , they also log the results in the executor.log -- is there a way to flag the result as sensitive? | 13:45 |
| *** pliu_ has quit IRC | 13:45 | |
| alee_ | ie. dont put passwords in my log! | 13:45 |
| *** lblanchard has joined #tripleo | 13:46 | |
| *** lblanchard has quit IRC | 13:48 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: [DNM] Fix ansible inventory clean up https://review.openstack.org/575407 | 13:48 |
| abishop | alee_: whoops >_< | 13:48 |
| abishop | alee_: sorry, I don't have answer, hopefully d0ugal can help | 13:49 |
| *** bogdando has joined #tripleo | 13:49 | |
| mwhahaha | weshay|ruck, quiquell|rover, EmilienM: CI is no appears to no longer be running on tripleo-docs | 13:50 |
| *** moshele has quit IRC | 13:50 | |
| weshay|ruck | fun | 13:50 |
| mwhahaha | that was terrible | 13:50 |
| mwhahaha | it appears taht ci is no longer running on tripleo-docs | 13:50 |
| *** ccamacho has quit IRC | 13:51 | |
| *** leanderthal has quit IRC | 13:52 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: [DNM] Fix ansible inventory clean up https://review.openstack.org/575407 | 13:53 |
| EmilienM | should not be hard to debug | 13:54 |
| *** anande has quit IRC | 13:54 | |
| EmilienM | last merge was 2 days ago | 13:55 |
| *** toure|biab is now known as toure | 13:56 | |
| *** leanderthal has joined #tripleo | 13:57 | |
| toure | alee_ that has been a tricky topic which thrash has been working on | 13:57 |
| toure | alee_ https://specs.openstack.org/openstack/mistral-specs/specs/pike/approved/secure-sensitive-data.html | 13:58 |
| toure | there are a few chicken or egg problem which need to be solved in mistral to allow this to work | 13:58 |
| *** shardy has quit IRC | 14:01 | |
| *** chem has quit IRC | 14:01 | |
| *** chem has joined #tripleo | 14:01 | |
| *** rajinir has joined #tripleo | 14:01 | |
| *** brault has quit IRC | 14:03 | |
| d0ugal | alee_: yeah, there isn't really a good solution | 14:03 |
| *** alee_ has quit IRC | 14:03 | |
| *** jcoufal has quit IRC | 14:04 | |
| *** brault has joined #tripleo | 14:04 | |
| *** ykarel is now known as ykarel|away | 14:04 | |
| fultonj | in oooq if --config foo.yaml contains an extra_args: bar.yaml, then would you expect the PREPARE_ARGS in overcloud-prep-containers.sh to have that extra env file? | 14:04 |
| beagles | bandini: dumb question - does updating reboot nodes? | 14:06 |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates stable/queens: Default environment/services/* to docker https://review.openstack.org/567800 | 14:06 |
| bandini | beagles: no. manual intervention is required, we don't do it automagically | 14:06 |
| *** ksambor has quit IRC | 14:07 | |
| beagles | bandini: ok whew - that's what I thought | 14:07 |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates master: Adds docker SFC and cleans up ODL envs https://review.openstack.org/570609 | 14:07 |
| beagles | bandini: thanks | 14:08 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo master: Adding wrapper script for haproxy in OVN metadata agent https://review.openstack.org/570942 | 14:08 |
| *** skramaja_ has quit IRC | 14:10 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 14:10 |
| dtantsur | hi folks, could you merge these two patches https://review.openstack.org/#/c/541570/ and https://review.openstack.org/#/c/541565/ | 14:14 |
| dtantsur | these add a new driver | 14:14 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: [DNM] Fix ansible inventory clean up https://review.openstack.org/575407 | 14:15 |
| *** links has quit IRC | 14:15 | |
| *** morazi_ has quit IRC | 14:15 | |
| *** mcornea has joined #tripleo | 14:18 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Fix ansible inventory clean up https://review.openstack.org/575407 | 14:19 |
| *** leanderthal has quit IRC | 14:19 | |
| *** leanderthal has joined #tripleo | 14:20 | |
| *** morazi has joined #tripleo | 14:24 | |
| flaper87 | EmilienM: hey, so, I noticed that scenario009 was not using os-net-config so I went ahead and changed the net config to use multinode-os-net-config: https://review.openstack.org/#/c/574233/ and, more specifically, https://review.openstack.org/#/c/575361/1 | 14:26 |
| flaper87 | does that look good so far? | 14:26 |
| *** quiquell|rover is now known as quiquell|off | 14:26 | |
| flaper87 | EmilienM: my current prob is the last 2 runs of https://review.openstack.org/#/c/574233/ have failed due to lack of network and the node not being able to pull the container | 14:26 |
| *** owalsh has quit IRC | 14:29 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Switch scenario00{1,2,3,4,7} to containerized undercloud"" https://review.openstack.org/575330 | 14:30 |
| *** owalsh has joined #tripleo | 14:35 | |
| *** alee_ has joined #tripleo | 14:35 | |
| *** ramishra has quit IRC | 14:37 | |
| alee_ | toure, cool thanks - good to know | 14:38 |
| EmilienM | flaper87: I believe you want to use common/net-config-multinode.yaml as other jobs, isn't? | 14:39 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Make sure memcached log permission survive log rotation. https://review.openstack.org/575409 | 14:40 |
| EmilienM | flaper87: common/net-config-multinode.yaml is the one we use for all other multinode jobs, afik | 14:40 |
| *** alee__ has joined #tripleo | 14:40 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Introduces NovaComputeCpuSharedSet parameter https://review.openstack.org/575449 | 14:41 |
| flaper87 | EmilienM: I was using that one but I don't think os-net-config is used with net-config-multinode.yaml | 14:41 |
| *** dtrainor has joined #tripleo | 14:42 | |
| *** alee_ has quit IRC | 14:43 | |
| *** hamzy_ is now known as hamzy | 14:44 | |
| EmilienM | flaper87: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/net-config-multinode.yaml#L46-L65 | 14:44 |
| EmilienM | I think it does | 14:44 |
| *** ksambor has joined #tripleo | 14:45 | |
| bogdando | EmilienM, flaper87: hm, that doesn't look like os-net-config data | 14:45 |
| bogdando | just a command | 14:45 |
| florianf | EmilienM, marios: since both of you +2'd this patch, can one of you just quickly hit "approve"? :) https://review.openstack.org/#/c/564254/ | 14:46 |
| florianf | "Workflow +1" is what I mean... ;) ^ | 14:46 |
| openstackgerrit | Ade Lee proposed openstack/tripleo-common master: Add mistral job to rotate passwords on the overcloud https://review.openstack.org/574782 | 14:46 |
| EmilienM | bogdando: it's configuring a bridge and attaching an interface to the bridge right? | 14:47 |
| alee__ | toure, rbrady , d0ugal , jaosorior , mwhahaha, EmilienM ^^ please review | 14:47 |
| bogdando | via a command, not /etc/os-net-config | 14:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Revert "Revert "Pass connection info via ansible config file"" https://review.openstack.org/568781 | 14:48 |
| toure | alee__ I will take a look | 14:48 |
| d0ugal | alee__: will do | 14:49 |
| bogdando | EmilienM: just to compare http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/common/net-config-multinode-os-net-config.yaml#n92 that is the os net config data | 14:49 |
| alee__ | thanks | 14:50 |
| *** moshele has joined #tripleo | 14:51 | |
| EmilienM | flaper87: do you have logs of your networking failure? | 14:51 |
| *** rpioso|afk is now known as rpioso | 14:54 | |
| flaper87 | EmilienM: http://logs.openstack.org/33/574233/8/experimental/tripleo-ci-centos-7-scenario009-multinode-oooq/22f2687/logs/undercloud/var/lib/mistral/config-download-latest/openshift/playbook.log.txt.gz | 14:55 |
| flaper87 | EmilienM: just that for now, it fails to download the container | 14:55 |
| flaper87 | I've rechecked and requested an autohold on those nodes | 14:55 |
| florianf | EmilienM: thanks! | 14:55 |
| flaper87 | we'll be able to debug on the node in a bit | 14:55 |
| EmilienM | https://registry.fedoraproject.org/v1/_ping: dial tcp [2607:f188::dead:beef:cafe:fed1]:443: connect: network is unreachable" | 14:56 |
| EmilienM | I'm wondering if really os-net-config | 14:56 |
| EmilienM | to me it looks like https://registry.fedoraproject.org/v1/ is unreachable | 14:57 |
| EmilienM | flaper87: could you reproduce consistently? | 14:57 |
| flaper87 | let me back up 1 sec | 14:58 |
| flaper87 | I do not think os-net-config is the problem | 14:58 |
| flaper87 | I noticed this job was not using os-net-config (basedo n the fact that os-net-config data was not in the subnode-2 etc dir) and I need that to be there as some of the OpenShift steps depend on that data | 14:59 |
| flaper87 | so, I switched the network config to the one I mentioned | 14:59 |
| flaper87 | first question is: Does that make sense? Or did I misinterpreted the previous config? | 15:00 |
| flaper87 | Second thing is that the last attempt to execute this job failed because there was no network available and I've no idea whether it was related to the change I did or not. I've re-ran the check now and I'm waiting for the results | 15:01 |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Delete messages container on plan deletion https://review.openstack.org/566247 | 15:01 |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Stop assuming all containers are plans https://review.openstack.org/566345 | 15:01 |
| EmilienM | flaper87: probably. I'm not sure. I looked on a random job: http://logs.openstack.org/16/575416/1/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/d4539bf/logs/subnode-2/etc/ and there is no os-net-config on subnode-2 right? | 15:02 |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-upgrade stable/queens: Fix docker images validation script https://review.openstack.org/575461 | 15:03 |
| *** yprokule has quit IRC | 15:04 | |
| jrist | hey guys | 15:05 |
| jrist | whenever anyone gets a chance | 15:05 |
| jrist | can we get some eyes on this | 15:05 |
| jrist | https://review.openstack.org/#/c/498936 | 15:05 |
| jrist | it's a cleaner way of doing node operations | 15:05 |
| jrist | key for the UI | 15:05 |
| *** sanjayu_ has quit IRC | 15:06 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-heat-templates master: Adds support to configure enabled and default bios interfaces https://review.openstack.org/575462 | 15:07 |
| *** dxiri has joined #tripleo | 15:09 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 15:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| *** radeks_ has joined #tripleo | 15:14 | |
| *** radeks has quit IRC | 15:14 | |
| arxcruz | weshay|ruck: tdasilva so, found the problem, actually two problems, one related to swift, one related to security groups | 15:17 |
| arxcruz | in swift, on swift proxy, as tdasilva show me, the roles allowed are admin, ResellerAdmin and swiftoperator, on tempest is setting to Member | 15:17 |
| weshay|ruck | arxcruz++ | 15:18 |
| weshay|ruck | tdasilva++ | 15:18 |
| weshay|ruck | thanks guys!! | 15:18 |
| arxcruz | before was SwiftOperator | 15:18 |
| arxcruz | weshay|ruck: i found the problem, not the solution :D | 15:18 |
| arxcruz | after change it to swiftoperator, it creates the container, but it's not able to delete the resources in security groups | 15:18 |
| arxcruz | so, i can confirm the problem isn't with swift | 15:18 |
| *** salmankhan has quit IRC | 15:18 | |
| arxcruz | weshay|ruck: tdasilva on tearDown it's getting tempest.lib.exceptions.UnexpectedResponseCode: Unexpected response code received Details: 503 | 15:19 |
| arxcruz | calling this https://github.com/openstack/tempest/blob/master/tempest/lib/services/network/security_groups_client.py#L61 | 15:19 |
| *** dmacpher has joined #tripleo | 15:22 | |
| bogdando | mwhahaha, stevebaker: PTAL https://review.openstack.org/#/c/572404/ | 15:24 |
| *** pcaruana has quit IRC | 15:24 | |
| *** holser__ has quit IRC | 15:27 | |
| *** dparkes has quit IRC | 15:28 | |
| flaper87 | EmilienM: yes, it's not there, that's my point :D | 15:29 |
| *** zaneb has quit IRC | 15:30 | |
| *** ykarel_ has joined #tripleo | 15:31 | |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: Use the multinode-os-net-config configs https://review.openstack.org/575361 | 15:31 |
| EmilienM | flaper87: let me look after tc office hrs | 15:32 |
| *** Haresh has quit IRC | 15:32 | |
| tdasilva | arxcruz: what's the swift request that returns the 503? | 15:32 |
| *** ykarel|away has quit IRC | 15:34 | |
| *** zaneb has joined #tripleo | 15:34 | |
| *** moshele has quit IRC | 15:35 | |
| *** morazi has quit IRC | 15:36 | |
| arxcruz | tdasilva: not the swift request that is returning 503 | 15:38 |
| arxcruz | it's the call to https://github.com/openstack/tempest/blob/master/tempest/lib/services/network/security_groups_client.py#L61 | 15:38 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-heat-templates master: Adds support to configure enabled and default bios interfaces https://review.openstack.org/575462 | 15:41 |
| *** quiquell|off is now known as quiquell|rover | 15:41 | |
| *** jlibosva has quit IRC | 15:42 | |
| *** bogdando has quit IRC | 15:42 | |
| zshi_ | dtantsur, hi, Re BIOS config support in tripleo, do you know if it requires integrating cleaning step with mistral flow? | 15:43 |
| *** zshi_ is now known as zshi | 15:43 | |
| *** ukalifon has quit IRC | 15:43 | |
| dtantsur | zshi: only if you plan to use it in the undercloud | 15:44 |
| zshi | dtantsur, you mean config undercloud BIOS or use it in undercloud to configure oc ? | 15:44 |
| *** holser__ has joined #tripleo | 15:45 | |
| dtantsur | zshi: the latter | 15:47 |
| beagles | oh the heartbreak of running out of space on your SSD while doing update testing | 15:48 |
| dtantsur | zshi: but as the very minimum, we need a way to configure enabled_bios_interfaces via puppet | 15:48 |
| zshi | dtantsur, yes, yolanda is working on enable bios interface config via puppet : https://review.openstack.org/#/c/575438/ and https://review.openstack.org/#/c/575462/ | 15:49 |
| *** morazi has joined #tripleo | 15:49 | |
| dtantsur | cool! | 15:49 |
| zshi | dtantsur, I think it'd be natural choice for user to config BIOS in undercloud ? or are we expecting they do the BIOS config outside of UC | 15:50 |
| dtantsur | no idea, this is my question to you :) | 15:51 |
| *** quiquell|rover has quit IRC | 15:51 | |
| dtantsur | also sorry, I really have to run right now | 15:51 |
| *** quiquell has joined #tripleo | 15:51 | |
| *** dtantsur is now known as dtantsur|afk | 15:51 | |
| zshi | dtantsur, thanks! | 15:51 |
| yolanda | zshi, i think that this will be the primary case yes... they'd want to configure bios settings on the undercloud, to configure the oc nodes | 15:52 |
| yolanda | they can do manually anyway | 15:52 |
| zshi | yolanda, I think so if they use tripleo, it will be configured via undercloud | 15:52 |
| yolanda | then we better add it... | 15:52 |
| zshi | yolanda, gotcha, so it's better to have, concerning there is no drivers that support BIOS config, so it's not hurry :-) | 15:53 |
| arxcruz | weshay|ruck: problem in neutron http://paste.openstack.org/show/723479/ | 15:53 |
| weshay|ruck | arxcruz, ok.. so additional bug on that? | 15:54 |
| weshay|ruck | arxcruz, seems like we have found a number of things | 15:54 |
| marios | florianf: my apologies just came off a long call looks like EmilienM got to it | 15:54 |
| *** sshnaidm has quit IRC | 15:55 | |
| florianf | marios: yep, already done. but thanks! | 15:55 |
| arxcruz | weshay|ruck: yes, we need to switch the role_operator on python-tempestconf to either admin, swiftoperator OR in swift add Member as role operator allowed to do the containers | 15:55 |
| weshay|ruck | arxcruz, 1. permissions config in tempest for swift, 2. this neutron bug | 15:55 |
| arxcruz | tosky: chandankumar ^ | 15:55 |
| d0ugal | florianf, jtomasek: Do you allow users to set initial or change passwords in the GUI? | 15:55 |
| tosky | arxcruz: even if switching in swift, it would not work for older versions | 15:56 |
| * tosky is the Ghost of Past Versions | 15:56 | |
| jtomasek | d0ugal: users can set the parameter value which will save it in parameter_defaults of plan-environment.yaml | 15:56 |
| tosky | unless you force the option for those versions (as it happens now iirc) | 15:56 |
| arxcruz | tosky: i notice a few days ago, tempest was setting it to SwiftOperator instead of Member, not sure what change | 15:56 |
| d0ugal | jtomasek: right, so there is nothing "special" about a password | 15:56 |
| jtomasek | d0ugal: and the original generated password value is going to get overridden by this | 15:56 |
| jtomasek | right | 15:56 |
| d0ugal | jtomasek: thanks, just trying to get my head around the new action and workflow here: https://review.openstack.org/#/c/574782/3/workbooks/plan_management.yaml | 15:57 |
| d0ugal | I don't think the replace_passwords action/workflow is needed since you can do that already | 15:57 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 16:00 |
| jtomasek | d0ugal: only difference I see is that it will replace the passwords with new autogenerated ones | 16:01 |
| openstackgerrit | Merged openstack/os-net-config master: Restore the order of params in ifcfg file that was inadvertently changed https://review.openstack.org/575220 | 16:01 |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/queens: Pass custom roles_data during minor update https://review.openstack.org/574910 | 16:01 |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/queens: Add log artifacts for upgrade steps https://review.openstack.org/574970 | 16:01 |
| openstackgerrit | Merged openstack/tripleo-ui master: Update links in README https://review.openstack.org/574584 | 16:01 |
| jtomasek | d0ugal: I did not look properly, but do the actions just update the passwords in passwords section of plan-environment.yaml? | 16:02 |
| jtomasek | d0ugal: is that expected? IE. if user explicitly set the number via parameters, the rotation would not get that password changed | 16:03 |
| d0ugal | jtomasek: rotate passwords generates new passwords, but replace passwords replaces passwords with a user provided value iiuc | 16:03 |
| d0ugal | jtomasek: I am not sure what is expected :) I am just trying to get my head around it still | 16:03 |
| *** cylopez has quit IRC | 16:04 | |
| *** myoung is now known as myoung|lunch | 16:04 | |
| *** panda is now known as panda|off | 16:06 | |
| *** kopecmartin has quit IRC | 16:06 | |
| *** moshele has joined #tripleo | 16:07 | |
| *** lifeless has quit IRC | 16:07 | |
| *** jpich has quit IRC | 16:08 | |
| *** lifeless has joined #tripleo | 16:08 | |
| *** mschuppert has quit IRC | 16:08 | |
| arxcruz | weshay|ruck: is there a bug already opened for it? I can update there | 16:09 |
| *** suuuper has quit IRC | 16:09 | |
| weshay|ruck | arxcruz, there is a bug, but let's chat for a hot sec | 16:09 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1776596 | 16:10 |
| openstack | Launchpad bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| *** moshele has quit IRC | 16:11 | |
| *** dparkes has joined #tripleo | 16:12 | |
| jtomasek | d0ugal: I also think that replace_passwords is not needed at all. It is clearly possible to do via GUI form or via list of parameter values in custom environment file or by editting plan-environment. I don't really see any added value to having a specific workflow for that. The workflow is even worse as it currently updates passwords section rather than stores these explicit passwords in parameter_defaults | 16:15 |
| d0ugal | jtomasek: agreed. I am writing something similar in my review | 16:15 |
| jtomasek | d0ugal: cool | 16:16 |
| *** udesale has joined #tripleo | 16:20 | |
| *** matbu has quit IRC | 16:22 | |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common master: Adds no_proxy input to create_overcloudrc workflow https://review.openstack.org/575490 | 16:25 |
| openstackgerrit | Ryan Brady proposed openstack/python-tripleoclient master: Changes Creation Of overcloudrc File to Call Workflow https://review.openstack.org/545351 | 16:27 |
| *** udesale has quit IRC | 16:28 | |
| *** dparkes has quit IRC | 16:30 | |
| *** ffiore has quit IRC | 16:32 | |
| *** dparkes has joined #tripleo | 16:36 | |
| *** brault has quit IRC | 16:38 | |
| *** zzzeek has quit IRC | 16:38 | |
| *** brault has joined #tripleo | 16:39 | |
| *** sshnaidm has joined #tripleo | 16:40 | |
| tdasilva | arxcruz: please don't change swift config to add Member, that could be construed as a security issue. my guess is that most users are "Members" and should not really have a "operator" type role | 16:40 |
| arxcruz | tdasilva: no, i'll switch to admin | 16:41 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: update the gating repo name in container updates https://review.openstack.org/575492 | 16:41 |
| tdasilva | arxcruz: do we know why it was changed in tempest.conf? | 16:41 |
| arxcruz | tdasilva: i'm investigating | 16:42 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: update the gating repo name in container updates https://review.openstack.org/575492 | 16:42 |
| arxcruz | tdasilva: so, in swift upstream jobs, not tripleo, is set to Member and admin in proxy.conf | 16:44 |
| *** zzzeek has joined #tripleo | 16:44 | |
| *** brault has quit IRC | 16:44 | |
| tdasilva | arxcruz: ah interesting, i'm still curious about what has changed recently that is causing these tests to break? checking puppet-swift and tht there were no changes to Swift's configuration of operator_roles. Checking tempest config setup it has used Member as the default role for a long time too | 16:49 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-common master: ensure the updates workflow checks container changes https://review.openstack.org/575495 | 16:51 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-common master: DNM: test patch https://review.openstack.org/575242 | 16:51 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: add scen008 container job config https://review.openstack.org/569514 | 16:54 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: trigger scen008 job when config changes https://review.openstack.org/569512 | 16:55 |
| *** shreshtha has joined #tripleo | 16:56 | |
| chandankumar | mwhahaha: Hello | 16:56 |
| mwhahaha | chandankumar: what's up? | 16:57 |
| chandankumar | mwhahaha: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/utils.py#L126 regarding this code | 16:57 |
| chandankumar | mwhahaha: we can not delete this code but we need a proper replacement which will generate deployment tool specific valyes which can be used by tempestconf to geneate and set properly in tempest.conf | 16:58 |
| chandankumar | one example is swift operator role | 16:58 |
| chandankumar | other might be cinder backend and many more | 16:58 |
| *** zoli is now known as zli|gone | 16:59 | |
| *** zli|gone is now known as zoli|gone | 16:59 | |
| mwhahaha | chandankumar: ok well imho no one has touched this stuff in years nor knows what its' for. so its fine if it has to stay for now but it seems to be in need of attention | 16:59 |
| chandankumar | one solution is mistral workflow for creatng those values and writing in tempest deployer inpurt conf file | 16:59 |
| mwhahaha | it would be beneficial if the items here could be discoverable externally by tempest | 16:59 |
| mwhahaha | so we can drop it | 16:59 |
| *** zoli|gone is now known as zoli | 16:59 | |
| chandankumar | some things are dependent on deployument and environment which is out of scope of tempestconf | 17:00 |
| chandankumar | other is external network | 17:00 |
| mwhahaha | chandankumar: sure but it doesn't necessarily make sense for tripleo client to be generating/managing this stuff since it's currently hard coded | 17:00 |
| weshay|ruck | marios, ping re: https://review.openstack.org/#/c/566565/8/toci_gate_test-oooq.sh@220 | 17:00 |
| mwhahaha | so if a user changes it, it's going ot be wrong anyway | 17:00 |
| *** derekh has quit IRC | 17:00 | |
| *** trown is now known as trown|lunch | 17:01 | |
| chandankumar | mwhahaha: so let's keep it till we find a proper soltuin, we will work on a sprint to get it fixed | 17:01 |
| mwhahaha | chandankumar: yea that's fine, it's just something that we generate for overcloud deployments and it's nto clear it's usage (or who owns this thing) | 17:02 |
| tosky | mwhahaha: no, the solution, as I tried to explain many times, it's not to get rid of that file | 17:02 |
| tosky | mwhahaha: but it's having tripleo to write it down with the real values used in the deployment | 17:02 |
| tosky | there are things that are complicated to discover | 17:02 |
| tosky | unfortunately I took that task (at Denver time) and I didn't have the time to work on it | 17:03 |
| tosky | the complication of setting up the mistral development environment didn't help | 17:03 |
| mwhahaha | tosky: i didn't say drop the file for now, i just don't want to be managing this in tripleo client as it is | 17:03 |
| mwhahaha | this is specific to the tempest conf but it's not clear to those not well versed in tempest why/how this thing fits in | 17:03 |
| tosky | mwhahaha: I agree that it's not where it should be (it should be a mistral workflow) | 17:03 |
| *** dparkes has quit IRC | 17:03 | |
| tosky | because probably at the time when that code was added it was the "easiest" place; then the person who worked on that left the project | 17:04 |
| tosky | and no one else worked on that | 17:04 |
| mwhahaha | yea that's kinda my point to this | 17:04 |
| mwhahaha | if there is no owner, that needs to be addressed | 17:04 |
| mwhahaha | or it gets removed | 17:04 |
| tosky | but if it gets removed with no replacement, it will cause more problems | 17:05 |
| mwhahaha | then time to find an owner | 17:05 |
| tosky | an owner with time | 17:05 |
| *** holser__ has quit IRC | 17:05 | |
| mwhahaha | time to make time | 17:07 |
| *** mcornea has quit IRC | 17:07 | |
| *** dparkes has joined #tripleo | 17:07 | |
| mwhahaha | also mistral doesn't solve the standalone case where there is no mistral | 17:09 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: add fs051 to use emit_releases_file.py https://review.openstack.org/574417 | 17:09 |
| chandankumar | tosky: mwhahaha https://trello.com/c/XQWhmflp/807-replace-tripleo-provided-tempest-deployer-input-file-with-mistral-workflow | 17:09 |
| mwhahaha | so there needs to be more docs/understanding how this actually fits into our tempest | 17:09 |
| *** mcornea has joined #tripleo | 17:10 | |
| *** gkadam__ is now known as gkadam | 17:10 | |
| quiquell | chem: Can we close this ? https://bugs.launchpad.net/tripleo/+bug/1758899 | 17:11 |
| openstack | Launchpad bug 1758899 in tripleo "reproducer-quickstart.sh building wrong gating package." [High,In progress] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 17:11 |
| chem | quiquell: didn't get a chance to reverify that's working now | 17:12 |
| quiquell | chem: I have beign doing gating rpms stuff with reproducers, was working, also an export of ZUUL_CHANGES was missing | 17:12 |
| *** myoung|lunch is now known as myoung | 17:12 | |
| quiquell | chem: now it's merged in master | 17:12 |
| chem | quiquell: oki, so let an note that's worknig for you and off we go, 1 less lp :) | 17:13 |
| chandankumar | mwhahaha: can we get this one https://blueprints.launchpad.net/tripleo/+spec/os-client-config-switch | 17:13 |
| openstackgerrit | Matt Young proposed openstack/tripleo-upgrade stable/queens: Add timestamps to update log files https://review.openstack.org/575499 | 17:13 |
| quiquell | chem: Cool thanks man | 17:13 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo stable/pike: install python-neutron-lbaas before starting neutron https://review.openstack.org/575500 | 17:13 |
| mwhahaha | chandankumar: if you write it | 17:13 |
| mwhahaha | we'll need to continue to write out both for a while | 17:13 |
| mwhahaha | that being said, we generate that file like in 4 different ways | 17:14 |
| mwhahaha | so there's some effort to align those | 17:14 |
| *** jpena is now known as jpena|off | 17:14 | |
| *** leanderthal is now known as leanderthal|afk | 17:14 | |
| tosky | mwhahaha: oh, what is the standalone case of no mistral? | 17:14 |
| * chandankumar plate is fill | 17:15 | |
| chandankumar | *full | 17:15 |
| tosky | I thought that mistral is a central component for the deployment | 17:15 |
| mwhahaha | tosky: not for the udnercloud deployment | 17:15 |
| mwhahaha | so containerized undercloud we'll want to run some part f tempest but there is no mistral used | 17:15 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: kick standalone on fs52 changes https://review.openstack.org/575503 | 17:15 |
| tosky | mwhahaha: uhm, that's a slightly different problem, but the undercloud case is a bit special | 17:15 |
| *** dparkes has quit IRC | 17:16 | |
| tosky | it could be possible handled all with discovery | 17:16 |
| tosky | and the all-in-one deployment, does/will it use mistral? | 17:16 |
| *** amoralej is now known as amoralej|off | 17:16 | |
| mwhahaha | the all-in-one will not use mistral | 17:18 |
| mwhahaha | so the generation code needs to be executable outside of mistral | 17:18 |
| mwhahaha | it's likely better to leverage THT to properly collect this information | 17:18 |
| mwhahaha | and then we include it as an output of the deployment itself | 17:18 |
| tosky | argh | 17:19 |
| tosky | the entire discussion that I had some time ago with marios and gfidente was based on the usage of mistral | 17:19 |
| tosky | in order to let this action available both to the internal processing and to tripleo-ui | 17:19 |
| tosky | the point at that time was that everything should go through mistral which provides the API | 17:20 |
| tosky | did this change? | 17:20 |
| mwhahaha | not necessarily, but it depends on what you're considering | 17:20 |
| mwhahaha | so this goes back to this needs to be documented and communicated about wtf this file is and how it plays into the usage for tempest runs we want to do in tripleo | 17:21 |
| mwhahaha | if it's a config file generation, mistral doesn't necessarily make sense | 17:21 |
| mwhahaha | if there' smore to it, then maybe mistral can be used to pass it around to the UI, but we need alternative ways of dropping this file | 17:21 |
| tosky | ok, so where should this code fit? Which part of the entire engine is always there and can access to the values set for the deployment? | 17:22 |
| tosky | (and query them) | 17:22 |
| mwhahaha | THT | 17:22 |
| chandankumar | what about putting as a plugin with tripleo validations? | 17:22 |
| chandankumar | it checks the configs and other stuff on the deployment itself | 17:23 |
| mwhahaha | so my thought on this file is that the contents of it assumes config items that come from THT right? we need to have it be an output of the deployment processing | 17:23 |
| mwhahaha | which means it would be part of THT and an output | 17:23 |
| *** sanjayu_ has joined #tripleo | 17:23 | |
| mwhahaha | which means it would also be queriable via mistral and outside of that | 17:23 |
| tosky | yes, the code which generates this file should be able to access the THT configuration to find out what was really deployed | 17:24 |
| weshay|ruck | EmilienM, weird.. filter on 571613 in the zuul queue.. it's in check and gate at the same time | 17:28 |
| weshay|ruck | wtf | 17:28 |
| *** mhenkel_ has quit IRC | 17:28 | |
| EmilienM | weshay|ruck: because gerrit dependencies :-) | 17:29 |
| EmilienM | 571613 is in grey in check | 17:29 |
| EmilienM | it's because it's tested by another patch on top of 571613 | 17:29 |
| EmilienM | but not actually 571613 | 17:29 |
| EmilienM | 571613 testing is in gate | 17:29 |
| weshay|ruck | ok | 17:29 |
| *** tesseract has quit IRC | 17:30 | |
| *** owalsh has quit IRC | 17:31 | |
| *** janki has quit IRC | 17:31 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Fix logrotate_crond issues https://review.openstack.org/574835 | 17:32 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Upload validations to swift on undercloud install https://review.openstack.org/564254 | 17:32 |
| openstackgerrit | Merged openstack/tripleo-ui master: Amend the spelling error of a word https://review.openstack.org/573059 | 17:32 |
| EmilienM | weshay|ruck: can you tell us the URL again to monitor the timeouts? | 17:33 |
| EmilienM | I want to see how it is now, if the revert really really helped | 17:34 |
| EmilienM | or if we're just luck | 17:34 |
| EmilienM | lucky* | 17:34 |
| weshay|ruck | http://38.145.34.131:3000/d/pgdr_WVmk/ruck-rover?orgId=1 | 17:34 |
| weshay|ruck | EmilienM, seeing some timeouts today | 17:34 |
| EmilienM | weshay|ruck: not that much, eh? | 17:36 |
| EmilienM | weshay|ruck: interesting, in Upstream gate jobs you can see the zuul gate break, then the little rechecks I did before sleep and then all recheck when I woke up | 17:37 |
| weshay|ruck | EmilienM, live and breathe this stuff | 17:37 |
| EmilienM | lol | 17:37 |
| EmilienM | weshay|ruck: I can't find the exact # of timeouts the last 3 hours for ex. | 17:38 |
| weshay|ruck | EmilienM, if duration is red | 17:38 |
| weshay|ruck | EmilienM, http://38.145.34.131:3000/d/pgdr_WVmk/ruck-rover?orgId=1&panelId=61&fullscreen | 17:39 |
| weshay|ruck | EmilienM, however I think the gerrit review # and logs are right | 17:39 |
| *** sanjayu_ has quit IRC | 17:39 | |
| weshay|ruck | quiquell, ^ | 17:39 |
| quiquell | weshay|ruck: Let's check | 17:40 |
| weshay|ruck | quiquell, weird.. it looks ok on second check.. sorry | 17:40 |
| quiquell | weshay|ruck: This dashboard live and breath too :-) | 17:40 |
| *** florianf has quit IRC | 17:43 | |
| *** pcaruana has joined #tripleo | 17:43 | |
| weshay|ruck | quiquell, did anyone write up bugs on rdo phase 1 master 2 jobs failing? | 17:45 |
| EmilienM | weshay|ruck: only 2 timeouts so far | 17:45 |
| EmilienM | if it's true, we're not too bad | 17:45 |
| EmilienM | and always scenario001 | 17:46 |
| EmilienM | this scenario is always on the edge of timeouts | 17:46 |
| EmilienM | ceph, telemetry, heat, etc | 17:46 |
| weshay|ruck | oh it's true baby | 17:46 |
| weshay|ruck | girl you know it's true... ga ga girlllll | 17:47 |
| quiquell | weshay|ruck: Do we want alarms on builds near updatream timeout ? | 17:47 |
| weshay|ruck | wha? | 17:47 |
| weshay|ruck | oh | 17:47 |
| weshay|ruck | no | 17:47 |
| quiquell | ok | 17:47 |
| weshay|ruck | quiquell, what we want are the artifacts from the undercloud and overcloud deployments | 17:48 |
| weshay|ruck | and pull those into https://review.rdoproject.org/grafana/dashboard/db/tripleo-ci?orgId=1 | 17:48 |
| weshay|ruck | so we know if any one step of the undercloud or any one step of the overcloud deployment has started to run over the norm | 17:48 |
| quiquell | Ohh I see ok | 17:49 |
| weshay|ruck | EmilienM, I'm not sure what artifacts to grab from the undercloud/overcloud to do that though | 17:49 |
| *** radeks_ has quit IRC | 17:49 | |
| weshay|ruck | we could scrape the ansible logs I suppose | 17:49 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Switch scenario00{1,2,3,4,7} to containerized undercloud"" https://review.openstack.org/575330 | 17:49 |
| weshay|ruck | OH BOY | 17:50 |
| weshay|ruck | HERE WE GO | 17:50 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Deploy container-multinode (fs010) with a containerized undercloud"" https://review.openstack.org/560264 | 17:50 |
| weshay|ruck | no worries.. let's just do this all at once and see what happens | 17:51 |
| EmilienM | lol | 17:51 |
| EmilienM | and leave on pto | 17:51 |
| EmilienM | 'good luck folks' | 17:51 |
| *** salmankhan has joined #tripleo | 17:51 | |
| weshay|ruck | EmilienM, hey... do you have a minute to talk to me and quiquell ? | 17:52 |
| weshay|ruck | re: perf of undercloud/overcloud | 17:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Implement fetching of networks https://review.openstack.org/548685 | 17:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Initial Network configuration views https://review.openstack.org/569124 | 17:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: [WIP] Add Network Topology diagram https://review.openstack.org/569125 | 17:52 |
| EmilienM | is it me or gerrit is super slow | 17:53 |
| weshay|ruck | EmilienM, gerrit is hosted on docker.io | 17:53 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Fix ansible inventory clean up https://review.openstack.org/575407 | 17:53 |
| EmilienM | lol | 17:53 |
| EmilienM | but we mirror it! | 17:53 |
| EmilienM | weshay|ruck: yeah in 5m | 17:54 |
| weshay|ruck | quiquell, you have a sec to talk to milli vanilli ^ | 17:55 |
| *** morazi has quit IRC | 17:55 | |
| *** salmankhan has quit IRC | 17:56 | |
| quiquell | weshay|ruck: Yep sure that will do the day | 17:56 |
| *** ykarel_ has quit IRC | 17:58 | |
| quiquell | weshay|ruck: You bj ? | 17:58 |
| weshay|ruck | ya | 17:58 |
| *** moshele has joined #tripleo | 18:01 | |
| *** pcaruana has quit IRC | 18:01 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud: manage mysql user + revoke grant during upgrade https://review.openstack.org/575186 | 18:03 |
| EmilienM | quiquell, weshay|ruck : joining | 18:03 |
| *** vpickard is now known as vpickard_ | 18:03 | |
| *** leanderthal|afk has quit IRC | 18:04 | |
| openstackgerrit | Ade Lee proposed openstack/tripleo-common master: Add mistral job to rotate passwords on the overcloud https://review.openstack.org/574782 | 18:05 |
| *** jaganathan has quit IRC | 18:06 | |
| *** pcaruana has joined #tripleo | 18:06 | |
| openstackgerrit | Matt Young proposed openstack/tripleo-upgrade stable/pike: Add log artifacts for upgrade steps https://review.openstack.org/574969 | 18:09 |
| *** leanderthal|afk has joined #tripleo | 18:11 | |
| *** owalsh has joined #tripleo | 18:11 | |
| *** trown|lunch is now known as trown | 18:19 | |
| *** pkovar has quit IRC | 18:20 | |
| *** shreshtha has quit IRC | 18:22 | |
| *** mhenkel_ has joined #tripleo | 18:23 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-upgrade stable/queens: Add timestamps to update log files https://review.openstack.org/575499 | 18:24 |
| *** sshnaidm is now known as sshnaidm|off | 18:26 | |
| *** quiquell is now known as quiquell|off | 18:26 | |
| hamzy | so in undercloud.conf, there is for ctlplane-subnet a dhcp_start and a dhcp_end which is a simple range. However, for ~/templates/network_data.yaml you can define allocation_pools with a bunch of ranges. | 18:26 |
| hamzy | I am seeing "IpAddressGenerationFailureClient: resources.NovaCompute: No more IP addresses available on network 571792bd-c6c2-4514-9f38-cffa87ecdfd7." | 18:26 |
| hamzy | is that because of the small range that I have defined in ~/undercloud.conf? | 18:27 |
| *** mvenesio has joined #tripleo | 18:27 | |
| *** salmankhan has joined #tripleo | 18:32 | |
| *** morazi has joined #tripleo | 18:34 | |
| *** jaganathan has joined #tripleo | 18:38 | |
| *** jaganathan has quit IRC | 18:40 | |
| EmilienM | 2018-06-14 17:33:43 | time="2018-06-14T17:33:42Z" level=fatal msg="pinging docker registry returned: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.1:53: server misbehaving" | 18:42 |
| EmilienM | http://logs.openstack.org/92/572692/1/gate/tripleo-ci-centos-7-containerized-undercloud-upgrades/6b3b997/logs/undercloud/home/zuul/undercloud_upgrade.log.txt.gz#_2018-06-14_17_33_43 | 18:42 |
| EmilienM | another mistery why we hit that while we're supposed to hit mirrors | 18:42 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Refactor get-occ-config.sh https://review.openstack.org/574727 | 18:44 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: snmp: listen on ctrlplane https://review.openstack.org/572108 | 18:46 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/queens: Fix logrotate_crond issues https://review.openstack.org/575525 | 18:49 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/pike: Fix logrotate_crond issues https://review.openstack.org/575526 | 18:52 |
| *** Goneri has quit IRC | 18:52 | |
| *** dparkes has joined #tripleo | 18:55 | |
| *** leanderthal|afk has quit IRC | 18:56 | |
| *** aufi has quit IRC | 18:56 | |
| *** gfidente is now known as gfidente|afk | 19:00 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add BlacklistedHostnames stack output https://review.openstack.org/565582 | 19:04 |
| *** hkominos has joined #tripleo | 19:06 | |
| *** matbu has joined #tripleo | 19:06 | |
| *** sai_p has joined #tripleo | 19:07 | |
| hkominos | Hi all. I have a quick question regarding the disk image build. we have a yaml file describing an image (https://github.com/openstack/tripleo-common/blob/stable/queens/image-yaml/overcloud-images.yaml). In that yaml there is a list of elements and a list of packages. The elements are predifend in DIB source code. How does triple 0 then search for the remaining packages ? (using yum or apt) ?? | 19:08 |
| weshay|ruck | rlandy, EmilienM I've abandoned https://review.openstack.org/#/c/574101/ in favor of https://review.openstack.org/#/c/573819/5 | 19:10 |
| weshay|ruck | rlandy, fyi https://review.openstack.org/#/c/571631/ | 19:11 |
| rlandy | weshay|ruck: https://review.openstack.org/#/c/569514/ - pls see comment - | 19:13 |
| weshay|ruck | ha. thanks | 19:14 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: Correct tag on undercloud-prep-containers https://review.openstack.org/573820 | 19:15 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: Add doc for the libvirt reproducer https://review.openstack.org/566155 | 19:20 |
| *** moshele has quit IRC | 19:21 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add steps to allow downstream to use featuresets https://review.openstack.org/536452 | 19:22 |
| *** leanderthal has joined #tripleo | 19:22 | |
| EmilienM | weshay|ruck: k | 19:30 |
| *** leanderthal has quit IRC | 19:31 | |
| *** lifeless has quit IRC | 19:37 | |
| *** leanderthal has joined #tripleo | 19:38 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: use the tq ansible.cfg for the undercloud install https://review.openstack.org/575535 | 19:38 |
| *** brault has joined #tripleo | 19:40 | |
| *** jfrancoa has joined #tripleo | 19:42 | |
| *** brault has quit IRC | 19:45 | |
| *** leanderthal has quit IRC | 19:48 | |
| *** gkadam has quit IRC | 19:48 | |
| *** leanderthal has joined #tripleo | 19:48 | |
| *** lifeless has joined #tripleo | 19:53 | |
| *** leanderthal has quit IRC | 19:54 | |
| *** leanderthal has joined #tripleo | 19:56 | |
| *** agopi has quit IRC | 20:09 | |
| *** agopi has joined #tripleo | 20:09 | |
| *** liverpooler has quit IRC | 20:11 | |
| *** raildo has quit IRC | 20:12 | |
| EmilienM | weshay|ruck: please rebase https://review.openstack.org/#/c/573820/ on top of https://review.openstack.org/#/c/571613/ | 20:13 |
| weshay|ruck | k | 20:15 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: WIP In swift ringbuilder allow override of retries and timeouts https://review.openstack.org/575542 | 20:16 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates stable/queens: snmp: listen on ctrlplane https://review.openstack.org/575543 | 20:18 |
| weshay|ruck | EmilienM, hrm.. no need.. Steve made the exact change | 20:19 |
| weshay|ruck | err.. | 20:20 |
| weshay|ruck | rlandy, review this when you have a sec please https://review.openstack.org/#/c/573820/ | 20:21 |
| rlandy | looking | 20:24 |
| rlandy | weshay|ruck: ok - you abandoned that change - and you want me to review https://review.openstack.org/#/c/571613/? | 20:27 |
| weshay|ruck | rlandy, yes, please review and confirm that abandoning is the right move there | 20:28 |
| weshay|ruck | pretty sure it is | 20:28 |
| rlandy | weshay|ruck: that particular change is pretty irrelevant | 20:28 |
| rlandy | so abandoning it is fine | 20:28 |
| rlandy | I still think the tag is wring but it's fine either way | 20:28 |
| *** ansmith has quit IRC | 20:31 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Collect origin and NetworkManager dirs https://review.openstack.org/575015 | 20:37 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Collect /etc/os-*/ to get os-net-config https://review.openstack.org/575088 | 20:37 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: undercloud-upgrade job is stable, move to vote https://review.openstack.org/574868 | 20:37 |
| weshay|ruck | EmilienM, failed in the gate http://logs.openstack.org/13/571613/18/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/c7a7bcf/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz | 20:46 |
| EmilienM | yeah | 20:46 |
| *** jfrancoa has quit IRC | 20:47 | |
| weshay|ruck | 2018-06-14 18:59:21 | /usr/bin/docker-current: Error: image tripleomaster/centos-binary-heat-engine:39d396e0ae0d982d63cbcadef57e75e84a052911_033c532c not found. | 20:47 |
| EmilienM | weshay|ruck: I'll look shortly | 20:47 |
| weshay|ruck | https://hub.docker.com/r/tripleomaster/centos-binary-heat-engine/tags/ | 20:47 |
| EmilienM | stevebaker: hey morning | 20:47 |
| stevebaker | EmilienM: hewbrocca_afk | 20:47 |
| stevebaker | woah | 20:47 |
| stevebaker | EmilienM: hey | 20:47 |
| EmilienM | hey :) | 20:48 |
| weshay|ruck | http://logs.openstack.org/13/571613/18/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/c7a7bcf/logs/undercloud/var/log/journal.txt.gz#_Jun_14_18_56_01 | 20:49 |
| weshay|ruck | time="2018-06-14T18:56:01Z" level=error msg="response completed with error" err.code="blob unknown" | 20:50 |
| stevebaker | EmilienM, weshay|ruck: where are things at with container images in the gate? | 20:50 |
| weshay|ruck | stevebaker, this fully passed check https://review.openstack.org/#/c/571613/ | 20:51 |
| weshay|ruck | but just blew up the gate | 20:51 |
| stevebaker | (my internet connection is about to lose power) | 20:52 |
| weshay|ruck | Nooooooooooooooooooo | 20:53 |
| weshay|ruck | :) | 20:53 |
| stevebaker | weshay|ruck: so those same jobs passed in check? | 20:53 |
| weshay|ruck | aye | 20:53 |
| weshay|ruck | stevebaker, EmilienM that may indicate that from cloud to cloud this is not working | 20:56 |
| weshay|ruck | works on one and not the other? | 20:56 |
| weshay|ruck | Error: image tripleomaster/centos-binary-heat-engine:39d396e0ae0d982d63cbcadef57e75e84a052911_033c532c not found" | 20:56 |
| EmilienM | I'm not sure | 20:57 |
| EmilienM | let me look now at the failure, I was busy before | 20:57 |
| weshay|ruck | the containers are fresh | 20:57 |
| weshay|ruck | 39d396e0ae0d982d63cbcadef57e75e84a052911_033c532c | 20:57 |
| weshay|ruck | 276 MB | 20:57 |
| weshay|ruck | 4 hours ago | 20:57 |
| weshay|ruck | busy doing his hair | 20:57 |
| EmilienM | :D | 20:57 |
| EmilienM | ah it reseted | 20:58 |
| EmilienM | http://logs.openstack.org/13/571613/18/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/c7a7bcf/logs are the logs? | 20:58 |
| weshay|ruck | ya | 20:58 |
| stevebaker | weshay|ruck: openstack tripleo container image prepare should have failed if the remote->undercloud upload failed, but I'm seeing container-check failing on fetching the image from the undercloud http://logs.openstack.org/13/571613/18/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/c7a7bcf/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-06-14_18_59_21 | 20:58 |
| tonyb | It looks like bug 1776596 is resolved now or is there more I can do on the nova side? | 21:00 |
| openstack | bug 1776596 in tripleo "[QUEENS] Promotion Jobs failing at overcloud deployment with AttributeError: 'IronicNodeState' object has no attribute 'failed_builds'" [Critical,Fix committed] https://launchpad.net/bugs/1776596 - Assigned to yatin (yatinkarel) | 21:00 |
| weshay|ruck | tonyb, ya.. that looks good https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload/296/console | 21:02 |
| weshay|ruck | tonyb, thanks | 21:03 |
| tonyb | weshay|ruck: woot! | 21:03 |
| *** gfidente|afk has quit IRC | 21:07 | |
| *** abishop has quit IRC | 21:14 | |
| *** trown is now known as trown|outtypewww | 21:20 | |
| *** tzumainn has quit IRC | 21:23 | |
| *** ansmith has joined #tripleo | 21:23 | |
| stevebaker | tonyb: hey, I'm still up for reviewing this series when its refreshed https://review.openstack.org/#/q/status:open+project:openstack/python-tripleoclient+branch:master+topic:bp/multiarch-support | 21:27 |
| *** alee__ has quit IRC | 21:27 | |
| tonyb | stevebaker: Thanks. I'll get them rebased next week | 21:28 |
| stevebaker | k | 21:28 |
| *** mhenkel_ has quit IRC | 21:29 | |
| *** mhenkel_ has joined #tripleo | 21:29 | |
| *** slaweq has quit IRC | 21:29 | |
| *** slaweq has joined #tripleo | 21:30 | |
| stevebaker | urf, this broke the reproducer script https://review.openstack.org/#/c/572732/ | 21:32 |
| EmilienM | oh no | 21:33 |
| EmilienM | mwhahaha: we should make standalone working with existing interface no? ^ | 21:33 |
| EmilienM | I think we should revert that | 21:33 |
| mwhahaha | eventually | 21:33 |
| *** mhenkel_ has quit IRC | 21:33 | |
| mwhahaha | right now it assumes the same as the undercloud | 21:34 |
| mwhahaha | which uses 2 interfaces | 21:34 |
| mwhahaha | eth1 | 21:34 |
| EmilienM | uh so why did we that patch | 21:34 |
| *** cdearborn_ has joined #tripleo | 21:35 | |
| *** mvenesio has quit IRC | 21:35 | |
| *** mvenesio has joined #tripleo | 21:36 | |
| stevebaker | it breaks "Wait for provisioned hosts to become reachable" because the new private IP is in the inventory instead of the public IP | 21:36 |
| stevebaker | won't that break every multinode job? | 21:37 |
| mwhahaha | no cause it's just the reproducer i think | 21:37 |
| mwhahaha | because the reproducer uses heat to spin up subnodes, that patch isn't used by CI | 21:38 |
| *** cdearborn_ has quit IRC | 21:38 | |
| *** cdearborn has quit IRC | 21:38 | |
| mwhahaha | since we already have nodes with that extra interface (i think) | 21:38 |
| * mwhahaha doesn't remember anymore | 21:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: kolla_builder: use any templates except puppet https://review.openstack.org/575552 | 21:39 |
| EmilienM | stevebaker: ^ | 21:39 |
| *** toure is now known as toure|gone | 21:40 | |
| *** mvenesio has quit IRC | 21:40 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Add correct subnode public IP https://review.openstack.org/575553 | 21:41 |
| *** myoung is now known as myoung|off | 21:45 | |
| *** hkominos has quit IRC | 21:46 | |
| *** slaweq has quit IRC | 21:46 | |
| *** lifeless has quit IRC | 21:49 | |
| *** lifeless has joined #tripleo | 21:51 | |
| *** edmondsw has quit IRC | 21:54 | |
| *** akhilaki has joined #tripleo | 21:56 | |
| *** jbadiapa_ has joined #tripleo | 22:03 | |
| *** jbadiapa has quit IRC | 22:06 | |
| *** limao has joined #tripleo | 22:10 | |
| *** akhilaki has quit IRC | 22:11 | |
| *** limao_ has joined #tripleo | 22:13 | |
| *** limao has quit IRC | 22:14 | |
| *** alee__ has joined #tripleo | 22:14 | |
| openstackgerrit | Sai Ram Peesapati proposed openstack/tripleo-heat-templates stable/pike: Conditionally Enable Nuage VRS https://review.openstack.org/572653 | 22:16 |
| *** rcernin has joined #tripleo | 22:20 | |
| EmilienM | 571631 is passing check so far | 22:21 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: kolla_builder: use any templates except puppet https://review.openstack.org/575552 | 22:29 |
| *** pradk has quit IRC | 22:37 | |
| EmilienM | weshay|ruck: re scenarios/standalone in CI | 22:37 |
| EmilienM | weshay|ruck: with alex we agreed we could switch scenarios except fs010 to standalone | 22:37 |
| EmilienM | and keep fs010 with an overcloud, and see how it works | 22:37 |
| EmilienM | weshay|ruck: do we have someone looking at tempest for standalone? | 22:38 |
| EmilienM | i don't think it's a big deal, we basically need to re-use what was done for containerized undercloud I guess | 22:39 |
| EmilienM | mwhahaha: any way we can revert https://review.openstack.org/#/c/572732/ you think? I'm confused why we need it for standalone and not for containerized undercloud | 22:39 |
| mwhahaha | if it's causing problems, file a bug and revert | 22:40 |
| mwhahaha | i didn't see tha tone so i don't know what it is for | 22:40 |
| EmilienM | weshay|ruck: you +1-ed the patch, do you have more contexgt? | 22:41 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo stable/queens: Remove share_backend_name from Dell-EMC manila backends https://review.openstack.org/572931 | 22:42 |
| *** brault has joined #tripleo | 22:43 | |
| EmilienM | mwhahaha: we could probably try with eth0 | 22:43 |
| mwhahaha | we can get away with it i think | 22:43 |
| mwhahaha | let me figure out this masq stuff first | 22:43 |
| mwhahaha | stop getting ahead | 22:43 |
| mwhahaha | it's likely we should be ok just setting the local ip to the eth0 | 22:44 |
| mwhahaha | though what are we doing for the undercloud jobs? | 22:44 |
| mwhahaha | this stuff should be the same | 22:44 |
| EmilienM | mwhahaha: we use br-ex :D | 22:44 |
| mwhahaha | we don't use br-ex for undercloud | 22:45 |
| mwhahaha | (show me the configs) | 22:45 |
| EmilienM | http://logs.openstack.org/95/575495/1/check/tripleo-ci-centos-7-undercloud-containers/06ccaca/logs/undercloud/home/zuul/undercloud.conf.txt.gz | 22:45 |
| EmilienM | local_interface = br-ex | 22:45 |
| * mwhahaha facepalms | 22:45 | |
| mwhahaha | what creates br-ex? | 22:46 |
| mwhahaha | oooq? | 22:46 |
| EmilienM | no idea but I'll try something out | 22:46 |
| mwhahaha | so yea that should work the same | 22:46 |
| mwhahaha | if you set the NeutronPublicINterface to br-ex | 22:46 |
| *** jbadiapa_ has quit IRC | 22:46 | |
| mwhahaha | still wondering what is creating that | 22:47 |
| *** brault has quit IRC | 22:47 | |
| *** sanjay__u has joined #tripleo | 22:48 | |
| mwhahaha | so yea we create that in our vxlan setup | 22:49 |
| mwhahaha | http://logs.openstack.org/95/575495/1/check/tripleo-ci-centos-7-undercloud-containers/06ccaca/logs/undercloud/home/zuul/vxlan_networking.sh.log.txt.gz#_2018-06-14_19_42_50 | 22:49 |
| EmilienM | mwhahaha: when you created standalone_parameters, why did you have to override NeutronPublicInterface ? | 22:49 |
| mwhahaha | because i don't have br-ex create | 22:49 |
| mwhahaha | NeutronPublicInterface is used to create the br-ctlplane interface | 22:49 |
| EmilienM | on the undercloud we do: http://git.openstack.org/cgit/openstack/python-tripleoclient/tree/tripleoclient/v1/undercloud_config.py#n67 | 22:49 |
| *** tosky has quit IRC | 22:49 | |
| EmilienM | oh ok | 22:49 |
| *** mhenkel_ has joined #tripleo | 22:50 | |
| mwhahaha | that's what udnercloud_config generates for undercloud_paramters.yaml | 22:50 |
| EmilienM | I guess it's fine to have eth1 now, we can live with it | 22:50 |
| mwhahaha | so local_interface from undercloud.conf gets outputted to NeutronPublicInterface ina .yaml | 22:50 |
| mwhahaha | so for CI we can set that to br-ex | 22:50 |
| mwhahaha | because that's created | 22:50 |
| mwhahaha | on my host, it's an actual interface that has a real network associated with it | 22:50 |
| mwhahaha | but we might be able to get away with eth0 | 22:51 |
| EmilienM | the standalone job is still failing anyway: http://logs.openstack.org/64/560264/37/check/tripleo-ci-centos-7-standalone/904e9e7/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-06-14_20_30_00 | 22:51 |
| mwhahaha | once i figure out the masq rules | 22:51 |
| mwhahaha | likely because we're doing something in ci | 22:51 |
| mwhahaha | so it's a similar problem | 22:52 |
| EmilienM | I think we need to use br-ex | 22:52 |
| EmilienM | to do like cont/uc | 22:52 |
| mwhahaha | so standalone is doing what the undercloud does just without the pretty wrapper | 22:52 |
| mwhahaha | so we should use br-ex for NeutronPublicInterface | 22:53 |
| mwhahaha | and that's what the uc does | 22:53 |
| *** mhenkel_ has quit IRC | 22:54 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add tripleo-ci-centos-7-standalone to tripleo-multinode-container-full https://review.openstack.org/575563 | 22:55 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Reliably build the subnode_ip_pair https://review.openstack.org/575553 | 22:56 |
| *** threestrands has joined #tripleo | 22:56 | |
| *** threestrands has quit IRC | 22:56 | |
| *** threestrands has joined #tripleo | 22:56 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: standalone: switch default NeutronPublicInterface to br-ex https://review.openstack.org/575564 | 22:56 |
| EmilienM | stevebaker: ^ fyi | 22:57 |
| *** threestrands has quit IRC | 22:57 | |
| *** threestrands has joined #tripleo | 22:57 | |
| *** threestrands has quit IRC | 22:57 | |
| *** threestrands has joined #tripleo | 22:57 | |
| *** jbadiapa_ has joined #tripleo | 23:02 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: standalone: run keystone tempest tests https://review.openstack.org/575565 | 23:03 |
| EmilienM | weshay|ruck: I probably won't have time before july to work on it ^ but here is a start | 23:03 |
| EmilienM | mhh I don't like that, the gate starts to be backed up again slowly | 23:07 |
| EmilienM | almost 7 hours now | 23:07 |
| *** limao_ has quit IRC | 23:17 | |
| EmilienM | zuul was restarted :( | 23:20 |
| EmilienM | queue is come back | 23:25 |
| *** itlinux has joined #tripleo | 23:30 | |
| beagles | anybody around familiar with debugging minor update setups? After attempts on two different deployments, I got the same error on one of my controller nodes http://paste.openstack.org/show/723516/ | 23:30 |
| itlinux | hello all..ExternalNetworkVlanID if I do not use vlans going out.. what should I have here? | 23:31 |
| itlinux | thanks | 23:31 |
| beagles | itlinux: a couple of things about that question: The ExternalNetwork is primarily about public access to the openstack APIs running in the overcloud. While it is sometimes combined with floating IPs or "tenant->internet" access it is not necessarily so keep that in mind with planning out your network configs... | 23:40 |
| itlinux | yes that's fine this is for my test env.. | 23:40 |
| itlinux | In production I do have vlans | 23:40 |
| itlinux | thanks beagles: | 23:40 |
| beagles | itlinux: ah okay.. so what you likely want to do is use a network template that doesn't actually use the ExternalNetworkVlanID parameter at all for configuring that on the nodes | 23:41 |
| beagles | itlinux: in which case the value of the ExternalNetworkVlanID doesn't matter... | 23:42 |
| itlinux | yea I have the network now and set it to ID 1 | 23:42 |
| itlinux | which is the default shall see if that works.. | 23:42 |
| beagles | ack that was my second suggestion ;) | 23:42 |
| *** lifeless_ has joined #tripleo | 23:42 | |
| itlinux | trying to build this as close as to what I have in DEV so the new guys can understand | 23:42 |
| beagles | right makes sense | 23:43 |
| *** lifeless has quit IRC | 23:43 | |
| itlinux | if I change too many things they are not happy.. | 23:43 |
| *** dxiri has quit IRC | 23:47 | |
| *** dxiri has joined #tripleo | 23:48 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: WIP/DNM: Do not create metrics pool telemetry is disabled https://review.openstack.org/575571 | 23:50 |
| *** dxiri has quit IRC | 23:52 | |
| *** rpioso is now known as rpioso|afk | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!