| *** achadha has quit IRC | 00:00 | |
| *** leitan has joined #tripleo | 00:01 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 00:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 00:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 00:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 00:10 |
| *** rhallisey has quit IRC | 00:12 | |
| *** toure is now known as toure_biab | 00:21 | |
| *** garyk has joined #tripleo | 00:24 | |
| *** lblanchard has joined #tripleo | 00:41 | |
| *** limao has joined #tripleo | 00:43 | |
| *** ecerquei has joined #tripleo | 00:43 | |
| *** Hazelesque has quit IRC | 00:44 | |
| *** Hazelesque has joined #tripleo | 00:46 | |
| *** leitan has quit IRC | 00:52 | |
| *** psahoo has joined #tripleo | 00:53 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 01:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 01:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 01:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 01:10 |
| *** achadha has joined #tripleo | 01:11 | |
| *** ecerquei has quit IRC | 01:14 | |
| *** achadha has quit IRC | 01:15 | |
| openstackgerrit | Merged openstack/tripleo-common master: Add stunnel to redis image for tls https://review.openstack.org/496519 | 01:22 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: manila: set "host" to "hostgroup" https://review.openstack.org/497482 | 01:23 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Enable ceilometer compute agent https://review.openstack.org/498119 | 01:24 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Update reno>=2.5.0 https://review.openstack.org/495715 | 01:24 |
| *** ebarrera has quit IRC | 01:33 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Change Delorean repo link to consistent for single job pipelines https://review.openstack.org/497982 | 01:35 |
| *** achadha has joined #tripleo | 01:45 | |
| *** achadha has quit IRC | 01:49 | |
| *** achadha has joined #tripleo | 02:02 | |
| *** yamahata has quit IRC | 02:08 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 02:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 02:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 02:10 |
| *** achadha_ has joined #tripleo | 02:12 | |
| *** achadha has quit IRC | 02:15 | |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo stable/pike: Enable config for docker daemon debug https://review.openstack.org/498653 | 02:20 |
| *** gbarros has joined #tripleo | 02:30 | |
| *** gbarros has quit IRC | 02:34 | |
| *** achadha_ has quit IRC | 02:37 | |
| *** ramishra has joined #tripleo | 02:38 | |
| *** lblanchard has quit IRC | 02:43 | |
| *** tzumainn has quit IRC | 02:45 | |
| *** rlandy|bbl is now known as rlandy | 02:54 | |
| *** yamahata has joined #tripleo | 03:06 | |
| *** garyk has quit IRC | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 03:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 03:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 03:10 |
| *** limao has quit IRC | 03:11 | |
| *** itlinux has joined #tripleo | 03:13 | |
| *** pradk has quit IRC | 03:13 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Add stunnel to redis image for tls https://review.openstack.org/498662 | 03:27 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: manila: set "host" to "hostgroup" https://review.openstack.org/498664 | 03:27 |
| *** achadha has joined #tripleo | 03:27 | |
| *** achadha has quit IRC | 03:27 | |
| *** achadha has joined #tripleo | 03:28 | |
| *** achadha has quit IRC | 03:32 | |
| *** pradk has joined #tripleo | 03:33 | |
| *** gkadam_ has joined #tripleo | 03:41 | |
| *** jbcraig1 has quit IRC | 03:49 | |
| *** udesale has joined #tripleo | 03:50 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates master: DNM, for testing purposes only https://review.openstack.org/498671 | 03:56 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates master: DNM, for testing purposes only https://review.openstack.org/498671 | 03:59 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Upload remove default for pull_source https://review.openstack.org/489837 | 04:01 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Discover a versioned container image tag https://review.openstack.org/498678 | 04:01 |
| EmilienM | stevebaker: how things are going | 04:02 |
| stevebaker | EmilienM: hey, good | 04:02 |
| *** links has joined #tripleo | 04:03 | |
| EmilienM | stevebaker: do you need any help or review? | 04:04 |
| stevebaker | EmilienM: I'm about to propose a fix to this, its a new command though, so even though its a "feature" I still think its worth landing for rc2 https://bugs.launchpad.net/tripleo/+bug/1708967 | 04:06 |
| openstack | Launchpad bug 1708967 in tripleo "prepare command needs to discover the versioned tag from the latest tag" [High,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:06 |
| *** pdeore has joined #tripleo | 04:06 | |
| EmilienM | stevebaker: I was more interested by the 2 current CI alerts that you're assigned of | 04:07 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1689556 | 04:07 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:07 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1691403 | 04:07 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:07 |
| stevebaker | EmilienM: this change should fix both, just waiting to merge https://review.openstack.org/#/c/493695/ | 04:07 |
| EmilienM | stevebaker: cool. We have https://bugs.launchpad.net/tripleo/+bug/1712910 which is blocking our gate right now... weshay is on it | 04:09 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 04:09 |
| EmilienM | stevebaker: please keep doing recheck, we have no fix now and the issue is random | 04:09 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 04:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 04:10 |
| stevebaker | EmilienM: OK, you mean recheck my change until it lands? | 04:11 |
| EmilienM | stevebaker: yea | 04:11 |
| stevebaker | oki | 04:11 |
| EmilienM | and we all count on weshay to fix the world | 04:11 |
| EmilienM | weshay: no pressure | 04:11 |
| openstackgerrit | wes hayutin proposed openstack/instack-undercloud master: DNM, test commit only https://review.openstack.org/498680 | 04:12 |
| weshay | :P | 04:12 |
| stevebaker | looks like weshay is just committing random crap now 0_o | 04:13 |
| weshay | stevebaker, like that's a change in behavior ;) | 04:14 |
| stevebaker | :) | 04:15 |
| stevebaker | weshay: so the virt-customize fails randomly and we're not capturing enough yet to know why? | 04:15 |
| weshay | stevebaker, k.. waiting on the status page to look again | 04:16 |
| stevebaker | weshay: ok, i'll keep an eye out | 04:16 |
| weshay | dang.. overall we're at 91.4% pass rate EmilienM :) | 04:17 |
| EmilienM | stevebaker: that's exactly that | 04:17 |
| EmilienM | weshay: on which one | 04:17 |
| weshay | overall | 04:17 |
| EmilienM | weshay: we had so much failures today | 04:17 |
| EmilienM | ah | 04:17 |
| EmilienM | that's good man | 04:17 |
| weshay | the undercloud-oooq job is failing 30% of the time | 04:17 |
| EmilienM | that's a lot | 04:17 |
| weshay | yes | 04:18 |
| weshay | EmilienM, thanks for doing the math for me | 04:18 |
| EmilienM | weshay: I'm working on running tempest on more jobs | 04:19 |
| EmilienM | weshay: good progress today | 04:19 |
| *** itlinux has quit IRC | 04:19 | |
| weshay | I saw | 04:19 |
| EmilienM | weshay: I hope before ptg we can run tempest on all scenarios (container + baremetal) | 04:19 |
| weshay | EmilienM, stevebaker it's not random | 04:21 |
| EmilienM | stevebaker: could you review weshay's work on https://review.openstack.org/#/c/498074/ please ? | 04:21 |
| weshay | it's only failing on instack-undercloud changes | 04:21 |
| EmilienM | weshay: ah I thought, sorry | 04:21 |
| EmilienM | weshay: good work btw on it ^ | 04:22 |
| stevebaker | EmilienM: sure | 04:22 |
| EmilienM | thx | 04:22 |
| weshay | thanks for the reviews.. | 04:22 |
| *** ykarel has joined #tripleo | 04:23 | |
| weshay | EmilienM, the dlrn issue only hit us once? in 30 days? is this correct | 04:26 |
| weshay | http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22ERROR:dlrn:Received%20exception%20Error%20in%5C%22%20AND%20tags:%5C%22console%5C%22%20AND%20build_queue:%5C%22gate%5C%22%20AND%20voting:1&from=864000s | 04:26 |
| EmilienM | weshay: let me check | 04:27 |
| EmilienM | weshay: which launchpad is it? | 04:27 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1708832 ? | 04:28 |
| openstack | Launchpad bug 1708832 in tripleo "DLRN build failures in gate" [High,In progress] | 04:28 |
| EmilienM | yeah | 04:28 |
| weshay | Bug 1708832 - DLRN build failures in gate | 04:28 |
| EmilienM | 0 fails in 24 hrs / 1 fails in 10 days | 04:28 |
| EmilienM | sounds like yes | 04:28 |
| EmilienM | weshay: can you update the LP please (assignment + link the patch) | 04:28 |
| EmilienM | so we can keep track of what who where | 04:28 |
| EmilienM | and thanks :) | 04:28 |
| weshay | hrm.. there are two bugs on it.. will I mess up the elestic recheck if I mark it a dupe of https://bugs.launchpad.net/tripleo/+bug/1713130 | 04:31 |
| openstack | Launchpad bug 1713130 in tripleo "pip install DLRN don't git clone from github.com" [High,In progress] - Assigned to wes hayutin (weshayutin) | 04:31 |
| EmilienM | weshay: yeah use the ES one true | 04:31 |
| EmilienM | or good I mean | 04:32 |
| weshay | ok.. I have to update the commit message then | 04:32 |
| EmilienM | weshay: no need | 04:33 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Command to discover the versioned tag from latest https://review.openstack.org/498683 | 04:33 |
| EmilienM | weshay: just link it in comment | 04:33 |
| EmilienM | so we can merge it | 04:33 |
| weshay | that is done | 04:33 |
| EmilienM | ok thx | 04:34 |
| *** morazi has quit IRC | 04:37 | |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Command to discover the versioned tag from latest https://review.openstack.org/498683 | 04:38 |
| *** achadha has joined #tripleo | 04:40 | |
| EmilienM | stevebaker: if you can review https://review.openstack.org/#/c/498440/2 please | 04:43 |
| *** achadha has quit IRC | 04:44 | |
| *** achadha has joined #tripleo | 04:44 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: the kolla container playbook needs release variables https://review.openstack.org/498688 | 04:48 |
| EmilienM | scenario004 is not working in container because the neutron plugin is not part of the container | 04:51 |
| EmilienM | http://logs.openstack.org/75/498575/1/check/gate-tripleo-ci-centos-7-scenario004-multinode-oooq-container/ffd691c/logs/subnode-2/var/log/containers/neutron/neutron-server.log.txt.gz#_2017-08-28_21_13_54_320 | 04:51 |
| EmilienM | stevebaker: can you help with this one today if you have time? Otherwise I'll look at it tomorrow | 04:51 |
| EmilienM | that's part of this effort: https://review.openstack.org/#/c/498575/ | 04:52 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-common master: Encode stdout/stderr to generate readable logs https://review.openstack.org/497395 | 04:55 |
| EmilienM | I reported the issue: https://bugs.launchpad.net/tripleo/+bug/1713612 | 04:56 |
| openstack | Launchpad bug 1713612 in tripleo "When neutron is containerized, No 'neutron.service_plugins' driver found" [High,Triaged] | 04:56 |
| *** shreshtha has joined #tripleo | 04:57 | |
| *** bcafarel has quit IRC | 04:57 | |
| *** skramaja has joined #tripleo | 04:57 | |
| *** jschlueter has quit IRC | 04:58 | |
| *** mburned has quit IRC | 04:58 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs016: run tempest like fs005 https://review.openstack.org/498569 | 04:58 |
| *** shreshtha has quit IRC | 04:58 | |
| *** pdeore has quit IRC | 04:58 | |
| *** bcafarel has joined #tripleo | 04:59 | |
| *** shreshtha has joined #tripleo | 04:59 | |
| *** limao has joined #tripleo | 04:59 | |
| *** achadha_ has joined #tripleo | 05:00 | |
| *** mburned has joined #tripleo | 05:02 | |
| *** jschlueter|znc has joined #tripleo | 05:02 | |
| *** achadha has quit IRC | 05:03 | |
| weshay | so.. EmilienM does this need to come from a mirror? https://github.com/openstack/tripleo-quickstart-extras/blame/master/roles/build-images/defaults/main.yml#L7 | 05:05 |
| * weshay looks at toci | 05:05 | |
| *** Vijayendra has joined #tripleo | 05:06 | |
| *** achadha_ has quit IRC | 05:07 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Drop step_config as top level docker requirement https://review.openstack.org/442716 | 05:07 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: docker: Add unit tests on service_name https://review.openstack.org/442755 | 05:07 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add docker templates to configure Ironic inspector https://review.openstack.org/457822 | 05:07 |
| EmilienM | weshay: I'll look later | 05:08 |
| EmilienM | weshay: go to bed now, ttyl | 05:08 |
| EmilienM | jaosorior: https://review.openstack.org/#/c/498440/2 | 05:09 |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 05:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 05:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 05:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 05:10 |
| jaosorior | EmilienM: sounds like a reasonable change. +2 | 05:10 |
| jaosorior | * looks | 05:10 |
| *** dparkes has quit IRC | 05:11 | |
| *** achadha has joined #tripleo | 05:11 | |
| *** pdeore has joined #tripleo | 05:14 | |
| *** achadha has quit IRC | 05:15 | |
| *** dpawar has joined #tripleo | 05:23 | |
| *** ratailor has joined #tripleo | 05:24 | |
| *** masco has joined #tripleo | 05:28 | |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy OpenShift using OOO on the overcloud https://review.openstack.org/494470 | 05:31 |
| *** pgadiya has joined #tripleo | 05:36 | |
| *** janki has joined #tripleo | 05:37 | |
| *** daidv has quit IRC | 05:41 | |
| *** jfrancoa has joined #tripleo | 05:41 | |
| *** daidv has joined #tripleo | 05:42 | |
| *** mdnadeem has joined #tripleo | 05:44 | |
| *** iranzo has joined #tripleo | 05:46 | |
| *** garyk has joined #tripleo | 05:49 | |
| *** ykarel_ has joined #tripleo | 05:49 | |
| *** jfrancoa has quit IRC | 05:50 | |
| *** jfrancoa has joined #tripleo | 05:50 | |
| *** ykarel has quit IRC | 05:51 | |
| *** yprokule has joined #tripleo | 05:54 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Use [[ for =~ matches https://review.openstack.org/498333 | 05:59 |
| *** ykarel_ is now known as ykarel | 06:00 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Set docker-puppet --health-cmd = /bin/true https://review.openstack.org/498137 | 06:01 |
| *** dparkes has joined #tripleo | 06:04 | |
| *** cschwede has joined #tripleo | 06:07 | |
| *** dsneddon has quit IRC | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 06:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 06:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 06:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 06:10 |
| *** marios has joined #tripleo | 06:10 | |
| *** iranzo has quit IRC | 06:16 | |
| *** agurenko has joined #tripleo | 06:17 | |
| *** pgadiya has quit IRC | 06:18 | |
| *** iranzo has joined #tripleo | 06:20 | |
| *** rlandy has quit IRC | 06:27 | |
| *** pgadiya has joined #tripleo | 06:34 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add yaml validation in docker upgrade_tasks. https://review.openstack.org/497940 | 06:35 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add tags to baremetal cron removal tasks https://review.openstack.org/497936 | 06:35 |
| *** ffiore has joined #tripleo | 06:39 | |
| *** pcaruana has joined #tripleo | 06:39 | |
| *** Vijayendra has quit IRC | 06:41 | |
| *** fzdarsky|afk has joined #tripleo | 06:41 | |
| *** Vijayendra has joined #tripleo | 06:42 | |
| *** jaganathan has joined #tripleo | 06:44 | |
| jaosorior | adarazs: could you check this out https://review.openstack.org/#/c/496639/ ? | 06:48 |
| *** pgadiya has quit IRC | 06:54 | |
| *** aditya_r has joined #tripleo | 06:58 | |
| *** fzdarsky|afk is now known as fzdarsky | 07:04 | |
| *** pgadiya has joined #tripleo | 07:06 | |
| *** ebarrera has joined #tripleo | 07:06 | |
| *** rcernin has joined #tripleo | 07:10 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 07:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 07:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 07:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 07:10 |
| *** jprovazn has joined #tripleo | 07:10 | |
| *** florianf has joined #tripleo | 07:13 | |
| *** aufi has joined #tripleo | 07:13 | |
| *** cylopez has joined #tripleo | 07:15 | |
| sshnaidm|afk | mwhahaha, hi, can you please un-workflow this patch? https://review.openstack.org/#/c/498562 it requires a discussion.. | 07:18 |
| *** sshnaidm|afk is now known as sshnaidm | 07:19 | |
| therve | jaosorior, Hi, you around? Wanted to chat about that swift ssl issue? | 07:21 |
| *** bkopilov has quit IRC | 07:21 | |
| jaosorior | therve: I'm around | 07:22 |
| jaosorior | what's up? | 07:22 |
| therve | jaosorior, So it's https://bugzilla.redhat.com/show_bug.cgi?id=1485189 | 07:23 |
| openstack | bugzilla.redhat.com bug 1485189 in rhosp-director "rhosp-director: HA overcloud +SSL+ceph fails when undercloud was also deployed with SSL: os-collect-config[3057]: requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)" [Unspecified,New] - Assigned to gfidente | 07:23 |
| *** dpawar has quit IRC | 07:23 | |
| jaosorior | therve: well, regarding that issue. I did comment on the BZ, giving the CAMap option, which is what you would anyway have to do if you have an external swift endpoint (the overcloud has to trust it) | 07:23 |
| jaosorior | therve: check comment #8 | 07:23 |
| therve | You mentioned it should be using internal endpoints for that though | 07:23 |
| therve | I was wondering why you thought that | 07:23 |
| jaosorior | therve: well, if it's swift deployed on the undercloud, it should probably be using the internal endpoint. | 07:24 |
| therve | jaosorior, But it's contacted by the overcloud | 07:24 |
| jaosorior | therve: although.... now that I think about it, there's very little chance that the undercloud will be configured with network isolation. So really it's not that big of a deal. | 07:24 |
| *** tesseract has joined #tripleo | 07:25 | |
| therve | So arguably you'd want the communication to be secured | 07:25 |
| therve | Not sure I get your comment about network isolation | 07:25 |
| therve | You mean the IPs will be the same between internal and public? | 07:26 |
| jaosorior | therve: they'll be in the same network. | 07:26 |
| therve | jaosorior, Can you configure the undercloud with network isolation? | 07:26 |
| *** bkopilov has joined #tripleo | 07:26 | |
| jaosorior | therve: that's what I just mentioned, I don't think you can (and if you can I don't know how) | 07:27 |
| therve | Basically, I want to know if there is a code change to be made to use internal endpoint | 07:27 |
| therve | Or if the answer is to properly use CaMap for security | 07:27 |
| jaosorior | therve: * CAMap | 07:27 |
| *** jpena|off is now known as jpena | 07:28 | |
| *** dpawar has joined #tripleo | 07:28 | |
| jaosorior | therve: so... here's the situation. ideally we should be using SSL everywhere for the undercloud. we're not cause of some constraints (long story). as a fallback we rely on that network being private (which seems to be just fine for many deployers) | 07:29 |
| *** anshul has joined #tripleo | 07:29 | |
| therve | Right | 07:30 |
| jaosorior | therve: due to the current situation, I don't have a very strong opinion on using public vs. internal endpoint there. I think using an internal endpoint will make things easier, since it is an internal service anyway (as far as I understood). | 07:30 |
| therve | Well | 07:30 |
| jaosorior | therve: it's not? | 07:31 |
| therve | jaosorior, Critical information is circulating on that communication though. If you care about undercloud ssl, that's probably what you want to protect | 07:31 |
| therve | Otherwise not sure why you'd want undercloud ssl | 07:31 |
| jaosorior | therve: for horizon access, from somewhere to the undercloud. | 07:31 |
| therve | Ok, that's fair | 07:32 |
| *** jpich has joined #tripleo | 07:32 | |
| therve | I guess you have more pressing issues if someone can read data between your nodes and the undercloud | 07:32 |
| *** dpawar has quit IRC | 07:32 | |
| *** ykarel is now known as ykarel|lunch | 07:33 | |
| jaosorior | therve: and we can't really enable TLS everywhere, cause that ends up breaking services like ironic... we need to modify the IPA images to trust the CA, and we can't cause then folks loose support | 07:34 |
| therve | Ah | 07:34 |
| therve | jaosorior, So ironic is already using the internal endpoints? | 07:34 |
| jaosorior | therve: it is | 07:34 |
| therve | Even from remote nodes | 07:34 |
| jaosorior | therve: well, for the overcloud deployment, yeah | 07:35 |
| therve | OK that's an interesting point | 07:35 |
| jaosorior | therve: I came up with this some time ago https://review.openstack.org/#/c/358457/ but I could never get the time to move it forward, and apparently folks are OK with that. | 07:35 |
| therve | jaosorior, So let's assume it should be using internal endpoints here | 07:35 |
| therve | jaosorior, Should we use them all the time, or allow configuration? | 07:35 |
| jaosorior | therve: we should allow configuration | 07:35 |
| therve | Considering it can work if you setup it properly | 07:35 |
| * therve nods | 07:36 | |
| jaosorior | therve: well, it's a matter of usability too | 07:36 |
| *** egonzalez has joined #tripleo | 07:36 | |
| jaosorior | therve: if you decide to use a remote swift endpoint. (swift deployed outside of the undercloud... which someone mentioned you can do... though I don't really know how) then you do need to use the CAMap anyway, and use public endpoints | 07:36 |
| *** jtomasek has quit IRC | 07:37 | |
| *** jtomasek_ has joined #tripleo | 07:37 | |
| therve | Looking at the code I don't see how's that possible :) | 07:37 |
| therve | But ok I'll look into doing that | 07:37 |
| *** yamahata has quit IRC | 07:37 | |
| therve | Thanks for the input! | 07:37 |
| jaosorior | therve: I thought that using the CAMap and making the undercloud's CA cert trusted by the overcloud would be the easier path so I proposed this for quickstart https://review.openstack.org/#/c/496639/ | 07:38 |
| jaosorior | but as feedback I was told that would be quite a burden for deployers | 07:38 |
| jaosorior | I don't really think it is, as long as we document it | 07:38 |
| jaosorior | using the CAMap, stuff works, and no code changes are needed for tripleo or the services :D | 07:39 |
| therve | :) | 07:39 |
| jaosorior | only to quickstart and infrared, and the docs | 07:39 |
| jaosorior | which is fine by me | 07:39 |
| therve | Well I'll let you push that path, because I think it's worth it too | 07:39 |
| therve | I'll see if I can make the endpoints configurable | 07:39 |
| *** ratailor is now known as ratailor|Lunch | 07:39 | |
| jaosorior | therve: soo... if you have influence in this, lets use the CAMap. we use TLS there; it's something that should be done for remote Swift deployments anyway, and it seems like the easier solution | 07:40 |
| therve | jaosorior, Not sure about influence, but I have inertia if I don't write the code :D | 07:41 |
| jaosorior | therve: are you acquainted with infrared? there could be code written for that to allow using CAMap :D | 07:41 |
| therve | Hum no I don't know about it at all | 07:41 |
| jaosorior | therve: also, you said you looked at the code and you didn't see how that was possible. How so? | 07:42 |
| jaosorior | regarding the IPA patch. | 07:42 |
| therve | jaosorior, I don't see how is it possible to use an external swift | 07:42 |
| jaosorior | therve: aaaah | 07:43 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP create default settings for environment types https://review.openstack.org/495407 | 07:44 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Change devmode to deploy containerized services by default https://review.openstack.org/472412 | 07:45 |
| *** oidgar has joined #tripleo | 07:46 | |
| *** mcornea has joined #tripleo | 07:47 | |
| jaosorior | therve: commented on the patch. | 07:51 |
| jaosorior | * the BZ, not the patch | 07:51 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add yaml validation in docker upgrade_tasks. https://review.openstack.org/497940 | 07:52 |
| jaosorior | therve: is zaqar using mongodb by default? | 07:52 |
| therve | jaosorior, Depends where :) | 07:52 |
| therve | Yes in the overcloud, no in the undercloud | 07:52 |
| jaosorior | therve: did you see the mail of folks suggesting to get rid of mongodb? | 07:53 |
| therve | jaosorior, Ah thanks. I was in that thread but missed the last update | 07:54 |
| *** dpawar has joined #tripleo | 07:56 | |
| matbu | dciabrin: hello, can you take a look at https://review.openstack.org/#/c/498499 | 07:57 |
| matbu | when you have a moment | 07:57 |
| *** paramite has joined #tripleo | 08:02 | |
| *** ffiore has quit IRC | 08:04 | |
| *** suuuper has joined #tripleo | 08:07 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 08:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 08:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 08:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 08:10 |
| *** ffiore has joined #tripleo | 08:10 | |
| *** dsariel has joined #tripleo | 08:14 | |
| matbu | one easy review plz https://review.openstack.org/#/c/480461/ | 08:20 |
| matbu | and another one: https://review.openstack.org/#/c/479212/ | 08:21 |
| *** milan has joined #tripleo | 08:26 | |
| *** stendulker has joined #tripleo | 08:28 | |
| *** ffiore has quit IRC | 08:28 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use integers for rabbitmq ports https://review.openstack.org/498711 | 08:28 |
| openstackgerrit | Merged openstack/tripleo-docs master: Typo: removed backslash which ended up in the generated HTML version https://review.openstack.org/497939 | 08:28 |
| *** bogdando has joined #tripleo | 08:29 | |
| *** ratailor|Lunch is now known as ratailor | 08:29 | |
| honza | mwhahaha: apetrich: d0ugal: i just got back from vacation, is there anything i can do to help move this patch across the finish line? https://review.openstack.org/#/c/469608/ I feel rather unqualified to resolve the CI issues but I'm happy to learn. | 08:30 |
| sshnaidm | who is Steve Baker on IRC? | 08:31 |
| apetrich | honza, I'm on it. The CI is not logging the file that we need to understand the problem, I'm trying to reproduce the CI run locally so I can look at that file | 08:31 |
| apetrich | d0ugal, ^^ | 08:31 |
| honza | apetrich: perfect, thank you | 08:32 |
| mandre | sshnaidm: try stevebaker :) | 08:32 |
| sshnaidm | stevebaker, hi | 08:32 |
| sshnaidm | mandre, thanks :) | 08:32 |
| mandre | sshnaidm: he's in NZ, so his night now | 08:33 |
| sshnaidm | mandre, oh, ok.. 8 PM | 08:34 |
| mandre | sshnaidm: anything I can help you with maybe? | 08:35 |
| sshnaidm | mandre, no, just wanted him to remove +w from some patch.. | 08:35 |
| sshnaidm | nevermind | 08:35 |
| openstackgerrit | Lukas Bezdicka proposed openstack/python-tripleoclient master: Add setup-heat-outputs option to get heat stack outputs https://review.openstack.org/495658 | 08:35 |
| mandre | sshnaidm: iiuc if someone else -2 or -w over it, that won't merge | 08:36 |
| sshnaidm | mandre, good to know, thanks | 08:37 |
| d0ugal | apetrich: what file isn't logged? | 08:38 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-heat-templates master: DONT REVIEW: test build patch https://review.openstack.org/498712 | 08:39 |
| apetrich | d0ugal, so it fails on a disk thingmagick manipulation that is logged as > /var/lib/oooq-images/master/repo_setup.sh.log and var/lib is not on the logs | 08:40 |
| apetrich | d0ugal, it is set to collect but didn't | 08:40 |
| skramaja | have a question on container image building.. do we use directly kolla-build command to build images [or] invoked via "openstack overcloud container image build " command? | 08:42 |
| *** bogdando has quit IRC | 08:48 | |
| d0ugal | apetrich: oh, weird | 08:48 |
| *** gfidente has joined #tripleo | 08:49 | |
| *** gfidente has quit IRC | 08:49 | |
| *** gfidente has joined #tripleo | 08:49 | |
| *** bogdando has joined #tripleo | 08:53 | |
| *** tosky has joined #tripleo | 08:55 | |
| mrunge | skramaja: the guide says openstack overcloud container image build | 08:56 |
| mrunge | skramaja: but internally, it calls kolla-build | 08:56 |
| mrunge | as far as I found out | 08:57 |
| *** ykarel|lunch is now known as ykarel | 08:57 | |
| skramaja | thanks mrunge i am trying to find how the templates-override options is provided via "overcloud container image build" as i am unable to find the related option.. | 08:58 |
| mrunge | skramaja: good Q. | 08:58 |
| *** lucas-afk is now known as lucasagomes | 08:59 | |
| mrunge | I've always used kolla-build, and you can edit /etc/kolla/kolla-build.conf | 08:59 |
| skramaja | mrunge: the doc here https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html, referes to kolla-build command.. | 08:59 |
| mrunge | right | 08:59 |
| mrunge | skramaja: there is more: https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html | 09:01 |
| mrunge | meh, same link | 09:01 |
| *** ratailor_ has joined #tripleo | 09:01 | |
| *** ratailor_ has quit IRC | 09:01 | |
| *** ratailor_ has joined #tripleo | 09:02 | |
| *** hewbrocca_afk is now known as hewbrocca | 09:04 | |
| *** ratailor has quit IRC | 09:04 | |
| *** garyk1 has joined #tripleo | 09:07 | |
| skramaja | mrunge: hmm.. an option to do it, from dprince helper scripts - https://github.com/dprince/undercloud_containers/blob/master/build_kolla.sh#L23 | 09:10 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 09:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 09:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 09:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 09:10 |
| *** limao has quit IRC | 09:11 | |
| *** aditya_r has quit IRC | 09:16 | |
| *** aditya_r has joined #tripleo | 09:16 | |
| *** sshnaidm is now known as sshnaidm|afk | 09:18 | |
| *** fzdarsky has quit IRC | 09:20 | |
| *** ratailor has joined #tripleo | 09:22 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Enable TLS for rabbitmq's replication traffic https://review.openstack.org/498324 | 09:23 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Rabbitmq: Enable Erlang distribution TLS and add CA file https://review.openstack.org/498325 | 09:23 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use FQDN for nodename in rabbitmq configuration https://review.openstack.org/498726 | 09:25 |
| *** ratailor_ has quit IRC | 09:26 | |
| adarazs | jaosorior: https://review.openstack.org/496639 looks good, but gfidente has a question on it that's not answered yet. I can merge it otherwise. | 09:28 |
| mrunge | skramaja: so, he's basically providing a kolla-build.config | 09:28 |
| skramaja | mrunge: yes. and the conf file is provided as input to the "overcloud container image build" command | 09:29 |
| mrunge | exactly | 09:29 |
| *** fzdarsky has joined #tripleo | 09:30 | |
| *** ffiore has joined #tripleo | 09:40 | |
| *** pkovar has joined #tripleo | 09:43 | |
| *** psahoo has quit IRC | 09:47 | |
| adarazs | sshnaidm|afk: can you please take a look at this change: https://review.openstack.org/498688 -- we should have it in ASAP. | 09:50 |
| gfidente | leseb with jprovazn there is an env where the osd bootstrap seems to fail | 09:51 |
| gfidente | apparently the bootstrap osd keyring is missing in the osd container | 09:52 |
| gfidente | but I couldn't really figure out the sequence of operations from entrypoint.sh down to the command which failed | 09:52 |
| gfidente | you have time to check with him? | 09:52 |
| leseb | gfidente yes | 09:53 |
| jprovazn | leseb: cool, let me send you details | 09:54 |
| *** udesale has quit IRC | 10:02 | |
| bogdando | weshay: hi. Could you please re-vote https://review.openstack.org/#/c/492178/ ? | 10:03 |
| *** rook is now known as rook|pto | 10:06 | |
| *** ratailor has quit IRC | 10:07 | |
| *** ratailor has joined #tripleo | 10:08 | |
| jprovazn | gfidente: ansible tasks were logged in mistral executor.log on UC? | 10:09 |
| gfidente | hey sorry guys | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 10:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 10:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 10:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 10:10 |
| gfidente | leseb jprovazn read the tmux chat | 10:15 |
| leseb | gfidente looks like the problem is that the disk had partitions already | 10:15 |
| leseb | cpeh partitions | 10:15 |
| leseb | *ceph | 10:15 |
| gfidente | so yeah oooq reuses the same disks which are pre-partitioned and potentially with ceph if there has been a previous attempt | 10:15 |
| gfidente | right ironic can do the cleanup it'st just not enabled by default | 10:15 |
| gfidente | cause it's a bit time consuming | 10:15 |
| gfidente | so oooq does not | 10:16 |
| gfidente | let me find the setting | 10:16 |
| leseb | sure | 10:16 |
| jprovazn | should ceph-ansible repartition the disk (at least if an option is used)? | 10:17 |
| gfidente | jprovazn no I think that is risky | 10:17 |
| jprovazn | gfidente: if there is an option "--i-really-mean-it"? | 10:17 |
| gfidente | yeah but then if you pass a parameter like that from tht | 10:17 |
| sshnaidm|afk | adarazs, looking.. | 10:17 |
| gfidente | it's stuck in heat for every stack update | 10:17 |
| gfidente | I think cleanup on deletion is good approach | 10:18 |
| leseb | gfidente +1 | 10:18 |
| jprovazn | gfidente: well, many times I re-run oooq with an existing OC deployed | 10:18 |
| jprovazn | gfidente: so I'm afraid that cleanup in THT will not help | 10:18 |
| gfidente | in ironic? | 10:19 |
| jprovazn | gfidente: it depends when deletion of ironic nodes is done? | 10:19 |
| gfidente | when the node is released | 10:19 |
| *** sshnaidm|afk is now known as sshnaidm | 10:19 | |
| sshnaidm | adarazs, a few questions about https://review.openstack.org/#/c/498688/1/scripts/shell-builders/rdo-infra-config/build-containers-images.sh | 10:20 |
| jprovazn | gfidente: if it covers scenario when I have fully deployed oooq and then jsut run oooq again with -T all, then it's fine | 10:20 |
| jprovazn | gfidente: aha, ok | 10:20 |
| sshnaidm | adarazs, why do we set trunk_repository and then unconditionally override it? | 10:20 |
| jprovazn | back in 30 | 10:21 |
| sshnaidm | adarazs, that patch means we never run kolla for master, is it intentionally? | 10:21 |
| gfidente | jprovazn automated_clean=true | 10:21 |
| gfidente | jprovazn in [conductor] should do | 10:21 |
| gfidente | jprovazn https://docs.openstack.org/ironic/latest/admin/cleaning.html | 10:21 |
| openstackgerrit | Janki Chhatbar proposed openstack/python-tripleoclient master: Add --neutron-drvier argument to prepare command https://review.openstack.org/498743 | 10:23 |
| *** dtantsur|afk is now known as dtantsur | 10:23 | |
| *** garyk has quit IRC | 10:24 | |
| janki | mandre, ^ | 10:24 |
| adarazs | sshnaidm: hmm, I saw $STABLE_RELEASE defaulting to master if not set, but maybe that's not going to happen in that script. | 10:26 |
| *** aditya_r has quit IRC | 10:27 | |
| mandre | janki: this is great but we can even automate it a bit more by inspecting the list of services that are enabled, so you don't have to introduce a new --neutron-driver option | 10:28 |
| mandre | janki: does it make sense? | 10:28 |
| sshnaidm | adarazs, afaik it's never master, because it's *stable* branch | 10:29 |
| adarazs | sshnaidm: I'll rework this. | 10:29 |
| janki | mandre, I am submitting corresponding ooo-common patch. lets discuss there? Ya. it makes sense but I am not sure how to achieve that | 10:29 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui stable/pike: Imported Translations from Zanata https://review.openstack.org/498747 | 10:29 |
| sshnaidm | adarazs, we use in scripts ${STABLE_BRANCH:-'master'} | 10:29 |
| adarazs | sshnaidm: yeah, I saw some places where it's defaulted to master, but I don't think it propagates here from any of those scripts. | 10:30 |
| *** skramaja has quit IRC | 10:32 | |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-common master: Add ODL container images to TripleO-common. https://review.openstack.org/498383 | 10:32 |
| janki | mandre, ^ | 10:32 |
| adarazs | sshnaidm: thanks for the review. | 10:32 |
| mandre | janki: ack, all the tripleo-common cares about is that the neutron_driver variable is set, let's discuss the best way to set it in your python-tripleoclient patch | 10:32 |
| janki | mandre, sure | 10:33 |
| numans | janki, mandre can i follow this patch https://review.openstack.org/498383 and do the same for ovn ? | 10:38 |
| janki | numans, lets decide on how to pass neutron-driver to ooo-common first. currently the patch is half done. You can do that now or can do it later once the logic is fully implemented. | 10:40 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-common master: Add clustercheck healthcheck https://review.openstack.org/497468 | 10:40 |
| *** akrivoka has joined #tripleo | 10:45 | |
| *** jkilpatr has quit IRC | 10:46 | |
| *** ykarel is now known as ykarel|afk | 10:46 | |
| *** akrivoka has quit IRC | 10:46 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci master: Use release specific repos during containers build https://review.openstack.org/498688 | 10:47 |
| adarazs | sshnaidm: ^ can you check it out again? | 10:47 |
| sshnaidm | adarazs, it's playbook, you can't use env vars like "$RELEASE", you need the lookup('env', 'RELEASE') etc | 10:48 |
| adarazs | sshnaidm: it's being cat-ed and the variable is defined. let me fetch the config for it. | 10:49 |
| sshnaidm | adarazs, oh, right, sorry | 10:49 |
| *** akrivoka has joined #tripleo | 10:50 | |
| adarazs | sshnaidm: might be more elegant with a lookup though, but it shouldn't make a practical difference here :) | 10:51 |
| sshnaidm | adarazs, do you know where is $RELEASE defined? | 10:51 |
| sshnaidm | adarazs, yeah, you're right | 10:51 |
| adarazs | sshnaidm: yes, fetching that part of the config. | 10:51 |
| adarazs | just gitweb is slow. | 10:51 |
| sshnaidm | adarazs, I use usually github for it, it highlights the lines | 10:52 |
| mandre | janki: I pointed to code example in https://review.openstack.org/#/c/498743, I think you can easily reuse the service_filter variable | 10:52 |
| adarazs | sshnaidm: https://review.rdoproject.org/r/gitweb?p=config.git;a=blob;f=jobs/tripleo-upstream.yml;h=e883555e504d88c82ad6649f3fed4de32a1e165e;hb=HEAD#l76 | 10:52 |
| janki | mandre, checking the commit. let me try doing it this way. | 10:52 |
| adarazs | sshnaidm: yeah, but do we have this rdoproject config repo anywhere mirrored to github? :) | 10:52 |
| sshnaidm | adarazs, should be.. | 10:53 |
| *** fzdarsky is now known as fzdarsky|lunch | 10:54 | |
| sshnaidm | adarazs, is it only for me - when I open any link the rdo.review page it logs me in and redirect to "my changes", without going to page which I wanted to open | 10:55 |
| sshnaidm | adarazs, and without any chance to go to the original page.. | 10:55 |
| adarazs | sshnaidm: yeah, I just realized I have the same problem. I bookmarked the gitweb page of this repo and it doesn't work. | 10:56 |
| mandre | numans, janki: maybe we can add the ovn neutron image in https://review.openstack.org/#/c/498383/ as well since we'll have https://review.openstack.org/#/c/498743/ set neutron_driver to ovn when appropriate | 10:56 |
| sshnaidm | adarazs, yeah, I need to open link twice there.. | 10:56 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use FQDN for nodename in rabbitmq configuration https://review.openstack.org/498726 | 10:56 |
| adarazs | sshnaidm: second time it works, yep. not the best UX :) | 10:56 |
| adarazs | sshnaidm: that top bar iframe messes with things. | 10:57 |
| jprovazn | gfidente: I wonder which ironic clean step should be used for cleaning up non-root disk partitions in case of manual cleanup | 10:58 |
| jprovazn | dtantsur: ^ maybe you might know ? | 10:58 |
| sshnaidm | adarazs, +2 | 10:59 |
| dtantsur | jprovazn: ironic cleaning cleans all disks | 10:59 |
| dtantsur | well, all it can find | 10:59 |
| sshnaidm | adarazs, we push containers to trunk.registry.rdoproject.org, right? | 10:59 |
| adarazs | sshnaidm: yeah. I think we will need to push them to docker.io as well, but for now there. | 10:59 |
| sshnaidm | ok | 10:59 |
| adarazs | sshnaidm: also it seems to be working for the master jobs. | 11:00 |
| adarazs | sshnaidm: crap, now that I pushed that change I can't +2 it I guess. | 11:00 |
| sshnaidm | adarazs, you can merge it actually, ci jobs don't make sense there | 11:00 |
| adarazs | sshnaidm: I'm not talking about CI jobs, just another pair of eyes. but yeah, I think we could get it going. not much to lose but time. :) | 11:01 |
| dtantsur | jprovazn: http://tripleo.org/install/mistral-api/mistral-api.html#cleaning-nodes | 11:02 |
| *** jkilpatr has joined #tripleo | 11:04 | |
| lvdombrkr | guys, im trying enable ssl for overcloud, in ofic documentation i can read: To start, it is necessary to have a predictable public VIP. As of this writing, that means using network isolation. | 11:05 |
| *** ffiore has quit IRC | 11:05 | |
| lvdombrkr | is there any posibility enable ssl without network isolation? | 11:06 |
| *** ansmith has quit IRC | 11:06 | |
| jprovazn | dtantsur: nice. thanks | 11:07 |
| *** garyk1 has quit IRC | 11:08 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 11:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 11:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 11:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 11:10 |
| *** stendulker has quit IRC | 11:15 | |
| gfidente | dtantsur again on the automated_clean default | 11:17 |
| gfidente | do I remember correctly it is enabled by default in ironic, it is enabled by default in undercloud.conf but oooq is disabling it? | 11:17 |
| lvdombrkr | can my network. yaml look like this http://paste.openstack.org/raw/619745/ | 11:17 |
| lvdombrkr | if i want just add ssl | 11:18 |
| lvdombrkr | ? | 11:18 |
| jprovazn | gfidente: http://tripleo.org/install/mistral-api/mistral-api.html#cleaning-nodes says " As TripleO disables automated cleaning,...." | 11:18 |
| lvdombrkr | guys, im trying enable ssl for overcloud, can i do this without network isolation? or with simple .yaml like http://paste.openstack.org/raw/619745/ | 11:23 |
| lvdombrkr | ? | 11:23 |
| *** Vijayendra has quit IRC | 11:24 | |
| gfidente | jprovazn ah https://github.com/openstack/instack-undercloud/blob/1d8ad41db9cba8f23ccba4a78f7a13405e57ef5d/instack_undercloud/undercloud.py#L373 | 11:24 |
| jprovazn | gfidente: dtantsur: I can imagine that combination of disabled cleanup (by default) and ceph-ansible may cause quite a lot of pain to tripleo deployers | 11:25 |
| gfidente | but it is true in ironic https://github.com/openstack/ironic/blob/master/ironic/conf/conductor.py#L119 | 11:26 |
| gfidente | so I think the default in tripleo is a safe measure | 11:28 |
| sshnaidm | adarazs, do you know why was it added? https://review.openstack.org/#/c/475747/108/toci-quickstart/playbooks/multinode.yml | 11:29 |
| gfidente | we should probably add in the doc a comment about that pointing to how to cleanup the node disks in case of repeated ceph deployments | 11:29 |
| sshnaidm | adarazs, why do we need to build images in multinode and how is it related to that containers job? | 11:29 |
| dtantsur | gfidente, jprovazn, it's disabled by default (mostly historically) | 11:30 |
| dtantsur | I have nothing against enabling, but 1. it will make enrolling nodes longer, 2. it will make people who wipe their deployments very sad, as recovery won't be possible :) | 11:30 |
| *** eck` is now known as eck`gone | 11:31 | |
| *** Vijayendra has joined #tripleo | 11:32 | |
| *** garyk has joined #tripleo | 11:32 | |
| jaosorior | dciabrin, bandini: would using FQDNs for rabbitmq's NODENAME work? or will pacemaker have issues with it? | 11:34 |
| bandini | mmh hard to say. I'd be surprised if something did not break tbh | 11:35 |
| jaosorior | :( | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Heat] https://review.openstack.org/491399 | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Aodh] https://review.openstack.org/491411 | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Mistral] https://review.openstack.org/493471 | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Ceilometer] https://review.openstack.org/490392 | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Neutron] https://review.openstack.org/489957 | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Gnocchi] https://review.openstack.org/491433 | 11:36 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Mixed-1] https://review.openstack.org/495195 | 11:36 |
| *** udesale has joined #tripleo | 11:36 | |
| *** ratailor has quit IRC | 11:38 | |
| *** pdeore has quit IRC | 11:38 | |
| *** pgadiya has quit IRC | 11:39 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Multinode jobs should not build images https://review.openstack.org/498772 | 11:40 |
| adarazs | sshnaidm: this is an interesting question. so we have a couple of multinode promotion jobs that are not child jobs of the image building job. don't they need images for the overcloud nodeS? | 11:42 |
| sshnaidm | adarazs, neither of multinode jobs needs images, we use deployed server ^^ | 11:42 |
| jaosorior | bandini: I think we'll need a cluster_nodes option like the galera resource agent :/ | 11:42 |
| jaosorior | dciabrin: ^^ | 11:42 |
| bandini | oh boy | 11:43 |
| *** pdeore has joined #tripleo | 11:43 | |
| adarazs | sshnaidm: hm, okay. then it's probably not necessary. I checked some multinode job and indeed it doesn't build the image. | 11:43 |
| sshnaidm | adarazs, maybe we need to build them for promotion only..? but we do it in ovb jobs, never did this in multinode, because no way to test them | 11:43 |
| *** shreshtha has quit IRC | 11:44 | |
| sshnaidm | adarazs, actually it's not configured right for building images and it fails every patch for tripleo-common, etc | 11:44 |
| adarazs | sshnaidm: looks like the image building job is running on multinode and that needs it: https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-images-build/136/consoleFull | 11:45 |
| *** ykarel|afk is now known as ykarel | 11:46 | |
| *** rhallisey has joined #tripleo | 11:46 | |
| *** trown|outtypewww is now known as trown | 11:47 | |
| sshnaidm | adarazs, why does it build them? | 11:47 |
| adarazs | sshnaidm: then it also fetches them, sooo I'm a bit confused. :) | 11:47 |
| trown | sshnaidm: it builds images in periodic job | 11:47 |
| sshnaidm | trown, why does multinode job build images? | 11:47 |
| trown | sshnaidm: https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-images-build/ | 11:47 |
| trown | sshnaidm: that job | 11:48 |
| adarazs | trown, weshay: FYI we merged this: https://review.openstack.org/498688 -- let me know if it doesn't look all right, I can probably still -w it.. | 11:48 |
| trown | adarazs: nice, that was the first thing I was going to look at this morning | 11:49 |
| sshnaidm | trown, I see, but it doesn't answer the question - why *multinode* job? | 11:49 |
| trown | adarazs: saw that pike image build "worked", but built master images | 11:50 |
| trown | sshnaidm: because the job I pointed to is a multinode job | 11:50 |
| sshnaidm | trown, it broke CI for tripleo-common, instack-undercloud and other repos that usually build images, if it's required for this job we need to limit it for rdo cloud only | 11:50 |
| adarazs | sshnaidm: why does it matter which job builds the image? it doesn't use them, just builds them. | 11:50 |
| trown | sshnaidm: but there is really no reason to build the image there... since we are not testing it there anyways | 11:50 |
| sshnaidm | trown, adarazs ovb job build the images and test them by deploying overcloud, multinode jobs don't do it | 11:51 |
| trown | sshnaidm: but then we need some way to upload from the ovb jobs in periodic pipeline | 11:51 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Remove puppet run and workarounds from tripleo_upgrade_node.sh https://review.openstack.org/498776 | 11:51 |
| trown | sshnaidm: ya the job I pointed to is just a parent job which builds and sub jobs which test | 11:51 |
| trown | sshnaidm: but we didnt add jobs which test qcow image only containers | 11:51 |
| sshnaidm | trown, adarazs so we build images that nobody uses? I'm confused.. | 11:51 |
| trown | sshnaidm: take a deep breath | 11:52 |
| adarazs | sshnaidm: I am a bit as well. I assumed that we will chain up the OVB jobs on the image build job eventually and use those built images. | 11:52 |
| trown | adarazs: sshnaidm, ^ that was the plan | 11:52 |
| trown | adarazs: sshnaidm, but we dont have to stick with that... | 11:52 |
| adarazs | but now until the image build job was working it was easier to build in each job :) | 11:52 |
| sshnaidm | trown, so what is the current plan? | 11:53 |
| trown | sshnaidm: s/was/is currently/ just saying we can change it if we have a way to upload from other jobs | 11:53 |
| trown | sshnaidm: but right now we upload from the job I linked, so if we dont build images there, we have no images to cache at the end of periodic | 11:54 |
| sshnaidm | trown, why do we need to cache them in the end of periodic..? seems like I miss the whole picture | 11:55 |
| trown | sshnaidm: I was asking for your feedback on that review for weeks fwiw | 11:55 |
| trown | sshnaidm: umm... so we have images to use when we promote? | 11:55 |
| sshnaidm | trown, yeah, and I for a weeks asked for explanation of this review | 11:55 |
| adarazs | trown: where are we uploading them? | 11:55 |
| trown | adarazs: to images.rdoproject.org | 11:56 |
| adarazs | trown: just found the upload part in the console log. so they are these, right? https://images.rdoproject.org/master/delorean/tripleo-upstream/testing/ | 11:57 |
| trown | adarazs: ya | 11:57 |
| *** abishop has joined #tripleo | 11:57 | |
| jaosorior | jistr, mwhahaha, EmilienM: Can you check this out https://review.openstack.org/#/c/498711/ ? | 11:57 |
| trown | adarazs: then on promote we would move those to passed-ci dir or the like | 11:57 |
| sshnaidm | trown, can't we upload from ovb job? | 11:57 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Enable TLS for rabbitmq's replication traffic https://review.openstack.org/498324 | 11:57 |
| adarazs | trown: ack. | 11:58 |
| trown | sshnaidm: can you spend some time to understand what we have setup in rdoproject.org? | 11:58 |
| sshnaidm | trown, yeah, if you have some information for me | 11:58 |
| *** lucasagomes is now known as lucas-hungry | 11:58 | |
| trown | sshnaidm: we could upload from an ovb job... but how do we know the status of all jobs from a single job | 11:58 |
| trown | sshnaidm: but I guess in this case we dont care about that actually... so ya we could add to one of the ovb jobs to upload its images to testing link | 11:59 |
| trown | sshnaidm: promote is a later step anyways | 12:00 |
| sshnaidm | trown, ok, anyway, I need either to remove build image part from multinode playbook, or limit it for rdocloud only and configure it correctly | 12:00 |
| sshnaidm | trown, right now it's bug | 12:00 |
| sshnaidm | trown, so let's decide either we use multinode job or not | 12:00 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Rabbitmq: Enable Erlang distribution TLS https://review.openstack.org/498325 | 12:01 |
| trown | sshnaidm: well if you remove it you have to fix jjb in rdoproject.org | 12:01 |
| trown | sshnaidm: the upload is called from the jjb and it will fail if there are not images to upload | 12:01 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Rabbitmq: Enable Erlang distribution TLS https://review.openstack.org/498325 | 12:01 |
| *** ffiore has joined #tripleo | 12:02 | |
| sshnaidm | trown, just curios, do you have the current workflow described anywhere that I can see it? | 12:03 |
| trown | sshnaidm: ya rdoproject.org jjb | 12:04 |
| sshnaidm | trown, it's code, not docs | 12:04 |
| sshnaidm | it's not even code, it's *jjb* | 12:05 |
| *** fzdarsky|lunch is now known as fzdarsky | 12:07 | |
| *** ffiore has quit IRC | 12:09 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 12:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 12:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 12:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 12:10 |
| *** jlabarre has joined #tripleo | 12:11 | |
| openstackgerrit | Or Idgar proposed openstack/tripleo-common master: WIP Add Octavia post deployment steps to tripleo workflows https://review.openstack.org/487450 | 12:11 |
| *** morazi has joined #tripleo | 12:11 | |
| weshay | trown, sshnaidm reading through just checking https://review.openstack.org/#/c/498772/1/toci-quickstart/playbooks/multinode.yml addresses https://bugs.launchpad.net/tripleo/+bug/1713167 ? | 12:12 |
| openstack | Launchpad bug 1713167 in tripleo "build-images : Throw error msg if image build is failing *** Image Build Error ***" [High,Triaged] | 12:12 |
| *** atoth has joined #tripleo | 12:13 | |
| *** jpena is now known as jpena|lunch | 12:14 | |
| sshnaidm | weshay, yes | 12:16 |
| weshay | sshnaidm, trown k.. thanks guys | 12:16 |
| *** pchavva has joined #tripleo | 12:19 | |
| adarazs | weshay, sshnaidm: it addresses like cutting off a leg when you get a wound -- this would result in the periodic rdoproject.org jobs not building and uploading images for now and fail when trying to upload them. not that we currently use that images, but I think we should tune the to_build script to return stuff properly instead of removing it. | 12:20 |
| *** MVenesio has joined #tripleo | 12:20 | |
| *** ffiore has joined #tripleo | 12:20 | |
| sshnaidm | adarazs, weshay I don't so like the idea to build images with multinode job | 12:21 |
| weshay | adarazs, sshnaidm ok.. that is what env is for :) | 12:21 |
| adarazs | sshnaidm: why does it matter what nodes the image build runs on if we verify them in other jobs? | 12:22 |
| sshnaidm | weshay, depends.. env couldn't help if it's calculated dynamically as it happens now | 12:22 |
| *** dtantsur is now known as dtantsur|busy | 12:23 | |
| sshnaidm | adarazs, if we verify them in other jobs it doesn't matter, but it will save us this step | 12:23 |
| sshnaidm | adarazs, actually we never built images in multinode, so need to check it before | 12:24 |
| weshay | the check vs periodic vs. rdo soft factory jobs could all run w/ diff env files | 12:24 |
| *** aditya_r has joined #tripleo | 12:24 | |
| *** bkopilov has quit IRC | 12:25 | |
| *** aditya_r has quit IRC | 12:25 | |
| *** bkopilov has joined #tripleo | 12:25 | |
| *** aditya_r has joined #tripleo | 12:25 | |
| weshay | adarazs, where are we failing on upload, do you have a log? | 12:28 |
| adarazs | weshay: we're not, but if sshnaidm would take out the part that builds the images then we would fail in the rdoproject.org image building jobs on the upload. | 12:28 |
| *** oidgar has quit IRC | 12:28 | |
| *** aditya_r has quit IRC | 12:29 | |
| sshnaidm | weshay, shouldn't we have this workflow described anywhere? like diagram or chart.. | 12:29 |
| sshnaidm | using jjb as source of information is a nightmare | 12:30 |
| weshay | sshnaidm, that would certainly be helpful | 12:30 |
| sshnaidm | jjb is a labyrinth | 12:30 |
| weshay | ya.. | 12:30 |
| weshay | sshnaidm, adarazs could we not just set to_build appropriately in a env config? | 12:32 |
| *** pdeore has quit IRC | 12:32 | |
| weshay | major changes at this point of the release is not ideal | 12:33 |
| *** pdeore has joined #tripleo | 12:33 | |
| adarazs | weshay: I think that would be better, if we can. | 12:33 |
| weshay | well, I know we can.. adarazs sshnaidm want to hash it out? | 12:34 |
| * adarazs looks at the to_build script. | 12:34 | |
| *** aditya_r has joined #tripleo | 12:36 | |
| sshnaidm | weshay, to_build builds according to zuul changes, how is it related to building periodic? | 12:36 |
| *** ykarel has quit IRC | 12:36 | |
| *** janki has quit IRC | 12:37 | |
| sshnaidm | weshay, you want to build images always in periodic jobs, right? | 12:37 |
| adarazs | sshnaidm: well I assume that now that the to_build is added to the multinode job we have to just make a case for "when multinode don't build an image, unless it's periodic/promotion"? | 12:37 |
| adarazs | or unless it's that image building job specificially | 12:38 |
| sshnaidm | adarazs, weshay do we have a var that identifies rdo cloud? | 12:38 |
| weshay | sshnaidm, adarazs taking a step back, has anyone confirmed that fetch_images is actually getting the centos image | 12:38 |
| weshay | atm the error afaict is that the image was not found | 12:38 |
| sshnaidm | weshay, no, it doesn't build correctly at all, it's next step.. | 12:39 |
| weshay | ok | 12:39 |
| weshay | sshnaidm, and why is it not building correctly? | 12:39 |
| *** gkadam_ has quit IRC | 12:39 | |
| weshay | what is the error? | 12:39 |
| sshnaidm | weshay, because it's not configured correctly | 12:39 |
| weshay | sshnaidm, have an example? | 12:40 |
| sshnaidm | it's not configured at all and tries to build as it was in libvirt.. | 12:40 |
| sshnaidm | weshay, the logs you sent are an example | 12:40 |
| *** pdeore_ has joined #tripleo | 12:40 | |
| *** rlandy has joined #tripleo | 12:41 | |
| weshay | sshnaidm, so which log file indicates to you it's not configured correctly? | 12:41 |
| rnoriega | EmilienM, ping | 12:41 |
| sshnaidm | weshay, all of them :) | 12:41 |
| adarazs | sshnaidm: do we have a variable that is multinode/ovb so that we can check on it in to_build? | 12:41 |
| sshnaidm | adarazs, mm.. not sure I get it, what do you want to do..? | 12:41 |
| *** pdeore has quit IRC | 12:42 | |
| jaosorior | dciabrin: any reason why this was added https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/pacemaker/database/redis_bundle.pp#L136-L140 ? | 12:42 |
| weshay | sshnaidm, all I can see from the logs is that the isolation image is not present | 12:42 |
| adarazs | sshnaidm: wrap this in a "if multinode set_env false", else: https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/to_build#L12-L17 | 12:42 |
| jaosorior | bandini: is it possible to restart a service from a specific node with pacemaker? | 12:43 |
| sshnaidm | weshay, it's a symptom, not a problem | 12:43 |
| sshnaidm | weshay, this how the images build should be configured: https://github.com/openstack-infra/tripleo-ci/blob/3a18965dae750f17154eb10121266f6ce72255d4/toci-quickstart/config/testenv/ovb.yml#L24-L33 | 12:43 |
| *** eck`gone is now known as eck` | 12:43 | |
| *** tzumainn has joined #tripleo | 12:44 | |
| weshay | sshnaidm, ah.. so build_image_isolated should be false? | 12:44 |
| sshnaidm | adarazs, let's clarify the condition when we build - periodic=true && rdocliud=true && ... something else? | 12:45 |
| sshnaidm | weshay, of course | 12:45 |
| weshay | sshnaidm, ya.. we already have some variables defined to distinguish between jobs in upstream vs rdo software factory | 12:46 |
| weshay | and periodic vs. check | 12:46 |
| sshnaidm | weshay, adarazs should we build in every periodic rdocloud job? | 12:46 |
| sshnaidm | or on specific job? | 12:47 |
| adarazs | sshnaidm: I think only in the image build job. /me looks at the beloved JJB definitions :) | 12:47 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci master: WIP -- switch multinode composable upgrade job to oooq https://review.openstack.org/463580 | 12:47 |
| sshnaidm | or maybe just let's build on ovb jobs.. | 12:47 |
| weshay | sshnaidm, atm that is what we were doing because there was no parent / child jobs upstream | 12:47 |
| sshnaidm | adarazs, then maybe let's take TOCI_JOBTYPE into consideration | 12:47 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci master: WIP -- switch multinode composable upgrade job to oooq https://review.openstack.org/463580 | 12:47 |
| weshay | you can't build the image in one job, upload it and then have other jobs consume it w/ the older style tripleo promotion jobs | 12:47 |
| *** cdearborn has joined #tripleo | 12:48 | |
| weshay | sshnaidm, so afaict.. there wasn't much of a choice | 12:48 |
| sshnaidm | weshay, it saves us 30 mins and brings problems for a days.. | 12:48 |
| sshnaidm | weshay, does it worth this 30 min? | 12:48 |
| weshay | sshnaidm, it's not something we could change afaict while both set of jobs are running | 12:49 |
| weshay | it would have to be done after we bring sf jobs online | 12:49 |
| weshay | both sets have to be functional atm | 12:50 |
| *** ecerquei has joined #tripleo | 12:50 | |
| sshnaidm | weshay, sorry, I don't understand what you mean.. | 12:51 |
| jaosorior | eck`: I just tested out the patches (non-containerized, containerized testing coming next) and they work. Now, the only issue is that I couldn't enable verification because pacemaker tracks shortnames... and we can only issue certificates for FQDNs | 12:51 |
| weshay | sshnaidm, let's bluejeans real quick | 12:51 |
| bandini | jaosorior: pcs resource restart foo node-bar | 12:52 |
| weshay | adarazs, trown fyi.. if you are interested in discussing the image build issue w/ the isolated image | 12:52 |
| *** ansmith has joined #tripleo | 12:52 | |
| jaosorior | eck`: so, the current state would meet the minimum requirement (encryption everywhere); but it's not as secure as I wish it was. So alternatively we need to add cluster-names for the rabbitmq resource agent, as we do for mysql | 12:52 |
| adarazs | weshay: joining | 12:52 |
| eck` | jaosorior: awesome, i will try to sanity check ipv6+tls if you haven't done so already | 12:52 |
| weshay | my bluejeans | 12:52 |
| jaosorior | bandini: how disruptive is that? thinking about using that for certmonger's post-save command | 12:53 |
| jaosorior | eck`: please do, I haven't checked that. | 12:53 |
| jaosorior | eck`: what do you think of the commits overall though? | 12:53 |
| eck` | jaosorior: looked fine to me! | 12:54 |
| jaosorior | great! | 12:54 |
| *** jmelvin has joined #tripleo | 12:55 | |
| bandini | jaosorior: meh, actually you cannot do it for bundle it appears | 12:55 |
| jaosorior | bandini: no??? :( | 12:56 |
| jaosorior | bandini: why? | 12:56 |
| bandini | not sure where the limitation comes from, but pcs for sure is not liking that | 12:56 |
| jaosorior | shit fuck | 12:57 |
| jaosorior | X_X | 12:57 |
| jaosorior | bandini: know who I can ask? | 12:57 |
| bandini | jaosorior: I just asked andrew, but he is in .au, so it will have to wait until tonight | 12:58 |
| jaosorior | ok | 12:59 |
| jaosorior | thanks dude | 12:59 |
| jaosorior | bandini: now... if we could get restarts for bundles in the near future, that would be reaally neat :D | 12:59 |
| jaosorior | lets see what answer we get | 13:00 |
| *** ebarrera has quit IRC | 13:00 | |
| bandini | ack | 13:00 |
| jaosorior | bandini: need reviews? | 13:00 |
| *** number80 has quit IRC | 13:00 | |
| bandini | no thanks, am debugging some resource agents changes | 13:00 |
| *** lucas-hungry is now known as lucasagomes | 13:01 | |
| *** jcoufal has joined #tripleo | 13:01 | |
| *** toure_biab is now known as toure | 13:01 | |
| *** ebarrera has joined #tripleo | 13:02 | |
| *** pblaho has quit IRC | 13:02 | |
| *** number80 has joined #tripleo | 13:02 | |
| cdearborn | Hey folks, I've run into a problem where I have a node that I want to deploy ceph to, and it has 2 disk controllers in it. | 13:04 |
| cdearborn | The order in which the disks are enumerated is either all disks on controller A, then all disks on controller B or vice versa | 13:05 |
| cdearborn | This makes it so the OSD/journal configuration ends up being messed up | 13:05 |
| cdearborn | because the OS drive could be /dev/sda or /dev/sds | 13:05 |
| cdearborn | I've been trying to find a solution to this | 13:06 |
| cdearborn | I was wondering if it is possible to specify the OSD/journal mapping separately for each ceph node? | 13:06 |
| *** garyk has quit IRC | 13:07 | |
| *** aditya_r has quit IRC | 13:07 | |
| *** aditya_r has joined #tripleo | 13:07 | |
| *** aditya_r has quit IRC | 13:08 | |
| *** aufi has quit IRC | 13:08 | |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: [WIP] Make curl healthchecks work with internal TLS https://review.openstack.org/498805 | 13:08 |
| *** morazi has quit IRC | 13:08 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 13:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 13:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 13:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 13:10 |
| *** morazi has joined #tripleo | 13:10 | |
| *** aufi has joined #tripleo | 13:11 | |
| jistr | cdearborn: hi, personally i haven't tried this, but i think this may solve the issue http://tripleo.org/install/advanced_deployment/node_specific_hieradata.html | 13:11 |
| jistr | cdearborn: assuming you deploy stable, not Pike yet | 13:12 |
| *** leitan has joined #tripleo | 13:12 | |
| cdearborn | jistr: Thanks! I'm actually using Newton. Any idea if this capability is in that release? | 13:13 |
| gfidente | cdearborn jistr++ that does it yes | 13:14 |
| adarazs | sshnaidm: FYI this is what we should remove to not upload images: https://review.rdoproject.org/r/gitweb?p=config.git;a=blob;f=jobs/tripleo-upstream.yml;h=e883555e504d88c82ad6649f3fed4de32a1e165e;hb=HEAD#l233 | 13:14 |
| gfidente | thanks | 13:14 |
| *** dpawar has quit IRC | 13:14 | |
| gfidente | cdearborn note the uuid you need to use is not the ironic node uuid, it is the system uuid | 13:14 |
| jistr | cdearborn: yea the file is present in stable/newton so i think it should work https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/puppet/extraconfig/pre_deploy/per_node.yaml | 13:14 |
| cdearborn | gfidente, jistr: thanks! my thought was i could probe the ceph nodes and build an OSD/journal config using /dev/disk/by-id/* device names | 13:16 |
| *** eck` is now known as eck`gone | 13:16 | |
| cdearborn | gfidente, jistr: any other thoughts on how to work around the issue? | 13:16 |
| gfidente | cdearborn you can also use the dev aliases but not in newton I think | 13:17 |
| cdearborn | gfidente: can you clarify? | 13:17 |
| gfidente | cdearborn when specifying the disk names, you can refer to the aliases which udev creates | 13:18 |
| gfidente | instead of /dev/sdX | 13:18 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Multinode jobs should not build images https://review.openstack.org/498772 | 13:18 |
| *** links has quit IRC | 13:18 | |
| cdearborn | gfidente, right, that's what I meant above by using /dev/disk/by-id/* | 13:18 |
| gfidente | cdearborn yeah but then you'd still need per-node data right? | 13:19 |
| *** catintheroof has joined #tripleo | 13:19 | |
| cdearborn | gfidente: yes, would love to solve this without having to use per-node data | 13:19 |
| *** eck`gone is now known as eck` | 13:19 | |
| *** ecerquei_ has joined #tripleo | 13:20 | |
| gfidente | cdearborn yeah but I don't think that is possible because I don't think we have control over the disk discovery | 13:20 |
| cdearborn | gfidente: I looked into using /dev/disk/by-path/* for device names, but those names are specific by node too in RHEL 7.4 because they include the SASAddress of the dirve | 13:20 |
| *** jpena|lunch is now known as jpena | 13:20 | |
| gfidente | cdearborn in pike you should be able to use regexp | 13:21 |
| gfidente | but earlier I think the only option is per-node data | 13:21 |
| cdearborn | gfidente, really! regexp would be very nice | 13:22 |
| *** ecerquei has quit IRC | 13:22 | |
| gfidente | cdearborn yeah though that is pike | 13:22 |
| *** lblanchard has joined #tripleo | 13:24 | |
| cdearborn | gfidente, jistr, The only other solution I could come up with is to modify the udev rules so that the by-path names generated are names that are the same across nodes - no SASAddresses in names, maybe bay numbers or something like that | 13:25 |
| *** lucasagomes is now known as lucas-brb | 13:26 | |
| cdearborn | gfidente, jistr, maybe switch udev rules back to what they were in RHEL 7.3, because the by-path names did not include the SASAddresses in that version of RHEL | 13:26 |
| cdearborn | gfidente, jistr, any thoughts on this approach? | 13:26 |
| gfidente | cdearborn this is actually interesting | 13:26 |
| gfidente | as I didn't know the behavior changed in newer RHEL | 13:27 |
| *** oidgar has joined #tripleo | 13:27 | |
| *** oidgar has quit IRC | 13:27 | |
| *** oidgar has joined #tripleo | 13:28 | |
| cdearborn | gfidente, hmmm - one other thing that changed was the generation of server that I'm using, so i guess that could have changed due to the driver and not due to RHEL version. will check into that | 13:29 |
| *** udesale has quit IRC | 13:30 | |
| *** mrcc has joined #tripleo | 13:30 | |
| gfidente | cdearborn++ thanks let us know how it goes | 13:32 |
| *** masco has quit IRC | 13:32 | |
| *** ykarel has joined #tripleo | 13:34 | |
| *** morazi has quit IRC | 13:35 | |
| *** morazi has joined #tripleo | 13:36 | |
| *** garyk has joined #tripleo | 13:37 | |
| cdearborn | gfidente, ah, i see the difference. rhel 7.4, but the by-path names in one server are scsi, and the by-path names in the other server are sas with SASAddress | 13:38 |
| cdearborn | gfidente, the scsi names are the same across older server, but the sas names are unique per node on newer servers. ugh | 13:39 |
| gfidente | cdearborn so you don't have good aliases to use anymore | 13:39 |
| gfidente | I see | 13:39 |
| gfidente | well if you have different 'group' of servers | 13:40 |
| *** dparkes has quit IRC | 13:40 | |
| gfidente | another option is to use a custom role | 13:40 |
| gfidente | and pass two different list of disks to each role type | 13:40 |
| cdearborn | gfidente, that's a thought... that would help with the 2 different generations of the servers, but still wouldn't solve the original problem of the OS drive moving around in the enumeration on the newer server | 13:42 |
| *** ykarel has quit IRC | 13:42 | |
| EmilienM | hi | 13:42 |
| gfidente | cdearborn yeah | 13:42 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Set docker-puppet --health-cmd = /bin/true https://review.openstack.org/498812 | 13:43 |
| gfidente | cdearborn or wait for pike | 13:43 |
| *** gbarros has joined #tripleo | 13:44 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP create default settings for environment types https://review.openstack.org/495407 | 13:44 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Change devmode to deploy containerized services by default https://review.openstack.org/472412 | 13:44 |
| cdearborn | gfidente very true. will dig in to the per node config and see what i can do there | 13:44 |
| gfidente | cdearborn you can try upstream code | 13:45 |
| gfidente | to check if the regexp works | 13:45 |
| *** anshulbehl has joined #tripleo | 13:45 | |
| gfidente | it's basically ceph-ansible | 13:45 |
| *** anshul has quit IRC | 13:46 | |
| *** ffiore has quit IRC | 13:47 | |
| cdearborn | gfidente, ok, unfortunately I'm constrained to using Newton for now | 13:48 |
| *** aditya_r has joined #tripleo | 13:53 | |
| *** agurenko has quit IRC | 13:53 | |
| EmilienM | tripleo meeting in 5 min, here, you can bring topics on https://etherpad.openstack.org/p/tripleo-meeting-items | 13:55 |
| *** garyk has quit IRC | 13:56 | |
| cdearborn | gfidente, jistr, thanks for all your help! | 13:56 |
| *** ykarel has joined #tripleo | 13:57 | |
| *** jprovazn has quit IRC | 13:58 | |
| *** ffiore has joined #tripleo | 13:59 | |
| EmilienM | meeting time | 14:00 |
| EmilienM | #startmeeting tripleo | 14:00 |
| openstack | Meeting started Tue Aug 29 14:00:57 2017 UTC and is due to finish in 60 minutes. The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:01 | |
| openstack | The meeting name has been set to 'tripleo' | 14:01 |
| EmilienM | #topic agenda | 14:01 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:01 | |
| EmilienM | * review past action items | 14:01 |
| EmilienM | * CI status | 14:01 |
| EmilienM | * bugs | 14:01 |
| EmilienM | * Projects releases or stable backports | 14:01 |
| EmilienM | * Specs / blueprints | 14:01 |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Make curl healthchecks work with internal TLS https://review.openstack.org/498805 | 14:01 |
| EmilienM | * one off agenda items | 14:01 |
| EmilienM | * open discussion | 14:01 |
| EmilienM | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:01 |
| *** aufi has quit IRC | 14:01 | |
| EmilienM | Hi everyone! who is around today? | 14:01 |
| mwhahaha | hi | 14:01 |
| adarazs | \ö | 14:01 |
| *** garyk has joined #tripleo | 14:01 | |
| jkilpatr | \o | 14:02 |
| sshnaidm | 8/ | 14:02 |
| mandre | hello | 14:02 |
| pradk | o/ | 14:02 |
| jrist | o/ | 14:02 |
| cdearborn | \o | 14:02 |
| jaosorior | o/ | 14:02 |
| EmilienM | there is no action taken recorded in last week meeting | 14:02 |
| marios | o/ | 14:02 |
| EmilienM | so I'll skip "review past action items" item in the agenda | 14:02 |
| EmilienM | #topic CI status | 14:03 |
| *** openstack changes topic to "CI status (Meeting topic: tripleo)" | 14:03 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Add support for deploying RGW with ceph-ansible https://review.openstack.org/480804 | 14:03 |
| EmilienM | allright, we have a bunch of alerts which affect CI status, let's review it | 14:03 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1689556 | 14:03 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 14:03 |
| EmilienM | #link https://review.openstack.org/#/c/493695/ is the fix | 14:03 |
| EmilienM | so this one is covered | 14:03 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1691403 | 14:04 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 14:04 |
| saneax | o/ | 14:04 |
| trown | o/ | 14:04 |
| *** japestinho has joined #tripleo | 14:04 | |
| EmilienM | this one needs review | 14:04 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Derive Params OvsPmdCoreList compute issue https://review.openstack.org/498816 | 14:04 |
| beagles | o/ | 14:04 |
| EmilienM | #action team to help with reviewing https://review.openstack.org/#/q/topic:bug/1691403 (CI alert fix) | 14:04 |
| EmilienM | if you're familiar with docker & networking, please have a look ^ | 14:05 |
| EmilienM | there are some patches on quickstart as well, sshnaidm, trown, adarazs ^ | 14:05 |
| EmilienM | next one: https://bugs.launchpad.net/tripleo/+bug/1702955 | 14:06 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 14:06 |
| EmilienM | matbu: any update on this one? | 14:06 |
| * adarazs is looking | 14:06 | |
| EmilienM | the last one will be https://bugs.launchpad.net/tripleo/+bug/1712910 | 14:06 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 14:06 |
| EmilienM | which is I think wip by weshay and ssh | 14:06 |
| EmilienM | sshnaidm: ^ | 14:06 |
| sshnaidm | EmilienM, yeah, patch is on the gate I think | 14:07 |
| EmilienM | sshnaidm: https://review.openstack.org/#/c/498772 ? | 14:07 |
| sshnaidm | EmilienM, oh, yeah | 14:07 |
| abishop | o/ | 14:07 |
| EmilienM | sshnaidm: when merged and fixed, please update the launchpad | 14:07 |
| sshnaidm | EmilienM, sure | 14:08 |
| EmilienM | do we have any other CI issues at this time? | 14:08 |
| EmilienM | alright - is there anything else to discuss about CI this week? | 14:09 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Remove semicolons https://review.openstack.org/498822 | 14:09 |
| EmilienM | reminder: we have a 1h meeting, focused on tripleo ci work, on thursday. ping adarazs or me to get details | 14:09 |
| EmilienM | moving on | 14:10 |
| EmilienM | #topic bugs | 14:10 |
| *** openstack changes topic to "bugs (Meeting topic: tripleo)" | 14:10 | |
| EmilienM | #link https://launchpad.net/tripleo/+milestone/pike-rc2 | 14:10 |
| *** jaganathan has quit IRC | 14:10 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 14:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 14:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 14:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 14:10 |
| adarazs | CI Squad meeting details: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:10 |
| EmilienM | adarazs: thanks | 14:10 |
| *** skramaja has joined #tripleo | 14:10 | |
| EmilienM | 76 Triaged, 62 In Progress, currently for RC2 - it's a lot | 14:10 |
| *** skramaja has quit IRC | 14:11 | |
| EmilienM | I would use some help to triage them otherwise the script will move them to queens-1 automatically when we release final pike | 14:11 |
| EmilienM | but we want critical / high bugs fixed and backported to stable/pike, as a goal | 14:11 |
| *** jmelvin has quit IRC | 14:11 | |
| EmilienM | any bug to discuss this week in particular? | 14:12 |
| EmilienM | no bug? perfect. Let's move on ;-) | 14:12 |
| EmilienM | #topic projects releases or stable backports | 14:12 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:12 | |
| EmilienM | pike: we released RC1 last week (thanks mwhahaha for the huge help) - which means we have stable/pike branch everywhere in tripleo now | 14:13 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Add 'discover and enroll nodes' workflow https://review.openstack.org/488526 | 14:13 |
| EmilienM | which means: please take care of backporting patches you want in stable/pike (need FFE for blueprints or a launchpad bug for bugs) | 14:13 |
| *** aufi has joined #tripleo | 14:13 | |
| EmilienM | we aim to release RC2 by next week, we still have a bunch of things to land | 14:14 |
| EmilienM | any question about Pike release? | 14:14 |
| EmilienM | if any, ping me or mwhahaha anytime | 14:14 |
| *** cylopez has quit IRC | 14:14 | |
| *** garyk has quit IRC | 14:15 | |
| EmilienM | I also plan to release new tags for Newton and Ocata - probably this week | 14:15 |
| EmilienM | jschlueter|znc: ^ fyi | 14:15 |
| *** jschlueter|znc is now known as jschlueter | 14:15 | |
| EmilienM | any question / feedback about release management and backports? | 14:15 |
| jschlueter | EmilienM: ack | 14:15 |
| EmilienM | #topic specs / blueprints | 14:16 |
| *** openstack changes topic to "specs / blueprints (Meeting topic: tripleo)" | 14:16 | |
| EmilienM | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:16 |
| EmilienM | #link https://launchpad.net/tripleo/+milestone/pike-rc2 | 14:16 |
| EmilienM | so we still have 5 open blueprints | 14:16 |
| *** ecerquei__ has joined #tripleo | 14:16 | |
| EmilienM | Dell blueprints are going to be implemented in Pike, some patches are missing, but all in gate now. | 14:17 |
| EmilienM | https://blueprints.launchpad.net/tripleo/+spec/websocket-logging will be closed in Pike iiuc | 14:17 |
| *** jjoyce has joined #tripleo | 14:17 | |
| EmilienM | https://blueprints.launchpad.net/tripleo/+spec/node-discovery-by-range as well (closed in Pike) though the UI part will be postponed to queens | 14:18 |
| EmilienM | honza, jrist ^ correct? | 14:18 |
| jrist | yes | 14:18 |
| EmilienM | good | 14:18 |
| *** ecerquei_ has quit IRC | 14:18 | |
| jrist | literally only 1 patch for websocket-logging | 14:18 |
| EmilienM | please update the launchpad when done, otherwise I'll look myself next week | 14:18 |
| EmilienM | jrist: good! | 14:18 |
| jrist | ok not true. a backport :( | 14:18 |
| honza | #link https://blueprints.launchpad.net/tripleo/+spec/gui-node-discovery-by-range | 14:18 |
| EmilienM | honza: excellent | 14:19 |
| honza | new node discovery gui part blueprint | 14:19 |
| EmilienM | honza: approved | 14:19 |
| EmilienM | for queens-1, to start | 14:19 |
| jrist | :) | 14:20 |
| EmilienM | and https://blueprints.launchpad.net/tripleo/+spec/node-discovery-by-range closed | 14:20 |
| EmilienM | sorry I've lost internet 1min | 14:21 |
| EmilienM | can you see my msg? | 14:21 |
| mwhahaha | yea | 14:21 |
| EmilienM | ok thx | 14:21 |
| EmilienM | a last note about blueprints (I'll send a note to ML as well) | 14:22 |
| EmilienM | #action EmilienM sends a note about Queens blueprints / specs on ML | 14:22 |
| EmilienM | please prepare your blueprints / specs prior to the PTG | 14:22 |
| EmilienM | it really help our team when things are written and prepared before, so we can get into the right topics during face to face meetings | 14:22 |
| EmilienM | so if you have something you want to do in Queens, please create a blueprint (or a spec if you like) - before the PTG, and share it on the ML if you want | 14:23 |
| EmilienM | any question? | 14:24 |
| EmilienM | moving on | 14:24 |
| EmilienM | #topic one off agenda items | 14:24 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:24 | |
| EmilienM | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:24 |
| EmilienM | mwhahaha: go ahead | 14:24 |
| mwhahaha | As previously mentioned, the PTG is soon. Please see the ML post by EmilienM about the agenda. | 14:24 |
| mwhahaha | #link http://lists.openstack.org/pipermail/openstack-dev/2017-August/121574.html | 14:24 |
| mwhahaha | Also please take some time to review the proposed topics. | 14:24 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ptg-queens | 14:24 |
| mwhahaha | that is all | 14:24 |
| EmilienM | thanks for the reminder | 14:24 |
| EmilienM | can i use uppercase? | 14:24 |
| EmilienM | PLEASE READ THIS ETHERPAD AND CHECK THE SCHEDULE | 14:25 |
| mwhahaha | :o | 14:25 |
| EmilienM | that would be super helpful | 14:25 |
| EmilienM | I try to bring sessions together and create some slots that make sense | 14:25 |
| EmilienM | like CI topics or Heat / mistral things etc | 14:25 |
| *** aditya_r has quit IRC | 14:26 | |
| EmilienM | ANY feedback on the agenda / schedule is very welcome, you can ping me on IRC or email or on the ML in the thread | 14:26 |
| EmilienM | that is all | 14:26 |
| EmilienM | sshnaidm: go ahead please | 14:26 |
| *** anshulbehl has quit IRC | 14:26 | |
| sshnaidm | just a few reminders of merging policy in tripleo-quickstart/-extras and tripleo-ci repos | 14:26 |
| sshnaidm | if you gonna merge patches into oooq, oooq-extras or tripleo-ci repos, please attend our tripleo-ci-squad meetings every Thursday 14.30 UTC, so that you'll get the overview of what is going in CI, please take into account: http://lists.openstack.org/pipermail/openstack-dev/2017-July/119766.html | 14:27 |
| sshnaidm | if it's possible, please avoid from merging patches without review of one of oooq-cores, please remember that this framework is used both for a *few* CI systems and for running locally by tripleo developers, you may affect all of them. | 14:27 |
| sshnaidm | and general point of course - please be cautious with merging stuff that you don't have full understanding of it, even if it's something urgent | 14:27 |
| EmilienM | sshnaidm: it reminds me a patch ;-) let's discuss about this meeting | 14:27 |
| sshnaidm | ok | 14:27 |
| *** pdeore__ has joined #tripleo | 14:28 | |
| EmilienM | #topic open discussion | 14:28 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:28 | |
| EmilienM | any question or general feedback? | 14:28 |
| EmilienM | ok | 14:29 |
| EmilienM | #endmeeting | 14:29 |
| *** openstack changes topic to "CI Status: YELLOWish (undercloud-oooq failures) | "you can't land a Cessna with workarounds, so don't try in Quickstart" - TripleO : http://tripleo.org/ | https://wiki.openstack.org/wiki/TripleO | Meetings On Tuesdays at 14:00 UTC here." | 14:29 | |
| openstack | Meeting ended Tue Aug 29 14:29:31 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:29 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-08-29-14.00.html | 14:29 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-08-29-14.00.txt | 14:29 |
| EmilienM | now go back to work :P | 14:29 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-08-29-14.00.log.html | 14:29 |
| EmilienM | sshnaidm: I would love to get a review on https://review.openstack.org/#/c/498573/ as soon as we can, we got tempest working on scenario002-container (basically a copy paste from the other featureset) | 14:30 |
| *** pdeore_ has quit IRC | 14:30 | |
| *** dprince has joined #tripleo | 14:32 | |
| sshnaidm | EmilienM, looking.. also arxcruz and chandankumar are welcome to look, as it contains tempest config | 14:32 |
| EmilienM | sshnaidm: fine but we need to get this one merged really asap, if not today | 14:33 |
| * arxcruz loves to see tempest on scenarios :) | 14:34 | |
| EmilienM | sshnaidm: tbh, the time to get reviews on oooq is really slow | 14:34 |
| sshnaidm | EmilienM, this patch doesn't have reviewers added | 14:35 |
| EmilienM | sshnaidm: would it help if we add reviewers in gerrit? | 14:35 |
| sshnaidm | EmilienM, +w | 14:35 |
| sshnaidm | EmilienM, well, I get mail about it at least | 14:35 |
| sshnaidm | EmilienM, and usually firstly review where I've been added | 14:36 |
| sshnaidm | EmilienM, but not everybody do like this.. you can ping cores in IRC like now | 14:36 |
| sshnaidm | EmilienM, and if it's not urgent, we have squad meeting | 14:37 |
| sshnaidm | EmilienM, and if it's something big and complex, we have even oooq review meetings | 14:37 |
| EmilienM | sshnaidm: ok, fair enough | 14:37 |
| *** jmelvin has joined #tripleo | 14:38 | |
| EmilienM | for these who go to the PTG, I would like to share a nice reading about organization in Denver: https://ttx.re/queens-ptg.html - please take 5 min | 14:38 |
| *** chlong has joined #tripleo | 14:41 | |
| *** lucas-brb is now known as lucasagomes | 14:41 | |
| lvdombrkr | helo folks, if anyone tried deploy overcloud with ssl without network isalation? | 14:42 |
| *** mrcc has quit IRC | 14:42 | |
| jpich | Useful link, thanks. Nice work on the proposed schedule as well! | 14:42 |
| jrist | jtomasek_: ew ;) | 14:44 |
| jrist | jtomasek_: do you agree with the no semicolons thing? | 14:44 |
| jrist | jtomasek_: I like explicit | 14:44 |
| jtomasek_ | jrist: I have created the patch to start the discussion. I am not entirely convinced that we need to get rid of semicolons, on the other hand, the semicolons are not needed at all:) | 14:47 |
| jrist | yeah | 14:47 |
| jrist | I commented | 14:47 |
| jrist | This seems like complication for the sake of complication. The curmudgeon inside me is especially disturbed. I'm not particularly against it though. | 14:48 |
| *** itlinux has joined #tripleo | 14:49 | |
| *** Goneri has joined #tripleo | 14:50 | |
| jtomasek_ | jrist: the issue in the first link has a good discussion and feels discouraging, although when you look at the discussion under other 2 links, the problems are resolved. Also the problems are tied to prettier implementation rather than regular js without semicolons usage | 14:50 |
| *** ratailor has joined #tripleo | 14:51 | |
| jpich | jtomasek_: Seems like a lot of churn for not much gain. It's also going to cause a million conflicts and make the git history (git blame ;)) (for debugging) useless for a while | 14:51 |
| jtomasek_ | jpich: right, that is my biggest concern. good point | 14:52 |
| lvdombrkr | helo folks, if anyone tried deploy overcloud with ssl without network isalation? | 14:57 |
| *** iranzo has quit IRC | 15:01 | |
| *** yamahata has joined #tripleo | 15:01 | |
| *** jprovazn has joined #tripleo | 15:02 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras master: Update validate-tempest role docs https://review.openstack.org/498836 | 15:02 |
| *** iranzo has joined #tripleo | 15:03 | |
| *** iranzo has quit IRC | 15:03 | |
| *** iranzo has joined #tripleo | 15:03 | |
| *** ecerquei__ has quit IRC | 15:03 | |
| *** ecerquei has joined #tripleo | 15:04 | |
| *** limao has joined #tripleo | 15:09 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 15:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 15:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 15:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 15:10 |
| lvdombrkr | helo folks, if anyone tried deploy overcloud with ssl without network isalation? | 15:11 |
| *** trown is now known as trown|brb | 15:12 | |
| *** cdearborn_ has joined #tripleo | 15:13 | |
| *** dmarlin has joined #tripleo | 15:13 | |
| *** limao_ has joined #tripleo | 15:13 | |
| abishop | sshnaidm: hoping you can review https://review.openstack.org/474981 and https://review.openstack.org/486175 | 15:15 |
| *** itlinux has quit IRC | 15:15 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add DockerPuppetProcessCount defaults to 3 https://review.openstack.org/498139 | 15:15 |
| *** limao has quit IRC | 15:15 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/newton: Don't disable satellite repo after registration https://review.openstack.org/495452 | 15:17 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/newton: Workaround for RHEL registration as "localhost" https://review.openstack.org/495453 | 15:17 |
| bnemec | lvdombrkr: That would be a very uncommon configuration. I think we used to do it in ci so it should work though. | 15:17 |
| *** yprokule has quit IRC | 15:22 | |
| *** stee_3_ has joined #tripleo | 15:22 | |
| *** limao_ has quit IRC | 15:22 | |
| *** bugzy_ has joined #tripleo | 15:23 | |
| *** limao has joined #tripleo | 15:23 | |
| *** aufi has quit IRC | 15:25 | |
| *** stee_3 has quit IRC | 15:26 | |
| *** bugzy has quit IRC | 15:26 | |
| *** cdearborn has quit IRC | 15:26 | |
| *** trown|brb is now known as trown | 15:28 | |
| *** lucasagomes is now known as lucas-afk | 15:28 | |
| *** pdeore__ has quit IRC | 15:30 | |
| *** pdeore__ has joined #tripleo | 15:30 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add missing Docker service https://review.openstack.org/498842 | 15:31 |
| *** rlandy is now known as rlandy|mtg | 15:33 | |
| *** limao has quit IRC | 15:34 | |
| *** limao has joined #tripleo | 15:34 | |
| *** iranzo has quit IRC | 15:36 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: WIP: Remove package if service stopped and disabled https://review.openstack.org/479886 | 15:37 |
| *** mdnadeem has quit IRC | 15:37 | |
| lvdombrkr | bnemec: thanks, unfortinetly it didnt work for me...i generete certeficate and key, place both in enable-tls.yaml than create .yaml with ControlFixedIPs | 15:38 |
| *** beagles has left #tripleo | 15:38 | |
| bnemec | lvdombrkr: What is failing? | 15:38 |
| lvdombrkr | bnemec: my deploy command looks like: openstack overcloud deploy --templates -e one.yaml -e tls-endpoints-public-ip.yaml -e /home/stack/ssl-heat-templates/environments/ssl/enable-tls.yaml -e --compute-scale 1 | 15:38 |
| lvdombrkr | bnemec: and there is no luck | 15:39 |
| lvdombrkr | bnemec: nothing fails, i just cant reach dashboard by :443 | 15:39 |
| bnemec | lvdombrkr: That may be because you aren't using network isolation so it isn't listening on a public address. | 15:41 |
| bnemec | Without net-iso it only listens on the provisioning network, which frequently isn't routable. | 15:41 |
| lvdombrkr | bnemec: thanks. can i use such a simple network isalation .yaml only with sine parametrs: <bneme | 15:43 |
| lvdombrkr | bnemec: http://paste.openstack.org/raw/619745/ | 15:43 |
| bogdando | folks, I have an odd puppet augeas error making docker restart to not happen https://pastebin.com/SyLSkQQy | 15:45 |
| *** limao_ has joined #tripleo | 15:45 | |
| bogdando | I wonder, what's wrong? the config file looks correct | 15:45 |
| *** ffiore has quit IRC | 15:46 | |
| *** limao has quit IRC | 15:47 | |
| *** limao has joined #tripleo | 15:48 | |
| mwhahaha | bogdando: it's erroring on parsing the sysconfig file | 15:48 |
| mwhahaha | bogdando: /etc/sysconfig/docker-storage has a parse error in it, what does yours look like? | 15:49 |
| bogdando | mwhahaha: DOCKER_STORAGE_OPTIONS=-s overlay2 | 15:49 |
| bogdando | but I'm using standard packages :/ | 15:49 |
| *** limao_ has quit IRC | 15:49 | |
| mwhahaha | bogdando: so what added that, shouldn't it be quoted? | 15:49 |
| bogdando | mwhahaha: good question | 15:50 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Add ability to generate an environment index https://review.openstack.org/491925 | 15:50 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Begin adding environments with all params for a service https://review.openstack.org/475924 | 15:50 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Update generated ssl environments https://review.openstack.org/493245 | 15:50 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Convert enable-internal-tls.yaml to be generated https://review.openstack.org/493246 | 15:50 |
| mwhahaha | or do you know what added that | 15:50 |
| bnemec | We need the two ssl-related changes for Pike. ^ | 15:50 |
| bnemec | I just added a bug reference so we can backport them once they merge. | 15:50 |
| bogdando | mwhahaha: oh yeah, indeed. That's my user data script suxx | 15:51 |
| bogdando | thanks!! | 15:51 |
| bnemec | lvdombrkr: It is possible to deploy with just an external network, yes. | 15:51 |
| mwhahaha | bogdando: np | 15:51 |
| lvdombrkr | ls | 15:51 |
| bnemec | lvdombrkr: I have an example of doing that here: https://github.com/cybertron/tripleo-network-templates#simple | 15:51 |
| bnemec | The actual files are in https://github.com/cybertron/tripleo-network-templates/tree/master/simple | 15:51 |
| *** dtantsur|busy is now known as dtantsur|afk | 15:53 | |
| sshnaidm | abishop, I think it would help to review your patches if you pasted your oooq run where this works. Because it's not tested in CI. Some pastebin in the patch that shows the successful run with your patch. | 15:55 |
| *** ffiore has joined #tripleo | 15:56 | |
| *** limao has quit IRC | 15:56 | |
| *** ratailor has quit IRC | 15:56 | |
| *** limao has joined #tripleo | 15:57 | |
| *** rlandy|mtg is now known as rlandy | 16:01 | |
| *** marios has quit IRC | 16:02 | |
| *** marrusl has joined #tripleo | 16:03 | |
| lvdombrkr | bnemec: thanks, i will try right now | 16:04 |
| *** egonzalez has quit IRC | 16:06 | |
| *** limao has quit IRC | 16:07 | |
| *** limao has joined #tripleo | 16:08 | |
| *** dsneddon has joined #tripleo | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 16:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 16:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 16:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 16:10 |
| *** paramite has quit IRC | 16:12 | |
| *** tesseract has quit IRC | 16:15 | |
| *** thrash is now known as thrash|biab | 16:16 | |
| *** garyk has joined #tripleo | 16:16 | |
| *** fzdarsky is now known as fzdarsky|afk | 16:17 | |
| *** limao has quit IRC | 16:19 | |
| *** limao has joined #tripleo | 16:19 | |
| *** fzdarsky|afk has quit IRC | 16:22 | |
| *** itlinux has joined #tripleo | 16:23 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: use pip to install delorean https://review.openstack.org/498074 | 16:23 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/newton: Release 5.6.2 (newton) https://review.openstack.org/498858 | 16:25 |
| *** suuuper has quit IRC | 16:25 | |
| openstackgerrit | Michael Henkel proposed openstack/os-net-config master: This patch adds initial support for the Contrail vRouter interface https://review.openstack.org/492492 | 16:26 |
| *** achadha has joined #tripleo | 16:26 | |
| *** achadha has quit IRC | 16:27 | |
| *** achadha has joined #tripleo | 16:27 | |
| *** limao_ has joined #tripleo | 16:29 | |
| *** ykarel has quit IRC | 16:30 | |
| lvdombrkr | bnemec: no luck...if i understood right PublicVirtualFixedIPs should ping after deployment despite any problems/mistakes with certeficates/keys? | 16:31 |
| *** achadha_ has joined #tripleo | 16:31 | |
| *** achadha has quit IRC | 16:31 | |
| *** limao has quit IRC | 16:33 | |
| *** achadha_ has quit IRC | 16:33 | |
| *** achadha has joined #tripleo | 16:33 | |
| abishop | sshnaidm: sure, I'll do that. Thanks for feedback | 16:35 |
| *** achadha has quit IRC | 16:36 | |
| *** achadha has joined #tripleo | 16:36 | |
| *** achadha has quit IRC | 16:36 | |
| *** achadha has joined #tripleo | 16:37 | |
| *** gkadam_ has joined #tripleo | 16:37 | |
| *** achadha has quit IRC | 16:37 | |
| *** jpich has quit IRC | 16:38 | |
| *** achadha has joined #tripleo | 16:38 | |
| EmilienM | sshnaidm: if something is not tested by CI, I'm not sure it's really fair to ask authors to test themselves and past results in Gerrit | 16:38 |
| EmilienM | sshnaidm: I think we want to automate all the things and ensure CI covers what we need | 16:38 |
| sshnaidm | EmilienM, I think anyway who prepare the patch should test it firstly | 16:40 |
| sshnaidm | EmilienM, it doesn't contradict of course that we need to cover as much as possible in CI | 16:40 |
| *** mcornea has quit IRC | 16:40 | |
| *** milan has quit IRC | 16:43 | |
| *** pdeore__ has quit IRC | 16:44 | |
| mwhahaha | sshnaidm: yes they should test it, but then asking for verification because of a lack of CI seems odd. If you're concerned about it, perhaps you should take the time to make sure it works correctly if that's something you want to see before you approve it. | 16:46 |
| *** rcernin has quit IRC | 16:46 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: do not use a custom scripts dir in dlrn builds https://review.openstack.org/498866 | 16:46 |
| sshnaidm | mwhahaha, this is very special case for re-running quickstart locally twice, I doubt it will be covered in CI, unfortunately not every use case it's possible to cover. I ask actually for people to do CI work so that will help to merge patches. | 16:48 |
| sshnaidm | mwhahaha, I don't insist, it's not a requirement, but I think it could be helpful | 16:48 |
| *** itlinux has quit IRC | 16:48 | |
| *** pcaruana has quit IRC | 16:48 | |
| *** itlinux has joined #tripleo | 16:49 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Add DockerPuppetProcessCount defaults to 3 https://review.openstack.org/498867 | 16:49 |
| *** ffiore has quit IRC | 16:49 | |
| bnemec | lvdombrkr: Yes, the ip should exist regardless of what happens with the certificates. | 16:50 |
| EmilienM | dprince: doing backports for you on your docker patches, if I miss one please do it yourself | 16:50 |
| dprince | EmilienM: cool, thanks man | 16:53 |
| *** links has joined #tripleo | 16:54 | |
| *** jpena is now known as jpena|off | 16:59 | |
| *** hewbrocca is now known as hewbrocca_afk | 17:01 | |
| *** jfrancoa has quit IRC | 17:01 | |
| *** limao_ has quit IRC | 17:05 | |
| *** gbarros has quit IRC | 17:09 | |
| *** ramishra has quit IRC | 17:10 | |
| *** milan has joined #tripleo | 17:10 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 17:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 17:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 17:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 17:10 |
| *** pkovar has quit IRC | 17:11 | |
| *** gbarros has joined #tripleo | 17:11 | |
| *** gbarros has quit IRC | 17:12 | |
| *** gkadam_ has quit IRC | 17:15 | |
| *** sid1 has joined #tripleo | 17:20 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Use Python to compute release notes version https://review.openstack.org/498882 | 17:21 |
| *** trown is now known as trown|lunch | 17:26 | |
| EmilienM | bnemec: thx for your review, I was wondering if I missed something in setup.cfg but it looks good: http://docs-draft.openstack.org/82/498882/1/check/gate-tripleo-heat-templates-releasenotes/2cd0973//releasenotes/build/html/ | 17:26 |
| bnemec | EmilienM: Yeah, my review is obviously assuming ci passes. :-) | 17:27 |
| dprince | EmilienM: as for backporting containers patches I think we probably do want to be careful not to add new containers at this point | 17:27 |
| EmilienM | bnemec: it does (for reno jobs) | 17:27 |
| EmilienM | dprince: which ones are missing? | 17:27 |
| dprince | EmilienM: the cron-container was newish, and will now require some work to get it built downstream | 17:28 |
| dprince | EmilienM: cron-container was used by logrotate | 17:28 |
| EmilienM | dprince: ack for this one | 17:28 |
| EmilienM | dprince: anything else? | 17:28 |
| EmilienM | dprince: https://bugs.launchpad.net/tripleo/+bug/1713612 - I would like this one fixed | 17:28 |
| openstack | Launchpad bug 1713612 in tripleo "When neutron is containerized, No 'neutron.service_plugins' driver found" [High,Triaged] | 17:28 |
| EmilienM | rnoriega: ^ any progress? | 17:28 |
| EmilienM | bogdando: FYI about what dprince said | 17:29 |
| dprince | EmilienM: so long as neutron patch doesn't require a net-new container I think it would be a backport candidate | 17:29 |
| EmilienM | dprince: I hope not | 17:31 |
| bogdando | dprince, EmilienM: ack | 17:31 |
| bogdando | I'm not really know much of the downstream part of the things :( | 17:31 |
| bogdando | but we definitely need logrotate backported for pike | 17:32 |
| bogdando | (but I hoped it made it to the pike, didn't it?) | 17:33 |
| EmilienM | (I don't know) | 17:33 |
| *** yamahata has quit IRC | 17:33 | |
| EmilienM | dprince: I think yes we need a new container | 17:33 |
| EmilienM | dprince: because the bgp / l2gw things are external of neutron | 17:33 |
| EmilienM | they are neutron-l2gw something | 17:33 |
| EmilienM | let me check if we already have containers for them | 17:33 |
| dprince | EmilienM: then, I would suggest not backporting these fixes then | 17:34 |
| dprince | EmilienM: and just target them towards Queens | 17:34 |
| *** links has quit IRC | 17:34 | |
| EmilienM | dprince: ack, I'll warn neutron folks | 17:35 |
| dprince | EmilienM: should be no surprise I think. It is just a bit late to be backporting net-new containers at this point I think | 17:36 |
| EmilienM | yes I agree | 17:37 |
| *** sid1 has quit IRC | 17:37 | |
| EmilienM | dprince: I figured bgp/l2gw wasn't working in containers when enabling tests on scenario004 | 17:37 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-validations master: Lazy load the entire stack instead of single outputs https://review.openstack.org/498884 | 17:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Use Python to compute release notes version https://review.openstack.org/498882 | 17:47 |
| *** catintheroof has quit IRC | 17:49 | |
| *** tongl has joined #tripleo | 17:50 | |
| *** thrash|biab is now known as thrash | 17:52 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Use Python to compute release notes version https://review.openstack.org/498882 | 17:56 |
| *** bogdando has quit IRC | 17:57 | |
| *** yamahata has joined #tripleo | 18:09 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 18:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 18:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 18:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 18:10 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-common master: DO NOT MERGE - Add no-op change for containers update testing https://review.openstack.org/498898 | 18:12 |
| *** iranzo has joined #tripleo | 18:15 | |
| *** iranzo has quit IRC | 18:15 | |
| *** iranzo has joined #tripleo | 18:15 | |
| *** liverpooler has joined #tripleo | 18:15 | |
| *** catintheroof has joined #tripleo | 18:23 | |
| *** cschwede has quit IRC | 18:29 | |
| EmilienM | bnemec, mwhahaha : https://review.openstack.org/#/c/498882/ is ready for review again | 18:31 |
| pabelanger | EmilienM: weshay: sshnaidm: what is the status of 498074? Are we okay with moving that forward today? Another gate failure on github.com: http://logs.openstack.org/40/498440/2/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/e7dd2c4/console.html#_2017-08-29_17_53_58_453891 | 18:32 |
| weshay | pabelanger, I've addressed sshnaidm's concerns, I was looking at that yesterday in recheck, and I only saw one failure in the last 30 days | 18:33 |
| *** milan has quit IRC | 18:33 | |
| weshay | ah.. a few from today | 18:34 |
| weshay | pabelanger, so I have two patches now | 18:34 |
| EmilienM | in a mtg, bbl | 18:34 |
| weshay | pabelanger, two reviews now https://review.openstack.org/#/c/498866/ | 18:35 |
| pabelanger | great | 18:35 |
| *** trown|lunch is now known as trown | 18:36 | |
| pabelanger | artg_rdoinfo_repo_url: "https://github.com/redhat-openstack/rdoinfo" is still a potential issue | 18:36 |
| pabelanger | but I think we need to have RDO publish that to https://trunk.rdoproject.org so we can use the reverse proxy cache | 18:36 |
| weshay | rlandy, Slower https://trunk.rdoproject.org/centos7-master/c3/f6/c3f6ee141b463c781df9beade10a7d4086ce1968_548cd9ec/ | 18:36 |
| *** dprince has quit IRC | 18:36 | |
| weshay | pabelanger, agree | 18:45 |
| pabelanger | anybody better at mistral? http://logs.openstack.org/08/498308/1/gate/gate-tripleo-ci-centos-7-containers-multinode/f0b31ec/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz just failed in the gate, but I have no idea what that means | 18:48 |
| *** iranzo has quit IRC | 18:54 | |
| mwhahaha | pabelanger: that's a mighty fine Unexpected error you have there | 18:56 |
| *** gbarros has joined #tripleo | 18:56 | |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Add storage backends env files for containerized deployment https://review.openstack.org/495370 | 18:56 |
| mwhahaha | pabelanger: zaqar failed to put stuff in swift it looks like http://logs.openstack.org/08/498308/1/gate/gate-tripleo-ci-centos-7-containers-multinode/f0b31ec/logs/undercloud/var/log/zaqar/zaqar-server.log.txt.gz#_2017-08-29_18_33_21_134 | 18:57 |
| mwhahaha | and of course we're not capturing swift logs | 18:57 |
| tongl | Hi, I have a quick question. How to change the overcloud neutron core_plugins in yaml for quickstart? | 18:58 |
| mwhahaha | maybe they are in messages | 18:58 |
| *** jtomasek_ has quit IRC | 18:59 | |
| tongl | I am trying to setup a test env with other neutron plugins. | 18:59 |
| *** dobson has quit IRC | 18:59 | |
| *** jobewan has joined #tripleo | 19:00 | |
| *** jobewan has quit IRC | 19:00 | |
| *** dparkes has joined #tripleo | 19:01 | |
| *** dobson has joined #tripleo | 19:03 | |
| *** achadha has quit IRC | 19:04 | |
| *** achadha has joined #tripleo | 19:05 | |
| mwhahaha | tongl: changing the core_plugins would traditionally be something done via THT, the question i guess would be how do you include custom THT templates to the deploy command via quickstart | 19:05 |
| *** limao has joined #tripleo | 19:08 | |
| tongl | yes, should we include it in the yaml config file? Is there any sample yaml config that uses thirdparty neutron plugin? | 19:08 |
| *** achadha has quit IRC | 19:09 | |
| mwhahaha | tongl: i don't know, i'm trying to figure that out myself | 19:09 |
| *** dprince has joined #tripleo | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 19:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 19:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 19:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 19:10 |
| tongl | mwhahaha: thanks. we added our neutron plugin in pike release and would like to validate if everything works fine. | 19:10 |
| *** hjensas has joined #tripleo | 19:12 | |
| *** hjensas has quit IRC | 19:12 | |
| *** limao has quit IRC | 19:12 | |
| mwhahaha | tongl: so quickstart offers a few config options that lets you specify additional -e files during the deploy, example https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset031.yml#L42 | 19:14 |
| mwhahaha | tongl: the question (which i don't know the answer to) is how do you provide those when you run quickstart so they get copied to the undercloud when you want to deploy | 19:15 |
| *** ebarrera has quit IRC | 19:15 | |
| mwhahaha | tongl: effectively the examples i found seem to only use already existing files from either the tripleo-heat-template repo or tripleo-ci | 19:15 |
| mwhahaha | weshay, trown, sshnaidm: do you have an answer as to how to provide your own THT yaml to quickstart so it gets copied (and applied) -^ i couldn't find examples | 19:16 |
| *** Goneri has quit IRC | 19:17 | |
| *** garyk has quit IRC | 19:17 | |
| tongl | mwhahaha: Yeh, trying to see how can I provide yaml so that undercloud can deploy overcloud with thirdparty neutron plugin and customized configs. | 19:19 |
| *** achadha has joined #tripleo | 19:22 | |
| *** achadha has quit IRC | 19:26 | |
| *** gfidente is now known as gfidente|afk | 19:27 | |
| *** catintheroof has quit IRC | 19:30 | |
| *** achadha has joined #tripleo | 19:31 | |
| *** dprince has quit IRC | 19:32 | |
| *** catintheroof has joined #tripleo | 19:36 | |
| weshay | mwhahaha, let me know if this helps https://github.com/openstack/tripleo-quickstart-extras/tree/master/roles/overcloud-prep-config | 19:38 |
| mwhahaha | weshay: not really | 19:39 |
| mwhahaha | weshay: so is there not a way to specify 'copy this file over' via quickstart? | 19:39 |
| *** pchavva has quit IRC | 19:41 | |
| weshay | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-config/templates/overcloud-custom-tht-script.sh.j2 | 19:41 |
| mwhahaha | oh there it is extra_tht_configs | 19:41 |
| mwhahaha | tongl: extra_tht_configs is what you would define witha list of yaml files which should get included in the deploy | 19:43 |
| *** jprovazn has quit IRC | 19:43 | |
| mwhahaha | tongl: https://github.com/openstack/tripleo-heat-templates/blob/f3e084dc85c0d95fadd078c6297a40136779ee5c/environments/networking/neutron-midonet.yaml#L50 is ane xample how you can set the core_plugins via THT | 19:43 |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: OpenStack containerized qdrouterd service https://review.openstack.org/479049 | 19:48 |
| *** dprince has joined #tripleo | 19:51 | |
| weshay | arxcruz, there are two failing tempest tests on pike FYI | 19:52 |
| arxcruz | weshay: where's the logs? | 19:53 |
| weshay | arxcruz, http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha-tempest-oooq-pike/25b6333/logs/stackviz/#/stdin | 19:54 |
| *** japestinho has quit IRC | 19:54 | |
| *** iranzo has joined #tripleo | 19:54 | |
| *** iranzo has quit IRC | 19:54 | |
| *** iranzo has joined #tripleo | 19:54 | |
| *** Goneri has joined #tripleo | 20:08 | |
| tongl | mwhahaha: awesome. thanks. I will have a look at it. | 20:09 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 20:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 20:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 20:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 20:10 |
| *** sid1 has joined #tripleo | 20:11 | |
| *** eck` is now known as eck`gone | 20:12 | |
| openstackgerrit | Justin Kilpatrick proposed openstack/tripleo-quickstart-extras master: WIP: Disruption detection and some stack update roles https://review.openstack.org/497950 | 20:14 |
| pabelanger | mwhahaha: okay, I'll check logstash.o.o and see if that has happen before | 20:15 |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-common master: WIP Make some tweaks to swift logging https://review.openstack.org/498930 | 20:20 |
| *** iranzo has quit IRC | 20:24 | |
| *** iranzo has joined #tripleo | 20:25 | |
| *** iranzo has joined #tripleo | 20:25 | |
| *** jkilpatr has quit IRC | 20:28 | |
| *** iranzo has quit IRC | 20:31 | |
| *** ansmith has quit IRC | 20:33 | |
| pabelanger | mwhahaha: another failure in the gate: http://logs.openstack.org/96/498296/1/gate/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container/ebc5774/logs/undercloud/var/log/zaqar/zaqar-server.log.txt.gz | 20:35 |
| mwhahaha | pabelanger: seems like an ordering problem cause i looked in /var/log/messages and the post for that container is after the failure | 20:36 |
| *** ebarrera has joined #tripleo | 20:36 | |
| mwhahaha | pabelanger: seems like an intermittent zaqar/mistral issue | 20:36 |
| pabelanger | mwhahaha: ya, going to setup an elastic-recheck query if I can find the pattern | 20:36 |
| stevebaker | morning | 20:37 |
| stevebaker | sshnaidm: hi | 20:37 |
| stevebaker | EmilienM: anything you would like me to look at? | 20:38 |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-common master: Add workflow that wraps register, introspect and provide https://review.openstack.org/498936 | 20:38 |
| *** leitan has quit IRC | 20:46 | |
| *** leitan has joined #tripleo | 20:47 | |
| *** jkilpatr has joined #tripleo | 20:47 | |
| *** leitan has quit IRC | 20:47 | |
| *** leitan has joined #tripleo | 20:47 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-validations master: Prevent unnecessary calls to dynamic inventory host() https://review.openstack.org/498939 | 20:47 |
| *** leitan has quit IRC | 20:47 | |
| *** leitan has joined #tripleo | 20:48 | |
| *** leitan has quit IRC | 20:52 | |
| *** eck`gone is now known as eck` | 20:53 | |
| *** florianf has quit IRC | 20:59 | |
| *** ecerquei has quit IRC | 21:00 | |
| *** trown is now known as trown|outtypewww | 21:02 | |
| *** gfidente|afk has quit IRC | 21:06 | |
| arxcruz | weshay: one of the tests are legitm failing, but the luks one should not be running, because the job doesn't have encrypt enabled | 21:07 |
| *** lblanchard has quit IRC | 21:07 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 21:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 21:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 21:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 21:10 |
| *** rhallisey has quit IRC | 21:12 | |
| *** artom has quit IRC | 21:13 | |
| pabelanger | 3rd failure: http://logs.openstack.org/93/489393/9/gate/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container/043a4f5/logs/undercloud/var/log/zaqar/zaqar-server.log.txt.gz | 21:14 |
| pabelanger | hitting it pretty often now in gate | 21:14 |
| mwhahaha | pabelanger: please file a bug with alert ci tags | 21:17 |
| bnemec | Wow, 48 patches in the tripleo gate queue. That's...impressive. | 21:18 |
| *** jcoufal has quit IRC | 21:18 | |
| tosky | like https://images-na.ssl-images-amazon.com/images/I/41m3ebMv5XL.jpg ? | 21:19 |
| tosky | :) | 21:19 |
| *** abishop has quit IRC | 21:19 | |
| *** sid1 has quit IRC | 21:20 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart master: enable pingtest on fs019 https://review.openstack.org/498575 | 21:25 |
| pabelanger | mwhahaha: message:"u'subscription_create" AND message:"u'Object PUT failed" AND tags:"console" AND voting:1 | 21:25 |
| pabelanger | is the query | 21:25 |
| pabelanger | 17 hits over 10 days | 21:26 |
| pabelanger | mwhahaha: what would you like the bug to be? I'm not sure what is logic is doing :) | 21:26 |
| *** ansmith has joined #tripleo | 21:27 | |
| *** jlabarre has quit IRC | 21:27 | |
| mwhahaha | pabelanger: me neither so i usually just throw what i've figured out into a bug and let someone else RCA it up. it looks to be that zaqar is erroring when trying to post status to a swift container or something | 21:27 |
| pabelanger | k | 21:27 |
| mwhahaha | pabelanger: is it just the scenario002 containers jobs? | 21:28 |
| pabelanger | no, it is a mix | 21:28 |
| *** akrivoka has quit IRC | 21:29 | |
| *** morazi has quit IRC | 21:29 | |
| mwhahaha | ok yea just throw what you've found in the bug. it seems to be more of a downstream mistral/zaqar ordering issue | 21:29 |
| mwhahaha | but you never know | 21:29 |
| mwhahaha | i think it happens when we're doing introspection | 21:30 |
| pabelanger | bug 1713832 | 21:30 |
| openstack | bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Undecided,New] https://launchpad.net/bugs/1713832 | 21:30 |
| pabelanger | if you want to triage it | 21:30 |
| pabelanger | creating elastic-recheck now | 21:30 |
| mwhahaha | k | 21:31 |
| * mwhahaha adds it to the list o things to try and do today | 21:31 | |
| *** catintheroof has quit IRC | 21:34 | |
| pabelanger | mwhahaha: so, the bad thing here is, we've been seeing this issue in check since http://logs.openstack.org/43/496243/1/check/gate-tripleo-ci-centos-7-scenario004-multinode-oooq-container/0a075c5/console.html.gz but we didn't properly debug the failure. 2017-08-24 08:54:59 | 21:37 |
| *** dprince has quit IRC | 21:38 | |
| EmilienM | stevebaker: https://review.openstack.org/#/c/498575/ will fail because of https://bugs.launchpad.net/bugs/1713612 | 21:41 |
| openstack | Launchpad bug 1713612 in tripleo "Containers missing for Neutron L2GW and BPGVPN-API" [High,Triaged] - Assigned to Ricardo Noriega (rnoriega-b) | 21:41 |
| arxcruz | weshay: this should fix the tests https://review.rdoproject.org/r/#/c/9070/ | 21:42 |
| *** artom has joined #tripleo | 21:42 | |
| *** artom has quit IRC | 21:42 | |
| *** artom has joined #tripleo | 21:42 | |
| *** tbarron is now known as tbarron|errand | 21:46 | |
| *** pradk has quit IRC | 21:46 | |
| stevebaker | EmilienM: ok, it sounds like we should align upstream with downstream and not containerise neutron | 21:47 |
| EmilienM | stevebaker: neutron-server you mean? | 21:49 |
| EmilienM | stevebaker: or just disable these 2 services on scenario004-container | 21:50 |
| EmilienM | for pike. It should be fine as well | 21:50 |
| EmilienM | therve: if you around, we need your help on https://bugs.launchpad.net/tripleo/+bug/1713832 | 21:50 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,Triaged] | 21:50 |
| * EmilienM realizes it's super late in europe | 21:51 | |
| weshay | long zuul queues | 21:51 |
| EmilienM | mwhahaha, weshay: what are the patches we need RIGHT NOW for CI? | 21:51 |
| EmilienM | I'm going to kill the zuul queue | 21:51 |
| EmilienM | https://review.openstack.org/#/c/498772/ for sure | 21:52 |
| weshay | yes | 21:52 |
| weshay | that one | 21:52 |
| weshay | https://review.openstack.org/#/c/498688/ | 21:52 |
| EmilienM | weshay: that's all? | 21:52 |
| weshay | looking | 21:52 |
| EmilienM | weshay: that one is critical? | 21:53 |
| EmilienM | weshay: I don't care about promotion now | 21:53 |
| weshay | would help the container effort | 21:53 |
| EmilienM | weshay: I don't want to help anyone now, I want to see how we can bring the gate back | 21:53 |
| EmilienM | sounds like the zaqar issue and this patch https://review.openstack.org/#/c/498772/ are needed | 21:53 |
| weshay | a couple more... sec | 21:53 |
| EmilienM | ok | 21:53 |
| EmilienM | tell me because I'm about to kill the gate | 21:54 |
| EmilienM | mwhahaha: any thoughts before I proceed? | 21:54 |
| weshay | EmilienM, https://review.openstack.org/#/c/498074/ | 21:54 |
| EmilienM | probably too late for sshnaidm to review | 21:55 |
| EmilienM | weshay: but it sounds like it worked on his test patch | 21:56 |
| EmilienM | weshay: how many hits do we have for this one? https://review.openstack.org/#/c/498074/ | 21:56 |
| weshay | EmilienM, I have a follow up that address his concerns | 21:56 |
| EmilienM | in logstash | 21:56 |
| EmilienM | not as many iirc | 21:56 |
| weshay | EmilienM, paul pinged me today | 21:56 |
| EmilienM | ok | 21:56 |
| *** ecerquei has joined #tripleo | 21:56 | |
| EmilienM | mwhahaha: please review https://review.openstack.org/#/c/498074/ when you can | 21:56 |
| weshay | <pabelanger> EmilienM: weshay: sshnaidm: what is the status of 498074? Are we okay with moving that forward today? Another gate failure on github.com: http://logs.openstack.org/40/498440/2/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/e7dd2c4/console.html#_2017-08-29_17_53_58_453891 | 21:56 |
| stevebaker | EmilienM, weshay: I think it worked, not sure about that stderr though http://logs.openstack.org/12/498712/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/6516c0f/logs/ara_oooq/result/9621bc1f-97f3-46c2-a314-377b353409de/ | 21:57 |
| EmilienM | stevebaker: which one? | 21:57 |
| stevebaker | " packages not built correctly: not updating the consistent symlink" | 21:57 |
| weshay | stevebaker, look at a current build, and you'll see the same | 21:57 |
| weshay | was going to follow up w/ that too | 21:57 |
| weshay | you'll see that in all the jobs atm | 21:57 |
| stevebaker | looks like dlrn runs though, so I think the pip install works | 21:57 |
| weshay | EmilienM, sshnaidm wanted this https://review.openstack.org/#/c/498866/1 | 21:58 |
| stevebaker | http://logs.openstack.org/12/498712/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/6516c0f/logs/delorean_logs/e8/38/e838261c783eb7743d4999df5fd8284081dffb38_dev/ | 21:58 |
| stevebaker | ^^ dlrn logs from sshnaidm's trigger | 21:59 |
| EmilienM | weshay: I'm asking for critical issues we want merged ASAP like now | 21:59 |
| EmilienM | weshay: is https://review.openstack.org/#/c/498866 part of it? | 21:59 |
| EmilienM | there is no bug in the commit message so I think no | 21:59 |
| weshay | no | 21:59 |
| EmilienM | ok good | 21:59 |
| weshay | just referencing sshnaidm's objection | 21:59 |
| *** jmelvin has quit IRC | 21:59 | |
| EmilienM | so please (again) give me link of things we need merged now | 21:59 |
| EmilienM | and we need some help on https://bugs.launchpad.net/tripleo/+bug/1713832 | 22:00 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,Triaged] | 22:00 |
| weshay | so what is holding up the gates? | 22:00 |
| mwhahaha | container-oooq fails and then this zaqar fail thing | 22:00 |
| mwhahaha | and also possibly dlrn fails | 22:00 |
| EmilienM | mwhahaha: container-oooq ? | 22:00 |
| EmilienM | what exactly? | 22:01 |
| mwhahaha | the remove builds from multinode | 22:01 |
| EmilienM | ah indeed | 22:01 |
| EmilienM | so let's focus on these 3 issues | 22:01 |
| weshay | ya.. that should merge soon ISH | 22:01 |
| EmilienM | and I'm going to kill all other patches in the gate | 22:01 |
| EmilienM | @all please do not approve any patch | 22:01 |
| weshay | remove builds from multinode is here: https://review.openstack.org/#/c/498688/ | 22:02 |
| EmilienM | https://review.openstack.org/#/c/498772/ just failed in gate | 22:02 |
| stevebaker | EmilienM: could https://review.openstack.org/#/c/493695/ be next after the critical ones? | 22:02 |
| EmilienM | stevebaker: tbh I don't know right now we're trying to solve gate issue, so we can keep merging stuffs | 22:03 |
| EmilienM | stevebaker: because right now we don't merge anything | 22:03 |
| EmilienM | last patch which merged was 7h ago | 22:03 |
| EmilienM | everything between failed in gate | 22:04 |
| weshay | EmilienM, did tempest get enabled on the container-multinode job? | 22:05 |
| * weshay looks | 22:06 | |
| EmilienM | weshay: yes, one of them the main one | 22:07 |
| weshay | ya.. it worked, just looking at a very odd console log | 22:08 |
| EmilienM | ok gate is purged | 22:09 |
| *** Goneri has quit IRC | 22:10 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 22:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 22:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 22:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 22:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,Triaged] | 22:10 |
| weshay | looking for other jobs failing like http://logs.openstack.org/72/498772/2/gate/gate-tripleo-ci-centos-7-containers-multinode/701b480/console.html | 22:12 |
| *** Goneri has joined #tripleo | 22:13 | |
| *** itlinux has quit IRC | 22:14 | |
| weshay | EmilienM, I don't know why https://review.openstack.org/#/c/498772/ failed.. recheck? | 22:14 |
| EmilienM | weshay: I'm trying to get it promoted to the gate on #openstack-infra | 22:16 |
| EmilienM | can someone look at the zaqar thing? | 22:16 |
| weshay | k.. it just timed out | 22:16 |
| EmilienM | did we merge something that broke us? | 22:16 |
| EmilienM | ok 498772 promoted to gate now | 22:17 |
| EmilienM | I'm starting to investigate the zaqar problem | 22:17 |
| EmilienM | yeah, like pabelanger we have the issue since a bunch of days now :( | 22:19 |
| EmilienM | it could have happened in a promotion | 22:19 |
| *** Goneri has quit IRC | 22:20 | |
| mwhahaha | it's probably a promotion | 22:20 |
| mwhahaha | EmilienM: it looked like it's trying to read the queue before it gets created in swift | 22:20 |
| mwhahaha | but i don't know the order in which that stuff happens so it's all magic to me | 22:20 |
| pabelanger | did we just abandon a bunch of things? | 22:23 |
| mwhahaha | yea | 22:23 |
| EmilienM | pabelanger: I'm clearing the gate | 22:24 |
| EmilienM | pabelanger: so we keep focus on 3 problems: | 22:24 |
| EmilienM | 1) getting https://review.openstack.org/498772 merged | 22:24 |
| EmilienM | 2) https://bugs.launchpad.net/tripleo/+bug/1713832 | 22:25 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,Triaged] | 22:25 |
| EmilienM | and 3) https://review.openstack.org/498074 | 22:25 |
| EmilienM | pabelanger: could you promote 498074 to the gate as well please, I just approved this patch | 22:25 |
| pabelanger | 1 sec | 22:26 |
| EmilienM | pabelanger: 498772 was promoted by clarkb afik | 22:26 |
| EmilienM | ok it's in gate now | 22:26 |
| EmilienM | pabelanger: nevermind | 22:26 |
| *** dsneddon has quit IRC | 22:26 | |
| *** tongl has quit IRC | 22:29 | |
| EmilienM | mwhahaha: first time zaqar issue appeared was Aug 21 | 22:31 |
| EmilienM | mwhahaha: http://logs.openstack.org/59/471759/65/check/gate-tripleo-ci-centos-7-3nodes-multinode-nv/e045834/console.html#_2017-08-21_12_19_25_509975 | 22:32 |
| EmilienM | this one right? | 22:32 |
| mwhahaha | yea | 22:32 |
| EmilienM | k | 22:32 |
| pabelanger | EmilienM: https://review.openstack.org/498954/ is the elastic-recheck query | 22:33 |
| EmilienM | I nkow | 22:33 |
| EmilienM | but thanks | 22:33 |
| EmilienM | let's see what we changed between August 20 and 22 | 22:34 |
| *** achadha_ has joined #tripleo | 22:35 | |
| mwhahaha | EmilienM: since this is the validation workflow, did we update that then? | 22:36 |
| *** morazi has joined #tripleo | 22:37 | |
| EmilienM | mwhahaha: I'm doing rpm diffs and looking | 22:37 |
| EmilienM | https://www.diffchecker.com/JvMExY0X | 22:37 |
| EmilienM | maybe https://github.com/openstack/instack-undercloud/commit/3b4d191d986ee6a7f92e3314798057e58cf17dd9 | 22:38 |
| EmilienM | but not sure at all, I'm guessing at this stage | 22:38 |
| *** thrash is now known as thrash|g0ne | 22:38 | |
| EmilienM | is it always the same workflow? | 22:38 |
| *** achadha has quit IRC | 22:38 | |
| *** rlandy is now known as rlandy|bbl | 22:38 | |
| EmilienM | yes | 22:39 |
| EmilienM | Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: ebb5bed9-fae1-4e61-8617-3fb425527ffa | 22:39 |
| EmilienM | always this one | 22:39 |
| *** achadha_ has quit IRC | 22:39 | |
| mwhahaha | revert it | 22:41 |
| EmilienM | ok | 22:42 |
| EmilienM | I think that's the one | 22:42 |
| EmilienM | but I'm really unsure | 22:42 |
| EmilienM | mwhahaha: is the error random? | 22:42 |
| mwhahaha | yes | 22:42 |
| EmilienM | probably yes since gate is not 100% broken | 22:42 |
| mwhahaha | it's a race condition | 22:42 |
| EmilienM | kk | 22:42 |
| EmilienM | revert | 22:42 |
| mwhahaha | it's trying to query the results before the swift container gets created in zaqar | 22:42 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud master: Revert "Adds execution of post-upgrade validations on undercloud upgrade" https://review.openstack.org/498975 | 22:43 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud master: Revert "Adds execution of post-upgrade validations on undercloud upgrade" https://review.openstack.org/498975 | 22:43 |
| EmilienM | ok done ^ | 22:44 |
| EmilienM | mwhahaha: https://review.openstack.org/#/c/498975/ | 22:45 |
| EmilienM | pabelanger: my last request, promoting ^ to the gate please | 22:46 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/498772/ is failing again | 22:48 |
| *** dmarlin has quit IRC | 22:49 | |
| EmilienM | http://logs.openstack.org/72/498772/2/gate/gate-tripleo-ci-centos-7-containers-multinode/3d94563/console.html#_2017-08-29_22_43_24_373517 | 22:49 |
| EmilienM | is it a new one? | 22:49 |
| stevebaker | EmilienM: do you know what the story is with this missing ovs_vxlan_bridge command? http://logs.openstack.org/72/498772/2/gate/gate-tripleo-ci-centos-7-undercloud-oooq/a8af258/logs/undercloud/home/jenkins/vxlan_networking.sh.log.txt.gz | 22:49 |
| EmilienM | stevebaker: sounds like we're talking about the same thing now | 22:50 |
| pabelanger | EmilienM: haven't seen that before | 22:50 |
| EmilienM | pabelanger: http://logs.openstack.org/72/498772/2/gate/gate-tripleo-ci-centos-7-containers-multinode/3d94563/logs/undercloud/home/jenkins/vxlan_networking.sh.log.txt.gz#_2017-08-29_22_43_24 | 22:50 |
| EmilienM | 2017-08-29 22:43:24 | /home/jenkins/vxlan_networking.sh: line 91: ovs_vxlan_bridge: command not found | 22:50 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-common master: Accept glance image ID in addition to name https://review.openstack.org/498978 | 22:51 |
| pabelanger | ya, first time I seen that | 22:51 |
| EmilienM | I'm pretty sure it's a change in devstack-gate | 22:51 |
| EmilienM | that just merged | 22:51 |
| pabelanger | Oh, ya. | 22:51 |
| pabelanger | we just changed something | 22:51 |
| EmilienM | https://review.openstack.org/435933 | 22:51 |
| EmilienM | and broke us | 22:51 |
| pabelanger | lets move to #openstack-infra | 22:51 |
| EmilienM | so now we have a 4th issue | 22:51 |
| *** mwhahaha sets mode: +o EmilienM | 22:54 | |
| *** EmilienM changes topic to "CI Status: Red - please do not approve any patch" | 22:55 | |
| *** achadha has joined #tripleo | 22:58 | |
| *** cdearborn_ has quit IRC | 23:03 | |
| EmilienM | pabelanger: I'm filling a bug for the 4th issue | 23:05 |
| EmilienM | so we can have alerts here | 23:05 |
| EmilienM | pabelanger: https://bugs.launchpad.net/tripleo/+bug/1713868 | 23:08 |
| openstack | Launchpad bug 1713868 in tripleo "devstack-gate doesn't provide ovs_vxlan_bridge function anymore" [Critical,Triaged] - Assigned to Paul Belanger (pabelanger) | 23:08 |
| EmilienM | pabelanger: instead of using the function, would it be simpler to re-use the same bash code | 23:10 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1689556 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712910 | 23:10 |
| openstack | Launchpad bug 1689556 in tripleo "CI: containers overcloud can't start vm when deployed with multinic network isolation" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713868 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 23:10 |
| EmilienM | pabelanger: until we switch to an ansible task? | 23:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel) | 23:10 |
| openstack | Launchpad bug 1712910 in tripleo "oooq virt-customize repo setup logs are not saved " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 23:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 23:10 |
| openstack | Launchpad bug 1713868 in tripleo "devstack-gate doesn't provide ovs_vxlan_bridge function anymore" [Critical,Triaged] - Assigned to Paul Belanger (pabelanger) | 23:10 |
| EmilienM | weshay: removing alert on https://bugs.launchpad.net/tripleo/+bug/1713868 - no activity since more than a month | 23:10 |
| EmilienM | not this one | 23:10 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1702955 | 23:11 |
| weshay | k | 23:11 |
| mwhahaha | it's still a problem | 23:12 |
| mwhahaha | just no one seems to have time to fix it | 23:13 |
| * mwhahaha shrugs | 23:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci master: In-tree ovs_vxlan_bridge function from devstack-gate https://review.openstack.org/498985 | 23:13 |
| pabelanger | EmilienM: ^ should be our fix | 23:14 |
| mwhahaha | if only there was a code search folks to use to determine if things were being used outside of their project before removing them completely | 23:15 |
| * mwhahaha sighs and goes to flip tables elsewhere | 23:15 | |
| *** tosky has quit IRC | 23:15 | |
| EmilienM | weshay: please assign https://bugs.launchpad.net/tripleo/+bug/1708832 to yourself | 23:16 |
| openstack | Launchpad bug 1708832 in tripleo "DLRN build failures in gate" [High,In progress] | 23:16 |
| weshay | done | 23:16 |
| EmilienM | weshay: thank you | 23:17 |
| EmilienM | pabelanger: reviewing | 23:18 |
| pabelanger | EmilienM: hold up, I think swap is broken too | 23:18 |
| EmilienM | :( | 23:19 |
| pabelanger | okay, I'll have to abandon, restore | 23:21 |
| pabelanger | but want to make sure vxlan is setup before I do that | 23:21 |
| pabelanger | if we pass that, we can likey enqueue right into gate | 23:21 |
| EmilienM | ok | 23:22 |
| EmilienM | and swap? | 23:22 |
| pabelanger | http://status.openstack.org/elastic-recheck/#1706103 | 23:22 |
| pabelanger | just bad luck | 23:22 |
| pabelanger | known issue | 23:22 |
| EmilienM | ah ok | 23:22 |
| EmilienM | I thought it was something in tripleo again | 23:22 |
| EmilienM | :) | 23:22 |
| *** jeblair has joined #tripleo | 23:22 | |
| EmilienM | jeblair: o/ | 23:23 |
| *** yamahata has quit IRC | 23:34 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart-extras master: In-tree function ovs_vxlan_bridge https://review.openstack.org/498986 | 23:35 |
| pabelanger | EmilienM: okay, ^ is the other place it needs to be fixed | 23:35 |
| *** lblanchard has joined #tripleo | 23:35 | |
| EmilienM | pabelanger: ok | 23:36 |
| EmilienM | pabelanger: if we need it at multiple places, the patches won't land I guess | 23:37 |
| EmilienM | well or maybe | 23:37 |
| EmilienM | since we run all gate jobs in oooq now | 23:37 |
| pabelanger | EmilienM: ya, possible something will need to be force merged to unwedge | 23:38 |
| EmilienM | pabelanger: how can we do that? infra roots? | 23:38 |
| pabelanger | lets get 498986 green in check first | 23:38 |
| EmilienM | k | 23:38 |
| pabelanger | EmilienM: ya, I can force merge, just need to sudo in gerrit | 23:38 |
| *** yamahata has joined #tripleo | 23:40 | |
| EmilienM | ok | 23:40 |
| EmilienM | pabelanger: it would really help to force merge https://review.openstack.org/#/c/498772/ | 23:41 |
| EmilienM | weshay: can you remind me, https://review.openstack.org/#/c/498772/ is fixing the instack-undercloud gate, right? | 23:41 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/498074/ as well | 23:42 |
| EmilienM | pabelanger: and https://review.openstack.org/#/c/498975/ - this is a revert | 23:42 |
| pabelanger | EmilienM: I'm confident that is going to pass, so once we fix networking, I can enqueue / promote again, and you can start +A again | 23:42 |
| EmilienM | I'm pretty sure it's safe to force merge at this stage | 23:42 |
| pabelanger | well, lets not force merge until it is needed. I don't really like doing that unless gate is wedged | 23:43 |
| pabelanger | everything should pass, with proper depends-On headers | 23:43 |
| EmilienM | pabelanger: gats is really wedge right now | 23:43 |
| EmilienM | pabelanger: I'm not sure it will pass | 23:43 |
| EmilienM | it's a spaguetti | 23:43 |
| EmilienM | but as you wish... | 23:43 |
| pabelanger | right, we need to make sure 498986 is good | 23:43 |
| pabelanger | because it is untested right now | 23:44 |
| pabelanger | so, as long as something sets up vxlan, then we can +A, promote and enqueue | 23:44 |
| EmilienM | pabelanger: I didn't ask to force merge this one | 23:44 |
| pabelanger | then, we can do same with other patches | 23:44 |
| EmilienM | I asked https://review.openstack.org/#/c/498772/ https://review.openstack.org/#/c/498975/ and https://review.openstack.org/#/c/498074/ | 23:44 |
| pabelanger | right, there is no need to force merge those, they don't wedge the gate | 23:45 |
| weshay | EmilienM, several of the gates, yes | 23:45 |
| weshay | but not all are affected | 23:45 |
| EmilienM | weshay: k | 23:45 |
| weshay | it's a must have atm | 23:46 |
| weshay | that is the top one from us | 23:46 |
| EmilienM | pabelanger: it would save time to force merge them imho | 23:46 |
| EmilienM | so we would just have to focus on the d-g thing | 23:46 |
| EmilienM | and we wouldn't hit the 3 other problems | 23:46 |
| pabelanger | EmilienM: once they are enqueue into gate, anything behind them will work | 23:46 |
| pabelanger | we just need to make sure 498986,1 is good | 23:47 |
| EmilienM | k | 23:47 |
| EmilienM | pabelanger: before you leave for your end of day, please let me know | 23:48 |
| EmilienM | (I know it's late for you) | 23:48 |
| pabelanger | EmilienM: don't worry, I am here until this is all fixed | 23:48 |
| EmilienM | thanks ... | 23:49 |
| pabelanger | blarg | 23:51 |
| pabelanger | I see an issue | 23:51 |
| pabelanger | 1 sec | 23:51 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart-extras master: In-tree function ovs_vxlan_bridge https://review.openstack.org/498986 | 23:52 |
| EmilienM | ok still lgtm | 23:54 |
| pabelanger | https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-undercloud-oooq-master-nv/231/console | 23:55 |
| pabelanger | that is surprisingly fast | 23:55 |
| pabelanger | not sure why | 23:55 |
| pabelanger | EmilienM: okay, vxlan was setup | 23:58 |
| *** achadha has quit IRC | 23:59 | |
| *** achadha has joined #tripleo | 23:59 | |
| EmilienM | yep indeed | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
