| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 00:00 |
|---|---|---|
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 00:00 |
| openstackgerrit | Dan Sneddon proposed openstack/os-net-config master: Add Policy Based Routing Support to os-net-config https://review.openstack.org/575712 | 00:01 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 00:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 00:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 00:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 00:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
| mwhahaha | jtcressy: yea we should probably documents that | 00:10 |
| *** gbarros has joined #tripleo | 00:12 | |
| *** itlinux has joined #tripleo | 00:15 | |
| *** medberry has joined #tripleo | 00:24 | |
| *** medberry has quit IRC | 00:24 | |
| *** medberry has joined #tripleo | 00:24 | |
| *** itlinux has quit IRC | 00:25 | |
| *** gbarros has quit IRC | 00:26 | |
| *** gbarros has joined #tripleo | 00:26 | |
| * dpeacock pats zuul good gatekeeper | 00:28 | |
| * dpeacock does the finally merged happy dance | 00:28 | |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-redhat-subscription master: Zuul: add missing jobs for release https://review.openstack.org/586379 | 00:44 |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-tripleo-modify-image master: Zuul: add missing jobs for release https://review.openstack.org/586380 | 00:45 |
| *** lblanchard has joined #tripleo | 00:55 | |
| *** khyr0n has quit IRC | 01:01 | |
| *** slaweq has joined #tripleo | 01:05 | |
| *** gbarros has quit IRC | 01:07 | |
| *** artom has quit IRC | 01:09 | |
| *** gbarros has joined #tripleo | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 01:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 01:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 01:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 01:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
| *** slaweq has quit IRC | 01:10 | |
| *** mrsoul has joined #tripleo | 01:12 | |
| *** medberry has quit IRC | 01:13 | |
| *** toure is now known as toure|gone | 01:14 | |
| *** mrsoul` has quit IRC | 01:15 | |
| *** Petersingh has joined #tripleo | 01:19 | |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-tripleo-modify-image master: Zuul: add missing jobs for release https://review.openstack.org/586380 | 01:19 |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-tripleo-modify-image master: Zuul: add missing jobs for release https://review.openstack.org/586380 | 01:19 |
| openstackgerrit | Merged openstack/ansible-role-redhat-subscription master: Zuul: add missing jobs for release https://review.openstack.org/586379 | 01:25 |
| *** gbarros has quit IRC | 01:27 | |
| *** gbarros has joined #tripleo | 01:28 | |
| *** jtcressy has quit IRC | 01:30 | |
| *** jtcressy has joined #tripleo | 01:33 | |
| openstackgerrit | Merged openstack/python-tripleoclient stable/pike: Use only update/upgrade ansible queue instead https://review.openstack.org/585386 | 01:35 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix a race when upgrading Neutron & Ironic on the undercloud https://review.openstack.org/585583 | 01:35 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Add ODL puppet lib functions to zuul path https://review.openstack.org/584859 | 01:35 |
| *** yamahata has quit IRC | 01:48 | |
| *** medberry has joined #tripleo | 01:57 | |
| *** medberry has quit IRC | 01:57 | |
| *** medberry has joined #tripleo | 01:57 | |
| *** lblanchard has quit IRC | 02:05 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 02:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 02:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 02:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: Include 'tripleo_role_name' in the inventory https://review.openstack.org/583287 | 02:12 |
| *** Petersingh is now known as Petersingh|afk | 02:17 | |
| *** pliu_ has joined #tripleo | 02:17 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: deploy-steps: switch to tripleo_role_name https://review.openstack.org/582428 | 02:20 |
| *** Petersingh|afk is now known as Petersingh | 02:21 | |
| *** psachin`` has joined #tripleo | 02:23 | |
| *** psachin`` has quit IRC | 02:33 | |
| *** sanjayu_ has joined #tripleo | 02:34 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Optimized Ansible tasks in deplay-steps-tasks.yaml https://review.openstack.org/582429 | 02:37 |
| *** medberry has quit IRC | 02:39 | |
| *** myoung has quit IRC | 02:39 | |
| *** khyr0n has joined #tripleo | 02:39 | |
| *** thrash is now known as thrash|g0ne | 02:42 | |
| *** Petersingh is now known as Petersingh|afk | 02:59 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix deploy health checks https://review.openstack.org/584119 | 03:08 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 03:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 03:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 03:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
| *** jtcressy has quit IRC | 03:16 | |
| *** Petersingh|afk is now known as Petersingh | 03:17 | |
| *** sanjayu_ has quit IRC | 03:18 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use global ansible.cfg for nodes-uuid playbook https://review.openstack.org/583552 | 03:24 |
| *** Petersingh is now known as Petersingh|brb | 03:31 | |
| *** jtcressy has joined #tripleo | 03:32 | |
| *** gbarros has quit IRC | 03:39 | |
| *** psahoo has joined #tripleo | 03:42 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Action to perform container image prepare https://review.openstack.org/585913 | 03:44 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Add tripleoclient to mistral image https://review.openstack.org/586417 | 03:44 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Add container_image_prepare_params to deployment workflow https://review.openstack.org/586418 | 03:44 |
| *** links has joined #tripleo | 03:52 | |
| *** yamahata has joined #tripleo | 03:53 | |
| *** mschuppert has joined #tripleo | 04:06 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 04:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 04:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 04:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 04:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
| *** slaweq has joined #tripleo | 04:11 | |
| *** slaweq has quit IRC | 04:16 | |
| *** mdnadeem has joined #tripleo | 04:21 | |
| *** itlinux has joined #tripleo | 04:22 | |
| *** jtcressy has quit IRC | 04:28 | |
| *** pcaruana has joined #tripleo | 04:28 | |
| *** Petersingh|brb is now known as Petersingh | 04:30 | |
| *** pcaruana has quit IRC | 04:30 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow to remove xinetd service https://review.openstack.org/586057 | 04:39 |
| *** jaganathan has joined #tripleo | 04:45 | |
| *** shyamb has joined #tripleo | 04:46 | |
| *** jaganathan has quit IRC | 04:47 | |
| *** jaganathan has joined #tripleo | 04:47 | |
| *** jtcressy has joined #tripleo | 05:01 | |
| *** itlinux has quit IRC | 05:05 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: Add some logging for the preflight checks https://review.openstack.org/586285 | 05:09 |
| *** psahoo has quit IRC | 05:09 | |
| *** psahoo has joined #tripleo | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 05:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 05:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 05:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 05:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
| *** jfrancoa has joined #tripleo | 05:12 | |
| *** slaweq has joined #tripleo | 05:13 | |
| *** cshastri has joined #tripleo | 05:14 | |
| *** Petersingh is now known as Petersingh|afk | 05:15 | |
| *** pgadiya has joined #tripleo | 05:16 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New preflight validation: disk space on undercloud https://review.openstack.org/582917 | 05:17 |
| *** slaweq has quit IRC | 05:17 | |
| *** skramaja has joined #tripleo | 05:20 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New preflight validation: disk space on undercloud https://review.openstack.org/582917 | 05:24 |
| *** Petersingh|afk is now known as Petersingh | 05:33 | |
| *** quiquell has joined #tripleo | 05:41 | |
| *** kopecmartin has joined #tripleo | 05:49 | |
| openstackgerrit | Jose Luis Franco proposed openstack/python-tripleoclient master: Handle negative/wrong answer in question when upgrading undercloud. https://review.openstack.org/586249 | 05:49 |
| quiquell | jfrancoa: No logs for updates... can we check the tht version is installing with output log ? | 05:51 |
| *** Haresh has joined #tripleo | 05:56 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: [DNM] To debug fix tags https://review.openstack.org/585955 | 05:57 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow to remove xinetd service https://review.openstack.org/586057 | 05:58 |
| Tengu | humpf. not being able to run "tox -epep8" in t-h-t is a pain -.-. | 05:58 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Duplicate of I8330687ec64f1ebb7e3b100410d5bdc3e67ae7f0 https://review.openstack.org/586444 | 05:59 |
| *** ksambor has joined #tripleo | 06:05 | |
| jfrancoa | quiquell: yes, I saw...the releases.sh file was there I believe. let me check again | 06:07 |
| jfrancoa | jfrancoa: btw, good morning o/ | 06:07 |
| jfrancoa | quiquell: ^^^ | 06:07 |
| quiquell | jfrancoa: Good morning man, let's get this sort out | 06:07 |
| jfrancoa | quiquell: export OVERCLOUD_TARGET_HASH="959e1d7f755ee681b6f23b498d262a9e4dd6326f_4cbb1814" | 06:08 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 06:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 06:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 06:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 06:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
| quiquell | jfrancoa: Running duplicate of EmilienM fix with the Depends-On again https://review.openstack.org/#/c/586444/ | 06:12 |
| quiquell | jfrancoa: Just to check, I am stubborn | 06:12 |
| jfrancoa | quiquell: hahaha better to check it twice :-D, btw in the RDO CI job it failed with the same reason we saw yesterday in the morning https://logs.rdoproject.org/28/585528/7/openstack-check/legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master/b21ba7a/logs/undercloud/home/zuul/overcloud_update_run_Controller.log.txt.gz#_2018-07-27_02_32_22 | 06:12 |
| jfrancoa | quiquell: it's already too much coincidence to happen twice..I'll try to run the reproducer for the job in my RDO cloud | 06:13 |
| quiquell | jfrancoa: I have problems with the reproducer... now, perfect timing | 06:13 |
| *** shyamb has quit IRC | 06:15 | |
| *** bogdando has joined #tripleo | 06:15 | |
| *** shyamb has joined #tripleo | 06:15 | |
| quiquell | jfrancoa: I will take a look why we don't have logs at updates | 06:16 |
| quiquell | jfrancoa: Maybe a recheck can yelp ? | 06:16 |
| jfrancoa | quiquell: cool, thanks | 06:16 |
| quiquell | jfrancoa: This is the success update job with the Depends-On http://logs.openstack.org/28/585528/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/2e91dbe/ | 06:17 |
| quiquell | jfrancoa: If it's of any help | 06:17 |
| *** lvdombrkr has joined #tripleo | 06:18 | |
| jfrancoa | quiquell: they have the very same hashes in release.sh...but why isn't the minor update failing in this one? | 06:18 |
| quiquell | jfrancoa: Forget about them, tht install newer version | 06:18 |
| quiquell | jfrancoa: https://logs.rdoproject.org/28/585528/7/openstack-check/legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master/b21ba7a/logs/undercloud/var/log/yum.log.txt.gz | 06:18 |
| quiquell | It has yesterdays patches, the problem is elsewhere | 06:19 |
| quiquell | At least with the Depends-On we can see the error logs | 06:19 |
| quiquell | When it finishes | 06:19 |
| *** shyamb has quit IRC | 06:22 | |
| *** mhenkel_ has joined #tripleo | 06:23 | |
| *** jtcressy has quit IRC | 06:23 | |
| *** mhenkel_ has quit IRC | 06:27 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow to remove xinetd service https://review.openstack.org/586057 | 06:29 |
| *** lvdombrkr89 has joined #tripleo | 06:31 | |
| *** lvdombrkr has quit IRC | 06:33 | |
| *** shyamb has joined #tripleo | 06:35 | |
| Tengu | hmmm. one thing would be really, really cool in zuul: run the tox tests early, and block the other (deploy/foo) until tox is finished AND green. i.e. do not run the functional tests as long as tox isn't green. | 06:37 |
| Tengu | meaning early failure, and less lost resources. | 06:37 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Hard link environments to keep backwards compat https://review.openstack.org/586450 | 06:38 |
| *** shyambiradar has joined #tripleo | 06:44 | |
| *** shyamb has quit IRC | 06:47 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow to remove xinetd service https://review.openstack.org/586057 | 06:49 |
| *** amoralej|off is now known as amoralej | 06:50 | |
| *** brault has joined #tripleo | 06:51 | |
| *** tesseract has joined #tripleo | 06:52 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-common master: Add default for ceph_ansible_playbook in update/upgrade prepare workflow https://review.openstack.org/585367 | 06:53 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-common master: Remove ceph-specific logic from update/upgrade prepare workflow https://review.openstack.org/585368 | 06:53 |
| flaper87 | ok, what's the trick to create an external network that would let an overcloud VM instance access the public endpoints of the overcloud services? I've followed these docs but it ain't working for me: https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html#creating-floating-ip-networks | 06:55 |
| *** rcernin has quit IRC | 07:00 | |
| *** Petersingh is now known as Petersingh|afk | 07:02 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Fix TAG and playbooks logic for all upgrade type jobs https://review.openstack.org/585528 | 07:04 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Update/upgrade commands for services with external installers https://review.openstack.org/582284 | 07:05 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Drop obsolete `overcloud ceph-upgrade run` https://review.openstack.org/582619 | 07:05 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Remove parameter for ceph-ansible playbook from update/upgrade prepare https://review.openstack.org/585366 | 07:05 |
| *** mhenkel_ has joined #tripleo | 07:05 | |
| *** mhenkel_ has quit IRC | 07:05 | |
| *** mhenkel_ has joined #tripleo | 07:06 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 07:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 07:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 07:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 07:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
| *** jtcressy has joined #tripleo | 07:12 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Update and upgrade tasks for services deployed via external deploy tasks https://review.openstack.org/579170 | 07:14 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Drop obsolete environment for `overcloud ceph-upgrade run` https://review.openstack.org/582624 | 07:14 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Ceph update and upgrade in config-download era https://review.openstack.org/583321 | 07:14 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow to remove xinetd service https://review.openstack.org/586057 | 07:15 |
| quiquell | Hello one question regarding tripleo-heat-templates | 07:17 |
| quiquell | It's ok if TRIPLEO_DEPLOY_IDENTIFIER is empty ? | 07:17 |
| quiquell | Comparing docker_config a two jobs | 07:17 |
| quiquell | Looks like it was TRIPLEO_DEPLOY_IDENTIFIER=1532451032 | 07:18 |
| *** ccamacho has joined #tripleo | 07:20 | |
| *** dtantsur|afk is now known as dtantsur | 07:21 | |
| quiquell | jfrancoa: So looks like get_param: DeployIdentifier is not working | 07:22 |
| Tengu | hmm, DeployIdentifier isn't in use anymore I think | 07:24 |
| Tengu | or maybe I misunderstood a comment on a review. | 07:24 |
| quiquell | Tengu: Yep, looks like bad path, we have now skip_deploy_identifier I think | 07:24 |
| *** pliu_ has quit IRC | 07:24 | |
| Tengu | maybe. Not that used to those templates for now. still having some struggles :). | 07:25 |
| *** khyr0n has quit IRC | 07:25 | |
| Tengu | hey btw - does anyone have a way to run tox on t-h-t repository? Apparently I'm lacking something in my env, because it fails due to parsing error unrelated to the change. | 07:25 |
| quiquell | jfrancoa: I think it's related to Ronelles findings | 07:30 |
| *** zoli|gone is now known as zoli | 07:30 | |
| quiquell | jfrancoa: There is an bad extra start_order: 1 at docker_config | 07:30 |
| jfrancoa | Tengu, do you have python3-devel.x86_64 installed? also, in my case, it sometimes helps removing the .tox directory before running tox again | 07:33 |
| jfrancoa | quiquell: what are Ronelle's findings? are they listed in the lp? | 07:34 |
| Tengu | jfrancoa: yep, I have that package. | 07:34 |
| quiquell | jfrancoa: Yep, bad docker_config.yaml | 07:34 |
| Tengu | jfrancoa: while running `tox -epep8' I have that kind of error: http://paste.openstack.org/show/726736/ | 07:37 |
| Tengu | jfrancoa: and this leads to false-positive errors, and just make tox unusable -.-' | 07:37 |
| *** shardy has joined #tripleo | 07:37 | |
| Tengu | maybe because python isn't fixed in the tox.ini task? | 07:38 |
| jfrancoa | Tengu: that looks to me like something is missing in that template "Validating ./docker/services/octavia/octavia-deployment-config.yaml" , did you modify it locally? | 07:39 |
| Tengu | jfrancoa: not at all | 07:39 |
| Tengu | jfrancoa: ok, fixing the basepython in pep8 task in the tox.ini kind of sorted it out. | 07:40 |
| Tengu | -.- | 07:40 |
| Tengu | and now I have a relevent error. damn. python version mix is a pain. | 07:40 |
| *** dparkes has joined #tripleo | 07:41 | |
| *** waleedm has joined #tripleo | 07:42 | |
| *** jtcressy has quit IRC | 07:47 | |
| *** shyambiradar has quit IRC | 07:47 | |
| *** jtomasek has joined #tripleo | 07:48 | |
| *** jtomasek has quit IRC | 07:48 | |
| *** shyambiradar has joined #tripleo | 07:48 | |
| *** jtomasek has joined #tripleo | 07:49 | |
| *** gkadam has joined #tripleo | 07:49 | |
| *** suuuper has joined #tripleo | 07:50 | |
| *** dciabrin has quit IRC | 07:51 | |
| *** pblaho has joined #tripleo | 07:58 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 08:00 |
| openstackgerrit | Thomas Herve proposed openstack/instack-undercloud master: Set connect_timeout in mysql https://review.openstack.org/586275 | 08:00 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Hard link environments to keep backwards compat https://review.openstack.org/586450 | 08:00 |
| *** dtantsur is now known as dtantsur|bbl | 08:00 | |
| *** paramite has joined #tripleo | 08:02 | |
| bogdando | shardy: hi! can you please point me the tripleo common templates processing method? | 08:04 |
| bogdando | I want to propose a mirror change https://review.openstack.org/586450 but for tc | 08:04 |
| bogdando | PTAL, btw, I hope that might work for stable/queens and osp13 minor upgrades become non breaking | 08:04 |
| bogdando | marios: ^^ | 08:05 |
| bogdando | see also http://post-office.corp.redhat.com/archives/rhos-df/2018-July/msg00210.html | 08:05 |
| *** jpich has joined #tripleo | 08:06 | |
| shardy | bogdando: sure it's here: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/templates.py#L89 | 08:09 |
| bogdando | thanks! | 08:10 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 08:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 08:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 08:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 08:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
| shardy | bogdando: so we're expecting to hard-link all environment files, forever? | 08:10 |
| shardy | bogdando: AFAIK we've handled upgrades moving/removing files via deprecation warnings and release notes previously | 08:11 |
| bogdando | 1) that's for transitioning period only so LTS users have time to adopt their resources registries before breaking changes in Rocky | 08:11 |
| bogdando | 2) that's for queens/osp13 only | 08:11 |
| bogdando | 3) links are deploy-time only | 08:12 |
| shardy | bogdando: I think having links which are written during deploy time could be confusing, e.g if the files don't exist in the tht path? | 08:12 |
| shardy | just thinking of the operator experience | 08:12 |
| bogdando | shardy: we missed OSP13 GA so now the topic is late in z2 and becomes breaking minor release update | 08:13 |
| shardy | bogdando: so we removed them without deprecation in Queens, and this is a workaround? | 08:13 |
| bogdando | right | 08:14 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow to remove xinetd service https://review.openstack.org/586057 | 08:14 |
| bogdando | ofc we can revert all and keep it only as of rocky, but then, we all will have hard times backporting everything for LTS time | 08:14 |
| *** paramite has quit IRC | 08:15 | |
| bogdando | here is an example for you https://review.openstack.org/#/c/575122/ | 08:15 |
| *** ledo_ has joined #tripleo | 08:15 | |
| shardy | bogdando: personally I think a revert may be appropriate - was this a backport or just a mistake during queens? | 08:15 |
| bogdando | consider the insane complexity of backporting that having services & services-baremetal in rocky vs services & services-docker in queens | 08:15 |
| shardy | I don't think the services environments should be regularly touched for backports | 08:16 |
| shardy | bogdando: we never backport new services | 08:16 |
| bogdando | I've come through that once, while trying to align things | 08:16 |
| *** shyambiradar has quit IRC | 08:16 | |
| bogdando | shardy: that's not for new ones, but changing default params or resource registry switches | 08:16 |
| bogdando | also note, that reverting that alignment will keep the downstream only changes | 08:18 |
| shardy | This isn't the place to discuss downstream changes, lets focus on the upstream branches :) | 08:19 |
| *** waleedm has quit IRC | 08:19 | |
| bogdando | so we will have a 3 different ways of switching to containers defaults: 1) rocky - as is, 2) stable/queens - no switched at all, BM is default, 3) downstream OSP13 - switched another way as of osp12 | 08:19 |
| bogdando | those backports just aligned all into 1 way | 08:19 |
| shardy | bogdando: No, we can switch the defaults in overcloud-resource-registry-puppet.j2.yaml, the issue appears to be we did that and also removed the environments/services-docker/* | 08:20 |
| shardy | I'm just trying to find out if that was a backport, sec | 08:20 |
| bogdando | shardy: thank you, I'd very appreciate any help or other proposals than just reverting things | 08:21 |
| shardy | bogdando: Sure, but it sounds like we made a mistake - environment files should never be removed between a stable branch GA and a z-stream release | 08:22 |
| shardy | if we did that, a revert is probably appropriate | 08:22 |
| bogdando | shardy: right. | 08:22 |
| bogdando | it didn't make it into GA | 08:22 |
| shardy | I just want to be clear on the root cause before approving of the workaround ;) | 08:22 |
| bogdando | so we have to either fix with hard links, or revert | 08:22 |
| bogdando | I brought some concerns against revert | 08:23 |
| bogdando | we barely will be happy maintaining backports having 3 different code bases as of default containerization environments | 08:23 |
| *** derekh has joined #tripleo | 08:23 | |
| bogdando | for a 5 years | 08:23 |
| openstackgerrit | Luigi Toscano proposed openstack/tripleo-quickstart master: Fix/enable the Tempest tests for Manila https://review.openstack.org/509554 | 08:23 |
| shardy | can we focus on the stable branch policy as a first step? | 08:24 |
| bogdando | mind elaborating please? | 08:24 |
| bogdando | for that particular case, not the policy | 08:24 |
| jaosorior | shardy: where in instack-undercloud are the ctlplane network addresses added to the interface? (the addresses that haproxy listens on) | 08:24 |
| bogdando | shardy: I'd say we've made a fatal mistake earlier, with that downstream only switch to containerized defaults as of OSP12 | 08:25 |
| bogdando | but I may be missing other solutions to align things | 08:25 |
| shardy | bogdando: https://review.openstack.org/#/c/567810/ and https://review.openstack.org/#/c/567811/ broke stable branch policy by backporting a backwards incompatible change | 08:26 |
| shardy | probably the correct solution is to revert those, deprecate the env files (which are now a no-op) for rocky, and remove them in stein | 08:26 |
| *** Petersingh|afk is now known as Petersingh | 08:26 | |
| bogdando | makes sense | 08:27 |
| bogdando | but I'm affraid the topic needs to be reverted all, or breaking things | 08:27 |
| shardy | well have to document anyone doing fast-forward upgrades from queens -> T to adjust their environments and/or we can add a warning now to say remove them | 08:28 |
| shardy | well we just revert the removal of the environment files | 08:28 |
| shardy | so z-stream updates don't break | 08:28 |
| bogdando | right, I hope that might work | 08:28 |
| shardy | bogdando: Ok, IMHO that's the right solution, but others may have other ideas | 08:28 |
| shardy | I'll comment on the patch | 08:28 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Enable logging to stdout/stderr in memcached https://review.openstack.org/585154 | 08:29 |
| bogdando | I'll try to propose those reverts but do that smart, some changes need to be omitted , like for https://review.openstack.org/#/c/567811/ we'd only need to revert the single removed file , shardy | 08:30 |
| bogdando | thanks! | 08:30 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Revert "Delete not-used services-docker files" https://review.openstack.org/586480 | 08:31 |
| *** alee has joined #tripleo | 08:32 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Revert "Delete not-used services-docker files" https://review.openstack.org/586480 | 08:36 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Revert "Delete not-used services-docker files" https://review.openstack.org/586480 | 08:38 |
| *** jaosorior has quit IRC | 08:38 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Revert "Move remnants of environments/services-docker" https://review.openstack.org/586484 | 08:40 |
| *** dciabrin has joined #tripleo | 08:43 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Revert "Move remnants of environments/services-docker" https://review.openstack.org/586484 | 08:45 |
| *** shyamb has joined #tripleo | 08:48 | |
| openstackgerrit | Dan Sneddon proposed openstack/os-net-config master: Add Policy Based Routing Support to os-net-config https://review.openstack.org/575712 | 08:52 |
| openstackgerrit | Dan Sneddon proposed openstack/os-net-config master: Add Policy Based Routing Support to os-net-config https://review.openstack.org/575712 | 08:53 |
| shardy | bogdando: ack yeah I think a partial revert that just re-adds the files would be fine - they could even be empty (or just specify an empty resource_registry mapping | 08:54 |
| *** lifeless has quit IRC | 08:54 | |
| bogdando | shardy: that's interesting idea, but may be we should consider emptying it in follow ups | 08:55 |
| shardy | bogdando: yeah probably a clean revert or partial revert is safest | 08:55 |
| *** jaosorior has joined #tripleo | 08:58 | |
| panda|rover | we need help with this, https://bugs.launchpad.net/tripleo/+bug/1783866 a yaml file ends up being malformed and a mistral workflow fails as consequence. We can't understand what's causing the yaml to be malformed | 09:03 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:03 |
| *** ledo_ has quit IRC | 09:05 | |
| *** d0ugal has joined #tripleo | 09:09 | |
| *** salmankhan has joined #tripleo | 09:09 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 09:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 09:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 09:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 09:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
| *** lifeless has joined #tripleo | 09:11 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: [DNM] To debug fix tags https://review.openstack.org/585955 | 09:14 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Duplicate of I8330687ec64f1ebb7e3b100410d5bdc3e67ae7f0 https://review.openstack.org/586444 | 09:18 |
| *** pliu_ has joined #tripleo | 09:22 | |
| Tengu | hmm, does this CI error rings bells? http://logs.openstack.org/57/586057/9/check/tripleo-ci-centos-7-containers-multinode/54aeac0/job-output.txt.gz#_2018-07-27_08_43_11_053487 | 09:23 |
| quiquell | Tengu: looks like infra issue, recheck fixed it for us yesterday | 09:24 |
| quiquell | Tengu: Do you have some tht expertise ? | 09:24 |
| Tengu | quiquell: more or less, still learning | 09:25 |
| Tengu | will do a recheck later once the current run is over. | 09:25 |
| *** leanderthal has joined #tripleo | 09:25 | |
| quiquell | Tengu: Do you know about docker_config.yaml composition on updates ? | 09:25 |
| Tengu | quiquell: err.. not yet :(. sorry | 09:26 |
| quiquell | Tengu: Who can help us ? | 09:26 |
| *** quiquell has quit IRC | 09:27 | |
| *** quiquell has joined #tripleo | 09:27 | |
| Tengu | shardy: maybe you can help quiquell ? | 09:28 |
| quiquell | shardy: This is the bug https://bugs.launchpad.net/tripleo/+bug/1783866 | 09:29 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:29 |
| *** Petersingh is now known as Petersingh|afk | 09:30 | |
| *** holser_ has joined #tripleo | 09:31 | |
| *** zoli is now known as zoli|lunch | 09:32 | |
| shardy | quiquell: ack looking at the bug | 09:42 |
| Tengu | anyone for some review? https://review.openstack.org/#/c/582917/ thank you :) | 09:43 |
| quiquell | shardy: Weird thing, if we run with a Depends-On the docker_config.yaml is correct | 09:44 |
| shardy | quiquell: are we sure it's not some service with a bad docker_config that's been fixed in master? | 09:45 |
| panda|rover | shardy: we are trying to understand as first step, what adds a spurious start_order: 1 at the end of https://logs.rdoproject.org/95/583195/11/openstack-check/legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master/696681e/logs/undercloud/var/lib/mistral/e29e9ccd-2725-4522-9db6-32034e3971d2/Controller/docker_config.yaml.txt.gz | 09:45 |
| panda|rover | shardy: looks like something external is adding that part, it's not genreated, because it's malformed yaml | 09:46 |
| quiquell | shardy: We have a fresh run, and it's still failing | 09:47 |
| shardy | panda|rover: to clarify you mean start_order: 1 on line 10? | 09:47 |
| quiquell | http://logs.openstack.org/28/585528/8/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/c57633c/ | 09:47 |
| panda|rover | shardy: at the end of the file I pasted, ther's a start_order: 1 and a user: root | 09:48 |
| quiquell | shardy: The last logs I pasted are run from latest tht e87fcba, so it's not tht | 09:48 |
| shardy | panda|rover: ah sorry yeah I see now | 09:48 |
| panda|rover | shardy: that part is clearly out of place, and the indentation makes it a malformed yaml, anything that tried to parse it, fails | 09:48 |
| panda|rover | I suppose mistral is creating those files ? | 09:49 |
| quiquell | shardy: Maybe there is a moment when we copy the one created at mistral/Controller/docker_config.yaml to mistral/xxx/Controller/docker_config.yaml | 09:49 |
| *** jaosorior has quit IRC | 09:49 | |
| quiquell | And add it the lines | 09:49 |
| shardy | panda|rover: they should just come from config-download AFAIK, so they're just generated from the heat outputs | 09:50 |
| quiquell | config-download one is broken too | 09:50 |
| shardy | but I'm not sure if extra magic is added for the updates case ;) | 09:50 |
| shardy | quiquell: yeah I wonder if we copy a new file over an existing file without truncating or something | 09:51 |
| quiquell | shardy: Do you where can be that magic ? tripleo-upgrade role ? | 09:51 |
| quiquell | shardy: Also why the Depends-On a tht change works :-/ | 09:51 |
| shardy | quiquell: not sure tbh, AFAIK this should all just be config download followed by ansible-playbook but I've not been involved with updates/upgrades lately | 09:52 |
| shardy | marios: ^^ perhaps you know? | 09:52 |
| * shardy lookingg | 09:52 | |
| panda|rover | marios is PTO today | 09:52 |
| panda|rover | jistr: maybe ? | 09:52 |
| jistr | heya | 09:53 |
| jistr | i'm just discussing it on private chat with jfrancoa, didn't notice there was discussion here too | 09:53 |
| shardy | we recently introduced git controlled config download dirs, so I wonder if something is causing the new content to get mixed with the old file | 09:53 |
| shardy | e.g the diff vs overwriting the entire file | 09:54 |
| jistr | yea that's exactly what i'm thinking | 09:54 |
| jistr | we had a bad bug like this in the past | 09:54 |
| jistr | but i can't recall where it was | 09:54 |
| jistr | basically it was something like: we opened the file for writing, but it didn't truncate it to 0 size first | 09:55 |
| *** Haresh has quit IRC | 09:55 | |
| quiquell | jistr: But why the Depends-On passes ? | 09:56 |
| quiquell | jistr: Maybe it prevents some post stuff to run ? | 09:56 |
| shardy | quiquell: maybe by luck the latest tht generates a docker_config that's 2 lines longer than the parent revision of your patch or something? | 09:57 |
| shardy | https://github.com/openstack/tripleo-common/commit/f8aa548ca692e330bdc47bf32b3f1c309e50d45c#diff-b0f1a537852dc17e04252b05d779fd65 | 09:57 |
| shardy | I'd be tempted to try a Depends-On with a revert of that | 09:57 |
| shardy | can you reproduce it locally? | 09:57 |
| quiquell | shardy: RDO cloud is down, we can try libvirt | 09:58 |
| *** holser_ has quit IRC | 09:59 | |
| quiquell | shardy: Going to try a review with the revert | 10:00 |
| panda|rover | I love when the cavalry comes to the rescue. | 10:01 |
| shardy | panda|rover: heh, we've not fixed anything yet :D | 10:02 |
| shardy | you could probably also test this by doing config download, then removing some services and updating the stack, and running it again | 10:03 |
| jistr | ROFL | 10:03 |
| jistr | it could be exactly the same kind of issue | 10:03 |
| jistr | i finally found the original fix which made me look for the right thing https://review.openstack.org/#/c/168328/ | 10:04 |
| jistr | 3.5 yrs ago :) | 10:04 |
| shardy | I also wonder how the new git approach handles when some files are removed between updates | 10:04 |
| jistr | i guess it has to be this: | 10:04 |
| jistr | https://github.com/openstack/tripleo-common/blob/284020b6b66803860f33241cea64dc861bc3a671/tripleo_common/utils/config.py#L109 | 10:04 |
| jistr | do we have a bug open? I'd try posting a fix | 10:04 |
| quiquell | jistr: https://bugs.launchpad.net/tripleo/+bug/1783866 | 10:05 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:05 |
| quiquell | jistr: We need to test your fix from our gate fix, to ensure that we don't have a false postive | 10:06 |
| quiquell | jistr: past me the review when you have it | 10:06 |
| * quiquell crossing finger | 10:07 | |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Sharing BGPVPN Heat plugin volume https://review.openstack.org/585320 | 10:07 |
| shardy | Hmm but we've got 'w' for the fdopen and WRONLY | CREAT for the open so it shouldn't append? | 10:07 |
| * shardy probably needs more coffee | 10:07 | |
| jfrancoa | gosh jistr your memory is good :-D | 10:08 |
| shardy | oh it should be O_TRUNC ? | 10:08 |
| jistr | shardy: yes, along with the other ones | 10:08 |
| jistr | jfrancoa: that's so crazy bug that it's hard to forget :) | 10:08 |
| shardy | Ok that's confusing, e.g WRONLY != APPEND | 10:08 |
| shardy | only it does ;) | 10:08 |
| *** sshnaidm|afk has quit IRC | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 10:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 10:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 10:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 10:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-common master: Fix overwriting downloaded config files https://review.openstack.org/586499 | 10:14 |
| quiquell | jistr: ^ thanks | 10:14 |
| jistr | quiquell: thanks for the bug link, here's the fix attempt | 10:15 |
| quiquell | jistr: Let's run it from the fix for the gates, to ensure it fixes updates | 10:15 |
| quiquell | jistr: But again, maybe the Depends-On just make it pass :-/ | 10:15 |
| *** holser_ has joined #tripleo | 10:15 | |
| *** holser_ has quit IRC | 10:15 | |
| jfrancoa | jistr: thanks man, this confirms that memory is key for working in TripleO :-D | 10:16 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Avoid ifcfg files modification when a reboot is not required https://review.openstack.org/586500 | 10:17 |
| *** holser_ has joined #tripleo | 10:17 | |
| *** holser_ has quit IRC | 10:17 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Fix TAG and playbooks logic for all upgrade type jobs https://review.openstack.org/585528 | 10:17 |
| quiquell | jistr: ^ testing patch | 10:17 |
| *** leanderthal has quit IRC | 10:17 | |
| jistr | jfrancoa: yea... :D | 10:18 |
| jistr | quiquell: thanks | 10:18 |
| panda|rover | that's why we give 16GB to the undercloud | 10:19 |
| quiquell | panda|rover: So undercloud had the answer of it ? | 10:19 |
| jfrancoa | panda|rover: badam ptss :-D (I liked it indeed) | 10:19 |
| *** holser_ has joined #tripleo | 10:19 | |
| quiquell | jfrancoa: Also now the upgrade job is failing http://logs.openstack.org/28/585528/8/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/5a7fb08/logs/ | 10:20 |
| *** holser_ has quit IRC | 10:20 | |
| quiquell | What can go wrong... | 10:20 |
| *** Petersingh|afk is now known as Petersingh | 10:21 | |
| jistr | we need this for upgrades job https://review.openstack.org/#/c/585654/ | 10:21 |
| * jfrancoa checking | 10:21 | |
| jistr | there it's passing | 10:21 |
| quiquell | An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: Controller/step_config.pp"} | 10:21 |
| quiquell | jistr: for this ? ^ | 10:21 |
| *** Petersingh is now known as Petersingh|away | 10:21 | |
| jistr | oh that's different :) | 10:21 |
| jfrancoa | jistr: yes, it's what I was about to say | 10:22 |
| jfrancoa | jistr: it's failing in the deployment | 10:22 |
| quiquell | http://logs.openstack.org/28/585528/8/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/5a7fb08/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 10:22 |
| quiquell | jfrancoa, jistr: If it's legit, we are ok, with the fix, I mean that we didn't break anything with our CI scripts | 10:22 |
| *** Petersingh|away has quit IRC | 10:23 | |
| jfrancoa | quiquell: it seems to be legit, although we're missing logs in /undercloud/var/lib/ | 10:26 |
| *** bogdando has quit IRC | 10:26 | |
| *** bogdando_ has joined #tripleo | 10:26 | |
| quiquell | jfrancoa: Weird, update job have them :-/ | 10:27 |
| *** bogdando_ is now known as bogdando | 10:29 | |
| *** Haresh has joined #tripleo | 10:31 | |
| *** shyamb has quit IRC | 10:32 | |
| *** shyamb has joined #tripleo | 10:32 | |
| *** shyamb has quit IRC | 10:33 | |
| * jistr lunch, biab | 10:35 | |
| panda|rover | jfrancoa: in https://review.openstack.org/585654 were the static: no needed ? include_tasks: should already be dynamic | 10:37 |
| panda|rover | jfrancoa: nevermind, reading the comments | 10:38 |
| jfrancoa | panda|rover: I added it after your comment the other day in the chat about include vs include_tasks that it should be used with static: no , did I misunderstand it? | 10:39 |
| *** agurenko has joined #tripleo | 10:39 | |
| *** agurenko has quit IRC | 10:39 | |
| panda|rover | jfrancoa: oh, not what I meant :) you asked if include can be replaced by include_tasks. since Include_task is the dynamic version, it's ok to replace include with include_tasks when the original include contained a static: no. if instead you wanted to replace a static set of tasks you should have used import | 10:41 |
| jfrancoa | panda|rover: ok, I totally misunderstood it then. I'll remove it then, thanks for the info | 10:42 |
| Tengu | PTAL guys? https://review.openstack.org/#/c/582917/ thank you :) | 10:45 |
| *** dtantsur|bbl is now known as dtantsur | 10:49 | |
| *** thrash|g0ne is now known as thrash | 10:49 | |
| *** brault_ has joined #tripleo | 10:50 | |
| ccamacho | jfrancoa are you removing the "static: no" from https://review.openstack.org/#/c/585654 ? | 10:50 |
| openstackgerrit | Dan Sneddon proposed openstack/os-net-config master: Fix numbered NIC mapping when using dotted VLAN notation https://review.openstack.org/586507 | 10:50 |
| *** brault has quit IRC | 10:53 | |
| *** zoli|lunch is now known as zoli | 11:02 | |
| *** cshastri has quit IRC | 11:02 | |
| *** dhill_ has quit IRC | 11:03 | |
| jfrancoa | ccamacho: yes, give me a minute and I'll do it | 11:03 |
| ccamacho | cool | 11:04 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Change include for include_tasks when including loop controled scripts. https://review.openstack.org/585654 | 11:06 |
| jfrancoa | ccamacho: done ^^^ | 11:06 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 11:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 11:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 11:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 11:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 11:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 11:10 |
| *** Haresh has quit IRC | 11:12 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Fix TAG and playbooks logic for all upgrade type jobs https://review.openstack.org/585528 | 11:14 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: [WIP][DNM] Take playbooks from ansible https://review.openstack.org/586513 | 11:14 |
| *** bogdando has quit IRC | 11:16 | |
| *** bogdando has joined #tripleo | 11:16 | |
| *** dparkes has quit IRC | 11:16 | |
| *** cshastri has joined #tripleo | 11:17 | |
| ccamacho | jistr mind to revote here? v | 11:18 |
| ccamacho | https://review.openstack.org/#/c/585654/ | 11:19 |
| *** quiquell has quit IRC | 11:21 | |
| *** quiquell has joined #tripleo | 11:22 | |
| *** ledo_ has joined #tripleo | 11:22 | |
| *** quiquell has quit IRC | 11:24 | |
| *** quiquell has joined #tripleo | 11:24 | |
| *** Haresh has joined #tripleo | 11:26 | |
| *** morazi has joined #tripleo | 11:29 | |
| *** skramaja has quit IRC | 11:31 | |
| *** pradk has joined #tripleo | 11:32 | |
| *** nenad has joined #tripleo | 11:36 | |
| *** dprince has joined #tripleo | 11:38 | |
| *** quiquell has quit IRC | 11:40 | |
| *** d0ugal has quit IRC | 11:41 | |
| *** quiquell has joined #tripleo | 11:42 | |
| *** quiquell has quit IRC | 11:43 | |
| *** ledo__ has joined #tripleo | 11:43 | |
| EmilienM | hi | 11:43 |
| *** quiquell has joined #tripleo | 11:44 | |
| *** ledo_ has quit IRC | 11:46 | |
| *** rh-jelabarre has joined #tripleo | 11:47 | |
| *** atoth has joined #tripleo | 11:47 | |
| *** quiquell has quit IRC | 11:48 | |
| *** quiquell has joined #tripleo | 11:48 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Fix python2 for pep8 tests. https://review.openstack.org/586518 | 11:49 |
| *** quiquell_ has joined #tripleo | 11:49 | |
| *** quiquell has quit IRC | 11:49 | |
| *** quiquell has joined #tripleo | 11:52 | |
| *** quiquell_ has quit IRC | 11:52 | |
| *** amoralej is now known as amoralej|lunch | 11:54 | |
| *** quiquell_ has joined #tripleo | 11:54 | |
| *** rfolco|off is now known as rfolco|ruck | 11:55 | |
| *** lblanchard has joined #tripleo | 11:58 | |
| *** quiquell has quit IRC | 11:58 | |
| *** toure|gone is now known as toure | 12:00 | |
| *** quiquell_ has quit IRC | 12:01 | |
| *** quiquell has joined #tripleo | 12:02 | |
| *** medberry has joined #tripleo | 12:04 | |
| *** medberry has quit IRC | 12:04 | |
| *** medberry has joined #tripleo | 12:04 | |
| *** sshnaidm has joined #tripleo | 12:08 | |
| *** sshnaidm is now known as sshnaidm|off | 12:08 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 12:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 12:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 12:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 12:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 12:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 12:10 |
| *** quiquell has quit IRC | 12:13 | |
| *** dhill_ has joined #tripleo | 12:15 | |
| *** edmondsw has joined #tripleo | 12:17 | |
| *** jcoufal has joined #tripleo | 12:25 | |
| *** holser_ has joined #tripleo | 12:25 | |
| *** wolverineav has joined #tripleo | 12:26 | |
| *** rbrady has quit IRC | 12:32 | |
| *** mdnadeem has quit IRC | 12:32 | |
| *** mjturek has joined #tripleo | 12:32 | |
| *** pblaho has quit IRC | 12:33 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates stable/queens: undercloud: enable KernelIpNonLocalBind https://review.openstack.org/586531 | 12:33 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs050: enable TLS https://review.openstack.org/585333 | 12:34 |
| *** trown|outtypewww is now known as trown | 12:34 | |
| *** tzumainn has joined #tripleo | 12:34 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs050: enable TLS https://review.openstack.org/585333 | 12:36 |
| *** agopi has quit IRC | 12:37 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates stable/queens: OVN: Set sysctl value 'net.ipv4.ip_nonlocal_bind' to 1 https://review.openstack.org/586533 | 12:39 |
| fultonj | i'm deploying queens w 1cntrl+1cmpute and the VIPs in my /etc/puppet/hieradata/service_configs.json do not get assigned on my controller. | 12:44 |
| fultonj | ip a | grep 32 | wc -l --> 0 | 12:44 |
| fultonj | Thus deployment fails with aodh unable to connect to mysql on the vip | 12:44 |
| fultonj | So what part of tripleo assigns the VIPs so I can troubleshoot it? | 12:45 |
| shardy | fultonj: they're defined in the haproxy config and brought up by either pacemaker or keepalived | 12:45 |
| *** ohochman has joined #tripleo | 12:45 | |
| *** amoralej|lunch is now known as amoralej | 12:47 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/queens: Fix deploy health checks https://review.openstack.org/586536 | 12:47 |
| fultonj | thanks shardy | 12:47 |
| openstackgerrit | Merged openstack/ansible-role-tripleo-keystone master: Add basic jobs in Zuul Layout https://review.openstack.org/585548 | 12:47 |
| *** rlandy has quit IRC | 12:48 | |
| *** rlandy has joined #tripleo | 12:48 | |
| *** holser_ has quit IRC | 12:48 | |
| *** holser_ has joined #tripleo | 12:49 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New function: run_ansible_playbook https://review.openstack.org/586538 | 12:51 |
| *** ansmith has joined #tripleo | 12:52 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates stable/queens: DNM: Test Gnocchi Upgrades P->Q. https://review.openstack.org/565231 | 12:52 |
| *** holser_ has quit IRC | 12:53 | |
| *** jfrancoa has quit IRC | 12:59 | |
| *** jfrancoa has joined #tripleo | 12:59 | |
| *** pchavva has joined #tripleo | 13:01 | |
| *** quiquell has joined #tripleo | 13:02 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New preflight check for the undercloud: disk space https://review.openstack.org/586541 | 13:02 |
| Tengu | EmilienM: -^ :) | 13:02 |
| Tengu | hopefully this is right. | 13:02 |
| *** avivgt has quit IRC | 13:03 | |
| *** rpioso|afk is now known as rpioso | 13:03 | |
| EmilienM | Tengu: nice! I'll review after mtgs | 13:03 |
| Tengu | EmilienM: updated the trello card accordingly. | 13:04 |
| openstackgerrit | Thomas Herve proposed openstack/instack-undercloud master: Set connect_timeout in mysql https://review.openstack.org/586275 | 13:04 |
| therve | mwhahaha: ^^ opened a bug | 13:05 |
| therve | It'd be cool to have some help for the tht change too | 13:06 |
| Tengu | dpeacock: heya! are you available for some pythoning? :) | 13:06 |
| Tengu | oh wow. EmilienM pep8 with python3 and the small modification I made: pep8: commands succeeded | 13:08 |
| Tengu | dpeacock: guess I got it right in fact. my first python2 -> python3 script modification :D | 13:09 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 13:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 13:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 13:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 13:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 13:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 13:10 |
| *** edleafe is now known as figleaf | 13:11 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Fix python3 support in yaml-validate script. https://review.openstack.org/586518 | 13:11 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Fix python3 support in yaml-validate script. https://review.openstack.org/586518 | 13:12 |
| *** alee has quit IRC | 13:13 | |
| *** alee has joined #tripleo | 13:13 | |
| *** mjturek has quit IRC | 13:13 | |
| *** jcoufal has quit IRC | 13:16 | |
| *** mandre is now known as mandre_away | 13:18 | |
| *** alee is now known as alee_food | 13:20 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Fix python3 support in yaml-validate script. https://review.openstack.org/586518 | 13:20 |
| *** jcoufal has joined #tripleo | 13:20 | |
| *** ledo__ has quit IRC | 13:21 | |
| *** agopi has joined #tripleo | 13:22 | |
| *** mcornea has joined #tripleo | 13:23 | |
| *** pblaho has joined #tripleo | 13:25 | |
| *** pblaho has quit IRC | 13:25 | |
| *** pblaho has joined #tripleo | 13:25 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New function: run_ansible_playbook https://review.openstack.org/586538 | 13:25 |
| *** mjturek has joined #tripleo | 13:26 | |
| *** gbarros has joined #tripleo | 13:27 | |
| *** psahoo has quit IRC | 13:27 | |
| *** mjturek has quit IRC | 13:28 | |
| *** alee_food has quit IRC | 13:29 | |
| *** mjturek has joined #tripleo | 13:31 | |
| *** jistr is now known as jistr|mtg | 13:32 | |
| *** quiquell_ has joined #tripleo | 13:32 | |
| *** quiquell has quit IRC | 13:32 | |
| *** mjturek has quit IRC | 13:33 | |
| *** myoung has joined #tripleo | 13:35 | |
| *** quiquell_ has quit IRC | 13:35 | |
| *** quiquell has joined #tripleo | 13:35 | |
| *** medberry has quit IRC | 13:37 | |
| quiquell | jfrancoa: Just to confirm this is a legit error from upgrades ? http://logs.openstack.org/28/585528/10/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/b328978/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 13:40 |
| quiquell | "failed": true, "msg": "An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: Controller/step_config.pp" | 13:40 |
| quiquell | ccamacho: ^? | 13:40 |
| jfrancoa | quiquell: well, it's a legit error. or it seems, although it's not in upgrades workflow. It's happening during overcloud deployment | 13:41 |
| jfrancoa | quiquell: and again, there are no logs in /undercloud/var/lib -> http://logs.openstack.org/28/585528/10/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/b328978/logs/undercloud/var/lib/ | 13:41 |
| quiquell | Only unbound is there | 13:42 |
| *** bogdando has quit IRC | 13:43 | |
| *** dciabrin has quit IRC | 13:46 | |
| *** dciabrin has joined #tripleo | 13:46 | |
| *** moguimar has joined #tripleo | 13:47 | |
| *** mhenkel_ has quit IRC | 13:47 | |
| *** jaganathan has quit IRC | 13:48 | |
| openstackgerrit | Sylvain Afchain proposed openstack/tripleo-common master: Remove skydive workbook as using config-download now https://review.openstack.org/586556 | 13:49 |
| *** moguimar has quit IRC | 13:50 | |
| openstackgerrit | Sylvain Afchain proposed openstack/tripleo-common master: Add skydive-ansible to mistral base packages https://review.openstack.org/586561 | 13:54 |
| jrist | honza, jtomasek - fyi latest quickstart works, but there are some things broken with quickstart for the UI | 13:55 |
| honza | jrist: tell me more | 13:55 |
| jrist | honza: ran a regular quickstart - » bash quickstart.sh --clean --teardown all --no-clone -w /home/jrist/.quickstart-galena/ --release master 172.16.213.154 | 13:56 |
| jrist | honza: the UI config doesn't get changed https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/tasks/tripleo-ui.yml | 13:56 |
| jrist | not sure exactly how to fix it, but going to file against launchpad | 13:56 |
| derekh | dtantsur: I'm taking a look at https://trello.com/c/eRPY8uQz/84-ironic-features-checklist to check some items off | 13:57 |
| derekh | dtantsur: I think you put that list together | 13:57 |
| honza | jrist: cool! | 13:57 |
| honza | jrist: https://bugs.launchpad.net/tripleo/+bug/1782438 | 13:57 |
| openstack | Launchpad bug 1782438 in tripleo "tripleo ui endpoints misconfigured in containerized undercloud" [High,Triaged] | 13:57 |
| derekh | dtantsur: the ramdisk logs for both inspect and deploy and captured on success, is this enough or do we specifically need to trigger an error? | 13:57 |
| dtantsur | derekh: it's enough | 13:57 |
| derekh | dtantsur: ack, thanks | 13:58 |
| *** alee_food has joined #tripleo | 13:58 | |
| honza | jrist: i'm trying the libvirt reproducer right now | 13:59 |
| *** kopecmartin has quit IRC | 14:01 | |
| EmilienM | rfolco|ruck: why tripleo-ci-centos-7-3nodes-multinode is always failing? | 14:01 |
| jrist | honza: oh nice you already did, glad I mentioned that to you | 14:01 |
| jrist | honza: also no default plan? is that a new thing? | 14:02 |
| jrist | or did something else not get fired off | 14:02 |
| jrist | jtomasek: ^ | 14:02 |
| rfolco|ruck | EmilienM, will take a look | 14:02 |
| honza | jrist: weird, haven't seen that one | 14:02 |
| jrist | https://i.imgur.com/1fTk2Vi.png | 14:03 |
| *** mcornea has quit IRC | 14:04 | |
| Tengu | mwhahaha: heya! would you have a couple of minute for a BJ regarding the test coverage? dpeacock and I are a bit lost... | 14:04 |
| mwhahaha | Tengu: sure gimme 30 mins | 14:04 |
| *** mcornea has joined #tripleo | 14:04 | |
| Tengu | mwhahaha: great, thanks! | 14:04 |
| rfolco|ruck | EmilienM, apparently its tempest tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops... will confirm if this is consistent | 14:05 |
| quiquell | jrist: Looks like your review is working https://review.openstack.org/#/c/586499 | 14:05 |
| quiquell | jidar: Some reviewing is needed there | 14:06 |
| jrist | quiquell: itym jistr :) | 14:06 |
| jrist | how do you pronounce your name quiquell | 14:06 |
| jrist | kick-ell? | 14:06 |
| quiquell | jrist: ll is from the second name | 14:08 |
| quiquell | jrist: kick-ehhh is good | 14:08 |
| *** gbarros has quit IRC | 14:08 | |
| jrist | cool thanks | 14:08 |
| jrist | :) | 14:08 |
| *** alee_food is now known as alee | 14:10 | |
| *** links has quit IRC | 14:10 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 14:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 14:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 14:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 14:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 14:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 14:10 |
| panda|rover | EmilienM: shardy mwhahaha fishing for reviews on https://review.openstack.org/#/c/586499 to unblock updates checks | 14:13 |
| panda|rover | jistr|mtg: your patch worked. | 14:13 |
| jistr|mtg | whohoo \o/ | 14:13 |
| *** pblaho has quit IRC | 14:14 | |
| *** shardy has quit IRC | 14:18 | |
| *** moguimar has joined #tripleo | 14:18 | |
| *** moguimar has quit IRC | 14:18 | |
| jrist | woothoo! | 14:19 |
| Tengu | o_O I thought "w" mode overrode the content - the "a" is for "append"... ?! | 14:21 |
| *** lblanchard has quit IRC | 14:23 | |
| *** suuuper has quit IRC | 14:25 | |
| *** mjturek has joined #tripleo | 14:28 | |
| *** Haresh has quit IRC | 14:28 | |
| *** links has joined #tripleo | 14:29 | |
| *** mhenkel_ has joined #tripleo | 14:30 | |
| *** ramishra has quit IRC | 14:31 | |
| *** mhenkel_ has quit IRC | 14:34 | |
| *** mjturek has quit IRC | 14:38 | |
| *** jistr|mtg is now known as jistr | 14:39 | |
| *** shardy has joined #tripleo | 14:40 | |
| *** mhenkel_ has joined #tripleo | 14:43 | |
| mwhahaha | great ovb is broken because we don't have the latest dib | 14:43 |
| mwhahaha | https://logs.rdoproject.org/99/586499/1/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/18a383e/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz#_2018-07-27_11_54_22 | 14:43 |
| * mwhahaha blames owalsh | 14:43 | |
| *** Haresh has joined #tripleo | 14:45 | |
| owalsh | mwhahaha: ah crap | 14:45 |
| jistr | Tengu: i'm not sure what's the low-level workings there, but it seems that the inner os.open() matters in terms of flags. If we used open(..., 'w') -- not os.open, as there seems to be a difference -- it would work just fine. But we wouldn't be able to specify the file permissions directly in the call when using open(), so i guess that's why we use the os.fdopen(os.open()) pattern | 14:46 |
| owalsh | mwhahaha: should I revert the image patch, or is dib likely to get promoted soon? | 14:47 |
| mwhahaha | owalsh: well we need to fix promotion issues anyway, not sure if we should just add dib back into the includpkgs list or not | 14:47 |
| Tengu | jistr: hmm ok. funky. and good catch! | 14:48 |
| *** ohochman has quit IRC | 14:48 | |
| mwhahaha | EmilienM: -^ should we revert https://review.openstack.org/#/c/571779/ because we don't have the latest dib so it broke ovb or do we add dib back into the includepkgs? Thoughts? | 14:49 |
| *** ramishra has joined #tripleo | 14:49 | |
| EmilienM | looking | 14:49 |
| EmilienM | oh damn | 14:49 |
| EmilienM | yeah rever | 14:50 |
| EmilienM | and wait for promotion | 14:50 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Enable decoupled deployment plans e.g for OpenShift, DPDK etc https://review.openstack.org/574753 | 14:50 |
| EmilienM | mwhahaha: ^ | 14:50 |
| EmilienM | quiquell: thanks for your help on the tag issue | 14:51 |
| *** ksambor has quit IRC | 14:52 | |
| mwhahaha | owalsh: so yea, do you need it right now or can we back it out until promotion? | 14:53 |
| owalsh | mwhahaha: yea, can back it out | 14:54 |
| * mwhahaha files a bug and revert | 14:54 | |
| *** mjturek has joined #tripleo | 14:56 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Revert "Add disable-nouveau element to tripleo images" https://review.openstack.org/586591 | 14:56 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Revert "Add disable-nouveau element to tripleo images" https://review.openstack.org/586591 | 14:56 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Revert "Revert "Add disable-nouveau element to tripleo images"" https://review.openstack.org/586595 | 14:58 |
| mwhahaha | owalsh: -^ the revert-revert for post-promotion | 14:58 |
| *** Haresh has quit IRC | 14:59 | |
| *** dparkes has joined #tripleo | 14:59 | |
| *** links has quit IRC | 15:02 | |
| quiquell | EmilienM: team effort, tried to unfuck my fucked up, thanks for your patience though | 15:06 |
| EmilienM | it's all good | 15:08 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 15:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 15:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 15:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 15:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 15:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 15:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 15:10 |
| *** Haresh has joined #tripleo | 15:11 | |
| *** nenad has quit IRC | 15:12 | |
| *** ramishra has quit IRC | 15:13 | |
| *** cshastri has quit IRC | 15:15 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Begin adding environments with all params for a service https://review.openstack.org/475924 | 15:16 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Add ability to generate an environment index https://review.openstack.org/491925 | 15:16 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Add service config env with all Designate settings https://review.openstack.org/555008 | 15:16 |
| EmilienM | shardy: how do you deploy ansible-openshift in the container on the undercloud? | 15:21 |
| EmilienM | shardy: I saw https://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/config/general_config/featureset033.yml#n53 | 15:21 |
| EmilienM | but it's on the host | 15:21 |
| EmilienM | so I wonder how you can use external_deploy_tasks without having ansible-openshift in the mistral container | 15:21 |
| shardy | EmilienM: currently we don't, we use openshift-ansible from either a package or source, but I think flaper87 has been looking at changing to use the container | 15:22 |
| EmilienM | ok so the external_deploy_tasks use the role from the host | 15:22 |
| EmilienM | shardy: safchain is trying to do the same with Skydive | 15:22 |
| shardy | EmilienM: yeah, but I think it'll be better in future if we can run ansible in a container with osa inside | 15:23 |
| shardy | for now it runs on the host | 15:23 |
| EmilienM | shardy: ok so best path for them is to deploy the role in the mistral container, right? | 15:23 |
| EmilienM | for now | 15:23 |
| shardy | https://github.com/openstack/tripleo-heat-templates/commit/10ebdc81abd7d89c87510703cf48e124508814d4#diff-185ebdd6080da3d955fc5814b391c3d5 | 15:24 |
| EmilienM | oh | 15:24 |
| shardy | EmilienM: it's probably best to ensure it's installed to /usr/share/ansible on the host I think | 15:24 |
| EmilienM | that's new | 15:24 |
| shardy | or testing new ansible stuff becomes very inconvenient | 15:25 |
| EmilienM | I missed that commit | 15:25 |
| shardy | EmilienM: yeah we had to add it because the switch to containers broke the openshift CI job | 15:25 |
| EmilienM | safchain: so yeah, please retry with latest THT ^ | 15:25 |
| shardy | for exactly the reason you mentioned | 15:25 |
| EmilienM | shardy: ok | 15:25 |
| safchain | shardy, ok thanks weird I'm not seeing this bind mount | 15:25 |
| safchain | shardy, will recheck | 15:26 |
| *** alee is now known as ade_lee | 15:29 | |
| *** suuuper has joined #tripleo | 15:32 | |
| quiquell | EmilienM: openstack infra issues at zuul.o.o, problems at gates, we are sooo lucky | 15:35 |
| *** quiquell is now known as quique|luckyluck | 15:36 | |
| *** quique|luckyluck is now known as quiquell | 15:37 | |
| *** gkadam has quit IRC | 15:38 | |
| *** quiquell has quit IRC | 15:39 | |
| EmilienM | :( | 15:41 |
| -openstackstatus- NOTICE: A zuul config error slipped through and caused a pile of job failures with retry_limit - a fix is being applied and should be back up in a few minutes | 15:41 | |
| *** ChanServ changes topic to "A zuul config error slipped through and caused a pile of job failures with retry_limit - a fix is being applied and should be back up in a few minutes" | 15:41 | |
| *** mhenkel_ has quit IRC | 15:42 | |
| *** dprince has quit IRC | 15:42 | |
| *** pliu_ has quit IRC | 15:43 | |
| *** lvdombrkr89 has quit IRC | 15:45 | |
| *** mhenkel_ has joined #tripleo | 15:46 | |
| *** mhenkel_ has quit IRC | 15:50 | |
| *** pradk has quit IRC | 15:51 | |
| *** gkadam has joined #tripleo | 15:53 | |
| *** gkadam has quit IRC | 15:54 | |
| *** gkadam has joined #tripleo | 15:55 | |
| *** pradk has joined #tripleo | 15:56 | |
| *** rwsu has joined #tripleo | 15:57 | |
| *** Haresh has quit IRC | 15:59 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 16:00 |
| *** shardy has quit IRC | 16:01 | |
| *** jfrancoa has quit IRC | 16:02 | |
| *** jfrancoa has joined #tripleo | 16:03 | |
| *** openstackgerrit has quit IRC | 16:04 | |
| *** jpich has quit IRC | 16:06 | |
| *** openstackgerrit has joined #tripleo | 16:08 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New function: run_ansible_playbook https://review.openstack.org/586538 | 16:08 |
| Tengu | mwhahaha: -^^ not finished yet, and I think it's maybe even really ugly and bad. Will take that back Monday. | 16:08 |
| Tengu | have a great day and greater weekend folks :). | 16:09 |
| *** zoli is now known as zoli|gone | 16:09 | |
| *** zoli|gone is now known as zoli | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 16:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 16:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 16:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 16:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 16:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 16:10 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Mock deployment action in upgrade tests https://review.openstack.org/586609 | 16:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 16:10 |
| *** amoralej is now known as amoralej|off | 16:11 | |
| *** links has joined #tripleo | 16:14 | |
| *** trown is now known as trown|lunch | 16:15 | |
| *** ade_lee has quit IRC | 16:16 | |
| *** links has quit IRC | 16:17 | |
| *** links has joined #tripleo | 16:17 | |
| *** artom_ has joined #tripleo | 16:22 | |
| *** yolanda has quit IRC | 16:22 | |
| *** links has quit IRC | 16:23 | |
| *** yolanda has joined #tripleo | 16:29 | |
| *** derekh has quit IRC | 16:30 | |
| jtomasek | jrist: I'll need to run a new setup next week, to get in touch with the containerized undercloud + quickstart quirks | 16:30 |
| jrist | jtomasek: there appear to be a few. | 16:31 |
| jrist | luckly quickstart + quickstart-extras is mostly working | 16:31 |
| *** mhenkel_ has joined #tripleo | 16:31 | |
| jrist | will need to do a few patches | 16:31 |
| jrist | jtomasek: I'll work with weshay next week when he's back from PTO | 16:31 |
| jtomasek | jrist: ok, cool | 16:31 |
| openstackgerrit | Jill Rouleau proposed openstack/ansible-role-openstack-operations master: Add missing license file https://review.openstack.org/585827 | 16:31 |
| *** tesseract has quit IRC | 16:32 | |
| jrist | jtomasek: no plans are showing up on my plans page by default (after quickstart) https://i.imgur.com/1fTk2Vi.png | 16:32 |
| jtomasek | jrist: what do you get when running openstack container list? | 16:33 |
| jrist | jtomasek: tons of stuff | 16:34 |
| jtomasek | jrist: anything in browser console? | 16:34 |
| jrist | oh wait | 16:34 |
| jrist | nothing | 16:34 |
| jrist | | tripleo-ui-logs | | 16:35 |
| jrist | that's the only thing when openstack container list, jtomasek | 16:35 |
| jtomasek | jrist: ok, so there are no deployment plans created, on undercloud install | 16:35 |
| jtomasek | that's deffinitely a bug | 16:35 |
| jrist | jtomasek: http://paste.openstack.org/show/726760/ | 16:35 |
| jrist | when I do `sudo docker ps` | 16:36 |
| jrist | jtomasek: what do you get on `openstack container list` | 16:36 |
| jrist | jtomasek: fyi honza already filed https://bugs.launchpad.net/tripleo/+bug/1782438 | 16:36 |
| openstack | Launchpad bug 1782438 in tripleo "tripleo ui endpoints misconfigured in containerized undercloud" [High,Triaged] | 16:36 |
| jtomasek | yep, I saw that | 16:36 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add pauch to the includepkgs https://review.openstack.org/579223 | 16:37 |
| *** zaneb has quit IRC | 16:38 | |
| *** zaneb has joined #tripleo | 16:39 | |
| *** jfrancoa has quit IRC | 16:40 | |
| *** holser_ has joined #tripleo | 16:40 | |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-openstack-operations master: Add missing license file https://review.openstack.org/585827 | 16:42 |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-openstack-operations master: Add missing license file https://review.openstack.org/585827 | 16:42 |
| *** mhenkel_ has quit IRC | 16:43 | |
| trozet | EmilienM, mwhahaha: did you guys see my email about including aarch64 undercloud/overcloud and containers in OOO? | 16:43 |
| mwhahaha | trozet: yea we don't handle the container availability, you'll probably want to work with weshay & company to figure out how to do that | 16:44 |
| mwhahaha | trozet: it's unlikely that we'll be able to include them in our promotion process (unless we have builders available in rdo) | 16:44 |
| trozet | mwhahaha: yeah i figured promotion would probably come later if we had the resources | 16:45 |
| trozet | mwhahaha: but we can periodically just build and upload | 16:45 |
| openstackgerrit | Merged openstack/ansible-role-openstack-operations master: Refactor docker_facts.py https://review.openstack.org/585500 | 16:47 |
| openstackgerrit | Merged openstack/ansible-role-openstack-operations master: Correct variable names in module and tasks file https://review.openstack.org/583724 | 16:48 |
| *** karthiks has quit IRC | 16:48 | |
| openstackgerrit | Andrew Austin proposed openstack/tripleo-heat-templates stable/queens: Mount my.cnf.d into the db_sync container for Barbican and Octavia. https://review.openstack.org/586621 | 17:00 |
| *** panda|rover is now known as panda|rover|off | 17:03 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add Browbeat enable settings to fs053 https://review.openstack.org/583578 | 17:04 |
| *** trown|lunch is now known as trown | 17:05 | |
| *** yamahata has quit IRC | 17:07 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 17:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 17:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 17:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 17:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 17:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 17:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 17:10 |
| *** dtantsur is now known as dtantsur|afk | 17:10 | |
| *** gbarros has joined #tripleo | 17:11 | |
| *** athomas has quit IRC | 17:12 | |
| *** dparkes has quit IRC | 17:16 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates master: Fix manila update, upgrade, and FFU tasks https://review.openstack.org/586623 | 17:19 |
| *** dprince has joined #tripleo | 17:22 | |
| *** gbarros has quit IRC | 17:23 | |
| *** artom has joined #tripleo | 17:23 | |
| *** khyr0n has joined #tripleo | 17:25 | |
| *** artom_ has quit IRC | 17:26 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add feature set file to run HA validations https://review.openstack.org/538952 | 17:30 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add feature set file to run HA validations https://review.openstack.org/538952 | 17:30 |
| *** mhenkel_ has joined #tripleo | 17:40 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add feature set file to run HA validations https://review.openstack.org/538952 | 17:42 |
| *** gbarros has joined #tripleo | 17:42 | |
| *** yamahata has joined #tripleo | 17:43 | |
| *** salmankhan has quit IRC | 17:44 | |
| *** mhenkel_ has quit IRC | 17:44 | |
| *** morazi has quit IRC | 17:47 | |
| *** dprince has quit IRC | 17:48 | |
| *** atoth has quit IRC | 17:54 | |
| *** atoth has joined #tripleo | 17:55 | |
| *** gbarros_ has joined #tripleo | 18:05 | |
| *** gbarros has quit IRC | 18:09 | |
| *** gbarros_ has quit IRC | 18:09 | |
| *** myoung is now known as myoung|lunch | 18:09 | |
| *** jtomasek has quit IRC | 18:10 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 18:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 18:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 18:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 18:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 18:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 18:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 18:10 |
| *** gbarros has joined #tripleo | 18:10 | |
| openstackgerrit | Merged openstack/ansible-role-openstack-operations master: Add missing license file https://review.openstack.org/585827 | 18:10 |
| *** gbarros_ has joined #tripleo | 18:12 | |
| *** gbarros__ has joined #tripleo | 18:13 | |
| *** gbarro___ has joined #tripleo | 18:14 | |
| *** gbarros has quit IRC | 18:15 | |
| *** gbarros has joined #tripleo | 18:15 | |
| *** gbarros_ has quit IRC | 18:16 | |
| *** gbarros__ has quit IRC | 18:18 | |
| *** gbarro___ has quit IRC | 18:18 | |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-quickstart master: Fix/enable the Tempest tests for Manila https://review.openstack.org/509554 | 18:18 |
| *** Haresh has joined #tripleo | 18:25 | |
| *** artom has quit IRC | 18:26 | |
| trown | EmilienM: is there a way that a mistral task can run with root privledges? trying to figure out how to make this containerized openshift-ansible patch work https://review.openstack.org/583868 | 18:32 |
| *** sshnaidm|off has quit IRC | 18:32 | |
| trown | EmilienM: or do we need to make that container run on the "overcloud" and run ansible against localhost to install openshift? | 18:33 |
| *** artom has joined #tripleo | 18:37 | |
| *** mhenkel_ has joined #tripleo | 18:47 | |
| *** Haresh has quit IRC | 18:47 | |
| *** alee has joined #tripleo | 18:50 | |
| *** gbarros has quit IRC | 18:55 | |
| *** v1a4 has joined #tripleo | 18:55 | |
| v1a4 | hello. | 18:56 |
| v1a4 | does TripleO for Queens deploy Overcloud only on containers? | 18:57 |
| v1a4 | or is there also the 'clasic' directly on baremetal type of deployment | 18:57 |
| mwhahaha | v1a4: In queens we only support containers | 18:57 |
| mwhahaha | the code is still there to some extent but we don't test it | 18:57 |
| mwhahaha | and there is no upgrade support | 18:58 |
| v1a4 | So no way to deploy it directly on baremetal like for older versions | 18:58 |
| mwhahaha | not officially | 18:58 |
| *** alee has quit IRC | 18:58 | |
| v1a4 | thank you for the quick answer | 18:59 |
| v1a4 | I am doing some deployment tests, and for some reason some of the services for compute are not deploying | 18:59 |
| v1a4 | and I figured that instead of debugging container, I could try the 'classic' way of deployment | 18:59 |
| mwhahaha | v1a4: which services/are there any specific errors? | 19:00 |
| *** alee has joined #tripleo | 19:00 | |
| *** alee has quit IRC | 19:00 | |
| v1a4 | well basically the deployment stops on .AllNodesDeploySteps.ComputeDeployment_Step3 | 19:00 |
| v1a4 | not giving any error | 19:01 |
| v1a4 | just hangs | 19:01 |
| mwhahaha | that's odd | 19:01 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: DNM - Testing adding properties to flavors https://review.openstack.org/579406 | 19:01 |
| v1a4 | and on the compute node | 19:01 |
| v1a4 | it's the neutron container that is not running | 19:01 |
| v1a4 | container logs does not give any errors | 19:02 |
| v1a4 | just a notice | 19:02 |
| v1a4 | Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend | 19:02 |
| *** ade_lee has joined #tripleo | 19:02 | |
| mwhahaha | that's not related | 19:02 |
| v1a4 | I figured | 19:02 |
| mwhahaha | you could check the os-collect-config logs | 19:03 |
| mwhahaha | they are in /var/log/messages | 19:03 |
| v1a4 | I did | 19:03 |
| v1a4 | no errors there | 19:03 |
| mwhahaha | what was the last thing running for os-collect-config? | 19:03 |
| v1a4 | they last entries there are "starting post-deployment steps" | 19:04 |
| mwhahaha | are there any docker-puppet.py processes running? | 19:04 |
| v1a4 | I guess there is where it hangs | 19:04 |
| *** ade_lee_ has joined #tripleo | 19:04 | |
| *** ade_lee has quit IRC | 19:04 | |
| *** ade_lee_ has quit IRC | 19:04 | |
| *** gbarros has joined #tripleo | 19:06 | |
| v1a4 | meantime I deleted the stack and started a re-run | 19:07 |
| *** gbarros has quit IRC | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 19:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 19:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 19:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 19:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 19:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 19:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 19:10 |
| openstackgerrit | Toure Dunnon proposed openstack/tripleo-common master: Improve output when registering nodes. https://review.openstack.org/569811 | 19:11 |
| *** ade_lee has joined #tripleo | 19:14 | |
| *** myoung|lunch is now known as myoung | 19:18 | |
| *** salmankhan has joined #tripleo | 19:18 | |
| *** jtomasek has joined #tripleo | 19:18 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: [DNM] Add local network-environment file for fs001 https://review.openstack.org/577230 | 19:18 |
| *** jtomasek has quit IRC | 19:19 | |
| ayoung | anyone have good a HOWTO for using virt-dib? | 19:22 |
| *** salmankhan has quit IRC | 19:23 | |
| ayoung | I want to "freshen" the RHEL image with latest RPMs | 19:23 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: make reproducer bash syntax more portable https://review.openstack.org/581012 | 19:24 |
| *** jtcressy has joined #tripleo | 19:34 | |
| jtcressy | mwhahaha, fultonj, gfidente, florianf: Thank you all! We did it! I got a successful deploy of my overcloud for the first time last night!!!!! :D | 19:36 |
| mwhahaha | hooray | 19:36 |
| fultonj | \o/ | 19:36 |
| jtcressy | Now I just have to test everything, which starts with launching an instance. | 19:37 |
| *** zshi has quit IRC | 19:37 | |
| jtcressy | now, do you guys know anything about images getting stuck in "state=queued"? | 19:37 |
| jtcressy | i'll probably pop over to #openstack since it's not tripleo specific. | 19:38 |
| *** dprince has joined #tripleo | 19:39 | |
| *** benone has joined #tripleo | 19:40 | |
| v1a4 | mwhahaha, deployment was hanging due to the use of the 'low-memory' template | 19:41 |
| v1a4 | now I got an error | 19:41 |
| mwhahaha | v1a4: not sure why that would cause things other than increasing the timeouts so it just takes longer to fail | 19:42 |
| mwhahaha | jtcressy: usually means the backend is bad | 19:42 |
| jtcressy | mwhahaha: trying to find glance/cinder/ceph logs on one of the overcloud controllers and i'm having no luck | 19:44 |
| jtcressy | their folders in /var/log are empty | 19:44 |
| mwhahaha | jtcressy: /var/log/containers/<service> | 19:44 |
| jtcressy | hmm, glance only has the api log and all of it is requests to /healthcheck | 19:45 |
| mwhahaha | jtcressy: it's HA so you need to likely check all the controllers | 19:45 |
| jtcressy | ah right | 19:46 |
| mwhahaha | jtcressy: but queued usually means it was accepted by glance but isn't getting saved in the backend. did you switch glance to use ceph or is it using swift? | 19:46 |
| jtcressy | I didnt change anything specifically, I just assumed the ceph templates would do that for me. | 19:47 |
| jtcressy | it created a glance pool at least P | 19:47 |
| *** zshi has joined #tripleo | 19:50 | |
| mwhahaha | so it's likely that glance can't write to ceph for whatever reason | 19:51 |
| mwhahaha | or the ceph pool is bad | 19:51 |
| jtcressy | I ran the validation to check ceph and it passed. | 19:51 |
| mwhahaha | you may want ot make sure the glance backend is set to ceph (or whatever teh correct backend is for ceph) | 19:52 |
| mwhahaha | i think by default it's swift | 19:52 |
| jtcressy | Is there a way I can check what backend it's using? | 19:52 |
| mwhahaha | docker exec -it glance_api cat /etc/glance/glance.conf | grep backend | 19:52 |
| mwhahaha | i think that'll work | 19:52 |
| mwhahaha | the config files are in /var/lib/config-data/puppet-generated/<container>/etc/glance (i think) | 19:53 |
| jtcressy | Cant find any lines that aren't commented | 19:54 |
| jtcressy | btw the docker exec version of that says no such file | 19:55 |
| *** rlandy is now known as rlandy|brb | 19:55 | |
| mwhahaha | might be glance-api.conf | 19:55 |
| *** dprince has quit IRC | 19:55 | |
| mwhahaha | anyway | 19:55 |
| mwhahaha | jtcressy: did you use environment/storage-environment.yaml? | 19:55 |
| jtcressy | not specifically | 19:56 |
| mwhahaha | maybe your problem | 19:56 |
| mwhahaha | jtcressy: https://github.com/openstack/tripleo-heat-templates/blob/8566458b6badd9a98f0908303e69256c8fb65a5c/environments/storage-environment.yaml#L14-L30 | 19:56 |
| mwhahaha | if you didn't specify those backends, they aren't properly configured | 19:56 |
| jtcressy | Ok, am I able to patch this in with `openstack stack update overcloud --exsisting -e $THT/environments/storage-environment.yaml` ? | 19:58 |
| jtcressy | minus my typo in 'existing' ofc | 19:58 |
| mwhahaha | or did you use ceph-ansible/ceph-ansible.yaml instead maybe? | 19:59 |
| mwhahaha | https://github.com/openstack/tripleo-heat-templates/blob/8566458b6badd9a98f0908303e69256c8fb65a5c/environments/ceph-ansible/ceph-ansible.yaml#L7-L13 | 19:59 |
| jtcressy | yes: $THT/environments/ceph-ansible/ceph-ansible.yaml | 19:59 |
| *** pchavva has quit IRC | 19:59 | |
| mwhahaha | ok so then you already have those ig uess | 19:59 |
| jtcressy | and, $THT/environments/ceph-ansible/ceph-rgw.yaml | 19:59 |
| v1a4 | mwhahaha, this time it failed because the controller could not pull the docker images from undercloud | 20:00 |
| jtcressy | I know there's inheritance in some of these templates so I thought that the ceph-ansible templates would've pulled in those settings. | 20:00 |
| v1a4 | (configured as local registry for docker) | 20:00 |
| mwhahaha | jtcressy: they should unless there was an ordering issues | 20:00 |
| *** ccamacho1 has joined #tripleo | 20:00 | |
| v1a4 | related to Network Isolation configuration not applied on controller node | 20:00 |
| jtcressy | mwhahaha: here's my initial deployment command again: https://hastebin.com/pobemoxira | 20:01 |
| *** ccamacho has quit IRC | 20:01 | |
| *** itlinux has joined #tripleo | 20:03 | |
| mwhahaha | yea should be ok | 20:03 |
| *** jcoufal has quit IRC | 20:05 | |
| *** morazi has joined #tripleo | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 20:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 20:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 20:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 20:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 20:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 20:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 20:10 |
| jtcressy | mwhahaha: I looked at the parameters in the plan and found that "GlanceBackend" is set to "rbd" as it should be. | 20:10 |
| mwhahaha | jtcressy: ok so yea you'll have to debug the glance configuration. in the past when i've seen it stuck, it's been a problem saving to the store | 20:12 |
| jtcressy | and I don't know how to do that..... I cant seem to find glance logs | 20:13 |
| mwhahaha | jtcressy: is there nothing in /var/log/containers/glance* on all the controllers? | 20:13 |
| jtcressy | in /var/log/containers/glance there's only the api.log and grepping for "fail" and "error" gets me nothing | 20:14 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Create a Timesync service declaration https://review.openstack.org/586679 | 20:15 |
| *** slaweq has joined #tripleo | 20:15 | |
| jtcressy | I think `registry.log` would show me something but I can't find it | 20:15 |
| mwhahaha | it may not be an error | 20:16 |
| *** rlandy|brb is now known as rlandy | 20:17 | |
| mwhahaha | there isn't a separate registry service anymore so i think it's just the api log now | 20:17 |
| jtcressy | literally EVERY line in api.log is "GET /healthcheck HTTP/1.0" | 20:17 |
| jtcressy | I just scrolled through all of it | 20:17 |
| mwhahaha | you could try turning GlanceDebug on | 20:18 |
| *** ansmith has quit IRC | 20:19 | |
| mwhahaha | but yea it's just api.log | 20:19 |
| jtcressy | mwhahaha: BINGO. I tail -f'd every controller node's glance api log and re-uploaded an image. Immediately got an error on one of them.... | 20:25 |
| mwhahaha | dat ha | 20:26 |
| jtcressy | mwhahaha: https://hastebin.com/sogotoyinu | 20:26 |
| itlinux | jtcressy: I saw you are trying to use a diff backend.. I tried that since default is swift. tried cinder, it looked ok and I checked the DB do that works using glance image-list glance image-create... | 20:26 |
| itlinux | however I was not able to spin up VMs. | 20:26 |
| fultonj | jtcressy: https://review.openstack.org/#/c/585911 | 20:27 |
| jtcressy | My backend should be RBD all the way | 20:27 |
| itlinux | looks like you need permissions.. | 20:27 |
| jtcressy | oh so because i disabled gnocchi it's having permission issues? | 20:28 |
| fultonj | not exactly | 20:28 |
| fultonj | jtcressy: i suspect your ceph openstack keyring is malformed | 20:28 |
| jtcressy | I cleaned my nodes before the most recent deployment though.. It should be a brand new ceph cluster. Anything malformed is the fault of the template configurations. not like I did it by hand. | 20:29 |
| fultonj | yes a bug malformed it | 20:29 |
| fultonj | did you pass disable-telemtry.yaml ? | 20:30 |
| jtcressy | yes | 20:30 |
| jtcressy | because gnocchi kept failing | 20:30 |
| fultonj | i broke it and https://review.openstack.org/#/c/585911 unbreaks it | 20:30 |
| jtcressy | actually, mwhahaha, gnocchi might work now that I fixed the overcloud deploy images | 20:30 |
| fultonj | and by "broke it" i mean some HOT misshandled the empty string and created a ceph keyring which was not valid | 20:31 |
| *** trown is now known as trown|outtypewww | 20:31 | |
| jtcressy | What would be a good way to remove disable-telemetry.yaml from my plan and update the stack? | 20:31 |
| jtcressy | i only know how to add templates to it :P | 20:32 |
| *** artom has quit IRC | 20:32 | |
| fultonj | jtcressy: look at your openstack keyring , can we confirm it's malformed? | 20:33 |
| jtcressy | how do I do that? | 20:33 |
| *** dparkes has joined #tripleo | 20:33 | |
| fultonj | as per https://bugs.launchpad.net/tripleo/+bug/1776987 | 20:33 |
| openstack | Launchpad bug 1776987 in tripleo "Wrong ceph auth client when disable telemetry" [High,In progress] - Assigned to Giulio Fidente (gfidente) | 20:33 |
| fultonj | ssh controller node | 20:33 |
| fultonj | cat /etc/ceph/ceph.client.openstack.keyring | 20:33 |
| fultonj | does it contain... | 20:33 |
| fultonj | what is descirbed in said bug? | 20:34 |
| fultonj | if so, then you're hitting bug/1776987 | 20:34 |
| * mwhahaha sighs | 20:34 | |
| mwhahaha | sceanio001 and 003 are broken due to unhealthy containers | 20:34 |
| * mwhahaha file sbugs | 20:34 | |
| jtcressy | fultonj: It seems to be different: https://hastebin.com/deteqovubi | 20:35 |
| fultonj | jtcressy: that's the same bug | 20:35 |
| fultonj | profile rbd pool= | 20:35 |
| fultonj | is not valid | 20:35 |
| fultonj | it should be... | 20:35 |
| fultonj | pool=foo | 20:35 |
| fultonj | where foo is some pool | 20:35 |
| fultonj | jtcressy: you can do a mnaul fix to prove it will work... | 20:36 |
| fultonj | replace it with this | 20:36 |
| fultonj | http://paste.openstack.org/show/726765/ | 20:36 |
| fultonj | jtcressy: see the difference? | 20:36 |
| fultonj | manual* | 20:37 |
| jtcressy | aha. So it should be set to images | 20:37 |
| fultonj | look at your example agin, it is set to images | 20:37 |
| fultonj | but then it set it to nothing | 20:37 |
| fultonj | that's the problem | 20:37 |
| fultonj | i didn't set it to images | 20:38 |
| fultonj | i chopped off: ", profile rbd pool=" | 20:38 |
| jtcressy | Oh so I just need to remove that last "profile rbd pool=" because it would've been set to metrics | 20:38 |
| fultonj | exactly | 20:38 |
| mwhahaha | panda|rover|off, weshay, EmilienM: https://bugs.launchpad.net/tripleo/+bug/1784072 and https://bugs.launchpad.net/tripleo/+bug/1784073 | 20:38 |
| openstack | Launchpad bug 1784072 in tripleo "scenario001 fails due to unhealthy collectd container" [Critical,Triaged] | 20:38 |
| openstack | Launchpad bug 1784073 in tripleo "scenario003 fails because many containers are unhealthy" [Critical,Triaged] | 20:38 |
| fultonj | jtcressy: you could do this | 20:39 |
| fultonj | curl http://paste.openstack.org/raw/726765 > $file | 20:39 |
| fultonj | on all nodes which use that file | 20:39 |
| fultonj | controllers, computes etc | 20:39 |
| *** slaweq has quit IRC | 20:40 | |
| fultonj | and you'd be fixed | 20:40 |
| fultonj | you can do that quickly to get glance working to verify you're on the right path | 20:40 |
| fultonj | cinder should then work too | 20:40 |
| jtcressy | sweet. I'll just do `ansible -i inventory all -b -m shell -a "curl http://paste.openstack.org/raw/726765 > /etc/ceph/ceph.client.openstack.keyring"` | 20:40 |
| *** v1a4 has quit IRC | 20:40 | |
| fultonj | ok | 20:41 |
| fultonj | so tripleo would set up the wrong thing for you until the bug is fixed so keep that in mind if you do a stack update or redeploy | 20:41 |
| fultonj | if you want gnochi, just don't include disable telemetry and it should be ok | 20:41 |
| jtcressy | I think after this i'll just try to enable telemetry again | 20:42 |
| jtcressy | is there a way to remove disable-telemetry.yaml from my current plan? | 20:42 |
| fultonj | when you deploy don't -e it | 20:42 |
| jtcressy | i'd prefer to update the stack instead of fully tearing it down though... | 20:42 |
| fultonj | it will do a stack update | 20:43 |
| fultonj | openstack overcloud deploy --templates -e feature.foo.yaml | 20:43 |
| fultonj | openstack overcloud deploy --templates -e feature.foo.yaml -e feature.bar.yaml | 20:43 |
| fultonj | so you can run the first command and deploy your overcloud | 20:43 |
| fultonj | thenyou can run the second command and add the bar feature to the same overcloud | 20:44 |
| fultonj | it should just update the stack | 20:44 |
| jtcressy | ok | 20:45 |
| fultonj | i haven't tested if simply doing a stack update an excluding the disable-telemetry enables telemetry TBH | 20:46 |
| fultonj | but the pattern i described works for other features | 20:46 |
| fultonj | anyway, that's why you had a ceph permissions issue | 20:46 |
| jtcressy | well, i'll 1. fix this current issue manually 2. test if I can spawn an instance and log into it etc. 3. update the stack by removing the disable-telemetry.yaml 4. hammer a square block (gnocchi) into a round hole (my deployment) and go from there. | 20:47 |
| fultonj | cool | 20:47 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Integrate Validate-HA work into OVB jobs https://review.openstack.org/537669 | 20:50 |
| *** mwhahaha changes topic to "Welcome to Rocky | CI Status: Generally RED. | https://docs.openstack.org/tripleo-docs/latest/" | 20:51 | |
| *** mwhahaha changes topic to "Welcome to Rocky | CI Status: Generally RED. - Scenario001/003 broken, OVB questionable | https://docs.openstack.org/tripleo-docs/latest/" | 20:51 | |
| jtcressy | fultonj: do I need to restart the glance/cinder services after fixing the keychain? | 20:52 |
| jtcressy | keyring* | 20:52 |
| fultonj | you shouldn't | 20:52 |
| jtcressy | ok, i'll give it another go... | 20:52 |
| jtcressy | fultonj: It works!! | 20:53 |
| fultonj | yay | 20:53 |
| *** mhenkel_ has quit IRC | 20:55 | |
| jtcressy | woah, holdup. It went from queued to saving back to queued | 20:55 |
| *** isq_ has joined #tripleo | 20:56 | |
| fultonj | if it's a qcow2 file i recommend converting it to raw before uploading | 20:56 |
| *** yamahata__ has quit IRC | 20:57 | |
| jtcressy | damn it! it still gives a permission error | 20:57 |
| fultonj | try creating a cinder volume | 20:57 |
| *** gkadam has quit IRC | 20:58 | |
| jtcressy | I don't know what to put for the host field | 20:58 |
| fultonj | http://docs.ceph.com/docs/master/rbd/rbd-openstack/ | 20:59 |
| fultonj | if you want to boot virtual machines in Ceph (ephemeral backend or boot from volume), the Glance image format must be RAW | 20:59 |
| *** ade_lee has quit IRC | 20:59 | |
| fultonj | source overcloudrc | 20:59 |
| fultonj | openstack volume create --size 1 test-volume | 20:59 |
| fultonj | openstack volume list | 21:00 |
| fultonj | to create the image to raw... | 21:00 |
| fultonj | https://github.com/fultonj/oooq/blob/master/over/sanity-check.sh#L56 | 21:00 |
| fultonj | s/create/convert | 21:00 |
| jtcressy | status is error on that volume | 21:00 |
| fultonj | might need to restart and ensure the key file was updated on the server that is hosting cinder (one of the 3 presumably) | 21:01 |
| fultonj | tail -f /var/log/cinder/volume.log | 21:02 |
| *** brault_ has quit IRC | 21:02 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates master: Fix manila update, upgrade, and FFU tasks https://review.openstack.org/586623 | 21:03 |
| jtcressy | I cat'd all of the files on the nodes and it was a malformed URL. running again and verifying... | 21:03 |
| *** ade_lee has joined #tripleo | 21:03 | |
| fultonj | oh | 21:04 |
| *** edmondsw has quit IRC | 21:04 | |
| *** edmondsw has joined #tripleo | 21:05 | |
| *** yamahata has quit IRC | 21:05 | |
| jtcressy | I've verified that the files are correct but image upload still fails with permission error. I guess i'll have to restart the services. | 21:05 |
| fultonj | did you convert the image? | 21:05 |
| jtcressy | no. Is there a raw image I can use to just test with then? | 21:07 |
| fultonj | 17:00 fultonj: https://github.com/fultonj/oooq/blob/master/over/sanity-check.sh#L56 | 21:07 |
| jtcressy | btw this is what the keyring looks like on all the hosts: https://hastebin.com/waceceyobu | 21:08 |
| *** suuuper has quit IRC | 21:08 | |
| fultonj | so download cirros and use qemu-img convert | 21:09 |
| fultonj | as per the example in that little script | 21:10 |
| *** edmondsw has quit IRC | 21:10 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783586 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783653 | 21:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784072 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784073 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 21:10 |
| openstack | Launchpad bug 1783586 in tripleo "CI: jobs fail with The Workflow errored and no messages were received." [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 21:10 |
| openstack | Launchpad bug 1783653 in tripleo "scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker) | 21:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 21:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 21:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 21:10 |
| openstack | Launchpad bug 1784072 in tripleo "scenario001 fails due to unhealthy collectd container" [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1784073 in tripleo "scenario003 fails because many containers are unhealthy" [Critical,Triaged] | 21:10 |
| mwhahaha | alright ci is massively hosed | 21:12 |
| fultonj | :( | 21:12 |
| fultonj | friday afternoon buzz kill...... | 21:12 |
| fultonj | sorry mwhahaha | 21:12 |
| *** yamahata has joined #tripleo | 21:18 | |
| jtcressy | fultonj: converted to raw and it's still not uploading | 21:19 |
| jtcressy | same permission error | 21:19 |
| mwhahaha | the key probably has to be in the container | 21:19 |
| jtcressy | also what's the name of the service I need to restart to restart glance | 21:19 |
| mwhahaha | so unless they are mounted into the container, you might needs to copy it | 21:19 |
| itlinux | hello guys, do we have single account for swift (shared) or ooo is configured -- keystone is config to use independent storage location? | 21:19 |
| mwhahaha | docker restart glance_api | 21:19 |
| fultonj | the container bind mounts the path | 21:20 |
| fultonj | +1 on restart to ensure it reads the path | 21:20 |
| fultonj | i wonder if selinux got up set at the curl | 21:22 |
| fultonj | as in it had a different context | 21:22 |
| jtcressy | fultonj: restarted all the glance containers. Nope. Still getting this: "PermissionError: [errno 1] error creating image" | 21:23 |
| mwhahaha | should be permissive (if centos) | 21:24 |
| jtcressy | all the controller nodes are selinux permissive | 21:25 |
| *** toure is now known as toure|gone | 21:26 | |
| jtcressy | If there's nothing else for me to try, i'm just going to try redeploying with telemetry enabled | 21:28 |
| fultonj | i guess i'd keep seeing if cinder had the same issue and while tailing the volume log and restarted various services reasserting the keys until whatever it was popped out | 21:30 |
| fultonj | it might be easier to just rebuild it and take measures to avoid that bug | 21:31 |
| jtcressy | yeah that's what i'm going to do. | 21:31 |
| fultonj | ok | 21:31 |
| jtcressy | first i'm going to try updating the stack and if that fails i'll tear the whole thing down again. | 21:32 |
| fultonj | if you're using diable-telemetry edit that file to remove the line that sets gnochirbdpool: '' | 21:32 |
| fultonj | else, try not using disable-telemetry | 21:32 |
| jtcressy | i'm removing disable-telemetry | 21:32 |
| fultonj | ok | 21:33 |
| *** rfolco|ruck is now known as rfolco|off | 21:33 | |
| jtcressy | The last time I tried telemetry, I was mistakenly using pike images during a queens install. | 21:33 |
| jtcressy | so it should work this time. | 21:33 |
| *** holser_ has quit IRC | 21:38 | |
| *** thrash is now known as thrash|g0ne | 21:42 | |
| mwhahaha | EmilienM, slagle: https://review.openstack.org/#/c/582429/ broke all the scenario jobs on stable queens | 21:45 |
| * mwhahaha is filing bug and revert | 21:45 | |
| *** salmankhan has joined #tripleo | 21:45 | |
| EmilienM | mwhahaha: we might need to run scenarios on this file then | 21:46 |
| mwhahaha | yes | 21:46 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/queens: Revert "Optimized Ansible tasks in deplay-steps-tasks.yaml" https://review.openstack.org/586714 | 21:48 |
| *** ansmith has joined #tripleo | 21:49 | |
| *** mhenkel_ has joined #tripleo | 21:50 | |
| *** marrusl has quit IRC | 21:52 | |
| *** salmankhan has quit IRC | 21:53 | |
| *** mhenkel_ has quit IRC | 21:54 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/queens: Add missing service for DockerCinderConfigImage https://review.openstack.org/586715 | 21:59 |
| *** sshnaidm|off has joined #tripleo | 22:04 | |
| *** itlinux has quit IRC | 22:05 | |
| *** rh-jelabarre has quit IRC | 22:07 | |
| *** ade_lee has quit IRC | 22:07 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 22:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784078 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 22:10 |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 22:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 22:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 22:10 |
| openstack | Launchpad bug 1784078 in tripleo "scenario jobs in stable/queens are broken with ansible file issues: could not locate file in lookup: Controller/step_config.pp" [Critical,Triaged] | 22:10 |
| *** itlinux has joined #tripleo | 22:10 | |
| *** figleaf is now known as edleafe | 22:15 | |
| *** rh-jelabarre has joined #tripleo | 22:15 | |
| *** agopi has quit IRC | 22:38 | |
| *** mcornea has quit IRC | 22:39 | |
| sl0w | EmilienM: so those patches I am working on take 2 minutes off the overcloud deploy time | 22:44 |
| sl0w | EmilienM: from 2m51s to upload the plan and start the deployment down to 46s | 22:45 |
| sl0w | this is on a vm | 22:46 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/queens: Mount my.cnf.d into the db_sync container for Barbican and Octavia. https://review.openstack.org/586621 | 22:48 |
| *** mschuppert has quit IRC | 23:03 | |
| jtcressy | Is there a reason the stack update tries to wipe the OSD disks? https://hastebin.com/epiludoboc | 23:08 |
| jtcressy | I smell data loss when I need to update my deployment later on down the road. | 23:09 |
| *** rpioso is now known as rpioso|afk | 23:09 | |
| jtcressy | look at line 55 specifically | 23:09 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783399 | 23:10 |
| openstack | Launchpad bug 1783399 in tripleo "containerized undercloud upgrade jobs isn't testing upgrades anymore" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783540 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783857 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1783866 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784015 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784078 | 23:10 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1783857 in tripleo "TripleO CI jobs false positives" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 23:10 |
| openstack | Launchpad bug 1783866 in tripleo "fs037 updates: Failed to update nodes - Controller" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 23:10 |
| openstack | Launchpad bug 1784015 in tripleo "ovb image build broken due to diskimage_builder.element_dependencies.MissingElementException: Element 'disable-nouveau' not found" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 23:10 |
| openstack | Launchpad bug 1784078 in tripleo "scenario jobs in stable/queens are broken with ansible file issues: could not locate file in lookup: Controller/step_config.pp" [Critical,Triaged] | 23:10 |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates master: Fix errors using multiple-nics templates w/o VLANs defined https://review.openstack.org/586721 | 23:10 |
| jtcressy | mwhahaha: can you look at my above paste when you get a chance | 23:12 |
| * mwhahaha defers to fultonj on ceph bits | 23:14 | |
| mwhahaha | no idea | 23:14 |
| mwhahaha | it shouldn't, so likely something else changed | 23:14 |
| * mwhahaha wanders off | 23:14 | |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates stable/queens: Fix errors using multiple-nics templates w/o VLANs defined https://review.openstack.org/586722 | 23:15 |
| *** myoung has quit IRC | 23:22 | |
| jtcressy | Well, it's getting late so i'm going to delete the current deployment and kick off a whole new one with my changes. | 23:29 |
| *** rh-jelabarre has quit IRC | 23:33 | |
| *** rlandy has quit IRC | 23:37 | |
| *** itlinux has quit IRC | 23:37 | |
| *** mjturek has quit IRC | 23:41 | |
| *** rwsu has quit IRC | 23:43 | |
| *** itlinux has joined #tripleo | 23:50 | |
| *** itlinux has quit IRC | 23:50 | |
| *** itlinux has joined #tripleo | 23:51 | |
| *** itlinux has quit IRC | 23:51 | |
| *** wolverineav has quit IRC | 23:53 | |
| *** wolverineav has joined #tripleo | 23:54 | |
| *** wolverineav has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
