| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO https://review.openstack.org/604293 | 00:00 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] ubuntu element - ignore missing openstack repos https://review.openstack.org/610430 | 00:02 |
| *** rlandy is now known as rlandy|bbl | 00:03 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 00:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 00:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 00:10 |
| pvc | hi anyone | 00:11 |
| *** Vorrtex has quit IRC | 00:12 | |
| *** agopi has quit IRC | 00:29 | |
| *** agopi has joined #tripleo | 00:30 | |
| *** mcornea has quit IRC | 00:31 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Add posibilities to set tunnel_csum in ovs agent https://review.openstack.org/609923 | 00:33 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Add posibilities to set tunnel_csum in ovs agent https://review.openstack.org/609935 | 00:36 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: [Documentation] Make featureset numbers links to the definition. https://review.openstack.org/482480 | 00:36 |
| *** ssbarnea has joined #tripleo | 00:48 | |
| *** ssbarnea|bkp2 has quit IRC | 00:48 | |
| *** agopi_ has joined #tripleo | 00:49 | |
| *** agopi has quit IRC | 00:50 | |
| *** lhinds has quit IRC | 00:51 | |
| *** lhinds has joined #tripleo | 00:55 | |
| openstackgerrit | Merged openstack/tripleo-common master: Set workflow status to success when manipulating deployment status https://review.openstack.org/607181 | 00:57 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Update deployment status when deploying or failed https://review.openstack.org/607182 | 00:57 |
| *** pvc has quit IRC | 01:01 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Convert with_dict tasks to use loop and be less chatty https://review.openstack.org/607920 | 01:02 |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 01:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 01:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 01:10 |
| *** slaweq has joined #tripleo | 01:11 | |
| *** khyr0n has joined #tripleo | 01:13 | |
| *** slaweq has quit IRC | 01:16 | |
| openstackgerrit | Merged openstack/tripleo-common master: Replace skopeo inspect with python https://review.openstack.org/609586 | 01:16 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: check mode: puppet host https://review.openstack.org/609022 | 01:16 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Use transport_url instead of rpc_backend https://review.openstack.org/610512 | 01:16 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Ensure the needed openshift resources are set https://review.openstack.org/610054 | 01:16 |
| openstackgerrit | Merged openstack/tripleo-common master: Add AllNodesConfig to config-download group vars https://review.openstack.org/605046 | 01:16 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Disable Swift auditors/replicators on undercloud https://review.openstack.org/610012 | 01:16 |
| *** thrash is now known as thrash|g0ne | 01:25 | |
| *** mrsoul has joined #tripleo | 01:26 | |
| *** mschuppert has quit IRC | 01:28 | |
| *** rlandy|bbl is now known as rlandy | 01:28 | |
| *** openstackgerrit has quit IRC | 01:35 | |
| *** pvc has joined #tripleo | 01:50 | |
| pvc | hi | 01:50 |
| *** nhicher has quit IRC | 01:54 | |
| *** phuongnh has joined #tripleo | 01:58 | |
| *** rh-jelabarre has joined #tripleo | 02:02 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 02:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 02:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 02:10 |
| *** rh-jelabarre has quit IRC | 02:15 | |
| *** pvc has quit IRC | 02:22 | |
| *** lblanchard has quit IRC | 02:23 | |
| *** openstackgerrit has joined #tripleo | 02:26 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds https://review.openstack.org/604268 | 02:26 |
| *** lblanchard has joined #tripleo | 02:26 | |
| *** lblanchard has quit IRC | 02:29 | |
| openstackgerrit | Filippo Inzaghi proposed openstack/instack-undercloud master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610792 | 02:49 |
| *** chem has quit IRC | 02:50 | |
| openstackgerrit | Filippo Inzaghi proposed openstack/instack master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610793 | 02:51 |
| openstackgerrit | Filippo Inzaghi proposed openstack/os-apply-config master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610794 | 02:51 |
| openstackgerrit | Filippo Inzaghi proposed openstack/os-collect-config master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610795 | 02:51 |
| *** huynq has joined #tripleo | 02:52 | |
| openstackgerrit | Filippo Inzaghi proposed openstack/os-net-config master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610796 | 02:52 |
| openstackgerrit | Filippo Inzaghi proposed openstack/os-refresh-config master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610797 | 02:52 |
| openstackgerrit | Filippo Inzaghi proposed openstack/paunch master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610798 | 02:53 |
| openstackgerrit | Filippo Inzaghi proposed openstack/python-tripleoclient master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610799 | 02:53 |
| openstackgerrit | Filippo Inzaghi proposed openstack/tripleo-common-tempest-plugin master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610800 | 02:54 |
| openstackgerrit | Filippo Inzaghi proposed openstack/tripleo-common master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610801 | 02:54 |
| openstackgerrit | Filippo Inzaghi proposed openstack/tripleo-image-elements master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610802 | 02:55 |
| openstackgerrit | Filippo Inzaghi proposed openstack/tripleo-puppet-elements master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610803 | 02:55 |
| openstackgerrit | Filippo Inzaghi proposed openstack/tripleo-repos master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610804 | 02:56 |
| openstackgerrit | Filippo Inzaghi proposed openstack/tripleo-validations master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610805 | 02:56 |
| *** ramishra has joined #tripleo | 03:02 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 03:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 03:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 03:10 |
| *** rlandy has quit IRC | 03:15 | |
| *** pvc has joined #tripleo | 03:16 | |
| pvc | is Emilien Macchi here? | 03:17 |
| openstackgerrit | zhongshengping proposed openstack/puppet-pacemaker master: Update min tox version to 2.0 https://review.openstack.org/610827 | 03:43 |
| openstackgerrit | zhongshengping proposed openstack/puppet-tripleo master: Update min tox version to 2.0 https://review.openstack.org/610845 | 03:43 |
| pvc | keystoneauth1.exceptions.discovery.DiscoveryFailure: Cannot use v2 authentication with domain scope | 04:06 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 04:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 04:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 04:10 |
| *** pvc has quit IRC | 04:10 | |
| *** pvc has joined #tripleo | 04:11 | |
| pvc | hi im trying to upgrade my undercloud | 04:11 |
| pvc | keystoneauth1.exceptions.discovery.DiscoveryFailure: Cannot use v2 authentication with domain scope error | 04:11 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Revert "iscsi: workaround to let podman read /dev/null" https://review.openstack.org/610633 | 04:16 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds https://review.openstack.org/604268 | 04:17 |
| *** agopi_ is now known as agopi | 04:18 | |
| pvc | agopi | 04:18 |
| pvc | hi | 04:18 |
| *** tzumainn has quit IRC | 04:19 | |
| *** khyr0n has quit IRC | 04:21 | |
| *** ykarel has joined #tripleo | 04:22 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add ANSIBLE0019 (rule id 102) to skip list -[102] No Jinja in when https://review.openstack.org/610472 | 04:33 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: NODES_FILE definition is missing https://review.openstack.org/609846 | 04:38 |
| *** Petersingh has joined #tripleo | 04:44 | |
| *** Petersingh is now known as Petersingh|afk | 04:44 | |
| *** hberaud has quit IRC | 04:51 | |
| Tengu | hello there | 05:02 |
| *** hberaud has joined #tripleo | 05:03 | |
| *** hberaud has quit IRC | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 05:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 05:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 05:10 |
| *** yprokule has joined #tripleo | 05:11 | |
| openstackgerrit | Dan Macpherson proposed openstack/ansible-role-openstack-operations master: Adding Backup and Restore Operations https://review.openstack.org/604439 | 05:21 |
| *** Petersingh|afk is now known as Petersingh | 05:23 | |
| *** iranzo has joined #tripleo | 05:27 | |
| *** ksambor has joined #tripleo | 05:30 | |
| *** ratailor has joined #tripleo | 05:30 | |
| *** agurenko has joined #tripleo | 05:37 | |
| *** jfrancoa has joined #tripleo | 05:38 | |
| *** apetrich has quit IRC | 05:40 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Fix ceph-nfs duplicate property https://review.openstack.org/609599 | 05:44 |
| *** janki has joined #tripleo | 05:47 | |
| *** paramite has joined #tripleo | 05:54 | |
| *** apetrich has joined #tripleo | 06:00 | |
| *** saneax has joined #tripleo | 06:02 | |
| *** aufi has joined #tripleo | 06:04 | |
| *** chkumar|off is now known as chandankumar | 06:05 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 06:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 06:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 06:10 |
| *** dalvarez has joined #tripleo | 06:10 | |
| chandankumar | Tengu: Good morning | 06:11 |
| chandankumar | Tengu: all tests passed but object storage failed after selinux fix | 06:11 |
| *** slaweq has joined #tripleo | 06:11 | |
| chandankumar | Tengu: http://logs.openstack.org/46/606046/12/check/tripleo-ci-centos-7-standalone/3fb541f/logs/tempest.html.gz | 06:11 |
| chandankumar | with Bad URL | 06:11 |
| chandankumar | I am 100% sure something glichy is happening around swift side | 06:12 |
| Tengu | chandankumar: hmm ok. lemme check that. | 06:16 |
| *** slaweq has quit IRC | 06:16 | |
| Tengu | chandankumar: any way to know a bit more about that "Bad URL" ? it's kind of... meaningless. | 06:16 |
| Tengu | chandankumar: I'm currently running tempest against a selinux-enabled standalone - no idea how long it will take. | 06:19 |
| openstackgerrit | Janki Chhatbar proposed openstack/puppet-tripleo stable/rocky: Add support for ODL-OVS IPv6 deployment https://review.openstack.org/610883 | 06:22 |
| *** apetrich has quit IRC | 06:23 | |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Add container images for openshift 3.11 https://review.openstack.org/610663 | 06:23 |
| *** noama has joined #tripleo | 06:24 | |
| *** mschuppert has joined #tripleo | 06:27 | |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart master: DNM: Testing OpenShift 3.11 https://review.openstack.org/610674 | 06:33 |
| *** rdopiera has joined #tripleo | 06:36 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Ensure the needed openshift resources are set https://review.openstack.org/610886 | 06:36 |
| *** iurygregory|away is now known as iurygregory | 06:36 | |
| *** aufi has quit IRC | 06:43 | |
| *** apetrich has joined #tripleo | 06:44 | |
| chandankumar | Tengu: it is a full tempest, then it is going to take more than a hour | 06:44 |
| chandankumar | Tengu: I am checking the swift logs | 06:44 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: DNM. Fix linters E102 Error https://review.openstack.org/610889 | 06:47 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-common stable/rocky: Fix nova-api healthcheck in case of nova metadata wsgi https://review.openstack.org/610890 | 06:48 |
| *** aufi has joined #tripleo | 06:48 | |
| Tengu | chandankumar: funky. tempest just crashed: CRITICAL vitrage [-] Unhandled error: AttributeError: 'TestSuite' object has no attribute 'id' guess I should blacklist some of the tests right? | 06:49 |
| chandankumar | Tengu: hmm | 06:49 |
| chandankumar | Tengu: you are running full tempest | 06:49 |
| Tengu | o_O no swift log in /var/log/containers/httpd | 06:50 |
| Tengu | funky. | 06:50 |
| chandankumar | Tengu: test_regex is tempest.api|tempest.scenario | 06:50 |
| chandankumar | Tengu: yes, it is coming from yesterday | 06:50 |
| Tengu | chandankumar: yeah, had to take the whitelist thingy. | 06:50 |
| chandankumar | http://logs.openstack.org/46/606046/12/check/tripleo-ci-centos-7-standalone/3fb541f/logs/undercloud/home/zuul/tempest/tempest.log.txt.gz | 06:50 |
| chandankumar | Tengu: Let me try this one https://review.openstack.org/#/c/610017/ | 06:52 |
| Tengu | chandankumar: will re-run the deploy here as well. | 06:54 |
| Tengu | with the patch you just pointed. | 06:54 |
| chandankumar | Tengu: https://review.openstack.org/#/c/609943/ testing here, it is super fast 1 hr 15 mins approx | 06:54 |
| therve | Tengu: swift doesn't run under httpd? | 06:54 |
| Tengu | therve: it should :/ | 06:55 |
| therve | Tengu: How so? | 06:55 |
| Tengu | chandankumar: well, as I'm building it from scratch, it will take ~2h30 in total (build+tempest) | 06:55 |
| chandankumar | therve: we are seeing this http://logs.openstack.org/46/606046/12/check/tripleo-ci-centos-7-standalone/3fb541f/logs/tempest.html.gz | 06:55 |
| chandankumar | on standalone jobs from yesterday | 06:55 |
| *** slaweq has joined #tripleo | 06:56 | |
| ramishra | chandankumar: it seems like genuine failures with 412 from swift http://logs.openstack.org/46/606046/12/check/tripleo-ci-centos-7-standalone/3fb541f/logs/undercloud/home/zuul/tempest/tempest.log.txt.gz#_2018-10-15_23_32_37_867, do those negative tests in tempest work? | 06:56 |
| ramishra | May be the behaviour has changed after https://github.com/openstack/swift/commit/cbfa585d3b48d2d8e2aedb9cedc62226dade8798 | 06:57 |
| chandankumar | ramishra: earlier swift tests were invoked only on overcloud, I am working on enabling them against puppet-swift | 06:58 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: Fix linters E102 Error https://review.openstack.org/610889 | 06:58 |
| ramishra | chandankumar: http://logs.openstack.org/46/606046/12/check/tripleo-ci-centos-7-standalone/3fb541f/logs/undercloud/var/log/swift/swift.log.txt.gz#_Oct_15_23_32_37 | 06:59 |
| chandankumar | ramishra: last friday it was working http://logs.openstack.org/43/609943/1/check/tripleo-puppet-ci-centos-7-standalone/a3cf485/logs/tempest.html.gz | 06:59 |
| chandankumar | on standalone | 06:59 |
| chandankumar | testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttestt HTTP/1.0 412 - - gAAAAABbxSOVkH2x... - 7 - tx012b515441ba4da4af1c8-005bc52395 - 0.0049 - - 1539646357.951894045 1539646357.956746101 - | 07:00 |
| ramishra | https://github.com/openstack/swift/commit/97f1c3be0057bab09de83b5be3ac23fabc4db9d5 | 07:00 |
| ramishra | this merged 5 days ago | 07:00 |
| *** rcernin has quit IRC | 07:01 | |
| chandankumar | my run was from 2018-10-12 | 07:01 |
| chandankumar | may be it merged after that | 07:01 |
| ramishra | chandankumar: May be it's not related to the patch, better to check if those negative tests still work | 07:03 |
| *** jtomasek has joined #tripleo | 07:05 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 07:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 07:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 07:10 |
| *** hberaud has joined #tripleo | 07:14 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: Remove artificial constrains around notification drivers https://review.openstack.org/606126 | 07:14 |
| *** jpena|off is now known as jpena | 07:20 | |
| *** quiquell|off is now known as quiquell | 07:24 | |
| *** amoralej|off is now known as amoralej | 07:28 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: ubuntu: Add option to ignore missing mirror components https://review.openstack.org/610430 | 07:28 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Set EPEL mirror during openstack-ci-mirrors https://review.openstack.org/609169 | 07:28 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Provide for an EPEL mirror during build https://review.openstack.org/608834 | 07:28 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add fedora28 standalone job definition https://review.openstack.org/610463 | 07:29 |
| therve | chandankumar: It does "UT /tempest-TestContainer-156486261", that's a non-sensical URL | 07:34 |
| therve | Not sure how it gets to that | 07:34 |
| therve | Maybe tempest is not configured correctly? | 07:34 |
| therve | Or swift | 07:34 |
| Tengu | therve: maybe the auto-conf of tempest? | 07:35 |
| therve | Maybe | 07:37 |
| therve | chandankumar: It's not a swift code change, at least, same in both run | 07:39 |
| therve | 2.19.1-0.20180928230948.16fe18a | 07:39 |
| *** bogdando has joined #tripleo | 07:40 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610491 | 07:43 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610491 | 07:43 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610491 | 07:43 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610491 | 07:44 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Corrected release config location for collection https://review.openstack.org/610038 | 07:45 |
| *** gfidente has joined #tripleo | 07:45 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610475 | 07:47 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Add release file for fedora 28 https://review.openstack.org/610468 | 07:47 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Enable use of python3 and dnf https://review.openstack.org/610018 | 07:47 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add fedora28 standalone job definition https://review.openstack.org/610463 | 07:47 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Add fedora28 support for quickstart https://review.openstack.org/591652 | 07:50 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610728 | 07:51 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Remove old config/release files https://review.openstack.org/606944 | 07:51 |
| *** skramaja has joined #tripleo | 07:51 | |
| *** tosky has joined #tripleo | 07:52 | |
| *** fhubik has joined #tripleo | 07:55 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Run wrappers non-privileged byt SYSADMIN cap added https://review.openstack.org/607139 | 07:55 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 08:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 08:00 |
| *** jpich has joined #tripleo | 08:00 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 08:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO https://review.openstack.org/604293 | 08:00 |
| *** dtantsur|afk is now known as dtantsur | 08:03 | |
| openstackgerrit | Yael Azulay proposed openstack/puppet-tripleo master: Changes are needed for performance and stability at large intervals https://review.openstack.org/603183 | 08:05 |
| *** Petersingh is now known as Petersingh|lunch | 08:06 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797525 | 08:10 |
| openstack | Launchpad bug 1797525 in tripleo "After the switch to podman & skopeo, undercloud deployment takes +60% longer" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 08:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 08:10 |
| noama | hi | 08:10 |
| noama | is anyone know of a work around https://bugs.launchpad.net/tripleo/+bug/1792343? | 08:11 |
| openstack | Launchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 08:11 |
| quiquell | noama: This is suppose to be the fix https://review.openstack.org/#/c/604455/ | 08:12 |
| quiquell | noama: Are you sure you have it ? | 08:12 |
| noama | yes | 08:12 |
| quiquell | Maybe we have to change the numbers a little | 08:12 |
| noama | i have chcek /usr/lib/python2.7/site-packages/tripleo_common/actions/ansible.py | 08:13 |
| noama | the 2 line are there | 08:13 |
| noama | i now set both to 15 instead of 5 don't know if it will help | 08:14 |
| *** dmacpher_ has joined #tripleo | 08:23 | |
| *** gchamoul has quit IRC | 08:24 | |
| *** cylopez has joined #tripleo | 08:25 | |
| *** dmacpher has quit IRC | 08:27 | |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-heat-templates master: Add posibilities to set default timeouts for octavia https://review.openstack.org/609431 | 08:28 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Use --limit with ceph-ansible to cope with blacklisted nodes https://review.openstack.org/610736 | 08:28 |
| Tengu | hmm, is there some option in order to get tempest HTML output?= | 08:30 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Add fedora28 support for quickstart https://review.openstack.org/591652 | 08:30 |
| tosky | Tengu: tempest does not output HTML; the test runner part, which is a tiny wrapper around stestr these days, output a subunit stream, which can then be converted to HTML | 08:31 |
| Tengu | tosky: care to provide a link for some doc? | 08:31 |
| *** gchamoul has joined #tripleo | 08:31 | |
| tosky | Tengu: to what exactly? | 08:32 |
| Tengu | reading tempest log is a "nope". | 08:32 |
| *** cylopez has quit IRC | 08:32 | |
| Tengu | tosky: how to get a human readable thing out of tempest | 08:32 |
| tosky | subunit2html is the command, but the jobs that I've seen around calls it at some point in their playbooks/roles | 08:32 |
| tosky | I was trying to point out that the HTML version is not done by tempest but by the jobs | 08:33 |
| Tengu | hmm ok. | 08:33 |
| tosky | and now I'm trying to understand whether you don't see the generated HTML or you are running tempest locally and you want to get the HTML output | 08:33 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Enable use of python3 and dnf https://review.openstack.org/610018 | 08:33 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add fedora28 standalone job definition https://review.openstack.org/610463 | 08:33 |
| Tengu | tosky: running locally on my test VM, and tryin to get something usable out of tempest in some way :) | 08:33 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Fix wrapper containers for podman w/o sockets https://review.openstack.org/606095 | 08:35 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Run wrappers non-privileged byt SYSADMIN cap added https://review.openstack.org/607139 | 08:35 |
| bogdando | lets merge https://review.openstack.org/#/c/610017/ folks | 08:38 |
| *** gkadam has joined #tripleo | 08:38 | |
| tosky | Tengu: this is in the tripleo documentation, but it should really live (hopefully soon) in the tempest documentation: https://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/tempest.html#generating-html-report-of-tempest-tests | 08:38 |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates master: [WIP] logging test DO NOT MERGED https://review.openstack.org/594836 | 08:38 |
| Tengu | tosky: ah, I think I get it: I have to add --subunit to the command, and.. | 08:38 |
| *** gkadam has quit IRC | 08:38 | |
| Tengu | tosky: yeah, found that one via duck :) | 08:38 |
| Tengu | so will modify the way I launch tempest, and hop. | 08:39 |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates stable/rocky: [WIP] logging test DO NOT MERGED https://review.openstack.org/598128 | 08:41 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add fedora28 standalone job definition https://review.openstack.org/610463 | 08:41 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Enable use of python3 and dnf https://review.openstack.org/610018 | 08:42 |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates stable/queens: [WIP] logging test DO NOT MERGED https://review.openstack.org/594838 | 08:42 |
| *** skramaja has quit IRC | 08:44 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates stable/pike: [WIP] logging test DO NOT MERGED https://review.openstack.org/594840 | 08:44 |
| *** skramaja has joined #tripleo | 08:44 | |
| chandankumar | Tengu: therve ramishra https://review.openstack.org/#/c/610017/ fixed the swift test failure | 08:46 |
| *** akrivoka has joined #tripleo | 08:46 | |
| chandankumar | Tengu: therve ramishra http://logs.openstack.org/43/609943/4/check/puppet-swift-tripleo-standalone/350c0a2/logs/tempest.html.gz | 08:46 |
| *** holser_ has joined #tripleo | 08:47 | |
| therve | Hum | 08:47 |
| therve | chandankumar: Do you get why? | 08:47 |
| chandankumar | therve: no idea, but it worked | 08:48 |
| chandankumar | therve: https://bugs.launchpad.net/tripleo/+bug/1797167 | 08:48 |
| openstack | Launchpad bug 1797167 in tripleo "swift timeout during undercloud deploy" [Critical,Fix released] - Assigned to Bogdan Dobrelya (bogdando) | 08:48 |
| chandankumar | may be timed out issue leading to Bad url | 08:49 |
| *** ykarel is now known as ykarel|mtg | 08:51 | |
| therve | No that sounds like a different issue | 08:52 |
| therve | Test failures were too reliable | 08:52 |
| ramishra | chandankumar: Isn't that one running only swift tests? | 08:52 |
| *** derekh has joined #tripleo | 08:52 | |
| chandankumar | ramishra: yes it is running only swift tests | 08:52 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Enable use of python3 and dnf https://review.openstack.org/610018 | 08:53 |
| ramishra | therve: yeah, my guess is the issue happens when you run other tests | 08:53 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/rocky: Disable Swift auditors/replicators on undercloud https://review.openstack.org/610909 | 08:53 |
| ramishra | along with the swift tests | 08:53 |
| chandankumar | ramishra: therve https://review.openstack.org/609943 only swift tests runned here | 08:53 |
| therve | ramishra: So you mean it's not fixed? | 08:55 |
| ramishra | therve: yeah | 08:56 |
| therve | That makes more sense to me :) | 08:59 |
| *** Petersingh|lunch is now known as Petersingh | 08:59 | |
| *** chem has joined #tripleo | 08:59 | |
| Tengu | chandankumar: ah, good news then :). Now I can concentrate on the selinux part with podman. | 09:00 |
| *** iurygregory has quit IRC | 09:01 | |
| openstackgerrit | Natal Ngétal proposed openstack/python-tripleoclient stable/pike: [Core] Fix the default namespace. https://review.openstack.org/610912 | 09:01 |
| *** ykarel|mtg is now known as ykarel | 09:03 | |
| noama | quiquell issue still reproduce with setting 5->15 | 09:04 |
| *** iurygregory has joined #tripleo | 09:05 | |
| quiquell | marios|rover: ^ do yoiu know more about it ? | 09:05 |
| marios | quiquell: is this the testenv thing or something else ? reading back | 09:10 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 09:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 09:10 |
| *** salmankhan has joined #tripleo | 09:10 | |
| noama | marios: https://bugs.launchpad.net/tripleo/+bug/1792343 | 09:11 |
| openstack | Launchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 09:11 |
| quiquell | marios: it's a recurrent issue | 09:11 |
| noama | marios|quiquell: i also tried to set 5->15 | 09:11 |
| quiquell | slagle: knows more about it | 09:12 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add fedora28 standalone job definition https://review.openstack.org/610463 | 09:12 |
| Tengu | yay... 83 tempest failures, most probably due to selinux. woohooo | 09:13 |
| Tengu | a rope, anyone, please? :S | 09:13 |
| marios|rover | noama: ack yeah i am not familiar with this bug or the fix, looks like slagle posted it at https://review.openstack.org/#/c/604455/ and looking at comment #18 it looks like tripleo-common you have isn't new enough. maybe its a repo issue (it merged into rocky 3 weeks ago so should be available ) | 09:14 |
| chandankumar | Tengu: https://review.openstack.org/#/c/610600/ -> selinux issue and swift https://review.openstack.org/#/c/610017/ | 09:14 |
| chandankumar | Tengu: then you have full green party :-) | 09:14 |
| Tengu | chandankumar: nope. | 09:14 |
| marios|rover | noama: i'd say add comment on the bug about what you're seeing (I note ben also commented 5th october, so might be we missed something somewhere) | 09:14 |
| Tengu | chandankumar: we can't ask ppl to deactivate selinux on their prod system. | 09:14 |
| Tengu | chandankumar: on the CI, well, ok, although I'm not a fan of deactivating selinux on it either, but ok. | 09:15 |
| chandankumar | Tengu: Earlier selinux was not configured in standalone last night mwhahaha proposed it https://review.openstack.org/#/c/610600/ | 09:15 |
| *** panda|off has quit IRC | 09:15 | |
| noama | marios|rover: i think network config failed | 09:15 |
| Tengu | chandankumar: this has nothing to do with the goal - we do NOT want permissive selinux on customer hardware. so if standalone doesn't pass with selinux+podman, an overcloud deploy won't pass either. | 09:16 |
| Tengu | chandankumar: I've already worked in order to get podman deployed for both undercloud and standalone - we can't deactivate selinux separation like we can do with docker. that's the big difference. | 09:17 |
| chandankumar | Tengu: Ah it is related to podman, I was thinking about standalone | 09:17 |
| Tengu | chandankumar: more over, I have issues with compute ^^'. That's a nasty one. | 09:17 |
| Tengu | chandankumar: yeah. I'm using standalone in order to get a better scope for podman support ;). | 09:18 |
| *** fhubik is now known as fhubik|brb | 09:18 | |
| *** panda has joined #tripleo | 09:18 | |
| *** iurygregory has quit IRC | 09:18 | |
| *** iurygregory has joined #tripleo | 09:20 | |
| *** hberaud has quit IRC | 09:20 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart-extras master: WIP: Adapt reproducer to new workflow https://review.openstack.org/606879 | 09:21 |
| *** hberaud has joined #tripleo | 09:24 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Replace bootstrap_nodeid with SERVICE_short_bootstrap_node_name https://review.openstack.org/605728 | 09:24 |
| chem | marios: hey, I cannot find the lint error there http://logs.openstack.org/36/604736/15/gate/openstack-tox-linters/2783d7c/job-output.txt.gz, and I don't want to blindly recheck https://review.openstack.org/#/c/604736/ | 09:25 |
| chem | marios: like the lint was passing on the 11th and not passing on the 15th | 09:26 |
| marios|rover | chem: yeah recheck if it was yesterday there was a bug | 09:26 |
| marios|rover | chem: sec | 09:26 |
| marios|rover | https://bugs.launchpad.net/tripleo/+bug/1797838 https://review.openstack.org/#/c/610472/ openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check | 09:26 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 09:26 |
| chem | marios|rover: thanks | 09:26 |
| marios|rover | chem: the skip merged this morning so recheck should do it. ssbarnea is doing the more proper fix with *=* 10:47:55 *=*=*= "Repair ansible when calls with jinja2 " https://review.openstack.org/#/c/610501/3 I6eb5719841c2f264e1efed4bf38726f71c840e36 tripleo-quickstart-extras | 09:26 |
| marios|rover | chem: (and re-enabling it) | 09:27 |
| marios|rover | chem: thanks for asking :) | 09:27 |
| chem | marios|rover: that ansible jinja parsing always baffles me | 09:28 |
| chem | marios|rover: sometime it works as expected sometime it doesn't because of some obscure rule | 09:29 |
| marios|rover | chem: yeah /me shrug... plus it was fun to even track that rule down because it says '102' but the skip id is something else :) https://review.openstack.org/#/c/610472/3/ci-scripts/ansible-lint.sh | 09:30 |
| marios|rover | https://github.com/willthames/ansible-lint/blob/a5828ff235d83624bcb080af2a5b199a3f4b257f/lib/ansiblelint/rules/NoFormattingInWhenRule.py#L11 | 09:30 |
| marios|rover | chem: here fwiw ^ | 09:30 |
| marios|rover | chem: oh i see how that works ... multiple tasks will be tagged with 0019 but where is _that_ defined | 09:31 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: DNM: To test new zuul_changes https://review.openstack.org/610917 | 09:31 |
| marios|rover | chem: ok ignore the last rant that part seems legit :) | 09:31 |
| *** sshnaidm|afk has joined #tripleo | 09:32 | |
| marios|rover | chem: hm only in that task it seems. rant partly re-instated then | 09:32 |
| marios|rover | aaaanyway ... :) | 09:32 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-docs master: Add diagram to fast forward upgrade dev docs https://review.openstack.org/610511 | 09:33 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-docs master: Fix wording in major upgrade dev docs https://review.openstack.org/610467 | 09:33 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610475 | 09:33 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates stable/rocky: DNM: To test zuul_changes https://review.openstack.org/610918 | 09:33 |
| chem | marios|rover: oki, so bottom line is no "{{" "}}" in when statement for ansible, let's just learn it by heart and forget about it :) | 09:34 |
| marios|rover | chem: ;) well ssbarnea is re-adding the rule so you don't even need to forget about it ! | 09:34 |
| marios|rover | bonus | 09:34 |
| chem | marios: rigth :) | 09:34 |
| chem | marios|rover: sshnaidm|afk hum, when you get a chance to look at that review https://review.openstack.org/#/c/606996/ it's needed for the standalone upgrade business | 09:35 |
| chem | marios|rover: sshnaidm|afk not a easy read though ... | 09:35 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610475 | 09:35 |
| marios|rover | chem: ack, its at the top of th enext review queue | 09:35 |
| chem | marios: ~350 lines of test code for 70 line of code | 09:36 |
| marios|rover | fantastic | 09:36 |
| * marios|rover grabs vodka | 09:37 | |
| ssbarnea | chem: that is the reason why the original hack was not closing the bug, it was only a temporary measure to fix the gate (took unexpectedly long time to get it merged) | 09:37 |
| *** shardy has joined #tripleo | 09:37 | |
| *** hberaud has quit IRC | 09:37 | |
| shardy | mandre: Hey did you see my comment on https://review.openstack.org/#/c/583868/ re the Depends-On? | 09:37 |
| huynq | Hi, Have you guys got this bug when install tripleO by quickstart ? (https://bugs.launchpad.net/tripleo/+bug/1714609) | 09:38 |
| openstack | Launchpad bug 1717550 in tripleo "duplicate for #1714609 Repositories have unresolved dependencies in centos 7.4" [Critical,Fix released] - Assigned to Lars Kellogg-Stedman (larsks) | 09:38 |
| shardy | Oh nvm I see you abandoned it :) | 09:38 |
| *** hberaud has joined #tripleo | 09:38 | |
| chem | ssbarnea: anyway, we won't forget about anymore, which is good :) | 09:39 |
| *** hberaud has quit IRC | 09:39 | |
| *** hberaud has joined #tripleo | 09:39 | |
| ssbarnea | chem: we still need a +W on it. and indeed a fullday of broken gate due to it makes it easier to remember :D | 09:40 |
| sshnaidm|afk | chem, will look | 09:41 |
| quiquell | marios|rover: Have you see issues like this aroiund ? http://logs.openstack.org/10/609310/1/check/tripleo-puppet-ci-centos-7-standalone/c5dcdfa/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-10-16_08_08_23 | 09:41 |
| marios|rover | quiquell: no not seen that one | 09:43 |
| quiquell | marios|rover: ack | 09:43 |
| quiquell | marios|rover: I would be rich is someone gave me one coin per 'recheck' | 09:44 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Add release file for fedora 28 https://review.openstack.org/610468 | 09:44 |
| therve | quiquell: Patch doing that merged yesterday | 09:46 |
| quiquell | therve: what patch ? | 09:48 |
| therve | quiquell: That created the traceback you linked | 09:48 |
| quiquell | therve: So it's known ? | 09:48 |
| therve | quiquell: I don't think so :) | 09:49 |
| *** hberaud has quit IRC | 09:50 | |
| therve | I wonder if it has something to do with registry-1 URLs | 09:51 |
| *** nawar has joined #tripleo | 09:51 | |
| nawar | hi | 09:52 |
| *** fhubik|brb has quit IRC | 09:52 | |
| quiquell | therve: I see it passing at other reviews | 09:52 |
| nawar | I'm trying to deploy queens using containerized undercloud, every step before deployment done with success but when the deployment start the nodes failed to pxe boot , but when I introspect the nodes they boot successfully | 09:52 |
| mandre | shardy: hey! yeah, thanks a lot, i managed to cause zuul to -2 my patch just because I submitted the backport, that was pretty silly of me | 09:53 |
| *** hberaud has joined #tripleo | 09:53 | |
| nawar | I checked logs from dnsmasq i got DHCPDISCOVER(br-ctlplane) 1c:98:ec:1b:9c:2c ignored | 09:54 |
| *** saneax has quit IRC | 09:54 | |
| *** saneax has joined #tripleo | 09:56 | |
| yolanda | hi, are there still issues with upstream tests? i have a pair of tests for images, that are not passing RDO third party ci | 09:56 |
| *** ykarel has quit IRC | 09:59 | |
| *** sshnaidm|afk is now known as sshnaidm | 10:03 | |
| marios|rover | d0ugal: bogdando anything further here? see comment #37 at https://bugs.launchpad.net/tripleo/+bug/1789680 or can we say fix release | 10:04 |
| openstack | Launchpad bug 1789680 in tripleo "mistral MessagingTimeout correlates with containerized undercloud uptime" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal) | 10:04 |
| *** morazi has quit IRC | 10:04 | |
| *** ykarel has joined #tripleo | 10:08 | |
| *** ykarel is now known as ykarel|lunch | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 10:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 10:10 |
| *** pfo has joined #tripleo | 10:14 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: WIP: Run reproducer at zuulv3 workflow https://review.openstack.org/606881 | 10:15 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Handle LP openvswitch meta-package on upgrade https://review.openstack.org/605200 | 10:15 |
| pfo | is there a good way how to separate tripleO and the secrets stored in the environment files? | 10:16 |
| *** Petersingh is now known as Petersingh|afk | 10:37 | |
| chandankumar | quiquell: bogdando Hello | 10:44 |
| chandankumar | quiquell: bogdando since in puppet-openstack-integration repo all puppet-projects are used | 10:44 |
| chandankumar | quiquell: bogdando there we want to run full tempest? | 10:45 |
| chandankumar | full tempest == tempest.api + scenario | 10:45 |
| chandankumar | no plugins | 10:45 |
| *** ykarel_ has joined #tripleo | 10:48 | |
| *** phuongnh has quit IRC | 10:50 | |
| *** ykarel|lunch has quit IRC | 10:50 | |
| bogdando | chandankumar: I think so | 10:56 |
| *** ykarel_ is now known as ykarel | 10:56 | |
| chandankumar | bogdando: ok, adding | 10:57 |
| chandankumar | bogdando: is 2hrs ok na? | 10:57 |
| chandankumar | i mean the job timing will go to approx 2 to 2 hrs 30 mins | 10:57 |
| mschuppert | hi, could I please get reviews on cherry-pick https://review.openstack.org/#/c/608455/ | 10:58 |
| bogdando | I don't think that's ok, but that has nothing to tempest | 10:58 |
| bogdando | it has to composability we are missing for standalone jobs | 10:58 |
| bogdando | I started thread for that, maybe we'll have time to look into that | 10:59 |
| chandankumar | bogdando: let me fire the review and see the timing then we can trim the testsing | 10:59 |
| bogdando | http://lists.openstack.org/pipermail/openstack-dev/2018-October/135644.html | 10:59 |
| bogdando | chandankumar: ack | 10:59 |
| * chandankumar checks | 10:59 | |
| Tengu | hmm. is there anyone from the security team? As apparently jaosorior is on PTO | 11:00 |
| *** sshnaidm has quit IRC | 11:00 | |
| *** sshnaidm has joined #tripleo | 11:01 | |
| *** quiquell is now known as quiquell|lunch | 11:01 | |
| chandankumar | Tengu: may be ade_lee can help | 11:01 |
| Tengu | bogdando: would it be possible to rebase https://review.openstack.org/#/c/606095/ on https://review.openstack.org/#/c/610017/ ? or the contrary - just in order to be able to fetch both for testing.. | 11:02 |
| *** shardy has quit IRC | 11:02 | |
| Tengu | oh. sho. they are in 2 repositories. sorry | 11:03 |
| *** boazel has joined #tripleo | 11:03 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-common stable/pike: [Pike Only] Do not use namespace var name in docker img template. https://review.openstack.org/608647 | 11:03 |
| bogdando | Tengu: I don't think so I want those kept independent | 11:05 |
| Tengu | bogdando: ;_; | 11:05 |
| bogdando | Tengu: you can update zuul patches manually | 11:05 |
| bogdando | for the repro script | 11:05 |
| bogdando | or create a DNM patch that depends on either | 11:05 |
| Tengu | not using the repro script... for now I'm able to fetch a change from gerrit, but pulling 2 will be a bit harzardous. | 11:06 |
| bogdando | Tengu: you can take https://review.openstack.org/#/c/609947/ as a base, to add more things | 11:07 |
| bogdando | but please not there :) | 11:07 |
| *** Petersingh|afk is now known as Petersingh | 11:08 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 11:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 11:10 |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 11:10 |
| *** nawar has quit IRC | 11:11 | |
| Tengu | bogdando: will just fetch all locally, and cherry-pick/rebase locally - my tripleo-lab supports rsync from the ansible controller. easier I guess :] | 11:13 |
| Tengu | but first: tea. | 11:14 |
| *** ansmith has quit IRC | 11:16 | |
| chandankumar | bogdando: quiquell|lunch All patches updated for standalone | 11:19 |
| quiquell|lunch | chandankumar: You mean added tempest in all of them ? | 11:21 |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: Removes EXTRA_TAGS from toci_gate_test&quickstart.sh.j2 https://review.openstack.org/604768 | 11:22 |
| openstackgerrit | Pranali Deore proposed openstack/tripleo-heat-templates master: Move metadata file creation for netapp to puppet-tripelo from THT https://review.openstack.org/588132 | 11:22 |
| *** nawar has joined #tripleo | 11:26 | |
| *** jpena is now known as jpena|lunch | 11:29 | |
| openstackgerrit | Pranali Deore proposed openstack/tripleo-heat-templates master: Move metadata file creation for netapp to puppet-tripelo from THT https://review.openstack.org/588132 | 11:29 |
| *** amoralej is now known as amoralej|lunch | 11:35 | |
| *** morazi has joined #tripleo | 11:36 | |
| *** quiquell|lunch is now known as quiquell | 11:37 | |
| *** sshnaidm has quit IRC | 11:38 | |
| *** sshnaidm has joined #tripleo | 11:38 | |
| *** abishop has joined #tripleo | 11:38 | |
| *** jistr is now known as jistr|afk | 11:47 | |
| *** huynq has quit IRC | 11:52 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Add release file for fedora 28 https://review.openstack.org/610468 | 11:58 |
| *** nhicher has joined #tripleo | 11:58 | |
| openstackgerrit | Kamil Sambor proposed openstack/puppet-tripleo stable/pike: Restart ovs agent to restore rules https://review.openstack.org/610936 | 11:58 |
| *** ratailor has quit IRC | 12:00 | |
| *** trown|outtypewww is now known as trown | 12:00 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Corrected release config location for collection https://review.openstack.org/610038 | 12:01 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Remove obsoleted code https://review.openstack.org/610937 | 12:01 |
| *** rh-jelabarre has joined #tripleo | 12:03 | |
| *** shyamb has joined #tripleo | 12:09 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 12:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 12:10 |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 12:10 |
| *** nawar has left #tripleo | 12:10 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: DNM: OVN test https://review.openstack.org/593056 | 12:10 |
| *** quiquell is now known as quiquell|off | 12:11 | |
| *** thrash|g0ne is now known as thrash | 12:13 | |
| *** dtrainor has joined #tripleo | 12:14 | |
| *** Petersingh is now known as Petersingh|afk | 12:16 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-common stable/pike: [Pike Only] Do not use namespace var name in docker img template. https://review.openstack.org/608647 | 12:16 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: DNM: OVN test https://review.openstack.org/593056 | 12:19 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Corrected release config location for collection https://review.openstack.org/610038 | 12:20 |
| *** bfournie has quit IRC | 12:20 | |
| *** ade_lee has quit IRC | 12:21 | |
| *** Petersingh|afk is now known as Petersingh | 12:22 | |
| bogdando | beagles: hi. Do you know how can I add OVN CI job there https://review.openstack.org/#/c/609947/ ? | 12:22 |
| bogdando | it's not getting triggered | 12:22 |
| beagles | bogdando, will it trigger if you just touch one of the ovn service files - like add a dummy comment or something? | 12:27 |
| bogdando | beagles: will try, thanks | 12:32 |
| trown | mandre: wrt 3.11 patches, I havent looked into what is happening in CI yet, but locally I hit the same SSL issues | 12:33 |
| trown | mandre: do you know if there is an option we can pass deploy to allow turning off SSL? | 12:34 |
| *** raildo has joined #tripleo | 12:35 | |
| *** rlandy has joined #tripleo | 12:35 | |
| *** sshnaidm has quit IRC | 12:36 | |
| *** sshnaidm has joined #tripleo | 12:39 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Invoke redhat-subscription role during upgrade. https://review.openstack.org/610493 | 12:40 |
| *** mcornea has joined #tripleo | 12:41 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-upgrade master: Set container_cli for undercloud https://review.openstack.org/608462 | 12:42 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Switch the undercloud to deploy Podman by default starting from Stein https://review.openstack.org/608452 | 12:44 |
| mandre | trown: which ssl issue? | 12:45 |
| *** dmacpher__ has joined #tripleo | 12:46 | |
| trown | mandre: I am not positive, but I think that is the issue I have with 3.11 control plane not coming up... there are messages in the logs where the cluster is not coming up because we get a connection refused | 12:47 |
| trown | mandre: /var/log/containers/master-controllers-subnode-1_kube-system_controllers-eb505465b710541b13f5e8008906356f5ebb45c3ec36e6327c468ed676aa1350.log:{"log":"E1016 02:19:12.618880 1 leaderelection.go:224] error retrieving resource lock kube-system/openshift-master-controllers: Get https://subnode-1:8443/api/v1/namespaces/kube-system/configmaps/openshift-master-controllers: dial tcp 192.168.24.3:8443: getsockopt: | 12:47 |
| trown | connection refused\n","stream":"stderr","time":"2018-10-16T02:19:12.619002537Z | 12:47 |
| trown | stuff like ^ | 12:47 |
| mandre | trown: from what I understand there is an issue in the image prepare step, possibly something to do with the quay.io registry, http://logs.openstack.org/82/610682/1/check/tripleo-ci-centos-7-scenario009-multinode-oooq/e2326ae/logs/undercloud/var/log/containers/mistral/executor.log.txt.gz#_2018-10-16_07_59_02_865 | 12:47 |
| trown | mandre: ya, I am tempted to recheck and see if it is transient... I have not ever seen that locally | 12:49 |
| *** dmacpher_ has quit IRC | 12:49 | |
| trown | mandre: we will need to do something about those quay.io images though... maybe just copy them to our docker mirror ... they dont seem to get updated too often | 12:49 |
| mandre | trown: yep... or kindly ask infra to mirror quay.io :) | 12:50 |
| trown | well the prometheus ones do... but some dont | 12:50 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Fix linters E102 Error https://review.openstack.org/610889 | 12:51 |
| mandre | trown: concerning your error, it's probably something else that prevents your openshift api to come up -- you're looking at the result not the cause | 12:51 |
| trown | mandre: ya though I think if we want to get 3.11 support in rocky we probably have to go the manual copy route, but longer term it would be better to have quay.io also mirrored in infra | 12:52 |
| trown | mandre: well the odd thing is I can curl the URL in those messages ... if I provide a cert from /etc/origin | 12:53 |
| trown | mandre: but maybe the api is just taking a long time to come up... | 12:54 |
| *** sambetts|afk has joined #tripleo | 12:55 | |
| sambetts|afk | is a way to update a specifc parameter in a heat overcloud without triggering a full minor update? The problem I'm trying to solve is that I have DeleteOnRHELUnregistration set to true, but need to set it change it to false | 12:55 |
| janki | HI. Can I please get some reviews for this cherry-pick https://review.openstack.org/#/c/610883/ | 12:55 |
| mandre | trown: if it helps, I can upload the ansible logs for my deployment | 12:56 |
| mandre | trown: I see it ran a "Create etcd CA certificate" task | 12:56 |
| *** tzumainn has joined #tripleo | 12:56 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: DNM: OVN test https://review.openstack.org/593056 | 12:56 |
| *** mfedosin has joined #tripleo | 12:57 | |
| trown | mandre: ya all the certs are there in my env ... I think you might be right that I am not looking at the cause but instead an effect | 12:57 |
| *** dprince has joined #tripleo | 12:59 | |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: Fix variable name for workload_launch task. https://review.openstack.org/610449 | 12:59 |
| *** sshnaidm is now known as sshnaidm|afk | 13:05 | |
| *** ansmith has joined #tripleo | 13:08 | |
| *** sshnaidm|afk has quit IRC | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 13:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 13:10 |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:10 |
| *** shyamb has quit IRC | 13:14 | |
| *** agopi has quit IRC | 13:15 | |
| weshay | bogdando++ | 13:15 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default https://review.openstack.org/610491 | 13:15 |
| bogdando | weshay: thanks man, what for was that? :) | 13:16 |
| weshay | bogdando, at least thinking about to improve the situation w/ ovb | 13:17 |
| weshay | rdo-cloud | 13:18 |
| bogdando | aye, that is nice to think on | 13:18 |
| *** jistr|afk is now known as jistr | 13:20 | |
| *** amoralej|lunch is now known as amoralej | 13:23 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: WIP - POC verify nova api migration finished before start placement https://review.openstack.org/610966 | 13:25 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Enable use of python3 and dnf https://review.openstack.org/610018 | 13:25 |
| *** ade_lee has joined #tripleo | 13:26 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: WIP - POC verify nova api migration finished before start placement https://review.openstack.org/610966 | 13:27 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-common master: Add crane service to overcloud_containers.yaml.j2 https://review.openstack.org/610028 | 13:28 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Fix ceph-nfs duplicate property https://review.openstack.org/609599 | 13:29 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: WIP: Remove obsoleted code https://review.openstack.org/610937 | 13:31 |
| *** agopi has joined #tripleo | 13:36 | |
| *** jpena|lunch is now known as jpena | 13:37 | |
| *** agopi_ has joined #tripleo | 13:38 | |
| *** toure|gone is now known as toure | 13:38 | |
| *** agopi has quit IRC | 13:40 | |
| *** mjturek has joined #tripleo | 13:42 | |
| *** bfournie has joined #tripleo | 13:49 | |
| *** ade_lee has quit IRC | 13:50 | |
| *** ykarel is now known as ykarel|afk | 13:50 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: WIP - POC verify nova api migration finished before start placement https://review.openstack.org/610966 | 13:52 |
| *** vinaykns has joined #tripleo | 13:53 | |
| *** ade_lee_ has joined #tripleo | 13:55 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 13:56 |
| *** evrardjp has joined #tripleo | 13:57 | |
| *** janki has quit IRC | 13:58 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: WIP - POC verify nova api migration finished before start placement https://review.openstack.org/610966 | 13:58 |
| *** saneax has quit IRC | 13:59 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Change ceph-ansible fetch directory format on upgrade from Q to R https://review.openstack.org/607764 | 13:59 |
| marios|rover | 3~ | 13:59 |
| abishop | weshay: can you help me understand what's happening with https://review.openstack.org/608074 | 13:59 |
| abishop | I have a -1 from RDO CI, but recheck and check-rdo don't trigger another run | 13:59 |
| mwhahaha | #startmeeting tripleo | 14:00 |
| mwhahaha | #topic agenda | 14:00 |
| mwhahaha | * Review past action items | 14:00 |
| mwhahaha | * One off agenda items | 14:00 |
| mwhahaha | * Squad status | 14:00 |
| mwhahaha | * Bugs & Blueprints | 14:00 |
| mwhahaha | * Projects releases or stable backports | 14:00 |
| mwhahaha | * Specs | 14:00 |
| mwhahaha | * open discussion | 14:00 |
| openstack | Meeting started Tue Oct 16 14:00:08 2018 UTC and is due to finish in 60 minutes. The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| mwhahaha | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:00 |
| mwhahaha | Hi everyone! who is around today? | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:00 | |
| openstack | The meeting name has been set to 'tripleo' | 14:00 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:00 | |
| marios|rover | abishop: https://bugs.launchpad.net/tripleo/+bug/1797918 | 14:00 |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:00 |
| Tengu | «o/ | 14:00 |
| matbu | o/ | 14:00 |
| marios|rover | o/ | 14:00 |
| rfolco | o/ | 14:00 |
| Tengu | you're a bit early mwhahaha :) | 14:00 |
| weshay | abishop, wait for the rdo-cloud status update | 14:00 |
| fultonj | o/ | 14:00 |
| weshay | 0/ | 14:00 |
| mwhahaha | am not | 14:01 |
| mwhahaha | i'm 8 seconds late | 14:01 |
| mwhahaha | Meeting started Tue Oct 16 14:00:08 2018 | 14:01 |
| EmilienM | o/ | 14:01 |
| beagles | o/ | 14:01 |
| ksambor | o/ | 14:01 |
| abishop | weshay, marios|rover: ack, thx! | 14:01 |
| abishop | o/ | 14:01 |
| *** huynq has joined #tripleo | 14:01 | |
| Tengu | mwhahaha: ah, maybe my server is a bit late? thanks ntpd -.- | 14:01 |
| *** Vorrtex has joined #tripleo | 14:01 | |
| mwhahaha | maybe you should be running chrony | 14:02 |
| mwhahaha | :D | 14:02 |
| Tengu | :] | 14:02 |
| Tengu | easy target ;) | 14:02 |
| panda | o/ | 14:02 |
| rlandy | o/ | 14:02 |
| ccamacho | o/ | 14:03 |
| mwhahaha | alright let's do this | 14:04 |
| owalsh | o/ | 14:04 |
| mwhahaha | #topic review past action items | 14:04 |
| mwhahaha | None! | 14:04 |
| *** openstack changes topic to "review past action items (Meeting topic: tripleo)" | 14:04 | |
| *** agopi_ is now known as agopi | 14:04 | |
| mwhahaha | since there were no action items lets move on | 14:04 |
| mwhahaha | #topic one off agenda items | 14:04 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:04 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:04 | |
| mwhahaha | (bogdando) replace ovb jobs with multinode plus HW-provisioning done on fake QEMU VMs locally on undercloud, then switched to predeployed servers | 14:04 |
| dprince | hi | 14:04 |
| bogdando | o/ | 14:04 |
| bogdando | I hope the topic is self explaining? | 14:05 |
| jrist | o/ | 14:05 |
| *** apevec has joined #tripleo | 14:05 | |
| mwhahaha | not really | 14:05 |
| mwhahaha | where would these multinode jobs run? | 14:05 |
| bogdando | ok, it's about removing network communications while testing HW prov | 14:05 |
| bogdando | once we are sure qemu vms can be introspected and provisioned, we switch to deployed servers and continue the multinode setup, instead of ovb | 14:06 |
| bogdando | so qemu are fakes to be thrown away | 14:06 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Run DLRN gate role only if to_build is true https://review.openstack.org/610728 | 14:06 |
| bogdando | the expectation is to see only ironic et al bits in action and having 0 interference from L2 networking | 14:06 |
| mwhahaha | so would this be an additional step to be added to like the udnercloud job? | 14:07 |
| dtantsur | this removes coverage for non-deployed-servers flow, right? | 14:07 |
| bogdando | dtantsur: not sure, thus asking/proposing | 14:07 |
| dtantsur | btw your idea is quite close to what derekh is doing for his ironic-in-overcloud CI job | 14:07 |
| bogdando | do we really test something of Ironic stack after the nodes introspected? | 14:07 |
| bogdando | interesting | 14:07 |
| mwhahaha | yes | 14:07 |
| dtantsur | tear down at least? | 14:07 |
| mwhahaha | we need to make sure the images are good | 14:07 |
| bogdando | I see | 14:08 |
| mwhahaha | and that the provisioning askpect works | 14:08 |
| bogdando | ok, could we provision those QEMU vms then? | 14:08 |
| mwhahaha | so if you just wanted to test introspection early in the udnercloud job, that might be ok | 14:08 |
| mwhahaha | but we need ovb because it provides actual provisioning coverage (and coverage for our image building) | 14:08 |
| dtantsur | bogdando: this what devstack does. nested virt is slooooooooooooooooo | 14:08 |
| dtantsur | ooooooooooooooooo | 14:08 |
| dtantsur | you got the idea :) | 14:08 |
| * mwhahaha hands dtantsur an ow | 14:08 | |
| dtantsur | yay, here it is: ooow! thanks mwhahaha | 14:09 |
| bogdando | well yes, but it hopefully takes not so much just to provision a node? got stats? | 14:09 |
| derekh | bogdando: the ironic is overcloud job can be seen here https://review.openstack.org/#/c/582294/ see tripleo-ci-centos-7-scenario012-multinode-oooq-container | 14:09 |
| dtantsur | no hard data, but expect 2-3 times slow down of roughly everything | 14:09 |
| mwhahaha | given our memory constraints i'm not sure we have enough to do the undercloud + some tiny vms for provisioning | 14:09 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Fix ceph-nfs duplicate property https://review.openstack.org/609599 | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797838 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 14:10 |
| openstack | Launchpad bug 1797838 in tripleo "openstack-tox-linters job is failing for the tripleo-quickstart-extras master gate check" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 14:10 |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:10 |
| bogdando | Would be nice to see it in real CI jobs logs, how long it takes | 14:10 |
| dtantsur | tiny? | 14:10 |
| bogdando | in real ovb jobs | 14:10 |
| dtantsur | our IPA images require 2G of RAM just to run... | 14:10 |
| mwhahaha | right we can't do that on our upstream jobs | 14:11 |
| bogdando | anyway, once the provisioning done the idea was to throw vms away and switch to multinode as usual | 14:11 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: WIP Fix ipv6 addrlabel for ceph-nfs https://review.openstack.org/610987 | 14:11 |
| bogdando | may be run some sanity checks before that... | 14:12 |
| dtantsur | this removes the ability to check that our overcloud-full images 1. are usable, 2. can be deployed by ironic | 14:12 |
| holser_ | o/ | 14:12 |
| dtantsur | (this is assuming that ironic does its job correctly, which is verified in our CI.. modulo local boot) | 14:12 |
| arxcruz | o/ | 14:13 |
| bogdando | dtantsur: ;-( for 1, but not sure for 2 | 14:13 |
| dtantsur | bogdando: ironic has some requirements for images. e.g. since we default to local boot, we need grub2 there. | 14:13 |
| dtantsur | and if/when we switch to uefi, fun will increase | 14:13 |
| bogdando | I thought that 2 will be covered by Ironic provisioning those images on qemu | 14:13 |
| weshay | hrm.. need scenario12 doc'd https://github.com/openstack/tripleo-heat-templates/blob/master/README.rst#service-testing-matrix | 14:13 |
| derekh | weshay: will do | 14:14 |
| weshay | thank you sir /me looks at code | 14:14 |
| bogdando | ok, how is that idea different to what derekh does? | 14:14 |
| dtantsur | bogdando: derekh only need to verify that ironic boots stuff. he does not need to verify the final result. | 14:15 |
| bogdando | also, an alternative may be to convert those qemu into images for the host cloud | 14:15 |
| bogdando | to continue with multi-node | 14:15 |
| weshay | fyi.. anyone else.. scenario12 defined here https://review.openstack.org/#/c/579603/ | 14:15 |
| bogdando | the take away is we avoided L2 dances :) | 14:15 |
| dtantsur | bogdando: what's the problem you're trying to solve? | 14:16 |
| dtantsur | these L2 dances you're referring to? | 14:16 |
| bogdando | avoiding networking issues | 14:16 |
| mwhahaha | i think we need to fix those and not just work around them in ci | 14:16 |
| dtantsur | ++ | 14:16 |
| bogdando | the main thing to solve is https://bugs.launchpad.net/tripleo/+bug/1797526 | 14:16 |
| openstack | Launchpad bug 1797526 in tripleo "Failed to get power state for node FS01/02" [Critical,Triaged] | 14:16 |
| mwhahaha | we need proper coverage for this | 14:16 |
| bogdando | none knows how to solve it it seems | 14:16 |
| bogdando | and bunch of similar issues hitting us periodically | 14:17 |
| mwhahaha | this also seems like a good thing to add some resilience to | 14:17 |
| dtantsur | that's bad, but won't we just move the problems to a different part? | 14:17 |
| dtantsur | I can tell you that nested virt does give us hard time around networking from time to time | 14:17 |
| dtantsur | e.g. ubuntu updated their ipxe ROM - and we're broken :( | 14:17 |
| weshay | noting that it's not easy to deliniate the issues we're seeing in 3rd party atm.. | 14:17 |
| bogdando | I do not know tbh, if solving network issues is domain of tripleo | 14:18 |
| *** Petersingh is now known as Petersingh|gone | 14:18 | |
| weshay | introspection errors can be caused by rdo-cloud networking | 14:18 |
| bogdando | the base OS rather | 14:18 |
| *** Petersingh|gone has quit IRC | 14:18 | |
| bogdando | and infrastructure | 14:18 |
| mwhahaha | so it seems like it would be better to invest in improved logging/error correction in the introspection process | 14:18 |
| bogdando | not tripleo | 14:18 |
| weshay | it's a very unstable env atm | 14:18 |
| dprince | is this something that a retry would resolve. Like ironic.conf could be tuned for this a bit perhaps? | 14:18 |
| weshay | ya.. we have a patch to tcpdump during introspection | 14:18 |
| mwhahaha | rather than coming up with a complex ci job to just skip it | 14:18 |
| bogdando | I do not mean control/data plance network ofc | 14:18 |
| bogdando | but provisioning and hardware layers | 14:18 |
| dtantsur | bogdando: OVB is an openstack cloud. if we cannot make it reliable enough for us.. we're in trouble | 14:19 |
| bogdando | if integration testing may be done w/o real networking involved, only virtual SDNs, may be we can accept that? dunno | 14:19 |
| derekh | but the problems in CI (at least the ones I've looked at) were where ipmi to the bmc wasn't working over a longer term, reties aint going to help | 14:19 |
| bogdando | just not sure if that would be moving the problems to a different part | 14:19 |
| weshay | dtantsur, agree.. there is work being done to address it. apevec has some details there however right now I agree it's unstable and painful | 14:20 |
| dtantsur | what's the problem with IPMI? the BMC is unstable? using UDP? | 14:20 |
| derekh | Can you make periodic jobs leave envs around on failure so they can be debugged ? | 14:20 |
| bogdando | dtantsur: it is not related to the testing patches | 14:20 |
| weshay | and it's causing a lot of false negatives | 14:20 |
| dtantsur | if the former, we can fix it. if the latter, we can switch to redfish (http based) | 14:20 |
| bogdando | only produces unrelevant noice and tones of grey failures | 14:20 |
| weshay | derekh, you can recreate those w/ the reproducer scripts | 14:20 |
| bogdando | weshay: ++ | 14:21 |
| weshay | in logs/reproduce-foo.sh | 14:21 |
| bogdando | for flase negatives | 14:21 |
| dprince | derekh: I'd like to know more on this. If its a long term connectivity issue.. Then that is our problem right? | 14:21 |
| derekh | weshay: yes, but we were not able to on friday | 14:21 |
| bogdando | basically, 99% of errors one can observe in those integrational builds had nothing to the subject of testing | 14:21 |
| bogdando | (taking numbers out of air) | 14:22 |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Remove ceph-ansible Mistral workflow https://review.openstack.org/604783 | 14:22 |
| dtantsur | are we use IPMI is always to blame in these 99% of made up cases? :) | 14:22 |
| weshay | derekh, k k.. not sure what you were hitting it's hit or miss w/ the cloud atm but also happy to help. derekh probably getting w/ panda locally will help too | 14:22 |
| derekh | It looks to me like either 1. ipmi traffic from the undercloud to the bmc node was dispupted or 2. traffic from the BMC to the rdo-cloud api was distrupted (perhapes because of DNS outages) | 14:22 |
| weshay | ya | 14:22 |
| derekh | *disrupted | 14:23 |
| bogdando | mwhahaha: so not sure for "we need to fix thoseЭ | 14:23 |
| bogdando | " :) | 14:23 |
| weshay | the next topic in the agenda is related | 14:23 |
| weshay | basically rdo-cloud status | 14:23 |
| mwhahaha | k let's dive into the next topic then | 14:24 |
| mwhahaha | RDO-Cloud and Third Party TripleO CI | 14:24 |
| mwhahaha | weshay: is that yours | 14:24 |
| weshay | ya.. just basically summarizing what most of you already know | 14:24 |
| weshay | we have an unstable env for 3rd party atm.. and it's causing pain | 14:24 |
| weshay | the rdo team has built a new ovs package based on osp and we're waiting on ops to install | 14:24 |
| mwhahaha | should we remove it from check until we can resolve stability? | 14:25 |
| weshay | nhicher, is also experimenting with other 3rd party clouds | 14:25 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-heat-templates master: Add scenario 012 - overlcoud baremetal+ansible-ml2 https://review.openstack.org/579603 | 14:25 |
| dtantsur | mwhahaha: if it's not gating, you can ignore it even if it's voting | 14:25 |
| mwhahaha | so we can actually devote efforts to trouble shoot rather than continue to add useless load and possibly hiding issues? | 14:25 |
| mwhahaha | dtantsur: i mean not even run it on patches | 14:25 |
| weshay | mwhahaha, I think thats on the table for sure.. however I can't say I'm very comfortable with that idea | 14:25 |
| dtantsur | ah | 14:25 |
| mwhahaha | if it's 25% what's the point of even running it other than wasting resources | 14:25 |
| dtantsur | well, that means not touching anything related to ironic until it gets fixed | 14:26 |
| dtantsur | (which may be fine) | 14:26 |
| weshay | testing and loading the cloud?? ya .. it's a fair question | 14:26 |
| weshay | mwhahaha, we still need rdo-cloud for master promotions and other branches | 14:26 |
| weshay | so the cloud NEEDS to work | 14:26 |
| weshay | I've had luck on the weekends | 14:26 |
| mwhahaha | if it's fine on the weekends that points to load related issues | 14:27 |
| weshay | fine is relative | 14:27 |
| derekh | in the mean time merge the previously mentioned scenario012 and use it to verify ironic in overcloud instead of the undercloud | 14:27 |
| weshay | better maybe | 14:27 |
| mwhahaha | weshay: do we know what has changed in the cloud itself since the stability issues started? | 14:28 |
| weshay | mwhahaha, personally I'd like to see the admins, tripleo folks, and ironic folks chat for a minute and see what we can do to improve the situation there | 14:28 |
| huynq | Hi, What's about Undercloud OS upgrade? | 14:28 |
| weshay | mwhahaha, I have 0 insight into any changes | 14:29 |
| dtantsur | I'm ready to join such chat, even though I'm not sure we can do much on ironic side | 14:29 |
| weshay | better lines of communication between tripleo, ops, and ironic, ci etc.. are needed.. | 14:29 |
| dtantsur | (well, retries for IPMI are certainly an option) | 14:29 |
| *** ykarel|afk has quit IRC | 14:29 | |
| weshay | dtantsur, agree however being defensive and getting the steps in ci that prove it's NOT ironic would be helpful I think | 14:30 |
| *** ykarel|afk has joined #tripleo | 14:30 | |
| weshay | it's too easy for folks to just think.. oh look.. introspection failed | 14:30 |
| weshay | imho | 14:30 |
| dtantsur | unstable IPMI can be hard to detect outside of ironic | 14:30 |
| weshay | when really it's the networking or other infra issue | 14:30 |
| dtantsur | as I said, if UDP is a problem, we have TCP-based protocols to consider | 14:30 |
| weshay | ah.. interesting | 14:31 |
| panda | maybe add a tcpdump log from a tcpdump in the background ? | 14:31 |
| weshay | panda, ya.. there is a patch up to do that | 14:31 |
| rlandy | we added that already | 14:31 |
| weshay | rascasoft, already added that code :) | 14:31 |
| bogdando | > as I said, if UDP is a problem, we have TCP-based protocols to consider | 14:31 |
| bogdando | good idea ot explore | 14:31 |
| derekh | along with a tcpdump on the BMC node | 14:31 |
| panda | I can never have an original idea :( | 14:31 |
| weshay | lolz | 14:31 |
| rlandy | would the IPMI retries go into the CI code or the ironic code? | 14:32 |
| rascasoft | actually I used it to debug a previous ironic problem so it should fit also this issue | 14:32 |
| rlandy | we had retries at one point | 14:32 |
| dtantsur | we have retries in ironic. we can have moar of them for the CI. | 14:32 |
| panda | rascasoft: past me the link to that change in pvt please | 14:33 |
| rascasoft | panda, it's merged into extras | 14:33 |
| weshay | lol | 14:33 |
| mwhahaha | dtantsur: are they on by default? | 14:33 |
| mwhahaha | or is it tunable in a way? | 14:33 |
| mwhahaha | (and are we doing that) | 14:33 |
| rascasoft | panda, https://github.com/openstack/tripleo-quickstart-extras/commit/78c60d2102f5e22c3abb30bb0c8179d4c999829c | 14:33 |
| dtantsur | mwhahaha: yes, yes, dunno | 14:34 |
| weshay | I like the idea of not using udp if possible | 14:34 |
| rlandy | dtantsur: what's involved with switching to TCP-based protocols - can we discuss after the meeting? we can try it | 14:34 |
| weshay | and maybe ironic error'ing out with nodes unreachable, please check network | 14:35 |
| bogdando | but https://github.com/openstack/tripleo-quickstart-extras/commit/78c60d2102f5e22c3abb30bb0c8179d4c999829c doesn't show the vbmc VM side | 14:35 |
| dtantsur | it's a different protocol, so the BMC part will have to be changed | 14:35 |
| dtantsur | hint: etingof may be quite happy to work on anything redfish-related :) | 14:35 |
| bogdando | and we need to place dstate there also IMO | 14:35 |
| bogdando | dstat | 14:35 |
| mwhahaha | #action rlandy, weshay, dtantsur to look into switching to TCP for IMPI and possibly tuning retries | 14:35 |
| mwhahaha | tag you're it | 14:35 |
| derekh | btw, yesterday I ploted these ipmi outages on a undercloud I'm running on rdo-cloud, they lasted hours, retries and tcp isn't going to help that https://plot.ly/create/?fid=higginsd:2#/ | 14:35 |
| weshay | can we just blame evilien? | 14:36 |
| weshay | derekh++ | 14:36 |
| dtantsur | yeah, hours is too much | 14:36 |
| derekh | in the case of the env about DNS wasn't working (using 1.1.1.1 , to the BMC couldn't talk to the rdo-cloud API) | 14:36 |
| EmilienM | :-o | 14:36 |
| derekh | /about/above/ | 14:37 |
| weshay | dtantsur, derekh so would you guys be open to the idea of an ovb no-change job that runs say every 4 hours or so and does some kind of health report on the our 3rd party env? | 14:37 |
| dtantsur | for the record: these two options together control ipmi timeouts: https://docs.openstack.org/ironic/latest/configuration/config.html#ipmi | 14:37 |
| weshay | until we have this solved? | 14:37 |
| dtantsur | weshay: you mean, instead of voting OVB jobs? | 14:37 |
| weshay | I think the two are independent proposal, both I think should be considered | 14:38 |
| *** etingof has joined #tripleo | 14:38 | |
| * dtantsur welcomes etingof | 14:38 | |
| derekh | weshay: if it could be left up afterwards it would be great, I'd be happy to jump on and try and debug it | 14:38 |
| weshay | I think ironic takes the brunt of at least the current issues in our 3rd party cloud | 14:38 |
| weshay | rock on.. | 14:39 |
| dtantsur | etingof: tl;dr a lot of ipmi outages in the ovb CI, thinking of various ways around | 14:39 |
| weshay | really happy to see you guys in the mix and proactive here.. this has been very painful for the project.. so thank you!!!!! | 14:39 |
| etingof | that reminds me of my ipmi patches | 14:39 |
| dtantsur | etingof: yep, this is also relevant | 14:39 |
| bogdando | https://imgflip.com/i/2k8a00 | 14:39 |
| derekh | weshay: actaully if you want I couldjust do that on my own rdo account | 14:39 |
| dtantsur | we may want to increase retries here, so we'd better sort out our side | 14:39 |
| weshay | I'm not sure if I see the same networking issues in personal tenants as I see in the openstack-infra tenant | 14:40 |
| weshay | not sure what others have noticed | 14:40 |
| derekh | weshay: ok | 14:40 |
| dtantsur | etingof: but also thinking if switching ovb to redfish will make things a bit better | 14:40 |
| etingof | so I have this killer patch that limits ipmitool run time -- https://review.openstack.org/#/c/610007/ | 14:40 |
| panda | getting tcpdumps from BMC is a bit more difficult | 14:41 |
| etingof | that patch might hopefully improve situation when we have many nodes down | 14:41 |
| weshay | https://docs.openstack.org/ironic/pike/admin/drivers/redfish.html ? | 14:41 |
| dtantsur | weshay: yep | 14:41 |
| *** sri_ has left #tripleo | 14:42 | |
| etingof | we would probably have to switch from virtualbmc to sushy-tools emulator | 14:42 |
| mwhahaha | the redfish stuff seems like a longer term investment | 14:43 |
| dtantsur | etingof: ovb does not use vbmc, but it's own bmc implementation | 14:43 |
| dtantsur | well, but you have a point: sushy-tools has a redfish-to-openstack bridge already | 14:43 |
| etingof | dtantsur, ah, right! because it manages OS instances rather than libvirt? | 14:44 |
| dtantsur | yep | 14:44 |
| *** agurenko has quit IRC | 14:44 | |
| etingof | luckly, sushy-tools can do both right out of the box \o/ | 14:44 |
| dtantsur | it's a big plus indeed, it means we can remove the protocol code from ovb itself | 14:45 |
| weshay | help me understand how sushy-tools and redfish would help introspection / provisioning when networking is very unstable | 14:45 |
| weshay | maybe offline, but that is not clear to me yet | 14:45 |
| dtantsur | weshay: simply because of using tcp instead of udp for power actions | 14:45 |
| weshay | ah k | 14:45 |
| dtantsur | I'm not sure if it's going to be a big contribution or not | 14:45 |
| dtantsur | but worth trying IMO | 14:46 |
| dtantsur | esp. since the world is (slowly) moving towards redfish anyway | 14:46 |
| weshay | agree, should be on the table as well | 14:46 |
| weshay | nhicher, ^^^ | 14:46 |
| derekh | sounds like a lot of work to take on before figuring out the problem | 14:46 |
| dtantsur | well, the proper fix is to make networking stable | 14:46 |
| derekh | although possibly work we'll do eventually anyways | 14:47 |
| weshay | we should wait for the admins to upgrade ovs imho | 14:47 |
| weshay | run for a day or two w/ that | 14:47 |
| mwhahaha | any ETA on the OVS upgrade? | 14:47 |
| etingof | will we have to maintain ipmi-based deployments for a long time (forever)? | 14:47 |
| weshay | asking admins to join | 14:47 |
| etingof | because people in the trenches still use ipmi | 14:47 |
| mwhahaha | yea we'll have to support it | 14:47 |
| weshay | amoralej, do you know? | 14:48 |
| *** ykarel|afk has quit IRC | 14:48 | |
| mwhahaha | but we don't necessarily need to rely on it 100% upstream | 14:48 |
| dtantsur | this ^^^ | 14:48 |
| *** ykarel|afk has joined #tripleo | 14:48 | |
| Tengu | well it needs to be tested anyway, if it's supported. | 14:48 |
| mwhahaha | anyway so it looks like there's some possible improvements to OVB that could be had, who wants to take a look at that? | 14:48 |
| weshay | <kforde> weshay, I need to do it in staging ... that will be tomorrow | 14:49 |
| amoralej | weshay, what's the question? | 14:49 |
| *** iranzo has quit IRC | 14:49 | |
| mwhahaha | Tengu: yea but we could limit it to a single feature set rather than all OVB jobs | 14:49 |
| amoralej | ovs update? | 14:49 |
| weshay | amoralej, when we're going to get the new ovs package in rdo-cloud | 14:49 |
| Tengu | mwhahaha: yup. | 14:49 |
| mwhahaha | Tengu: and we can just do a simple 1+1 deploy rather than full HA | 14:49 |
| Tengu | +1 | 14:49 |
| mwhahaha | Tengu: we techincally support the redfish stuff (i think) but we don't have any upstream coverage in tripleo | 14:49 |
| amoralej | weshay, no idea about the plan to deploy it in rdo-cloud | 14:49 |
| mwhahaha | anyway we need to move on | 14:49 |
| amoralej | alan did a build that hopefully will help | 14:50 |
| mwhahaha | who wants to take on the slushy-tools ovb review bits? dtantsur or etingof? | 14:50 |
| weshay | amoralej, k.. we need to expose these details more to the folks consuming 3rd party imho | 14:50 |
| * dtantsur pushes etingof forward :D | 14:50 | |
| mwhahaha | #action etingof to take a look at OVB+slushy-tools | 14:51 |
| mwhahaha | done | 14:51 |
| mwhahaha | moving on :D | 14:51 |
| bogdando | dtantsur: network is unreliable by definition :) | 14:51 |
| dtantsur | bogdando: yeah, there are grades of unreliability :) | 14:51 |
| bogdando | especially if thinking of edge cases in future, we barely can/should "fix" it | 14:51 |
| mwhahaha | in all seriousness we do need to move forward, please feel free to continue this discussion after the meeting or on the ML | 14:51 |
| dtantsur | k k | 14:51 |
| mwhahaha | (rfolco) CI Community Meeting starts immediately upon this meeting closing in #oooq. This is our team's weekly "open office hours." All are welcome! Ask/discuss anything, we don't bite. Agenda (add items freely) --> https://etherpad.openstack.org/p/tripleo-ci-squad-meeting ~ L49. | 14:52 |
| *** dprince has quit IRC | 14:52 | |
| mwhahaha | that's it on the meeting items | 14:52 |
| mwhahaha | moving on to the status portion of our agenda | 14:52 |
| mwhahaha | #topic Squad status | 14:52 |
| mwhahaha | ci | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:52 |
| mwhahaha | upgrade | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:52 |
| mwhahaha | containers | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-containers-squad-status | 14:52 |
| *** openstack changes topic to "Squad status (Meeting topic: tripleo)" | 14:52 | |
| mwhahaha | edge | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:52 |
| mwhahaha | integration | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:52 |
| mwhahaha | ui/cli | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status | 14:52 |
| mwhahaha | validations | 14:52 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:52 |
| mwhahaha | networking | 14:53 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:53 |
| mwhahaha | workflows | 14:53 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status | 14:53 |
| mwhahaha | security | 14:53 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-security-squad | 14:53 |
| mwhahaha | any particular highlights that anyone would like to raise? | 14:53 |
| *** aufi has quit IRC | 14:54 | |
| mwhahaha | sounds like no | 14:54 |
| mwhahaha | #topic bugs & blueprints | 14:55 |
| mwhahaha | #link https://launchpad.net/tripleo/+milestone/stein-1 | 14:55 |
| mwhahaha | For Stein we currently have 28 blueprints and about 743 open Launchpad bugs. 739 stein-1, 4 stein-2. 102 open Storyboard bugs. | 14:55 |
| mwhahaha | #link https://storyboard.openstack.org/#!/project_group/76 | 14:55 |
| *** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)" | 14:55 | |
| weshay | rfolco, post the link for the bluejeans ci here please | 14:55 |
| mwhahaha | please take a look at the open blueprints as there are a few without approvals and such | 14:55 |
| weshay | oops sorry | 14:56 |
| rfolco | https://bluejeans.com/5878458097 --> ci community mtg bj | 14:56 |
| mwhahaha | any specific bugs, other than the rdo cloud ones, that people want to point out? | 14:56 |
| mwhahaha | sounds like no on that as well | 14:57 |
| mwhahaha | #topic projects releases or stable backports | 14:57 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:57 | |
| mwhahaha | EmilienM: stable releases? | 14:57 |
| *** ccamacho has quit IRC | 14:57 | |
| EmilienM | mwhahaha: I do once a month now | 14:57 |
| EmilienM | no updates this week | 14:58 |
| mwhahaha | k | 14:58 |
| mwhahaha | #topic specs | 14:58 |
| mwhahaha | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:58 |
| *** openstack changes topic to "specs (Meeting topic: tripleo)" | 14:58 | |
| mwhahaha | please take some time to review the open specs, looks like there is only a few for stein | 14:58 |
| mwhahaha | #topic open discussion | 14:58 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:58 | |
| mwhahaha | anything else? | 14:58 |
| *** skramaja has quit IRC | 14:59 | |
| huynq | mwhahaha: What's about Undercloud/Overcloud OS upgrade? | 14:59 |
| mwhahaha | huynq: upgrade to what? | 14:59 |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-upgrade master: Add build option to plugin.spec https://review.openstack.org/611005 | 15:00 |
| slagle | thrash: therve : is it an ok idea to kick off workflow from within an action? | 15:00 |
| *** ccamacho has joined #tripleo | 15:00 | |
| thrash | slagle: Uhhh... I would say no | 15:00 |
| huynq | mwhahaha: e.g from CentOS7 to CentOS8 | 15:00 |
| mwhahaha | huynq: some folks are still looking into that )cc: chem, holser_) but i'm not sure we have a solid plan at the moment due to lack of CentOS8 | 15:01 |
| *** ramishra has quit IRC | 15:01 | |
| mwhahaha | alright we're out of time | 15:01 |
| mwhahaha | #endmeeting | 15:01 |
| *** openstack changes topic to "Welcome to Stein | CI Status: GREENish - don't mess this up | https://docs.openstack.org/tripleo-docs/latest/" | 15:01 | |
| openstack | Meeting ended Tue Oct 16 15:01:28 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:01 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-10-16-14.00.html | 15:01 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-10-16-14.00.txt | 15:01 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-10-16-14.00.log.html | 15:01 |
| EmilienM | mwhahaha: thx for running it | 15:02 |
| thrash | slagle: has that been done? Are were you thinking of doing it? | 15:02 |
| thrash | *or | 15:02 |
| weshay | rfolco, k.. now :) | 15:02 |
| slagle | thrash: thought that might be the case. this is very limiting as effectively we are locked out of using our own api logic | 15:02 |
| rfolco | https://bluejeans.com/5878458097 --> ci community mtg bj | 15:02 |
| thrash | slagle: in what way? | 15:02 |
| *** kforde has joined #tripleo | 15:02 | |
| weshay | thanks | 15:02 |
| thrash | slagle: I assume you are already calling this action from another workflow? | 15:03 |
| weshay | open office re: ci in https://bluejeans.com/5878458097 | 15:03 |
| slagle | thrash: i have an action, and i want to update the deployment status. and the only way to do that is through the tripleo.messaging.v1.send workflow | 15:03 |
| huynq | mwhahaha: thanks! | 15:03 |
| dtantsur | is https://github.com/cybertron/openstack-virtual-baremetal something still largely managed by bnemec personally? | 15:04 |
| bnemec | dtantsur: Largely, yes. | 15:04 |
| thrash | slagle: is it doing that via some type of loop in the action? | 15:05 |
| dtantsur | bnemec: have you seen the discussion about potentially using redfish there? | 15:05 |
| *** mcornea has quit IRC | 15:05 | |
| thrash | slagle: and what does the action do otherwise? | 15:05 |
| dtantsur | weshay: what is the range of openstack releases we support in the ovb ci now? | 15:05 |
| *** mcornea has joined #tripleo | 15:05 | |
| bnemec | dtantsur: I have not. There's no reason we couldn't though, as long as some sort of virtual bmc code for redfish exists. | 15:05 |
| weshay | we go back to pike atm | 15:05 |
| slagle | thrash: there's no loop. it's going to be a new action to sync the deployment status based on stack status, config_download status, and current deployment status | 15:06 |
| thrash | slagle: and the status is in swift, right? | 15:06 |
| thrash | slagle: does that really need to be an action? | 15:07 |
| slagle | thrash: the deployment status is in swift | 15:07 |
| thrash | Sounds like that could be done directly in a workflow | 15:07 |
| slagle | thrash: it possibly could be. with complex yaql and no unit test coverage | 15:07 |
| *** huynq has quit IRC | 15:07 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Corrected release config location for log collection https://review.openstack.org/610038 | 15:07 |
| *** boazel has quit IRC | 15:08 | |
| slagle | thrash: and...no way for anything else to consume the logic. as i am experiencing now :) | 15:08 |
| *** huynq has joined #tripleo | 15:08 | |
| slagle | it's not just "get an object from swift". | 15:08 |
| chandankumar | mwhahaha: weshay https://review.openstack.org/#/c/610017/ https://review.openstack.org/#/c/610600/ needs for standalone puppet tempest | 15:08 |
| *** yprokule has quit IRC | 15:09 | |
| thrash | slagle: I would suggest the action return the result back to the calling workflow, and then the workflow does the updating. | 15:09 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Resolve get_build_command output before container_build_id https://review.openstack.org/602734 | 15:09 |
| dtantsur | bnemec: we already have a redfish-to-openstack proxy | 15:09 |
| *** huynq has quit IRC | 15:10 | |
| dtantsur | etingof: did we end up packaging sushy-tools for RDO? | 15:10 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 15:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 15:10 |
| thrash | slagle: is it pre -> update status -> post? | 15:10 |
| thrash | Or just figure out status -> update status | 15:10 |
| etingof | dtantsur, I never worked on that | 15:10 |
| thrash | if it's the latter, then just have the figure out part be in an action, and the update be in the calling workflow (via send) | 15:11 |
| slagle | thrash: yea i'll give that a shot | 15:11 |
| dtantsur | etingof: wanna give it a try or should I? | 15:11 |
| etingof | dtantsur, I do not see sushy-tools in the rdo repo | 15:12 |
| etingof | dtantsur, yes, I will work on that | 15:12 |
| dtantsur | if either of us haven't packaged it, it's not there indeed :) | 15:12 |
| dtantsur | thanks! | 15:12 |
| dtantsur | ping me for review/help/whatever | 15:12 |
| etingof | most certainly | 15:12 |
| derekh | trying to run the reproducer script and getting this NODES_FILE | 15:17 |
| derekh | ERROR! Unable to retrieve file contents | 15:17 |
| derekh | Could not find or access '/tmp/reproduce-tmp.h5DCW/tripleo-quickstart' | 15:17 |
| derekh | which seems to be to lack of NODES_FILE being set | 15:17 |
| derekh | shouldn't the default just work? | 15:17 |
| derekh | $NODES_FILE is getting used here http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2#n251 | 15:19 |
| *** chandankumar is now known as chkumar|off | 15:19 | |
| bnemec | dtantsur: Note that right now we just pip install pyghmi in OVB anyway, so if sushy is available from pip then we can use it. | 15:20 |
| dtantsur | even simpler. but I do think we need to package it for future-proofing | 15:20 |
| bnemec | Yep, sounds good. | 15:21 |
| *** aufi has joined #tripleo | 15:22 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-heat-templates stable/rocky: Add tripleo-heat-templates dir to Mistral volumes https://review.openstack.org/611011 | 15:24 |
| florianf | honza: https://review.openstack.org/#/c/611011/ | 15:24 |
| *** dciabrin has joined #tripleo | 15:25 | |
| *** holser_ has quit IRC | 15:29 | |
| *** agurenko has joined #tripleo | 15:31 | |
| *** thrash is now known as thrash|f00dz | 15:33 | |
| *** ksambor has quit IRC | 15:33 | |
| *** sshnaidm has joined #tripleo | 15:37 | |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo stable/pike: Pin puppet-snmp https://review.openstack.org/611019 | 15:37 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo stable/pike: Rework GDPR compliant logrotate config https://review.openstack.org/606907 | 15:38 |
| *** hamzy has quit IRC | 15:42 | |
| openstackgerrit | weizj proposed openstack/tripleo-specs master: Update the hacking to latst https://review.openstack.org/611026 | 15:45 |
| openstackgerrit | weizj proposed openstack/tripleo-image-elements master: Update the hacking to latst https://review.openstack.org/611047 | 15:47 |
| openstackgerrit | weizj proposed openstack/os-net-config master: Update the hacking to latst https://review.openstack.org/611061 | 15:48 |
| *** dprince has joined #tripleo | 15:51 | |
| *** boazel has joined #tripleo | 15:57 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Add posibilities to set tunnel_csum in ovs agent https://review.openstack.org/609946 | 15:57 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/pike: GATE CHECK for TripleO https://review.openstack.org/602248 | 16:00 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: podman: stop the docker container during upgrade https://review.openstack.org/608319 | 16:01 |
| *** agurenko has quit IRC | 16:03 | |
| bnemec | Hey, can I get some reviews on the series starting at https://review.openstack.org/#/c/584026 ? | 16:04 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 16:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 16:10 |
| *** noama has quit IRC | 16:10 | |
| mwhahaha | kopecmartin|ruck, marios|rover, weshay: https://bugs.launchpad.net/tripleo/+bug/1798126 it's failing on the container upload but it says it's missing the containers with the dlrn hash being tested, are we sure there wasn't an issue uploading the test containers? | 16:10 |
| openstack | Launchpad bug 1798126 in tripleo "Undercloud deploy - 'NoneType' object has no attribute 'group'" [Critical,Triaged] | 16:10 |
| mwhahaha | EmilienM: -^ fyi | 16:10 |
| EmilienM | meh | 16:11 |
| EmilienM | let's see if docker://registry-1.docker.io/tripleomaster/centos-binary-cron:4f41f317d5ce4fccbd902052066df431b6ecdfa5_d48e5c18 exists | 16:12 |
| EmilienM | it doesn't | 16:12 |
| EmilienM | (https://hub.docker.com/r/tripleomaster/centos-binary-cron/tags/) | 16:12 |
| *** sshnaidm has quit IRC | 16:13 | |
| *** quiquell|off has quit IRC | 16:15 | |
| EmilienM | mwhahaha: should we run promotion jobs against current-tripleo-rdo-internal ? | 16:15 |
| mwhahaha | i'm likely to believe there was a failure in the promotion jobs because that code is tested in normal checks and we aren't having problems withe very job | 16:16 |
| mwhahaha | so i think we need the tripleoci folks to review why the testing containers are failing | 16:16 |
| * mwhahaha goes back to do other things | 16:16 | |
| EmilienM | https://hub.docker.com/r/tripleomaster/centos-binary-cron/tags/ | 16:16 |
| EmilienM | latest tag pushed for curren-tripleo-rdo-internal was 965941f1e62cef16967e7a7cd6d98263e52acb62_0989b280 | 16:16 |
| EmilienM | so my question is, is this tag used to test promotions? | 16:17 |
| EmilienM | weshay: not sure who can take a look from CI team | 16:17 |
| *** ykarel|afk is now known as ykarel | 16:18 | |
| *** apetrich has quit IRC | 16:18 | |
| *** d0ugal has quit IRC | 16:21 | |
| *** holser_ has joined #tripleo | 16:21 | |
| *** yolanda has quit IRC | 16:22 | |
| *** d0ugal has joined #tripleo | 16:23 | |
| *** Vorrtex has quit IRC | 16:24 | |
| *** gfidente is now known as gfidente|afk | 16:25 | |
| openstackgerrit | weizj proposed openstack/os-net-config master: Update the hacking to latest https://review.openstack.org/611061 | 16:26 |
| openstackgerrit | weizj proposed openstack/tripleo-image-elements master: Update the hacking to latst https://review.openstack.org/611047 | 16:29 |
| *** Vorrtex has joined #tripleo | 16:29 | |
| *** jpich has quit IRC | 16:29 | |
| *** apetrich has joined #tripleo | 16:30 | |
| weshay | EmilienM, sec.. in 1-1 panda can you assist EmilienM ? | 16:31 |
| panda | weshay: EmilienM yes | 16:31 |
| openstackgerrit | weizj proposed openstack/tripleo-image-elements master: Update the hacking to latest https://review.openstack.org/611047 | 16:32 |
| *** rlandy is now known as rlandy|biab | 16:32 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 16:34 |
| panda | EmilienM: yes, the hash is what has been tested in phase1 https://ci.centos.org/job/rdo-promote-get-hash-master-current-tripleo/547/console | 16:35 |
| mwhahaha | panda: yea but the failed jobs from the bug was not that hash | 16:35 |
| mwhahaha | panda: i used a reproducer from https://bugs.launchpad.net/tripleo/+bug/1798126 | 16:36 |
| openstack | Launchpad bug 1798126 in tripleo "Undercloud deploy - 'NoneType' object has no attribute 'group'" [Critical,Triaged] | 16:36 |
| mwhahaha | panda: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master/2dc60e2/job-output.txt.gz#_2018-10-16_11_43_01_925122 | 16:36 |
| *** rpioso|afk is now known as rpioso | 16:36 | |
| *** ykarel is now known as ykarel|away | 16:37 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 16:37 |
| panda | mwhahaha: the periodi job you linked takes the hash to test from the base job at the top of the pipeline. The base job takes the latest consistent and assign it the alias "tripleo-ci-testing". That's what all the various featuresets jobs are using. Emilien question was on current-tripleo-rdo-internal, which is tagged after phase2. | 16:41 |
| *** zaneb has quit IRC | 16:41 | |
| mwhahaha | panda: his question was should be using that for that job or not. Either way my original question was why is that hash missing from the periodic job | 16:41 |
| weshay | ok..back | 16:42 |
| * weshay readin | 16:42 | |
| weshay | g | 16:42 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-docs master: User docs for minor updates in Rocky+ https://review.openstack.org/610618 | 16:42 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-docs master: User docs for major upgrades in Rocky+ https://review.openstack.org/611082 | 16:42 |
| weshay | EmilienM, mwhahaha the containers used in promotion jobs are in docker.io, they exist only in the rdo registry | 16:44 |
| *** bogdando has quit IRC | 16:44 | |
| weshay | EmilienM, mwhahaha they are pushed to docker.io only during the actual promotion | 16:45 |
| weshay | that is why it's a 404 for Evilien | 16:45 |
| mwhahaha | k so the code we used to switch to use python assumes docker.io | 16:45 |
| * mwhahaha blames stevebaker | 16:45 | |
| *** jpena is now known as jpena|off | 16:45 | |
| weshay | why would we hardcode that | 16:45 |
| weshay | <sigh> | 16:45 |
| mwhahaha | well it might be configurable | 16:45 |
| mwhahaha | let me see | 16:45 |
| *** ykarel|away has quit IRC | 16:46 | |
| *** phuongnh has joined #tripleo | 16:46 | |
| mwhahaha | it's not hard coded | 16:47 |
| EmilienM | mhh I think he tried to mimic what skopeo inspect was doing | 16:47 |
| *** panda is now known as panda|off | 16:48 | |
| mwhahaha | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master/2dc60e2/logs/undercloud/home/zuul/containers-prepare-parameter.yaml.txt.gz | 16:48 |
| mwhahaha | so we don't appear to be honoring the namespace correctly then | 16:48 |
| *** thrash|f00dz is now known as thrash | 16:48 | |
| weshay | although.. the container in question is not in the rdo registry afaict https://console.registry.rdoproject.org/registry#/images/tripleomaster/centos-binary-cron | 16:49 |
| EmilienM | could be the _fix_dockerio_url | 16:49 |
| mwhahaha | shouldn't be unless the url was bad from the get go | 16:50 |
| weshay | is this the local tag? 4f41f317d5ce4fccbd902052066df431b6ecdfa5_d48e5c18 | 16:50 |
| weshay | or the tag I should see in the registry | 16:50 |
| mwhahaha | in the registry | 16:50 |
| *** phuongnh has quit IRC | 16:50 | |
| mwhahaha | cause we're doing an inspect on it | 16:50 |
| weshay | the job kicked again | 16:52 |
| weshay | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/5d9d5fc/job-output.txt.gz | 16:52 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Repair ansible when calls with jinja2 https://review.openstack.org/610501 | 16:52 |
| mwhahaha | weshay: what's the retention period on those hashes in rdo registry? | 16:54 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Export container_cli for update & upgrade tasks https://review.openstack.org/611089 | 16:54 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: DNM: test update https://review.openstack.org/611090 | 16:54 |
| weshay | mwhahaha, I was just going to ask panda if we ditch failed hashes | 16:55 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 16:55 |
| panda|off | the scripot taht implements the policy is called "nuk'em all!" | 16:55 |
| weshay | mwhahaha, I'll look into that | 16:55 |
| panda|off | weshay: we don't ditch failing hashes in the registry | 16:55 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: podman: rm Docker containers during upgrades https://review.openstack.org/611092 | 16:55 |
| panda|off | weshay: but they're going ot be deleted soon if they are not associated with any symlink | 16:56 |
| *** aufi has quit IRC | 16:56 | |
| EmilienM | holser_: see https://review.openstack.org/611092 | 16:56 |
| *** aufi has joined #tripleo | 16:56 | |
| weshay | panda|off, so ya.. EmilienM where did you pull this from? http://paste.openstack.org/show/732258/ | 16:56 |
| EmilienM | weshay: mwhahaha did with --debug | 16:56 |
| weshay | dang it.. that should be in all our promotion jobs.. /me puts in patch | 16:56 |
| *** chkumar|off is now known as chandankumar | 16:57 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - podman: rm Docker containers during upgrades https://review.openstack.org/611092 | 16:57 |
| holser_ | EmilienM already +1 | 16:58 |
| holser_ | :D | 16:58 |
| holser_ | EmilienM - now I am happy | 16:58 |
| EmilienM | good | 16:59 |
| holser_ | that's definitely the way I would like to see | 16:59 |
| holser_ | we may put validation before removing any docker containers | 16:59 |
| mwhahaha | EmilienM: does docker_container noop if docker is not installed? | 16:59 |
| holser_ | and remove after validation. or stop podman and quickly roll back to docker | 16:59 |
| holser_ | in case of upgrade troubles | 17:00 |
| EmilienM | mwhahaha: no I bet it fails | 17:00 |
| EmilienM | mwhahaha: I see why you ask | 17:00 |
| EmilienM | damn | 17:00 |
| *** aufi has quit IRC | 17:00 | |
| *** dtantsur is now known as dtantsur|afk | 17:03 | |
| *** kopecmartin|ruck is now known as kopecmartin|off | 17:05 | |
| *** hamzy has joined #tripleo | 17:05 | |
| *** paramite|clone has joined #tripleo | 17:05 | |
| *** paramite has quit IRC | 17:06 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 17:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 17:10 |
| mwhahaha | weshay: ok so the reproducer results in an incorrect containers-prepare-parameter.yaml | 17:14 |
| weshay | tmate? | 17:14 |
| mwhahaha | weshay: http://paste.openstack.org/show/732264/ | 17:14 |
| mwhahaha | weshay: vs https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master/2dc60e2/logs/undercloud/home/zuul/containers-prepare-parameter.yaml.txt.gz | 17:15 |
| weshay | ya.. k | 17:15 |
| weshay | k.. I'll write that up.. | 17:15 |
| weshay | mwhahaha, EmilienM so are we back to square one on the bug? | 17:15 |
| * mwhahaha fixes and tries again | 17:15 | |
| mwhahaha | yea | 17:15 |
| mwhahaha | so that explains that part | 17:16 |
| weshay | k | 17:16 |
| mwhahaha | let me try and reproduce the bug again | 17:16 |
| mwhahaha | so yea now i get teh NoneType error | 17:16 |
| mwhahaha | it's a bug in the image_uploader.py | 17:16 |
| weshay | kopecmartin|off, when finding bugs it's often helpful to run the reproducer to like EmilienM did | 17:16 |
| weshay | you find interesting things | 17:17 |
| mwhahaha | http://paste.openstack.org/show/732265/ | 17:17 |
| *** homeski has joined #tripleo | 17:20 | |
| * mwhahaha will address after lunch | 17:20 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Run update jobs for fuller set of services https://review.openstack.org/611100 | 17:21 |
| homeski | Hey team. I'm running the ffwd-upgrade process from Newton -> Queens | 17:21 |
| homeski | I ran the prepare command, which vets and uploads the new templates | 17:22 |
| homeski | during the "run" step though, I discovered I need to make a change to one of the templates | 17:22 |
| homeski | at this point, I cannot run the prepare command again though, because the Overcloud is not-operational due to steps the run command makes | 17:22 |
| homeski | so I can't run prepare anymore | 17:22 |
| weshay | mwhahaha, just so I'm SURE I'm looking at the same thing you are .. what's the url for the reproducer you pulled? | 17:23 |
| homeski | I tried updating the templates in the Swift containers, but that has no effect anymore | 17:23 |
| homeski | it seems like the actual params attached the the overcloud stack need to be manually updated .. | 17:23 |
| homeski | any way to do that? | 17:23 |
| *** panda|off has quit IRC | 17:24 | |
| homeski | from the Undercloud if I do `openstack stack show overcloud` I can see the stale "RoleData" value under outputs. Which looks likes the generated Ansible playbooks | 17:26 |
| homeski | I'm guessing "prepare" put that there | 17:26 |
| *** panda has joined #tripleo | 17:27 | |
| mwhahaha | weshay: wget https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master/2dc60e2/logs/reproducer-quickstart.sh | 17:28 |
| *** amoralej is now known as amoralej|off | 17:28 | |
| weshay | thanks | 17:28 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Run bashate via pre-commit https://review.openstack.org/583984 | 17:29 |
| *** vinaykns has quit IRC | 17:30 | |
| *** zaneb has joined #tripleo | 17:33 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Swift doesn't use standard logging. https://review.openstack.org/611110 | 17:36 |
| *** trown is now known as trown|lunch | 17:37 | |
| homeski | any ideas on how I can update the overcloud stack environment for the 'run' command during the ffwd-upgrade, without running 'prepare' again? | 17:41 |
| *** rlandy|biab is now known as rlandy | 17:46 | |
| homeski | scratch that, how can I update a parameter_defaults key for a stack manually? | 17:49 |
| *** rdopiera has quit IRC | 17:50 | |
| *** gfidente|afk has quit IRC | 17:52 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: DNM: OVN test https://review.openstack.org/593056 | 17:53 |
| *** jfrancoa has quit IRC | 17:59 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Ensure octavia flavor gets configured if composable octavia https://review.openstack.org/609696 | 17:59 |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add tripleo-ha-utils to extra-requirements https://review.openstack.org/610508 | 18:01 |
| *** salmankhan has quit IRC | 18:02 | |
| *** cylopez has joined #tripleo | 18:09 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 18:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| *** cylopez has left #tripleo | 18:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 18:10 |
| *** vinaykns has joined #tripleo | 18:17 | |
| stevebaker | weshay, mwhahaha: hey so it looks like I need to fix auth against the rdo registry? | 18:18 |
| mwhahaha | stevebaker: i've got a patch coming, sec | 18:19 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Make service optional for auth https://review.openstack.org/611119 | 18:19 |
| mwhahaha | stevebaker: -^ | 18:19 |
| *** lblanchard has joined #tripleo | 18:21 | |
| weshay | asdf1234 | 18:21 |
| weshay | PASS | 18:21 |
| *** naturalblue has quit IRC | 18:22 | |
| *** hamzy has quit IRC | 18:23 | |
| stevebaker | mwhahaha: looks good, I thought about doing that but decided that exploding noisily would show how different registries actually implement auth ;) | 18:25 |
| mwhahaha | pfffft | 18:25 |
| *** lblanchard has quit IRC | 18:26 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: overcloud-prep-images: use container_cli https://review.openstack.org/608460 | 18:29 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-upgrade master: Set container_cli for undercloud https://review.openstack.org/608462 | 18:32 |
| mwhahaha | stevebaker: did we not add retries to the image_labels stuff? http://logs.openstack.org/17/610017/8/gate/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/a20330f/logs/undercloud/home/zuul/install-undercloud.log.txt.gz | 18:40 |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo master: Dell EMC SC: Add use_multipath_for_image_xfer https://review.openstack.org/611126 | 18:43 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - podman: rm Docker containers during upgrades https://review.openstack.org/611092 | 18:43 |
| EmilienM | mwhahaha: ^ wdyt | 18:43 |
| EmilienM | (let's suppose we have docker during upgrade_tasks) | 18:44 |
| mwhahaha | yea better | 18:44 |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo master: Dell EMC SC: Add use_multipath_for_image_xfer https://review.openstack.org/611126 | 18:45 |
| *** hberaud has quit IRC | 18:48 | |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: Use external-(upgrade|update) commands for Ceph. https://review.openstack.org/604712 | 18:54 |
| *** ssbarnea_ has joined #tripleo | 18:56 | |
| *** pfo has quit IRC | 18:59 | |
| *** social has joined #tripleo | 19:01 | |
| *** trown|lunch is now known as trown | 19:07 | |
| weshay | stevebaker, we'll probably hit a third w/ osp :) | 19:09 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 19:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 19:10 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs https://review.openstack.org/583868 | 19:11 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix update tasks for openshift https://review.openstack.org/608658 | 19:11 |
| *** holser_ has quit IRC | 19:16 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Change log directory for OpenDaylight https://review.openstack.org/603907 | 19:16 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Collectd documentation refactor https://review.openstack.org/606956 | 19:16 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Dell EMC SC: Add support for excluded_domain_ips https://review.openstack.org/604444 | 19:16 |
| openstackgerrit | Merged openstack/puppet-tripleo master: ceilometer: escape % in crontab https://review.openstack.org/599421 | 19:16 |
| *** eck` has joined #tripleo | 19:27 | |
| *** homeski has quit IRC | 19:35 | |
| *** raildo has quit IRC | 19:46 | |
| *** hamzy has joined #tripleo | 19:54 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common stable/rocky: Set workflow status to success when manipulating deployment status https://review.openstack.org/611152 | 19:56 |
| *** akrivoka has quit IRC | 19:56 | |
| weshay | mwhahaha, EmilienM, bug the issue you found https://bugs.launchpad.net/tripleo/+bug/1798190 | 19:57 |
| openstack | Launchpad bug 1798190 in tripleo "tripleo ci reproducer for promotion jobs use the wrong containers" [Critical,Triaged] | 19:57 |
| *** akrivoka has joined #tripleo | 19:57 | |
| mwhahaha | weshay: actually just issue 1 | 19:57 |
| mwhahaha | weshay: when i fixed my prepare namespace and patches my tripleo-common it worked fine | 19:57 |
| openstackgerrit | Sean McGinnis proposed openstack/tripleo-common master: Update sphinx extension logging https://review.openstack.org/611153 | 19:57 |
| dtrainor | Is there any way to make staggered power management calls for a node, like only operating on N nodes at a time instead of all of them at the same time? I ask because some BMC interfaces (particularly, virtual ones with an API) may tend to throttle these requests, and I think I'm experiencing this with something I'm working on. | 19:58 |
| dtrainor | afaik my options are to introspect nodes one at a time, or bulk all of them, with no variation between these two methods | 19:58 |
| mwhahaha | dtrainor: there is a way to limit the max concurrent provisions but i don't think theres a way to limit it | 19:58 |
| mwhahaha | not sure for introspection tho | 19:59 |
| dtrainor | what kind of provisions exactly? | 19:59 |
| mwhahaha | like the number of concurrent systems to build | 19:59 |
| dtrainor | i see | 19:59 |
| dtrainor | ironic has a max_concurrency but it appears this only applies to the number of threads in the pool, not necessarily how many objects it would act on at any given time | 20:00 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient stable/rocky: Update deployment status when deploying or failed https://review.openstack.org/611154 | 20:04 |
| *** mjturek has quit IRC | 20:04 | |
| dtrainor | ooh, there's also an introspection_delay. that might work. | 20:05 |
| *** dciabrin has quit IRC | 20:08 | |
| slagle | EmilienM: did you add a rhel registration service? | 20:08 |
| mwhahaha | slagle: yes there is one now | 20:09 |
| *** boazel has quit IRC | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 20:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 20:10 |
| slagle | mwhahaha: called what? | 20:10 |
| slagle | grep'ing for all obvious names has failed me | 20:10 |
| mwhahaha | slagle: OS::TripleO::Services::Rhsm | 20:10 |
| mwhahaha | acronyms ftw | 20:11 |
| slagle | oh of course | 20:11 |
| tbarron | abishop: gfidente[m]: bandini: https://review.openstack.org/#/c/609599/ passed check queue :) | 20:16 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Inject undercloud user into the cloud image https://review.openstack.org/603069 | 20:20 |
| *** akrivoka has quit IRC | 20:21 | |
| dtrainor | mwhahaha, inspector.conf introspection_delay looks promising, it's starting introspection of nodes N seconds apart. | 20:23 |
| *** slaweq has quit IRC | 20:33 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-heat-templates master: Update to OpenShift 3.11 release https://review.openstack.org/610682 | 20:39 |
| *** holser_ has joined #tripleo | 20:41 | |
| *** ssbarnea has quit IRC | 20:42 | |
| *** ssbarne__ has joined #tripleo | 20:42 | |
| *** ssbarnea has joined #tripleo | 20:43 | |
| *** hamzy has quit IRC | 20:44 | |
| *** ssbarnea_ has quit IRC | 20:44 | |
| *** ansmith has quit IRC | 20:45 | |
| *** ssbarne__ has quit IRC | 20:57 | |
| *** sshnaidm has joined #tripleo | 21:01 | |
| *** sshnaidm is now known as sshnaidm|off | 21:01 | |
| *** trown is now known as trown|outtypewww | 21:04 | |
| *** holser_ has quit IRC | 21:06 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Sync state if needed during retrieval https://review.openstack.org/611168 | 21:06 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add chrony time service https://review.openstack.org/588111 | 21:09 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1798195 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1798198 | 21:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 21:10 |
| openstack | Launchpad bug 1798195 in tripleo "rdo-cloud yum repos unavailable during container updates and failing the undercloud install " [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1798198 in tripleo "pip installs failing in rdo-cloud based jobs" [Critical,Triaged] | 21:10 |
| openstackgerrit | Merged openstack/tripleo-docs master: Add diagram to minor update developer docs https://review.openstack.org/609693 | 21:13 |
| *** lblanchard has joined #tripleo | 21:14 | |
| *** lblanchard has quit IRC | 21:14 | |
| *** boazel has joined #tripleo | 21:21 | |
| *** Vorrtex has quit IRC | 21:21 | |
| *** dciabrin has joined #tripleo | 21:22 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Enable featureset override for tempest_{format, extra_override} https://review.openstack.org/609666 | 21:23 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Fix variable name for workload_launch task. https://review.openstack.org/610449 | 21:23 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Pin puppet-snmp https://review.openstack.org/611019 | 21:23 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add chrony time service https://review.openstack.org/588111 | 21:26 |
| *** toure is now known as toure|gone | 21:31 | |
| *** rh-jelabarre has quit IRC | 21:32 | |
| *** bfournie has quit IRC | 21:35 | |
| *** apevec has quit IRC | 21:40 | |
| *** sshnaidm|off has quit IRC | 21:48 | |
| *** dprince has quit IRC | 21:49 | |
| *** slaweq has joined #tripleo | 21:53 | |
| *** eck` is now known as eck`gone | 21:57 | |
| EmilienM | slagle: it's even documented https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/rhsm.html | 22:05 |
| *** morazi has quit IRC | 22:07 | |
| *** slaweq has quit IRC | 22:09 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 22:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1798195 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1798198 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 22:10 |
| openstack | Launchpad bug 1798195 in tripleo "rdo-cloud yum repos unavailable during container updates and failing the undercloud install " [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1798198 in tripleo "pip installs failing in rdo-cloud based jobs" [Critical,Triaged] | 22:10 |
| *** bugzy_ has joined #tripleo | 22:11 | |
| *** slaweq has joined #tripleo | 22:11 | |
| *** bugzy has quit IRC | 22:14 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: WIP: Flatten Aodh service configuration https://review.openstack.org/611188 | 22:19 |
| openstackgerrit | Merged openstack/tripleo-common stable/pike: [Pike Only] Do not use namespace var name in docker img template. https://review.openstack.org/608647 | 22:21 |
| *** ade_lee_ has quit IRC | 22:23 | |
| *** iranzo has joined #tripleo | 22:24 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: tripleo_deploy: remove defaults for roles_file & plan-environment https://review.openstack.org/611189 | 22:30 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: tripleo_deploy: remove defaults for roles_file & plan-environment https://review.openstack.org/611189 | 22:32 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Resolve get_build_command output before container_build_id https://review.openstack.org/602734 | 22:34 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Gather facts for the undercloud to make platform data available https://review.openstack.org/609488 | 22:35 |
| *** chem has quit IRC | 22:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: tripleo_deploy: remove defaults for roles_file & plan-environment https://review.openstack.org/611189 | 22:39 |
| *** slaweq has quit IRC | 22:44 | |
| *** tzumainn has quit IRC | 22:45 | |
| *** rcernin has joined #tripleo | 22:49 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add chrony time service https://review.openstack.org/588111 | 22:51 |
| *** agopi is now known as agopi|brb | 22:52 | |
| *** vinaykns has quit IRC | 22:54 | |
| *** agopi|brb has quit IRC | 22:56 | |
| openstackgerrit | Alex Schultz proposed openstack/ansible-role-chrony master: Fix missing folders https://review.openstack.org/611193 | 22:57 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add chrony time service https://review.openstack.org/588111 | 22:58 |
| openstackgerrit | Alex Schultz proposed openstack/ansible-role-chrony master: Fix missing folders https://review.openstack.org/611193 | 23:00 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add chrony time service https://review.openstack.org/588111 | 23:05 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797600 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1797918 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1798195 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1798198 | 23:10 |
| openstack | Launchpad bug 1797600 in tripleo "Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to set node power state to power on." [Critical,Incomplete] | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 23:10 |
| openstack | Launchpad bug 1798195 in tripleo "rdo-cloud yum repos unavailable during container updates and failing the undercloud install " [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1798198 in tripleo "pip installs failing in rdo-cloud based jobs" [Critical,Triaged] | 23:10 |
| *** tosky has quit IRC | 23:10 | |
| *** slaweq has joined #tripleo | 23:11 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: check mode: docker_puppet https://review.openstack.org/603253 | 23:12 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Fix ceph-nfs duplicate property https://review.openstack.org/609599 | 23:12 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: tripleo_deploy: remove defaults for roles_file & plan-environment https://review.openstack.org/611189 | 23:20 |
| EmilienM | slagle: ^ for the bug you reported, let me know how it works. i'll test it more tomorrow. | 23:21 |
| *** hamzy has joined #tripleo | 23:24 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Sync state if needed during retrieval https://review.openstack.org/611168 | 23:28 |
| *** slaweq has quit IRC | 23:44 | |
| *** ssbarnea has quit IRC | 23:46 | |
| *** ssbarnea|bkp2 has joined #tripleo | 23:46 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Rework GDPR compliant logrotate config https://review.openstack.org/606907 | 23:47 |
| *** iranzo has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!