| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 00:00 |
|---|---|---|
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 00:00 |
| weshay | mwhahaha, stop flipping tables man ur making a mess | 00:00 |
| weshay | I just bought that table cloth | 00:00 |
| mwhahaha | it's been a week | 00:00 |
| weshay | jebus it has | 00:00 |
| mwhahaha | at least the table is still in once peice | 00:00 |
| mwhahaha | looks like tempest conf should work with OS_CLOUD | 00:00 |
| mwhahaha | let me propose a patch or two | 00:00 |
| mwhahaha | not sure about the containerized tempest tho | 00:01 |
| mwhahaha | that appears to be an input somewhere | 00:01 |
| weshay | {% if tempest_format == "container" %} | 00:01 |
| weshay | export RCFILE="{{ rc_file }}" | 00:01 |
| weshay | cat <<'EOF' > {{ working_dir }}/tempest_container.sh | 00:01 |
| weshay | # Set the exit status for the command | 00:01 |
| weshay | set -e | 00:01 |
| weshay | # Get the list of | 00:01 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add support for OS_CLOUD https://review.openstack.org/600586 | 00:02 |
| weshay | ok.. deployer_input is hardcoded as an option in the validate-tempest jinja | 00:02 |
| mwhahaha | only if overcloud | 00:02 |
| mwhahaha | we'll see | 00:02 |
| mwhahaha | let me line up these patches | 00:02 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: use fs config to assign standalone role, tests https://review.openstack.org/572830 | 00:03 |
| mwhahaha | i think that might do it | 00:04 |
| weshay | --no-default-deployer | 00:04 |
| weshay | 00:04 | |
| weshay | Do not check for the default deployer input in | 00:04 |
| weshay | $HOME/tempest-deployer-input.conf | 00:04 |
| mwhahaha | oh wait i need to disable the container tempest format | 00:04 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: use fs config to assign standalone role, tests https://review.openstack.org/572830 | 00:05 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add tempest to standalone https://review.openstack.org/575588 | 00:05 |
| mwhahaha | there we go, let's give that a shot | 00:05 |
| weshay | mwhahaha, did you guys choose clouds.yaml just to mess w/ us? | 00:06 |
| weshay | :) | 00:06 |
| * weshay runs | 00:06 | |
| *** sri_ has quit IRC | 00:06 | |
| mwhahaha | and by us you mean the openstack community? sure | 00:06 |
| weshay | it's fair.. they f. w/ us every day | 00:06 |
| *** sri_ has joined #tripleo | 00:06 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base https://review.openstack.org/596422 | 00:07 |
| rlandy | weshay: ^^ with mock-etc-nodepool | 00:08 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 00:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 00:10 |
| *** karthiks has joined #tripleo | 00:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 00:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 00:10 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: WIP: cell_v2 multi-cell POC https://review.openstack.org/600588 | 00:10 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: cell_v2 multi-cell POC https://review.openstack.org/600589 | 00:11 |
| mwhahaha | weshay: so what would it take to get some zuul rules for the featureset* files to only run the affected feature sets? I don't think they currently even trigger their own jobs | 00:12 |
| weshay | mwhahaha, we may be missing some, but they are there, /me gets | 00:12 |
| mwhahaha | weshay: well it would be beneficial if they also ran all the branched versions of the featuresets too :o | 00:13 |
| weshay | mwhahaha, oooh | 00:14 |
| weshay | that is a swell idea | 00:14 |
| weshay | mwhahaha, some are here https://github.com/openstack/tripleo-quickstart/blob/master/zuul.d/layout.yaml#L11-L50 | 00:14 |
| weshay | some are in rdo | 00:14 |
| weshay | I'll work on kicking off the branched jobs w/ changes to the fs | 00:15 |
| weshay | that is really good | 00:15 |
| mwhahaha | i ask because that'd be the preventative action for https://review.openstack.org/#/c/599088/ :D | 00:15 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: cell_v2 multi-cell POC https://review.openstack.org/600589 | 00:16 |
| *** sri_ has quit IRC | 00:29 | |
| *** sri_ has joined #tripleo | 00:30 | |
| *** threestrands has joined #tripleo | 00:32 | |
| *** threestrands has quit IRC | 00:32 | |
| *** threestrands has joined #tripleo | 00:32 | |
| *** sri_ has quit IRC | 00:33 | |
| *** sri_ has joined #tripleo | 00:33 | |
| *** spsurya has quit IRC | 00:35 | |
| *** sai_p has quit IRC | 00:41 | |
| *** spartakos has quit IRC | 00:45 | |
| *** ducnv has joined #tripleo | 00:47 | |
| *** med_ has quit IRC | 00:48 | |
| openstackgerrit | Merged openstack/ansible-role-container-registry master: Correctly set registry_pkg_manager https://review.openstack.org/600334 | 00:52 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Initial support for Podman in docker-puppet https://review.openstack.org/588655 | 00:54 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: rabbit: do not mount /etc/rabbitmq in RO https://review.openstack.org/598327 | 00:54 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: DNM - install podman rpm https://review.openstack.org/590587 | 00:54 |
| *** hamzy has joined #tripleo | 00:55 | |
| *** palkar has quit IRC | 00:57 | |
| *** palkar has joined #tripleo | 00:57 | |
| *** tzumainn has quit IRC | 00:59 | |
| *** phuongnh has joined #tripleo | 01:00 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Make vlan_transparent in neutron.conf configurable from Undercloud https://review.openstack.org/591904 | 01:04 |
| openstackgerrit | Merged openstack/instack-undercloud stable/ocata: import zuul job settings from project-config https://review.openstack.org/596086 | 01:04 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Rebase patch in reproducing as zuul does in CI https://review.openstack.org/599460 | 01:07 |
| openstackgerrit | Merged openstack/tripleo-image-elements stable/ocata: import zuul job settings from project-config https://review.openstack.org/596101 | 01:07 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: import zuul job settings from project-config https://review.openstack.org/596097 | 01:07 |
| *** lblanchard has joined #tripleo | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 01:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 01:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 01:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 01:10 |
| *** colonwq has joined #tripleo | 01:18 | |
| *** dtrainor has quit IRC | 01:19 | |
| *** saneax has joined #tripleo | 01:19 | |
| *** saneax has quit IRC | 01:20 | |
| *** sri_ has quit IRC | 01:27 | |
| *** sri_ has joined #tripleo | 01:27 | |
| *** saneax has joined #tripleo | 01:33 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Persist ceph-ansible fetch_directory using config-download https://review.openstack.org/582811 | 01:36 |
| *** gbarros has joined #tripleo | 01:38 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Persist ceph-ansible fetch_directory using config-download https://review.openstack.org/582811 | 01:45 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Update build-test-package for non-centos7 https://review.openstack.org/589540 | 01:48 |
| EmilienM | mwhahaha: I had to export HOST_NET in docker run, see https://review.openstack.org/588655 - and now it works fine :) thx for the reviews. | 01:49 |
| EmilienM | lol in podman run | 01:50 |
| EmilienM | I need an alias | 01:50 |
| EmilienM | it sounds like python tripleoclient unit tests fail in delorean: http://logs.openstack.org/99/600499/3/check/tripleo-ci-centos-7-undercloud-containers/d300958/logs/delorean_logs/c4/2b/c42b8e9321a1eebed5ebc8feed9df9945cf514d3_dev/root.log.txt.gz | 01:52 |
| EmilienM | weshay: do we have a bug for that already? | 01:53 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add support for NODEPOOL_CBS_CENTOS_PROXY variable https://review.openstack.org/599896 | 01:54 |
| mwhahaha | EmilienM I think there is a bug for the empty container thing, sounds like we're checking for function calls in tripleo-common in tripleoclient | 01:58 |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Update swift_rings_backup workflow to also backup ceph fetch dir https://review.openstack.org/597221 | 02:01 |
| *** apetrich_ has joined #tripleo | 02:04 | |
| openstackgerrit | Merged openstack/tripleo-docs master: config-download User's Guide https://review.openstack.org/599053 | 02:06 |
| *** apetrich has quit IRC | 02:07 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 02:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 02:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 02:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 02:10 |
| *** phuongnh has quit IRC | 02:16 | |
| *** phuongnh has joined #tripleo | 02:17 | |
| *** ramishra has joined #tripleo | 02:22 | |
| *** phuongnh has quit IRC | 02:28 | |
| *** gbarros has quit IRC | 02:28 | |
| *** gbarros has joined #tripleo | 02:30 | |
| *** gbarros has quit IRC | 02:40 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add support for OS_CLOUD https://review.openstack.org/600586 | 02:45 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add tempest to standalone https://review.openstack.org/575588 | 02:47 |
| *** psachin has joined #tripleo | 02:49 | |
| *** ykarel|away has joined #tripleo | 02:54 | |
| mwhahaha | weshay, EmilienM -^ I think that'll get tempest working on the standalone | 02:55 |
| * mwhahaha gives up | 02:55 | |
| *** med_ has joined #tripleo | 02:57 | |
| *** sri_ has quit IRC | 03:01 | |
| *** sri_ has joined #tripleo | 03:01 | |
| openstackgerrit | yatin proposed openstack/tripleo-quickstart master: Use containers-prepare-parameters in upgrades/updates jobs. https://review.openstack.org/599605 | 03:09 |
| *** lblanchard has quit IRC | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 03:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 03:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 03:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 03:10 |
| *** ykarel|away is now known as ykarel | 03:11 | |
| *** sri_ has quit IRC | 03:14 | |
| *** sri_ has joined #tripleo | 03:15 | |
| *** gkadam has joined #tripleo | 03:21 | |
| *** sri_ has quit IRC | 03:24 | |
| *** sri_ has joined #tripleo | 03:25 | |
| openstackgerrit | Steve Baker proposed openstack/ansible-role-tripleo-modify-image master: Don't yum update files which are not installed https://review.openstack.org/599916 | 03:29 |
| openstackgerrit | Steve Baker proposed openstack/ansible-role-tripleo-modify-image master: Don't use copy module for /etc/yum.repos.d https://review.openstack.org/599917 | 03:29 |
| openstackgerrit | Steve Baker proposed openstack/ansible-role-tripleo-modify-image master: Cleanup modify directory https://review.openstack.org/599918 | 03:29 |
| openstackgerrit | Steve Baker proposed openstack/ansible-role-tripleo-modify-image master: Remove compare_host_packages strategy https://review.openstack.org/600273 | 03:29 |
| *** sri_ has quit IRC | 03:30 | |
| *** sri_ has joined #tripleo | 03:30 | |
| *** med_ has quit IRC | 03:31 | |
| *** skramaja has joined #tripleo | 03:31 | |
| *** med_ has joined #tripleo | 03:51 | |
| Tengu | hello there | 03:59 |
| *** pdeore has joined #tripleo | 04:00 | |
| weshay | Tengu, working too much.. are you going to ptg? | 04:03 |
| Tengu | weshay: have to get some hours done - won't be able to work during the day today, and I didn't work yesterday either ;). | 04:04 |
| Tengu | weshay: and unfortunately, nope can't go to the PTG :(. | 04:04 |
| *** jaganathan has joined #tripleo | 04:04 | |
| *** udesale has joined #tripleo | 04:05 | |
| weshay | hrm.. that's too bad I wanted to chat w/ you about the provisioning repo you have | 04:05 |
| Tengu | weshay: if it were in UE, I could go on my own, but it's a bit more expensive crossing the ocean | 04:05 |
| weshay | another time | 04:05 |
| weshay | aye | 04:05 |
| Tengu | ah, the tripleo-lab? | 04:05 |
| weshay | ya | 04:05 |
| Tengu | eh :). ppl are interested in it apparently. | 04:06 |
| weshay | ya.. as am i | 04:06 |
| weshay | I have to figure out a short term and long term goal | 04:06 |
| weshay | here | 04:06 |
| Tengu | want to chat a bit about it now? | 04:06 |
| weshay | short term goal is something like tripleo-lab, long term goal would be something along the lines of a nodepool plugin | 04:07 |
| weshay | Tengu, I need to mess around with a bit, I've only browsed through it | 04:07 |
| Tengu | nodepool plugin? | 04:07 |
| weshay | ya | 04:07 |
| weshay | nodepool libvirt plugin | 04:07 |
| Tengu | what's that? duck talk about jenkins, so I guess it's related ? | 04:08 |
| weshay | unrelated to jenkins entirely | 04:08 |
| Tengu | duh | 04:08 |
| Tengu | bad duck ^^ | 04:08 |
| *** redrobot has quit IRC | 04:08 | |
| weshay | nodepool is the upstream tool for provisioning nodes from an openstack cloud | 04:08 |
| Tengu | ah, thatone: https://github.com/openstack-infra/nodepool | 04:08 |
| weshay | aye | 04:08 |
| weshay | so we can keep coming up w/ one of repos to provision a test env... | 04:09 |
| weshay | or we can push infra into supporting a local libvirt based way to stand up nodes in the same way ci does upstream | 04:09 |
| Tengu | wait, you're telling me my small private work might be integrated in the upstream CI (with some modifications) ?! | 04:09 |
| weshay | no | 04:09 |
| weshay | well | 04:09 |
| weshay | maybe | 04:09 |
| Tengu | wow | 04:09 |
| weshay | I mean lots of folks have a local libvirt provisioner | 04:10 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 04:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 04:10 |
| weshay | that is not the hard part | 04:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 04:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 04:10 |
| Tengu | can imagine - I'm not the only one in need of a small thing for libvirt provisionning in a human way :) | 04:10 |
| weshay | Tengu, the idea would be to find interested parties to contribute to something that can be used generally to bring upstream ci to local libvirt environments | 04:11 |
| *** spsurya has joined #tripleo | 04:11 | |
| Tengu | oooq is nice, but as it's been discussed, it's more for automated things. | 04:11 |
| Tengu | weshay: know what? I was thinking about a way to inject packages built from reviews... | 04:11 |
| *** redrobot has joined #tripleo | 04:12 | |
| weshay | things can always improve.. however if we could get kolla, openstack-ansible, tripleo on board w/ a common way to libvirt provision environments then we have something | 04:12 |
| Tengu | as for now it's a bit manual. | 04:12 |
| weshay | any upstream project that uses more than one node, that is looking for something other than vagrant | 04:12 |
| weshay | may be interested | 04:12 |
| Tengu | gettin a common way is better in order to bug squash issues, whatever they are. | 04:13 |
| weshay | ya | 04:13 |
| *** pgadiya has joined #tripleo | 04:13 | |
| weshay | tough sell | 04:13 |
| *** sri_ has quit IRC | 04:13 | |
| Tengu | :) | 04:13 |
| weshay | but worth investigating | 04:13 |
| *** sri_ has joined #tripleo | 04:13 | |
| Tengu | well, I'm pleased to hear that. As it was kind of thrown on the table as an exercise for my ansible non-skills... That's pretty cool | 04:13 |
| weshay | you walked through the whole thing yourself w/ tripleo-lab so you have good insight | 04:13 |
| Tengu | and it's really simple in addition. | 04:14 |
| weshay | ya.. it's not a terribly complex problem, the hardest part would be to build a community behind "A" thing | 04:15 |
| Tengu | yup | 04:15 |
| *** rlandy has quit IRC | 04:16 | |
| *** med_ has quit IRC | 04:17 | |
| *** pdeore_ has joined #tripleo | 04:17 | |
| Tengu | weshay: still, I'll be up late next week (well, EMEA late), as I'll get a few talk to do remotely. So we can always find a time for some bluejeans if you want. | 04:18 |
| Tengu | for sure it would be best to meet face to face, but... | 04:19 |
| *** pdeore has quit IRC | 04:19 | |
| weshay | Tengu, ya.. I've been chatting about something like this w/ Paul Belenger for a bit.. and some other folks | 04:19 |
| weshay | just need to a little discussion .. on short term, long term goals | 04:20 |
| Tengu | yep. so ping me next week - I'll also be "late EMEA" today. | 04:22 |
| Tengu | so maybe we can discuss your tomorrow :). | 04:22 |
| *** ykarel has quit IRC | 04:24 | |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-heat-templates stable/queens: DNM Testing f132d6aae16ed85f637b54067a031e21d538acda https://review.openstack.org/600560 | 04:28 |
| *** spartakos has joined #tripleo | 04:28 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: add branched versions of scen01-04 jobs https://review.openstack.org/600641 | 04:29 |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-heat-templates stable/queens: DNM: Debugging scenario004 https://review.openstack.org/600642 | 04:30 |
| *** Petersingh has joined #tripleo | 04:30 | |
| *** Petersingh is now known as Petersingh|afk | 04:32 | |
| *** shreshtha has joined #tripleo | 04:33 | |
| *** psachin has quit IRC | 04:34 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: trigger branched version of each scenario job https://review.openstack.org/600643 | 04:35 |
| *** sri_ has quit IRC | 04:35 | |
| *** sri_ has joined #tripleo | 04:36 | |
| *** Petersingh|afk is now known as Petersingh | 04:36 | |
| *** pdeore__ has joined #tripleo | 04:37 | |
| *** psachin has joined #tripleo | 04:37 | |
| *** pdeore_ has quit IRC | 04:40 | |
| *** Petersingh is now known as Petersingh|brb | 04:40 | |
| *** ykarel has joined #tripleo | 04:44 | |
| *** sri_ has quit IRC | 04:47 | |
| Tengu | so, going offline. see you in a few hours :). | 04:47 |
| *** sri_ has joined #tripleo | 04:47 | |
| *** psachin has quit IRC | 04:51 | |
| *** psachin has joined #tripleo | 04:55 | |
| *** rcernin has quit IRC | 04:57 | |
| *** spartakos has quit IRC | 04:57 | |
| *** ratailor has joined #tripleo | 05:00 | |
| *** Petersingh|brb is now known as Petersingh | 05:06 | |
| *** psachin` has joined #tripleo | 05:08 | |
| *** psachin has quit IRC | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 05:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 05:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 05:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 05:10 |
| *** spartakos has joined #tripleo | 05:12 | |
| *** shyamb has joined #tripleo | 05:13 | |
| *** shyamb has quit IRC | 05:17 | |
| *** ssbarnea|bkp has quit IRC | 05:22 | |
| *** yprokule has joined #tripleo | 05:24 | |
| *** sri_ has quit IRC | 05:28 | |
| *** sri_ has joined #tripleo | 05:28 | |
| *** yprokule has quit IRC | 05:30 | |
| *** rcernin has joined #tripleo | 05:30 | |
| *** sanjayu_ has joined #tripleo | 05:33 | |
| *** saneax has quit IRC | 05:33 | |
| *** shyamb has joined #tripleo | 05:34 | |
| *** paramite has joined #tripleo | 05:35 | |
| *** sri_ has quit IRC | 05:37 | |
| *** sri_ has joined #tripleo | 05:38 | |
| *** quiquell has joined #tripleo | 05:45 | |
| *** gkadam_ has joined #tripleo | 05:46 | |
| *** gkadam has quit IRC | 05:48 | |
| *** psachin` has quit IRC | 05:50 | |
| *** brault has joined #tripleo | 05:51 | |
| *** dmacpher has quit IRC | 05:58 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates stable/queens: DNM: To test manila micro version in queens https://review.openstack.org/567224 | 05:59 |
| *** ratailor has quit IRC | 06:09 | |
| *** artom has joined #tripleo | 06:09 | |
| *** ratailor has joined #tripleo | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 06:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 06:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 06:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 06:10 |
| *** cshastri has joined #tripleo | 06:10 | |
| *** pcaruana has joined #tripleo | 06:13 | |
| *** lvdombrkr has joined #tripleo | 06:19 | |
| *** shyamb has quit IRC | 06:24 | |
| *** shyamb has joined #tripleo | 06:24 | |
| *** lvdombrkr89 has joined #tripleo | 06:24 | |
| *** donghm has quit IRC | 06:26 | |
| *** lvdombrkr has quit IRC | 06:27 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Refactor openshift services for composable roles https://review.openstack.org/599618 | 06:27 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Fix openshift new node detection https://review.openstack.org/600012 | 06:27 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Clean up previous osa inventory dir before deployment https://review.openstack.org/600028 | 06:27 |
| *** lvdombrkr89 has quit IRC | 06:28 | |
| *** lvdombrkr has joined #tripleo | 06:28 | |
| *** holser_ has joined #tripleo | 06:29 | |
| *** ksambor has joined #tripleo | 06:31 | |
| *** cylopez has joined #tripleo | 06:32 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 06:33 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates stable/queens: DNM: To test tempest manila miniversion https://review.openstack.org/600662 | 06:35 |
| *** quiquell is now known as quique|rover|bbl | 06:36 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: iscsi: workaround to let podman read /dev/null https://review.openstack.org/600544 | 06:37 |
| *** sanjayu_ has quit IRC | 06:40 | |
| *** aufi has joined #tripleo | 06:47 | |
| *** jfrancoa has joined #tripleo | 06:49 | |
| *** jtomasek has joined #tripleo | 06:59 | |
| *** shyamb has quit IRC | 07:00 | |
| openstackgerrit | Merged openstack/tripleo-common master: Refactor non-docker upload functions into base class https://review.openstack.org/590084 | 07:05 |
| *** quique|rover|bbl has quit IRC | 07:06 | |
| *** quiquell has joined #tripleo | 07:06 | |
| *** quiquell has quit IRC | 07:07 | |
| *** quiquell|rover has joined #tripleo | 07:07 | |
| *** quiquell|rover has quit IRC | 07:07 | |
| *** quiquell|rover has joined #tripleo | 07:08 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 07:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 07:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 07:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 07:10 |
| *** shyamb has joined #tripleo | 07:10 | |
| *** abishop has quit IRC | 07:11 | |
| *** f_ has joined #tripleo | 07:11 | |
| *** f_ is now known as florianf | 07:11 | |
| *** quiquell|rover has quit IRC | 07:11 | |
| *** quiquell has joined #tripleo | 07:12 | |
| *** quiquell is now known as quiquell|rover | 07:12 | |
| *** ccamacho has joined #tripleo | 07:17 | |
| *** abishop has joined #tripleo | 07:17 | |
| openstackgerrit | Tony Breeds proposed openstack/tripleo-common master: Handle missing or bad dmidecode https://review.openstack.org/599920 | 07:18 |
| *** akrivoka has joined #tripleo | 07:23 | |
| *** rdopiera has joined #tripleo | 07:25 | |
| *** tesseract has joined #tripleo | 07:27 | |
| *** palkar has quit IRC | 07:33 | |
| *** palkar has joined #tripleo | 07:33 | |
| *** shyamb has quit IRC | 07:34 | |
| *** Petersingh is now known as Petersingh|lunch | 07:40 | |
| *** dtantsur|afk is now known as dtantsur | 07:46 | |
| *** ipsecguy_ has joined #tripleo | 07:47 | |
| *** radeks has joined #tripleo | 07:50 | |
| quiquell|rover | jfrancoa: https://review.openstack.org/#/c/599605/ is a master promotion blocker, let's try to get merged | 07:50 |
| *** jpena|off is now known as jpena | 07:51 | |
| quiquell|rover | chkumar|out: ^ | 07:51 |
| jfrancoa | quiquell|rover: ack | 07:52 |
| quiquell|rover | jfrancoa: Something bad with OVB not infra related http://logs.rdoproject.org/05/599605/9/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/0ec085d/logs/undercloud/var/lib/mistral/overcloud/ansible.log.txt.gz | 07:56 |
| quiquell|rover | jq: error: Could not open file /var/lib/heat-config/deployed/c6ef6b9a-87bd-47e8-9af4-8ccbec33560f.notify.json: No such file or directory | 07:57 |
| quiquell|rover | Looks like it has problem deploying the compute node | 07:57 |
| *** ramishra has quit IRC | 07:58 | |
| *** tosky has joined #tripleo | 07:59 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 08:00 |
| *** sri_ has quit IRC | 08:00 | |
| *** jpich has joined #tripleo | 08:01 | |
| *** sri_ has joined #tripleo | 08:01 | |
| *** psahoo has joined #tripleo | 08:02 | |
| hjensas | marios: Is there a common pattern to ensure a service is restarted based on a different resource change? I think we need to make sure keepalived is restarted during update if os-net-config was changed. Atleast for the undercloud not to regress, see https://bugs.launchpad.net/tripleo/+bug/1791238 ? | 08:03 |
| openstack | Launchpad bug 1791238 in tripleo "Undercloud - changing os-net-config conf kills undercloud_[admin, public]_host IPs" [High,Triaged] | 08:03 |
| quiquell|rover | marios: Do you know wher to look at with this mistral failure ? http://logs.rdoproject.org/05/599605/9/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/0ec085d/logs/undercloud/var/lib/mistral/overcloud/ansible.log.txt.gz | 08:03 |
| marios | quiquell|rover: thats nasty | 08:04 |
| quiquell|rover | marios: The deployment uuid c6ef6b9a-87bd-47e8-9af4-8ccbec33560f does not exists, I suppose the compute has not deploy | 08:04 |
| marios | quiquell|rover: not without digging but i don'tknow if mistral is the root its just the workflow which fails but the error is like "jq: error: Could not open file /var/lib/heat-config/deployed/c6ef6b9a-87bd-47e8-9af4-8ccbec33560f.notify.json: No such file or directory" | 08:04 |
| quiquell|rover | marios: And this is that the check | 08:04 |
| quiquell|rover | marios: So the root cause is the compute node not deploying ? | 08:05 |
| quiquell|rover | marios: Maybe ironic has something to say | 08:05 |
| quiquell|rover | marios, jfrancoa: found this http://logs.rdoproject.org/05/599605/9/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/0c5b28e/logs/undercloud/var/log/containers/ironic/app.log.txt.gz | 08:06 |
| quiquell|rover | Server-side error: "'NoneType' object has no attribute '__getitem__'". Detail: | 08:06 |
| quiquell|rover | Traceback (most recent call last): | 08:06 |
| marios | hjensas: usually services are restarted 'as needed' i.e. we have individual/special cases in the update_tasks | 08:06 |
| marios | hjensas: we don't have some common pattern for handling that cc chem jistr (hjensas asking about "Is there a common pattern to ensure a service is restarted based on a different resource change?" | 08:07 |
| *** shardy has joined #tripleo | 08:08 | |
| marios | ack quiquell|rover its a new one? is it happening a lot | 08:08 |
| quiquell|rover | marios: Checking it's happening elsewhere | 08:09 |
| *** chem has quit IRC | 08:09 | |
| *** slaweq_ is now known as slaweq | 08:10 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 08:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 08:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 08:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 08:10 |
| *** ssbarnea|bkp has joined #tripleo | 08:11 | |
| quiquell|rover | marios: Do you see it elsewhere, I don't | 08:11 |
| marios | quiquell|rover: no haven't come across it | 08:12 |
| marios | quiquell|rover: thats the thing there are errors that are real but then only happen once | 08:12 |
| *** ykarel is now known as ykarel|lunch | 08:13 | |
| *** ramishra has joined #tripleo | 08:14 | |
| quiquell|rover | jfrancoa: Going to recheck RDO, I haven't see it before in the review, but put a -1 until we are sure | 08:16 |
| jfrancoa | quiquell|rover: I have the feeling that this patch is solving some error where this job legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master got stuck before and now we get into this new one | 08:18 |
| *** spartakos has quit IRC | 08:18 | |
| quiquell|rover | jfrancoa: Yep... that happends when we don't promote for some time... we are hidding stuff | 08:19 |
| jfrancoa | quiquell|rover: similar thing has happened with tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates, all the Depends-On patches were intented to unlock the job, but each time a new thing appears (that's the problem of having it non-voting, lots of stuff can be merged which break it) | 08:19 |
| quiquell|rover | jfrancoa: Maybe we can try to merge this to unlock some promotion jobs failing at the default yaml missing | 08:20 |
| quiquell|rover | jfrancoa: And continue | 08:20 |
| quiquell|rover | jfrancoa: Or try to si if the OVB issue is really related to the review (I suspect no) | 08:20 |
| jfrancoa | quiquell|rover: I think it's a good option, let me check other rdo jobs from different patches to see if these RDO jobs fail too | 08:20 |
| quiquell|rover | jfrancoa: I haven't find any | 08:21 |
| quiquell|rover | jfrancoa: This is one of the last reviews, and the job is passing https://review.openstack.org/#/c/572830/ | 08:22 |
| quiquell|rover | jfrancoa: That's why I feel it's related | 08:22 |
| jfrancoa | quiquell|rover: right, in other executions is passing...mmm..suspicious | 08:23 |
| quiquell|rover | jfrancoa: yep | 08:23 |
| jistr | marios, hjensas: re restarting services on resource change -- what in particular do we mean by "resource change"? | 08:24 |
| hjensas | marios: jistr: see https://bugs.launchpad.net/tripleo/+bug/1791238 | 08:25 |
| openstack | Launchpad bug 1791238 in tripleo "Undercloud - changing os-net-config conf kills undercloud_[admin, public]_host IPs" [High,Triaged] | 08:25 |
| hjensas | marios: jistr: prior to containerizing the undercloud changing things like undercloud_nameservers and re-running undercloud installer was'nt any issue. | 08:25 |
| hjensas | marios: jistr: Now, chaning the dnsserver cause os-net-config to down|up br-ctlplane and the keepalived vips are gone. So I guess we need to restart keepalived, and possibly a whole bunch of services that may no longer bind to the address? ... | 08:27 |
| *** Petersingh|lunch is now known as Petersingh | 08:28 | |
| jistr | hjensas: so the old keepalived vips are gone and we get new ones (different addresses)? | 08:31 |
| hjensas | jistr: the old addresses are still in etc/keepalived/keepalived.conf , but they are not there when doing ip addr show | 08:32 |
| jistr | hjensas: ack i understand now, it's just that they disappear, after we restart keepalived they'd re-appear with the same address, and then we need to restart the rest of the services which listened on them? | 08:33 |
| *** threestrands has quit IRC | 08:33 | |
| jfrancoa | quiquell|rover: it's really strange because the main patch doesn't modify featureset035 and all the job passes in all Depends-On patches...I don't understand how this change can impact in ironic | 08:33 |
| jfrancoa | quiquell|rover: should we recheck rdo to ensure? | 08:33 |
| quiquell|rover | jfrancoa: Now that RDO is at zuulv3 I think normal recheck also recheck's RDO | 08:34 |
| quiquell|rover | Yatin's rechecks have trigger RDO too | 08:34 |
| quiquell|rover | jfrancoa: btw we do modify featureset037 https://review.openstack.org/#/c/599605/9/config/general_config/featureset037.yml | 08:35 |
| jistr | hjensas: i don't think we have a pattern for that presently. (Even with old non-containerized undercloud/overcloud, i think we only restarted services on config change, we didn't watch for things like down/up of VIPs.) | 08:35 |
| jfrancoa | quiquell|rover: but not featuset035 which is the job you passed the log from | 08:35 |
| *** palkar has quit IRC | 08:36 | |
| quiquell|rover | jfrancoa: Ahh hehe make sense | 08:36 |
| *** palkar has joined #tripleo | 08:36 | |
| hjensas | jistr: yes, restarting keepalived container adds the ips again - see: https://etherpad.openstack.org/p/bug-1791238 | 08:37 |
| hjensas | jistr: not sure if we also need to restart other services as well. | 08:37 |
| *** sanjayu_ has joined #tripleo | 08:37 | |
| quiquell|rover | jfrancoa: Depends-On are not passing | 08:37 |
| quiquell|rover | jfrancoa: Maybe we can try to merge this to unlock some promotion jobs failing at the default yaml missinghttps://review.openstack.org/#/c/600467/ | 08:37 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: [WIP] Assert workloads are spawned in different stages https://review.openstack.org/600674 | 08:38 |
| quiquell|rover | jfrancoa: Same ironic issue http://logs.rdoproject.org/67/600467/1/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/d300458/logs/undercloud/var/log/containers/ironic/app.log.txt.gz | 08:38 |
| hjensas | jistr: I guess for services we had puppet dependendecies makeing sure things got restarted. But the os-net-config is bash ... :-/ | 08:38 |
| quiquell|rover | jfrancoa: here the issue is a overcloud-prep-images puff... | 08:39 |
| quiquell|rover | jfrancoa: The ironic issue http://logs.rdoproject.org/67/600467/1/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/d300458/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz | 08:40 |
| *** derekh has joined #tripleo | 08:40 | |
| hjensas | jistr: looks like just bumping keepalived is enough. Once I did that [Start containers for step 3] continued and the update is completing it seems. | 08:41 |
| jfrancoa | quiquell|rover: yes, it's exactly what I was checking. But seems related to ironic, in this case it does.. | 08:41 |
| quiquell|rover | jfrancoa: for OVB featureset001 is doing an multinode-overcloud-update ? | 08:41 |
| * hjensas goes to update the bug ... | 08:41 | |
| jfrancoa | quiquell|rover: I don't think so, updates job is fs037 | 08:42 |
| *** ssbarnea|bkp has quit IRC | 08:43 | |
| quiquell|rover | jfrancoa: Ha to be a new promotion, but I don't see it at any review :-( | 08:44 |
| *** shyamb has joined #tripleo | 08:45 | |
| *** pdeore__ has quit IRC | 08:45 | |
| jistr | hjensas: ok cool. So i wonder, how much we want to support this "automatically out of the box". Historically AFAIK we haven't really supported changing network config and having things to just continue working without manual intervention. But one could also argue that DNS server change is not a big deal and we should survive it. So i'm a bit on a fence... Perhaps we could have some "dirty flag" mechanism for | 08:45 |
| jistr | os-net-config which other services could observe. | 08:45 |
| *** pdeore__ has joined #tripleo | 08:46 | |
| quiquell|rover | jfrancoa: going to recheck the Depends-On too | 08:46 |
| jistr | hjensas: wondering how we'd react to it though. We have update_tasks/upgrade_tasks, but we don't have deploy_tasks AFAIK. | 08:47 |
| *** psachin has joined #tripleo | 08:47 | |
| hjensas | jistr: it's blocking scale up for routed networks as well. I.e if we add more subnets on the ctlplane in the undercloud, we need to add routes to the ctlplane interface ... this also bumps br-ctlplane. | 08:47 |
| hjensas | jistr: we need to revist https://review.openstack.org/152732 | 08:49 |
| *** ykarel|lunch is now known as ykarel | 08:49 | |
| quiquell|rover | jfrancoa: Did you see this at any other job ? https://bugs.launchpad.net/tripleo/+bug/1791115 | 08:50 |
| openstack | Launchpad bug 1791115 in tripleo "[ERROR] /opt/stack/new/devstack/functions-common:Detected fatal at RDO jobs" [Critical,New] - Assigned to Quique Llorente (quiquell) | 08:50 |
| hjensas | jistr: but for now I think we need a workaround that can be backported to Rocky. | 08:51 |
| jfrancoa | quiquell|rover: sure, in the very same patch we're trying to merge, wait I'll send you the logs | 08:51 |
| quiquell|rover | jfrancoa: Humm don't see it | 08:51 |
| jfrancoa | quiquell|rover: https://logs.rdoproject.org/05/599605/9/openstack-check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades-pike-branch/dddff4d/ | 08:51 |
| jfrancoa | quiquell|rover: https://logs.rdoproject.org/05/599605/9/openstack-check/legacy-tripleo-ci-centos-7-container-to-container-upgrades-master/3cb295e/ | 08:52 |
| quiquell|rover | Ahh they are non voting | 08:52 |
| jfrancoa | quiquell|rover: in these two jobs, check undercloud/home/zuul/vxlan_networking.sh.log | 08:52 |
| quiquell|rover | going to remove the critical from the bug | 08:52 |
| jistr | hjensas: yea that would be good re non-restarting changes. Alternative solution i'm thinking of right now (perhaps backportable), we could write a timestamp of when os-net-config last made some changes into something like /etc/puppet/hieradata/os-net-config.yaml, as a hiera value, and then we could write that out in docker-puppet which generates config for keepalived container, to some dummy file anywhere in | 08:53 |
| jistr | /etc. That would change the md5 checksum of the keepalived config volume, and Paunch would trigger a restart of keepalived container. | 08:53 |
| jistr | hjensas: it's a bit hacky though :) | 08:53 |
| *** ssbarnea|bkp has joined #tripleo | 08:54 | |
| hjensas | jistr: yup, that should work. Let me do some research on keepalived, maby it can monitor itself and put the ip's back up if the interface underneath was restarted. | 08:54 |
| *** colonwq has quit IRC | 08:55 | |
| jistr | hjensas: yea that might be nicer. Also a sidecar container running a bash script watch for this and do the restart would be possible, if keepalived can't do it by itself. | 08:55 |
| *** palkar has quit IRC | 08:56 | |
| *** palkar has joined #tripleo | 08:56 | |
| jistr | hjensas: re sidecar container, we could use this to write the script https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/nova-api.yaml#L145-L148 | 08:59 |
| hjensas | jistr: # Allow configuration to include interfaces that don't exist at startup. | 08:59 |
| hjensas | # This allows keepalived to work with interfaces that may be deleted and restored | 08:59 |
| hjensas | # and also allows virtual and static routes and rules on VMAC interfaces. | 08:59 |
| hjensas | dynamic_interfaces | 08:59 |
| hjensas | jistr: looks promising ... | 08:59 |
| jistr | hjensas: yea, nice | 08:59 |
| hjensas | jistr: thanks for the help. I'll try ^^. | 08:59 |
| jistr | sure thing ;) | 09:00 |
| dtantsur | shardy: hi! any chances you get to https://review.openstack.org/576856 again before the PTG? | 09:03 |
| shardy | dtantsur: Hi! Yes sure I'll try to get to it today | 09:03 |
| dtantsur | cool :) | 09:03 |
| shardy | dtantsur: sorry been a bit distracted with OpenShift things lately | 09:03 |
| shardy | but would be good to make progress on that | 09:03 |
| dtantsur | yeah | 09:05 |
| dtantsur | shardy: if there'll be a session on it, I can try answering questions in advance, if people have any | 09:06 |
| quiquell|rover | jfrancoa: #openstack-ironic commenting in the issue | 09:09 |
| shardy | dtantsur: ack - I added it as a topic, perhaps I can loop you in remotely if you're available | 09:09 |
| dtantsur | shardy: we can try. do you know the day and time? | 09:09 |
| shardy | mostly I wanted to raise visibility to get more folks to help with review/testing and figuring out the long term plan | 09:09 |
| shardy | dtantsur: yes sec there's an agenda | 09:10 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 09:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791191 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| shardy | hjensas: Hey will you be at the PTG next week? | 09:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 09:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 09:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1791191 in tripleo "instances stuck in node_failure for 9+ hours" [Critical,Triaged] | 09:10 |
| shardy | https://etherpad.openstack.org/p/tripleo-ptg-stein | 09:10 |
| hjensas | shardy: yes I will. | 09:10 |
| shardy | hjensas: great, it'd be good if you're able to help with the discussions re novaless deploy, thanks for your comments on the review | 09:11 |
| shardy | dtantsur: it looks like it'll be Thursday afternoon, but we can perhaps move the session if needed to fit better with your tz | 09:12 |
| dtantsur | shardy: 14:00 Denver is 10pm here :( | 09:12 |
| hjensas | shardy: yes, look forward to it. | 09:12 |
| shardy | dtantsur: yeah, lets chat with jaosorior about potentially moving it into the edge session | 09:12 |
| dtantsur | shardy: given that I have errands that evening, I can only be online 9am-10am Denver time | 09:14 |
| dtantsur | maybe we should just have a separate call | 09:14 |
| dtantsur | after the PTG | 09:14 |
| shardy | dtantsur: Ok yeah maybe we can sync after the session, I'll keep notes | 09:14 |
| *** rcernin has quit IRC | 09:16 | |
| quiquell|rover | jfrancoa: Latest periodic for OVB + featureset001 is working fine http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/66c220b/ | 09:27 |
| quiquell|rover | :-/ | 09:27 |
| *** sanjayu_ has quit IRC | 09:31 | |
| *** wxy-xiyuan has left #tripleo | 09:32 | |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Set container_images_file for containerized undercloud. https://review.openstack.org/584286 | 09:39 |
| *** yprokule has joined #tripleo | 09:39 | |
| *** ratailor_ has joined #tripleo | 09:39 | |
| *** ratailor has quit IRC | 09:40 | |
| openstackgerrit | Andreas Jaeger proposed openstack/tripleo-repos master: Use openstack-tox-cover template https://review.openstack.org/600682 | 09:48 |
| *** shyamb has quit IRC | 09:54 | |
| *** colonwq has joined #tripleo | 09:54 | |
| *** jtomasek has quit IRC | 09:55 | |
| *** shyamb has joined #tripleo | 09:59 | |
| *** shyamb has quit IRC | 10:04 | |
| *** gfidente has joined #tripleo | 10:07 | |
| *** devep has joined #tripleo | 10:08 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 10:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 10:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790988 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 10:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 10:10 |
| openstack | Launchpad bug 1790988 in tripleo ""packet_write_wait: Connection to port 22: Broken pipe\r\nrsync: connection unexpectedly closed" [Critical,Triaged] | 10:10 |
| *** Petersingh is now known as Petersingh|afk | 10:11 | |
| *** morazi has quit IRC | 10:11 | |
| *** pdeore__ has quit IRC | 10:12 | |
| *** pdeore__ has joined #tripleo | 10:12 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Improve output of vxlan networking tasks https://review.openstack.org/600039 | 10:13 |
| ssbarnea|bkp | jistr: if you can review https://review.openstack.org/#/c/583984/ it would be great (bug explains it in details and this is not the first openstack to use pre-commit). thanks. | 10:19 |
| owalsh | dtantsur: not going to the PTG? | 10:24 |
| dtantsur | owalsh: nope | 10:25 |
| owalsh | dtantsur: damn, was hoping to talk to you about some ironic-y stuff | 10:26 |
| hjensas | dtantsur: did you figure out an easy way to quickly change something in a container? (I tried exec into the running container and changed the configt, but on restarting the container it did'nt persist). | 10:27 |
| dtantsur | owalsh: sorry :( | 10:28 |
| owalsh | dtantsur: np, nothing urgent | 10:28 |
| dtantsur | hjensas: nope, I used https://etherpad.openstack.org/p/tripleo-rpm-container-image-notes but with limited success | 10:28 |
| quiquell|rover | Guys we need some reviewing here https://review.openstack.org/#/c/600407/ to close a promotion blocker bug. | 10:29 |
| owalsh | hjensas: which file? that works for me when hacking nova .py file | 10:29 |
| *** shyamb has joined #tripleo | 10:29 | |
| owalsh | hjensas: conf files are likely to get clobbered by kolla-config though, need to modifiy them in /var/iib/config-data | 10:30 |
| *** ykarel is now known as ykarel|session | 10:31 | |
| hjensas | owalsh: hm, I actually tried that first. Edit /var/lib/config-data/keepalived/etc/keepalived/keepalived.conf, then docker container restart <container-id>. When I exec into the container the config under /etc inside did not have my edit. | 10:31 |
| jistr | ssbarnea|bkp: looks good code-wise. Personally i don't use pre-commit hooks but i think it's fine to have it configured. I gave +1, i'll leave +2s to folks who submit patches to oooq-e more often. | 10:32 |
| owalsh | hjensas: https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/keepalived.yaml#L101 | 10:33 |
| owalsh | hjensas: needs to be that one ^^^ | 10:33 |
| ssbarnea|bkp | jistr sure, thanks. to be clear: nobody is forced to use them locally, freedom is kept. | 10:34 |
| hjensas | owalsh: that did it! Thanks! | 10:35 |
| owalsh | hjensas: np | 10:35 |
| quiquell|rover | gouthamr: Looks like the manila miniversion fix is not working | 10:36 |
| *** shyamb has quit IRC | 10:36 | |
| quiquell|rover | gouthamr: At a test review queens is still failing | 10:36 |
| quiquell|rover | gouthamr: https://review.openstack.org/#/c/600662/ | 10:36 |
| jistr | ssbarnea|bkp: cool :) | 10:37 |
| tbarron | quiquell|rover: gouthamr is on the west coast of us, pls. see my remark on that review - still using the old maniila-tests package I think :( | 10:37 |
| quiquell|rover | tbarron: didn't we merge it ? | 10:38 |
| tbarron | quiquell|rover: yes, why isn't it getting picked up? | 10:38 |
| *** radeks_ has joined #tripleo | 10:38 | |
| tbarron | quiquell|rover: promotion delay in rdo? | 10:38 |
| quiquell|rover | tbarron: This is going to be chicken egg | 10:40 |
| tbarron | quiquell|rover: yeah, seems like it | 10:40 |
| *** radeks has quit IRC | 10:41 | |
| quiquell|rover | not promoting because the issue so version cannot be pump :-/ | 10:41 |
| tbarron | quiquell|rover: the periodic job was using the right version though, do we have one of those with goutham's change? | 10:41 |
| *** devep has quit IRC | 10:42 | |
| *** ratailor__ has joined #tripleo | 10:42 | |
| quiquell|rover | tbarron: The problem yep but goutham cannot be merge, becasuse the version itself is not yet promoted :-/ | 10:43 |
| quiquell|rover | That's what I mean | 10:43 |
| quiquell|rover | tbarron: Will talk with Wes about commenting promotion criteria so we get the version bumped | 10:43 |
| tbarron | quiquell|rover: sure, thanks | 10:44 |
| *** ratailor_ has quit IRC | 10:45 | |
| ssbarnea|bkp | jistr or tbarron : please check https://review.openstack.org/#/c/600407/ -- fixing a "id: stack: no such user" error we seen recently. | 10:48 |
| *** ramishra_ has joined #tripleo | 10:51 | |
| *** jtomasek has joined #tripleo | 10:53 | |
| *** ramishra has quit IRC | 10:53 | |
| *** abishop has quit IRC | 10:58 | |
| *** abishop has joined #tripleo | 10:59 | |
| *** hkominos has joined #tripleo | 11:01 | |
| hkominos | Hi all. Is this the right channel to ask about tripleo-heat-templates ? | 11:01 |
| *** jpena is now known as jpena|lunch | 11:01 | |
| *** akrivoka has quit IRC | 11:03 | |
| *** akrivoka has joined #tripleo | 11:03 | |
| quiquell|rover | tbarron: This is different ? https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset018-queens/e83641e/logs/undercloud/home/zuul/tempest/tempest.html.gz | 11:04 |
| quiquell|rover | tbarron: now promotion featureset018 is failing :-/ changing criteria will not work | 11:04 |
| quiquell|rover | tbarron: but this can be related ? | 11:06 |
| quiquell|rover | https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/3ad2015/logs/undercloud/home/zuul/tempest/tempest.html.gz | 11:06 |
| openstackgerrit | Sam Betts proposed openstack/tripleo-heat-templates master: Add helper resource for converting string to map https://review.openstack.org/600713 | 11:06 |
| *** med_ has joined #tripleo | 11:10 | |
| jistr | ssbarnea|bkp, marios: re https://review.openstack.org/#/c/600407/ in what envs do we distinguish chrooted vs. non-chrooted? | 11:10 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 11:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 11:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 11:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 11:10 |
| jistr | ssbarnea|bkp: all looks good but i'm not very familiar with the "crhooted" boolean use so i'm only wondering about pulling it out of the `when: chrooted` block | 11:10 |
| *** ykarel|session is now known as ykarel | 11:12 | |
| *** shyamb has joined #tripleo | 11:12 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Update swift_rings_backup workflow to also backup ceph fetch dir https://review.openstack.org/597221 | 11:15 |
| *** udesale has quit IRC | 11:16 | |
| *** shyamb has quit IRC | 11:24 | |
| *** shyamb has joined #tripleo | 11:24 | |
| *** Petersingh|afk is now known as Petersingh | 11:25 | |
| *** EmilienM is now known as EvilienM | 11:25 | |
| EvilienM | hi | 11:25 |
| quiquell|rover | EvilienM: Who are you ? | 11:26 |
| quiquell|rover | :-P | 11:26 |
| EvilienM | you don't want to know | 11:26 |
| EvilienM | quiquell|rover: last night I reported an issue with unit tests failing in the gate for tripleoclient | 11:27 |
| quiquell|rover | :-X | 11:27 |
| EvilienM | have we reported a bug yet? | 11:27 |
| EvilienM | let me find the error again | 11:28 |
| quiquell|rover | Don't have news on it | 11:28 |
| *** Petersingh has quit IRC | 11:28 | |
| quiquell|rover | focusing in promotions issues | 11:28 |
| EvilienM | http://logs.openstack.org/39/600439/2/check/tripleo-ci-centos-7-undercloud-containers/b0642b9/logs/delorean_logs/6d/ff/6dff80803437de9b8524f54cbfaa01c1843db97f_dev/build.log.txt.gz | 11:28 |
| EvilienM | scroll down ^ | 11:28 |
| ykarel | EvilienM, i think mwhahaha has a patch for that | 11:29 |
| ykarel | https://review.openstack.org/#/c/600467/ | 11:30 |
| ykarel | https://bugs.launchpad.net/tripleo/+bug/1791120 | 11:30 |
| openstack | Launchpad bug 1791120 in tripleo "Deploying overcloud fails if there's a plan" [High,In progress] - Assigned to Brad P. Crochet (brad-9) | 11:30 |
| *** lvdombrkr has quit IRC | 11:30 | |
| EvilienM | thrash++ thanks | 11:30 |
| EvilienM | let's see if that helps to fix tripleoclient delorean build | 11:31 |
| ykarel | yes that would help | 11:31 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Introduce container_cli parameter for undercloud.conf https://review.openstack.org/600499 | 11:32 |
| quiquell|rover | EvilienM: Cool, I see a lot of those jobs passing in the gates tough | 11:32 |
| ykarel | quiquell|rover, any news for fs037 issue | 11:33 |
| ykarel | updates job timing out | 11:33 |
| *** dprince has joined #tripleo | 11:34 | |
| quiquell|rover | ykarel: nope | 11:34 |
| ykarel | quiquell|rover, that seems the only blocker for now, i guess we can promote considering the fact that fs037 is non-voting upstream and broken currently | 11:35 |
| ykarel | master is +7 days out | 11:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Disable sync_power_state_interval in containerized undercloud https://review.openstack.org/599423 | 11:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Improve logging for modify image during prepare https://review.openstack.org/599267 | 11:36 |
| *** larsks has left #tripleo | 11:36 | |
| quiquell|rover | ykarel: you talk about timeout or this https://bugs.launchpad.net/tripleo/+bug/1789881 ? | 11:36 |
| openstack | Launchpad bug 1789881 in tripleo "tripleo-quickstart fails to generate /home/stack/containers-default-parameters.yaml" [Critical,In progress] - Assigned to yatin (yatinkarel) | 11:36 |
| *** ducnv has quit IRC | 11:37 | |
| quiquell|rover | ykarel: there is a fix for it but looks like OVB is affected | 11:37 |
| ykarel | quiquell|rover, ovb failure is unrelated to the change | 11:37 |
| *** ratailor__ has quit IRC | 11:38 | |
| ykarel | the patch only changing other featuresets, not fs001 | 11:38 |
| quiquell|rover | ykarel: Yep feels like it, also it has some Depends-On that will take time to merge, so yes we can change promotion criteria | 11:38 |
| *** ratailor has joined #tripleo | 11:38 | |
| ykarel | quiquell|rover, yes it's good to refresh current-tripleo | 11:39 |
| ykarel | and i think current failures in the patch in updates jobs are related to some rabbitmq-bundle, may be bandini have some idea on it | 11:39 |
| quiquell|rover | ykarel, jfrancoa: Also OVB is passing in recheck | 11:40 |
| ykarel | quiquell|rover, bandini talking about: Error: Failed to initialize erlang distribution: {{shutdown,", ', | 11:41 |
| ykarel | http://logs.openstack.org/05/599605/9/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/4358b87/logs/undercloud/home/zuul/overcloud_update_run_Controller.log.txt.gz#_2018-09-07_06_42_19 | 11:41 |
| *** ratailor_ has joined #tripleo | 11:41 | |
| quiquell|rover | Argg now, one failing | 11:42 |
| *** ssbarnea|bkp has quit IRC | 11:43 | |
| *** ratailor has quit IRC | 11:44 | |
| quiquell|rover | ykarel: I see other featureset failing at master promotion | 11:46 |
| ykarel | quiquell|rover, i see only fs021 and fs037 | 11:46 |
| ykarel | in the current run | 11:47 |
| *** lvdombrkr has joined #tripleo | 11:47 | |
| ykarel | hmm fs020 failed | 11:47 |
| ykarel | 4 tests failed | 11:48 |
| *** lvdombrkr89 has joined #tripleo | 11:50 | |
| *** phuongnh has joined #tripleo | 11:50 | |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: [WIP] Assert workloads are spawned in different stages https://review.openstack.org/600674 | 11:50 |
| quiquell|rover | and 21 | 11:50 |
| ykarel | quiquell|rover, fs021 is not in promotion criteria | 11:51 |
| quiquell|rover | ykarel: ack | 11:51 |
| *** lvdombrkr has quit IRC | 11:52 | |
| quiquell|rover | jfrancoa: all good https://review.openstack.org/#/c/599605/ | 11:53 |
| ykarel | updates job timing out ^^ | 11:54 |
| *** jpena|lunch is now known as jpena | 11:54 | |
| jistr | quiquell|rover, ykarel: hi, i just reported https://bugs.launchpad.net/tripleo/+bug/1791292 -- it seems that on master branch t-h-t we lost the updates job (it's not even running) | 11:55 |
| openstack | Launchpad bug 1791292 in tripleo "minor update sc000 job not being run at all on master branch t-h-t" [Critical,Triaged] | 11:55 |
| quiquell|rover | jistr: ack let me check | 11:55 |
| jistr | quiquell|rover, ykarel: actually we lost it from rocky too as i look at latest patches... Last time i saw it, it was green, so we probably dropped it by accident. | 11:56 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Introduce undercloud_container_cli parameter https://review.openstack.org/600512 | 11:58 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 11:58 |
| *** sri_ has quit IRC | 12:00 | |
| ykarel | jistr, it's intentionally not running atleast on the patch u mentioed | 12:00 |
| ykarel | jistr, https://github.com/openstack/tripleo-heat-templates/blob/master/zuul.d/layout.yaml#L13-L19 | 12:00 |
| *** sri_ has joined #tripleo | 12:00 | |
| quiquell|rover | yep | 12:00 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 12:01 |
| quiquell|rover | jistr: you have to add more criteria there in the file list, if you want this to run | 12:01 |
| *** morazi has joined #tripleo | 12:02 | |
| jistr | ykarel, quiquell|rover: ah sorry yes, i see :) | 12:02 |
| quiquell|rover | jfrancoa: Do have any idea about the timeouts at updates at your review ? was not looking at it | 12:04 |
| jistr | ykarel, quiquell|rover: it's probably good as it is for now, just got confused on that :) if i notice that we're not running it on something where it would be helpful, i'll post a patch | 12:04 |
| ykarel | jistr, ack | 12:05 |
| ykarel | quiquell|rover, running fs020 here: https://review.rdoproject.org/r/#/c/13943/ just to check if the 4 failed tests reproduce | 12:06 |
| quiquell|rover | ykarel: cool thanks | 12:06 |
| ykarel | quiquell|rover, if that pass, we can promote | 12:06 |
| ykarel | if not please file bug file, fix | 12:07 |
| quiquell|rover | ykarel: will do | 12:07 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 12:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 12:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 12:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 12:10 |
| jistr | ykarel, quiquell|rover: we should probably move the sc000 updates job back to voting so that we don't break it | 12:13 |
| quiquell|rover | jistr: Before my PTO was voting :-) | 12:13 |
| ykarel | jistr, yes once it starts passing :) | 12:13 |
| EvilienM | can someone from CI approve https://review.openstack.org/#/c/599896/ ? small patch thanks | 12:14 |
| ykarel | jistr, can u check the current failure in the fix, rabbitmq not starting | 12:14 |
| jistr | quiquell|rover: we had to drop it. Updates were actually fine, but the ansible-modify-image role which we use in CI was broken. We only figured out what was wrong after we made the job non-voting. | 12:14 |
| jistr | ykarel: can you give mee a link? sorry i probably missed some discussion | 12:15 |
| *** trown|outtypewww is now known as trown | 12:15 | |
| ykarel | jistr, Notice: /Stage[main]/Tripleo::Profile::Pacemaker::Rabbitmq_bundle/Exec[rabbitmq-ready]/returns: Error: Failed to initialize erlang distribution: {{shutdown,", ', | 12:15 |
| quiquell|rover | jistr: I see, looks like we are in a better place now, let's see if we can fix it | 12:15 |
| ykarel | jistr, http://logs.openstack.org/05/599605/9/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/25b40d0/logs/undercloud/home/zuul/overcloud_update_run_Controller.log.txt.gz#_2018-09-07_11_24_39 | 12:15 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-specs master: WIP upgrades CI with the Standalone installer for Stein https://review.openstack.org/579854 | 12:16 |
| *** tzumainn has joined #tripleo | 12:16 | |
| *** jtomasek has quit IRC | 12:20 | |
| *** sri_ has quit IRC | 12:21 | |
| *** sri_ has joined #tripleo | 12:22 | |
| *** lvdombrkr89 has quit IRC | 12:22 | |
| *** lvdombrkr has joined #tripleo | 12:22 | |
| *** sri_ has quit IRC | 12:23 | |
| *** mathlin has joined #tripleo | 12:24 | |
| *** sri_ has joined #tripleo | 12:24 | |
| *** pdeore__ has quit IRC | 12:26 | |
| *** moguimar_ has joined #tripleo | 12:27 | |
| *** moguimar is now known as Guest30860 | 12:28 | |
| *** moguimar_ is now known as moguimar | 12:28 | |
| *** Guest30860 has joined #tripleo | 12:28 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-docs master: Add commands in the doc to generate the nic configs https://review.openstack.org/596188 | 12:29 |
| *** psachin has quit IRC | 12:31 | |
| jistr | http://logs.openstack.org/05/599605/9/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/25b40d0/logs/subnode-2/var/log/host_info.txt.gz | 12:35 |
| jistr | Failed Actions: | 12:35 |
| jistr | * rabbitmq-bundle-docker-0_start_0 on centos-7-inap-mtl01-0001864564 'unknown error' (1): call=22, status=complete, exitreason='failed to pull image 192.168.24.1:8787/tripleomaster/centos-binary-rabbitmq:pcmklatest', | 12:35 |
| jistr | last-rc-change='Fri Sep 7 10:15:06 2018', queued=0ms, exec=280ms | 12:36 |
| jistr | * galera-bundle-docker-0_start_0 on centos-7-inap-mtl01-0001864564 'unknown error' (1): call=23, status=complete, exitreason='failed to pull image 192.168.24.1:8787/tripleomaster/centos-binary-mariadb:pcmklatest', | 12:36 |
| jistr | last-rc-change='Fri Sep 7 10:15:06 2018', queued=0ms, exec=263ms | 12:36 |
| jistr | * haproxy-bundle-docker-0_start_0 on centos-7-inap-mtl01-0001864564 'unknown error' (1): call=32, status=complete, exitreason='failed to pull image 192.168.24.1:8787/tripleomaster/centos-binary-haproxy:pcmklatest', | 12:36 |
| jistr | last-rc-change='Fri Sep 7 10:15:10 2018', queued=0ms, exec=194ms | 12:36 |
| jistr | ^ ykarel | 12:36 |
| jistr | probably some issue with (re)tagging of images | 12:37 |
| jistr | ykarel, jfrancoa: either way we should get the patch chain merged | 12:38 |
| jfrancoa | quiquell|rover: ykarel: about the updates job timeout, I can see in the update run log the same log bandini referenced: "2018-09-07 11:24:39 | u' "Notice: /Stage[main]/Tripleo::Profile::Pacemaker::Rabbitmq_bundle/Exec[rabbitmq-ready]/returns: Error: Failed to initialize erlang distribution: {{shutdown,", '," | 12:38 |
| jistr | jfrancoa: see above ^ in host_info all the pcmk-managed containers are stopped, and complaining that they can't find the image | 12:39 |
| *** lblanchard has joined #tripleo | 12:40 | |
| *** rlandy has joined #tripleo | 12:40 | |
| *** moguimar has quit IRC | 12:41 | |
| shyamb | Hi | 12:41 |
| shyamb | In case of ssl enabled on public interface | 12:42 |
| jfrancoa | jistr: aha,we should check the tagging tasks for minor updates. thanks for the analysis | 12:42 |
| shyamb | do we need to provide ssl key and cert at service level? | 12:42 |
| shyamb | I think haproxy takes care of this | 12:42 |
| ykarel | jfrancoa, jistr, cool, so seems issue with the image tag being looked for | 12:43 |
| ykarel | jistr, agree to getting the patches merged, those are also blocking promotions | 12:44 |
| *** moguimar has joined #tripleo | 12:45 | |
| *** shyamb has quit IRC | 12:49 | |
| *** gkadam_ has quit IRC | 12:50 | |
| *** moguimar has quit IRC | 12:50 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Fix failure to read UID of non root user https://review.openstack.org/600730 | 12:54 |
| *** skramaja has quit IRC | 12:56 | |
| openstackgerrit | Merged openstack/instack master: fix tox python3 overrides https://review.openstack.org/572904 | 12:58 |
| *** ssbarnea has joined #tripleo | 12:59 | |
| *** ssbarnea|bkp2 has joined #tripleo | 12:59 | |
| *** ssbarnea|bkp2 has quit IRC | 12:59 | |
| *** ssbarnea|bkp has joined #tripleo | 13:00 | |
| jistr | ykarel, jfrancoa: do you know what's generating the containers-prepare-parameter.yaml file? It doesn't look right | 13:00 |
| jistr | ykarel, jfrancoa: or maybe i just don't understand how it should work :) | 13:01 |
| *** ansmith has joined #tripleo | 13:04 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 13:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 13:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 13:10 |
| *** agopi has quit IRC | 13:10 | |
| *** phuongnh has quit IRC | 13:12 | |
| *** aufi has quit IRC | 13:13 | |
| *** myoung has joined #tripleo | 13:13 | |
| *** mjturek has joined #tripleo | 13:16 | |
| *** shreshtha has quit IRC | 13:18 | |
| weshay | matbu, you around? | 13:18 |
| *** ratailor_ has quit IRC | 13:18 | |
| ykarel | jistr, /me don't know | 13:19 |
| *** gkadam has joined #tripleo | 13:19 | |
| matbu | weshay: yep | 13:19 |
| matbu | weshay: i'm in your bluejean :) | 13:20 |
| *** mcornea has joined #tripleo | 13:20 | |
| *** med_ has quit IRC | 13:21 | |
| fultonj | toure: regarding 1789680 do you want to see "top -b | 13:21 |
| fultonj | -o +%MEM | head -n 22" | 13:21 |
| fultonj | on hypervisor OR undercloud? | 13:21 |
| fultonj | fwiw, my uc has 12G of ram | 13:22 |
| openstackgerrit | Merged openstack/tripleo-common master: Allow networks in roles_data to be dict or list https://review.openstack.org/595570 | 13:25 |
| ykarel | jistr, btw what's wrong, stevebaker may know that | 13:26 |
| *** gbarros has joined #tripleo | 13:26 | |
| jistr | ykarel: so it seems like at some points we're using docker.io and at others we're using undercloud. E.g. we tag docker.io/....:pcmklatest and then we try to use 192.168.....:pcmklatest image | 13:28 |
| jistr | which obviously won't do any good :) | 13:28 |
| jistr | but i don't see the root cause, need to read up a bit on the ImagePrepare parameter for Heat, but i'm getting kinda bombarded with requests today | 13:28 |
| jistr | would still like to carve out some time though... | 13:29 |
| jistr | the update job is hi prio i think | 13:29 |
| ykarel | jistr, isn't the correct image be 192.168.24.1:8787/tripleomaster/centos-binary-haproxy:<container_build_id> | 13:30 |
| *** agopi has joined #tripleo | 13:30 | |
| ykarel | instead of pcmklatest | 13:30 |
| ykarel | okk i think it's tagged at some place | 13:31 |
| jistr | ykarel: <container_build_id> would be on undercloud registry, we then pull that into overcloud and tag it with :pcmklatest only locally on the overcloud nodes. It's a hack to make rolling container updates possible for pacemaker-managed containers. Pacemaker cannot run tag :abc on one node and tag :def on another node, but that's exactly what's needed to make rolling updates happen, so we hack it with | 13:32 |
| jistr | :pcmklatest -- the tag is the same, but during various phases of minor update, it may not always point to the same image on all overcloud nodes. | 13:32 |
| *** jistr is now known as jistr|call | 13:32 | |
| ykarel | jistr|call, ack | 13:33 |
| holser_ | jistr|call why should we use docker.io? why not quay.io | 13:33 |
| *** rh-jelabarre has joined #tripleo | 13:36 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Persist ceph-ansible fetch_directory using config-download https://review.openstack.org/582811 | 13:39 |
| *** aufi has joined #tripleo | 13:41 | |
| *** aufi has quit IRC | 13:41 | |
| toure | fultonj on the undercloud | 13:42 |
| fultonj | toure: ok | 13:42 |
| *** jtomasek has joined #tripleo | 13:42 | |
| toure | want to see if there is anything that is out of sorts | 13:42 |
| *** aufi has joined #tripleo | 13:42 | |
| *** gkadam has quit IRC | 13:47 | |
| tbarron | quiquell|rover: sorry for delay but I'm on PTO today and only "here" occasionally - sure | 13:50 |
| mandre | bandini: hi! quick question on the haproxy config, I'm trying to LB the openshift infra nodes but problem is, I do not have a service for it | 13:50 |
| tbarron | http://logs.openstack.org/62/600662/1/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/ee0f72c/logs/undercloud/var/log/extra/rpm-list.txt.gz | 13:50 |
| mandre | bandini: the infra node is kind of a specialization of a compute node | 13:50 |
| tbarron | is a different failure but it's mistral not manila | 13:50 |
| tbarron | quiquell|rover: ^^^ manila folks don't know much about mistral | 13:51 |
| ykarel | jistr|call, that file seems being generated via this template: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/containers-prepare-parameter.yaml.j2 | 13:51 |
| mandre | bandini: so question is, it it possible to provide a vip for just a selection of my worker nodes, or I must create a specialized service for the infra node that perhaps inherit from the openshift compute service | 13:51 |
| d0ugal | tbarron: What mistral problem is there? is there a bug? | 13:51 |
| quiquell|rover | tbarron: ack, go out and enjoy your PTO | 13:52 |
| tbarron | d0ugal: there's a ci failure that quiquell|rover knows about :) | 13:52 |
| d0ugal | quiquell|rover: Let me know if I can help | 13:52 |
| * quiquell|rover changing brains | 13:54 | |
| quiquell|rover | tbarron: this is the manila min version issue ? | 13:55 |
| *** chem has joined #tripleo | 13:55 | |
| *** ade_lee has joined #tripleo | 13:55 | |
| quiquell|rover | d0ugal: thanks | 13:59 |
| *** TheJulia is now known as needssleep | 14:00 | |
| *** boazel has joined #tripleo | 14:02 | |
| chem | weshay: hey, do we just recheck until tripleo-ci-centos-7-containers-multinode goes under 3h, or do I miss something ? | 14:05 |
| weshay | chem, let me see the review | 14:06 |
| weshay | chem, generally rechecking w/o knowing the root cause if not advised :) | 14:06 |
| chem | weshay: https://review.openstack.org/#/c/599348/ ... it's timeout during post overcloud action | 14:06 |
| *** lblanchard has quit IRC | 14:06 | |
| weshay | chem, ya.. don't recheck | 14:06 |
| weshay | chem, https://review.openstack.org/#/c/600546/ | 14:07 |
| * weshay adds alert tag to the bug https://bugs.launchpad.net/tripleo/+bug/1791169 | 14:07 | |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 14:07 |
| *** quiquell|rover has quit IRC | 14:08 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Stop relying on old legacy sub_nodes_private https://review.openstack.org/596906 | 14:08 |
| chem | weshay: oki, thanks for the info | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 14:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 14:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 14:10 |
| *** cshastri has quit IRC | 14:16 | |
| EvilienM | ykarel: sounds like Depends-On didn't help https://review.openstack.org/#/c/600499/ | 14:17 |
| EvilienM | mwhahaha: have you seen tripleoclient build failures lately? | 14:18 |
| EvilienM | http://logs.openstack.org/99/600499/4/check/tripleo-ci-centos-7-undercloud-containers/3e19984/logs/delorean_logs/4c/8d/4c8db54b2f108277378c2d200358e61c70caf2de_dev/build.log.txt.gz | 14:18 |
| *** boazel has quit IRC | 14:21 | |
| *** holser_ has quit IRC | 14:22 | |
| *** holser__ has joined #tripleo | 14:22 | |
| ykarel | EvilienM, hmm tripleo-common change not picked by dlrn while building tripleoclient(openstack-tripleo-common-9.3.1-0.20180907070809.b68c248.el7.noarch is installed but required is) | 14:23 |
| EvilienM | I think it's https://review.openstack.org/#/c/553616/ | 14:23 |
| EvilienM | dprince: ^ I think your patch broke tripleoclient build on delorean | 14:23 |
| EvilienM | I wonder if we need a new version of tripleo-common | 14:23 |
| dprince | EvilienM: new version of tripleo-common is probably the issue | 14:24 |
| dprince | EvilienM: It is common that python-tripleoclient and tripleo-common have this happen | 14:24 |
| dprince | EvilienM: because I think we build both in CI | 14:24 |
| ykarel | looks like --order in dlrn will solve this | 14:24 |
| ykarel | jpena, ^^ | 14:24 |
| ykarel | hmm depends-on packages are not even build | 14:25 |
| ykarel | dlrn --config-file projects.ini --head-only --package-name python-tripleoclient --local --info-repo rdoinfo --dev | 14:25 |
| EvilienM | it is a regression | 14:26 |
| EvilienM | they were before | 14:26 |
| weshay | ykarel, EvilienM the unit tests failed http://logs.openstack.org/99/600499/4/check/tripleo-ci-centos-7-standalone/7576fe5/logs/delorean_logs/4c/8d/4c8db54b2f108277378c2d200358e61c70caf2de_dev/build.log.txt.gz | 14:26 |
| ykarel | hmm i thing it's doing wrong currently, should run single dlrn command instead of multiple | 14:26 |
| *** sanjayu_ has joined #tripleo | 14:27 | |
| EvilienM | we need a bug report and an alert, it's pretty critical | 14:27 |
| weshay | EvilienM, is this happening in more reviews? | 14:27 |
| EvilienM | weshay: tripleoclient check | 14:28 |
| EvilienM | the delorean issue? everywhere | 14:28 |
| weshay | I'll look | 14:28 |
| weshay | jebus | 14:28 |
| jpena | I'm checking http://logs.openstack.org/99/600499/4/check/tripleo-ci-centos-7-standalone/7576fe5/logs/ara_oooq/ | 14:29 |
| EvilienM | I can't help much, I'm in a meeting and about to board in a plane :D | 14:29 |
| weshay | EvilienM, and flying | 14:29 |
| weshay | while doing pushups | 14:30 |
| EvilienM | lol | 14:30 |
| EvilienM | tonight ;-) | 14:30 |
| *** med_ has joined #tripleo | 14:30 | |
| jpena | actually, this is something where mwahahaha's patch should help -> https://review.openstack.org/592668 | 14:30 |
| jpena | by default, the oooq playbooks run dlrn one by one for each change | 14:30 |
| jpena | instead of using a single command-line for all | 14:31 |
| mwhahaha | See I am helpful | 14:31 |
| mwhahaha | Sometimes | 14:31 |
| weshay | mwhahaha, meh.. you have test patches that hit that? | 14:31 |
| jpena | well, now that I think of it... If we used the right order we could even do it in multiple runs | 14:31 |
| jpena | let me have a look | 14:31 |
| ykarel | yeah that should help | 14:31 |
| mwhahaha | weshay: I had it when I was doing my fedora28 stuff | 14:32 |
| weshay | jpena, ykarel do we still have an issue w/ rpms being built that are older than dlrn-current rpms? | 14:32 |
| mwhahaha | Rdo does it like that in their ci | 14:32 |
| weshay | k | 14:32 |
| weshay | mwhahaha, we need that post.yml timeout patch to merge | 14:33 |
| mwhahaha | We need a lot of things | 14:33 |
| ykarel | weshay, not sure about that, but ^^ is something different(order of packages to build) | 14:33 |
| *** pcaruana has quit IRC | 14:33 | |
| jpena | ok, so we could also fix it by inverting the order of artg_change_list in https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/build-test-packages/tasks/main.yml#L145-L156 | 14:34 |
| jpena | the issue here is that the playbook is getting all the changes in order (first the parent, then the depends-on), but we need to build in reverse order (first the depends-on, then the parent) | 14:34 |
| *** ksambor has quit IRC | 14:34 | |
| mwhahaha | so my dlrn change will address that cause it lets dlrn figure out the proper build order | 14:35 |
| mwhahaha | the packages could still end up being older than current if the patch itself has not been rebased in some time | 14:35 |
| *** athomas has joined #tripleo | 14:36 | |
| weshay | jpena, that sounds sane to me :) | 14:36 |
| weshay | nice | 14:36 |
| jpena | yep to both | 14:36 |
| mwhahaha | hey weshay, http://logs.openstack.org/88/575588/12/check/tripleo-ci-centos-7-standalone/34e7b52/logs/tempest.html.gz | 14:36 |
| weshay | mwhahaha, is the fucking man | 14:36 |
| weshay | ignore all those horrible things I've ever said about him | 14:37 |
| mwhahaha | :D | 14:37 |
| jpena | let me propose a quick change to tripleo-quickstart-extras to reverse the order, then we can test it on 600499 | 14:37 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-upgrade master: Increase post-timeout for upgrade jobs from 30min to 40min https://review.openstack.org/600791 | 14:40 |
| openstackgerrit | Javier Peña proposed openstack/tripleo-quickstart-extras master: Build packages using DLRN in reverse order https://review.openstack.org/600792 | 14:41 |
| jpena | EvilienM: I'm adding the depends-on to test it | 14:41 |
| openstackgerrit | Javier Peña proposed openstack/python-tripleoclient master: Introduce container_cli parameter for undercloud.conf https://review.openstack.org/600499 | 14:42 |
| *** moguimar has joined #tripleo | 14:43 | |
| EvilienM | jpena: ok thx | 14:43 |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base https://review.openstack.org/596422 | 14:44 |
| *** dtrainor has joined #tripleo | 14:48 | |
| *** lvdombrkr89 has joined #tripleo | 14:50 | |
| EvilienM | mandre, lol https://github.com/containers/libpod/pull/1426 | 14:51 |
| EvilienM | it changes everything ahah | 14:51 |
| EvilienM | bandini, Tengu^ FYI, we don't need to worry about absent directories, podman will create them like docker did :-O | 14:51 |
| EvilienM | I'll cleanup the code in our side to not create twice | 14:52 |
| *** spartakos has joined #tripleo | 14:52 | |
| *** hkominos has left #tripleo | 14:52 | |
| *** lvdombrkr has quit IRC | 14:52 | |
| mandre | EvilienM: oh wow, I didn't see it coming | 14:57 |
| mandre | good news, less potential bugs for us | 14:57 |
| EvilienM | yes | 14:58 |
| *** apetrich_ has quit IRC | 14:58 | |
| *** gkadam has joined #tripleo | 15:01 | |
| *** boazel has joined #tripleo | 15:02 | |
| *** holser__ has quit IRC | 15:03 | |
| jistr|call | ykarel, jfrancoa: o-oh, i see the problem. The image prepare service uses external_deploy_tasks to amend the deployment plan. That was never meant to happen and as currently implemented, it's incompatible with both minor updates and major upgrades. | 15:03 |
| *** jistr|call is now known as jistr | 15:03 | |
| *** holser_ has joined #tripleo | 15:04 | |
| *** d0ugal has quit IRC | 15:04 | |
| jfrancoa | jistr: so, is that the reason why both jobs timeout? | 15:05 |
| jistr | jfrancoa: it's definitely reason why update job is broken. Re upgrade job it's very likely too but i didn't check. | 15:06 |
| jfrancoa | jistr: /o\ | 15:07 |
| ykarel | mwhahaha, EvilienM posted some comments on https://review.openstack.org/#/c/592668/, it's good to test in some patch where packages are build to see if that works | 15:08 |
| mwhahaha | ykarel: it's disabled by default | 15:08 |
| jistr | jfrancoa: we don't execute the external_deploy_tasks during update prepare. In fact we can't. It's a tool for installing services so if we run it during upgrade prepare, we'd also be running ceph-ansible etc. | 15:08 |
| mwhahaha | ykarel: but i can confirm that it does work (i use it for my fedora28 work) | 15:08 |
| ykarel | mwhahaha, ack i meant to test that by setting artg_build_one: false | 15:09 |
| ykarel | mwhahaha, but i see syntax errors there | 15:09 |
| ykarel | did i missed something? | 15:09 |
| mwhahaha | ykarel: not sure let me review. | 15:10 |
| ykarel | ack | 15:10 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 15:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 15:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 15:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 15:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 15:10 |
| mwhahaha | ykarel: yea that would be why it's fine in ci cause it's disabled :D | 15:10 |
| *** paramite has quit IRC | 15:10 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add ability to have dlrn build all the packages at once https://review.openstack.org/592668 | 15:10 |
| mwhahaha | i fix | 15:11 |
| ykarel | mwhahaha, ack :) i think it would fail at least for 'is default' part when it's tested as depends-on on a project where packages get build | 15:11 |
| mwhahaha | ykarel: i can try and propse some patches to test it, unfortunately it'll require a bunch of patches all over the place | 15:12 |
| mwhahaha | cause i'll have to patch quickstart and then 2 projects to depends on | 15:12 |
| mwhahaha | gimme a few | 15:12 |
| ykarel | mwhahaha, no hurry, just wanted that it should merge with testing | 15:12 |
| ykarel | otherwise we would have failures | 15:12 |
| *** chem` has joined #tripleo | 15:13 | |
| ykarel | okk artg_build_one: true so will not fail | 15:13 |
| * mwhahaha gets out the featureset to job secret decoder ring | 15:13 | |
| * ykarel is that ring shareable :) | 15:14 | |
| *** ykarel is now known as ykarel|away | 15:15 | |
| mwhahaha | pffft get your own :D | 15:15 |
| ykarel|away | :D | 15:15 |
| mwhahaha | https://i.imgur.com/kY24jhz.jpg?1 | 15:16 |
| *** chem has quit IRC | 15:16 | |
| *** jtomasek has quit IRC | 15:17 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: DNM: testing build-test-packages https://review.openstack.org/600797 | 15:17 |
| jistr | jfrancoa: at least that's how it looks. I still can't wrap my brain around how can the external_deploy_tasks export the image parameters for a deployment which is already in progress... continuing to look | 15:18 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo master: DNM: testing build-test-packages https://review.openstack.org/600798 | 15:19 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: DNM: testing build-test-packages https://review.openstack.org/600799 | 15:19 |
| mwhahaha | ykarel|away: -^ | 15:20 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: DNM: testing build-test-packages https://review.openstack.org/600799 | 15:21 |
| *** shardy has quit IRC | 15:21 | |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo master: DNM: testing build-test-packages https://review.openstack.org/600798 | 15:22 |
| ykarel|away | mwhahaha, ack looks good | 15:22 |
| mwhahaha | now it should be good | 15:22 |
| mwhahaha | stupid ordering | 15:22 |
| *** chem` has quit IRC | 15:23 | |
| *** devep has joined #tripleo | 15:23 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base https://review.openstack.org/596422 | 15:24 |
| *** d0ugal has joined #tripleo | 15:25 | |
| *** gkadam has quit IRC | 15:26 | |
| *** shyamb has joined #tripleo | 15:26 | |
| jistr | i'm reading http://tripleo.org/install/advanced_deployment/container_image_prepare.html | 15:27 |
| jistr | still i don't quite get where do we set the DockerHAProxyImage etc. from that composable service | 15:27 |
| jistr | doc says "During deployment, this will lookup images in docker.io/tripleomaster tagged with current-tripleo and discover a versioned tag by looking up the label rdo_version. This will result in the heat image parameters in the plan being set with appropriate values," | 15:28 |
| jistr | and i can see the call to image-prepare command here | 15:28 |
| jistr | https://github.com/openstack/tripleo-heat-templates/blob/3ad979ac721c034a127b09b36bdda8e5fa488216/puppet/services/container-image-prepare.j2.yaml#L101 | 15:28 |
| jistr | but i don't follow where/how we "export" the Docker*Image parameters from there so that we're actually able to affect the deployment which is already in progress | 15:29 |
| jistr | mwhahaha, EvilienM: ^ do you guys have an idea? | 15:29 |
| *** ramishra_ has quit IRC | 15:30 | |
| mwhahaha | jistr: that's in tripleo-common | 15:30 |
| *** artom has quit IRC | 15:30 | |
| mwhahaha | jistr: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/image/kolla_builder.py#L67 | 15:31 |
| openstackgerrit | Javier Peña proposed openstack/tripleo-quickstart-extras master: Build packages using DLRN in reverse order https://review.openstack.org/600792 | 15:32 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Stop relying on old legacy sub_nodes_private https://review.openstack.org/596906 | 15:32 |
| jistr | mwhahaha: hmm ok but even if we write out some env file, how do we make it affect a deployment which is already in progress? e.g. right after external_deploy_tasks step 1, there comes normal step 1. That's already in config-download phase, after Heat stack outputs have been exported. If there's a Heat env file written in external_deploy_tasks step 1, how can we still affect anything? re-run Heat stack update | 15:35 |
| jistr | + config-download? I don't see where would that be happening, i'm probably missing something fundamental here :) | 15:35 |
| openstackgerrit | yatin proposed openstack/tripleo-quickstart master: [DNM] For testing manila-tempest-plugin 0.1.0 https://review.openstack.org/600803 | 15:36 |
| mwhahaha | jistr: that's probably a question best answered by stevebaker | 15:36 |
| mwhahaha | jistr: it is optional | 15:36 |
| *** devep has quit IRC | 15:37 | |
| mwhahaha | jistr: so if you don't set ContainerImagePrepare i don't think it triggers | 15:37 |
| *** aufi has quit IRC | 15:37 | |
| mwhahaha | jistr: but yea it'd be best to get an answer from steve fo ryour specific ordering questions | 15:38 |
| openstackgerrit | yatin proposed openstack/tripleo-heat-templates stable/queens: DNM: To test tempest manila miniversion https://review.openstack.org/600662 | 15:38 |
| jistr | mwhahaha: ack, thanks. I'm trying to figure out how it works to assess whether it could possibly ever work with updates+upgrades. As implemented right now, it's fundamentally incompatible... | 15:38 |
| jistr | mwhahaha: the reason why update job is broken, and probably upgrade too | 15:39 |
| jistr | not that sure about upgrade yet though | 15:39 |
| mwhahaha | it works on the undercloud updates/upgrades (i think) so it must be something specific to the overcloud stuff | 15:40 |
| *** cylopez has quit IRC | 15:41 | |
| *** ssbarnea|bkp has quit IRC | 15:43 | |
| itlinux | marios: are you around thanks | 15:45 |
| *** ykarel|away has quit IRC | 15:46 | |
| *** shyamb has quit IRC | 15:47 | |
| *** athomas has quit IRC | 15:48 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Stop relying on old legacy sub_nodes_private https://review.openstack.org/596887 | 15:50 |
| jistr | mwhahaha: hm i'm actually starting to think it may be running in two chunks, once during the mistral workflow before Heat, to generate correct Heat parameters, and 2nd time during external_deploy_tasks, to download+upload the images. That's the only explanation i can think of right now. stevebaker if you could confirm/refute if/when you read this, that'd be great :) i have ZNC so i'll read it later. | 15:53 |
| jistr | mwhahaha: thanks for the reply | 15:53 |
| mwhahaha | jistr: yea i thought that's what happened, is that mistral was used to process the list and we pull the containers down in an external task | 15:53 |
| mwhahaha | that's before everything | 15:54 |
| mwhahaha | so if that's a problem with upgrades/updates we'll need to figure something out | 15:54 |
| *** dtantsur is now known as dtantsur|afk | 15:54 | |
| jistr | mwhahaha: yea, main problem is that 1) during update/upgrade it's not really before everything 2) we can't really run any of the external tasks there at all in general. The `upgrade run` can be done many times (e.g. per node or per role). If we ran those tasks, then e.g. every time we update controller-0 it would also run full ceph-ansible... Then it would run ceph-ansible again when we upgrade controller-1 | 15:56 |
| jistr | etc. :D | 15:56 |
| jistr | mwhahaha: that's why external_*_tasks have a completely separate commands for updates/upgrades, that's the only sane way to run them | 15:57 |
| *** lvdombrkr89 has quit IRC | 15:57 | |
| *** yprokule has quit IRC | 15:57 | |
| mwhahaha | so that can be bypassed by not using the containerimageprepare variables, but yea we'll need to figure something out. steve's the guy so we'll see what he says next week | 15:58 |
| jistr | yea. thanks mwhahaha | 15:58 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/pike: GATE CHECK for TripleO https://review.openstack.org/564285 | 16:00 |
| *** moguimar has quit IRC | 16:06 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Stop relying on old legacy sub_nodes_private https://review.openstack.org/596906 | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 16:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 16:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 16:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 16:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 16:10 |
| *** holser_ has quit IRC | 16:14 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-heat-templates master: Add simple validation that OpenShift is deployed https://review.openstack.org/600067 | 16:15 |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-heat-templates stable/queens: DNM: Debugging scenario004 https://review.openstack.org/600642 | 16:17 |
| *** cschwede has quit IRC | 16:19 | |
| *** rdopiera has quit IRC | 16:20 | |
| jistr | mwhahaha, stevebaker: perhaps it could go like this https://bugs.launchpad.net/tripleo/+bug/1791332/comments/1 i'll follow up next week | 16:21 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 16:21 |
| jistr | thanks and have a good weekend | 16:21 |
| *** cschwede has joined #tripleo | 16:22 | |
| *** ssbarnea|bkp has joined #tripleo | 16:23 | |
| *** bnemec is now known as beekneemech | 16:24 | |
| *** dpeacock has quit IRC | 16:24 | |
| weshay | jistr++ | 16:25 |
| *** dpeacock has joined #tripleo | 16:26 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Stop relying on old legacy sub_nodes_private https://review.openstack.org/596887 | 16:26 |
| mwhahaha | weshay: so i got the standalone running on fedora28, did we have patches outstanding to try and pull in a fedora28 version of the standalone job? | 16:26 |
| weshay | mwhahaha, so fairly undeveloped patches ya | 16:27 |
| * weshay looks | 16:27 | |
| *** spartakos has quit IRC | 16:27 | |
| mwhahaha | the heat, heat-agent and tripleo-client python3 distgits still need merging | 16:28 |
| mwhahaha | and we're missing net-snmp int he fedora-stable repo | 16:28 |
| mwhahaha | but other than that it seems to work fine | 16:28 |
| weshay | k | 16:29 |
| mwhahaha | oh we need to figure out how to have a default cron package installed (cronie?) | 16:30 |
| mwhahaha | other wise the cron providers fail horribly in puppet | 16:30 |
| *** florianf has quit IRC | 16:30 | |
| mwhahaha | though i guess i should figure out what service is still installing cron jobs on the host | 16:31 |
| mwhahaha | cause i thought we had a container for that | 16:31 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 16:33 |
| *** jpich has quit IRC | 16:34 | |
| *** dpeacock has quit IRC | 16:35 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 16:35 |
| *** psahoo has quit IRC | 16:36 | |
| *** dpeacock has joined #tripleo | 16:36 | |
| *** ykarel has joined #tripleo | 16:40 | |
| *** dtrainor_ has joined #tripleo | 16:41 | |
| *** Jim137 has joined #tripleo | 16:44 | |
| *** dtrainor has quit IRC | 16:45 | |
| openstackgerrit | Nicolas Hicher proposed openstack/tripleo-quickstart master: DNM: disable clean_node for testing https://review.openstack.org/600815 | 16:45 |
| Jim137 | Hello | 16:45 |
| *** Jim137 is now known as Chaserjim | 16:46 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart master: Disable vxlan_networking in mixed_upgrade RDO cloud jobs. https://review.openstack.org/600816 | 16:46 |
| Chaserjim | Hi Tripleo | 16:47 |
| Chaserjim | Ill be attending my first PTG next week. | 16:48 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart master: Disable vxlan_networking in mixed_upgrade RDO cloud jobs. https://review.openstack.org/600816 | 16:48 |
| *** Chaserjim has quit IRC | 16:50 | |
| *** Chaserjim has joined #tripleo | 16:50 | |
| *** Chaserjim has quit IRC | 16:55 | |
| *** Chaserjim has joined #tripleo | 16:55 | |
| *** sanjayu_ has quit IRC | 16:57 | |
| *** jfrancoa has quit IRC | 16:58 | |
| *** Chaserjim has quit IRC | 16:58 | |
| *** med_ has quit IRC | 16:58 | |
| *** Chaserjim has joined #tripleo | 16:59 | |
| weshay | mwhahaha, meh.. not sure https://review.openstack.org/#/c/595374/ | 16:59 |
| *** myoung is now known as myoung|dr | 16:59 | |
| *** Chaserjim has quit IRC | 17:00 | |
| *** Chaserjim has joined #tripleo | 17:00 | |
| *** derekh has quit IRC | 17:01 | |
| *** jpena is now known as jpena|off | 17:01 | |
| Tengu | EvilienM: well, as long as it can create them with the right selinux label/context.... | 17:03 |
| *** saneax has joined #tripleo | 17:03 | |
| Tengu | EvilienM: not sure it's really a good news in fact... | 17:03 |
| *** psachin has joined #tripleo | 17:03 | |
| Tengu | (oh and, hello btw ;)) | 17:03 |
| slagle | Chaserjim: hi. i hope you can join some of our sessions if you're interested in tripleo | 17:05 |
| slagle | Chaserjim: we have an agenda at https://etherpad.openstack.org/p/tripleo-ptg-stein | 17:05 |
| Chaserjim | @slagle sounds great | 17:08 |
| *** trown is now known as trown|lunch | 17:09 | |
| mwhahaha | weshay: k I'll poke at it after lunch | 17:09 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 17:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 17:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 17:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 17:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 17:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 17:10 |
| *** sanjayu_ has joined #tripleo | 17:11 | |
| *** saneax has quit IRC | 17:11 | |
| *** med_ has joined #tripleo | 17:12 | |
| *** tesseract has quit IRC | 17:20 | |
| *** jtomasek has joined #tripleo | 17:24 | |
| *** ssbarnea|bkp2 has joined #tripleo | 17:25 | |
| *** ssbarnea|bkp has quit IRC | 17:26 | |
| *** dtrainor_ is now known as dtrainor | 17:27 | |
| *** khyr0n has joined #tripleo | 17:33 | |
| *** ykarel is now known as ykarel|away | 17:33 | |
| Tengu | EvilienM, bandini apparently I'm not the only one not really happy - PR creating directories in podman is closed: https://github.com/containers/libpod/pull/1426 | 17:34 |
| Tengu | mandre: -^ | 17:34 |
| Tengu | EvilienM: commented your card, moved to "blocked", I let you chose what to do next ;). | 17:35 |
| Tengu | so basically... we "just" need to push systemd in the loop in order to get proper service restart ). | 17:36 |
| Tengu | and that's a really good news. | 17:36 |
| *** psachin has quit IRC | 17:36 | |
| weshay | mwhahaha, fyi.. working w/ nhicher in #sf-dfg | 17:38 |
| weshay | on fed | 17:38 |
| openstackgerrit | Nicolas Hicher proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 17:38 |
| *** spartakos has joined #tripleo | 17:39 | |
| *** ykarel|away has quit IRC | 17:46 | |
| *** med_ has quit IRC | 17:46 | |
| *** dsneddon has joined #tripleo | 17:50 | |
| openstackgerrit | Nicolas Hicher proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 17:52 |
| *** sanjayu_ has quit IRC | 17:57 | |
| EvilienM | Tengu: right. Well we'll see | 17:57 |
| *** saneax has joined #tripleo | 17:58 | |
| *** saneax has quit IRC | 17:59 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: udpate reproducer to install required deps https://review.openstack.org/600836 | 18:02 |
| *** saneax has joined #tripleo | 18:03 | |
| *** jaganathan has quit IRC | 18:03 | |
| mwhahaha | weshay: so that's the issue you're working thorugh? | 18:04 |
| weshay | mwhahaha, the fedora image is not found for some reason | 18:05 |
| mwhahaha | weshay: for rdo? or upstream? | 18:05 |
| weshay | upstream | 18:05 |
| weshay | let's rock the standalone in upstream man | 18:06 |
| * mwhahaha is confused as so why you weren't asking in #openstack-infra then | 18:06 | |
| weshay | mwhahaha, oh wait.. they answer questions there? | 18:06 |
| mwhahaha | maybe? | 18:06 |
| weshay | lolz | 18:07 |
| *** saneax has quit IRC | 18:08 | |
| *** morazi has quit IRC | 18:08 | |
| *** trown|lunch is now known as trown | 18:09 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 18:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 18:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 18:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 18:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 18:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 18:10 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 18:11 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: udpate reproducer to install required deps https://review.openstack.org/600836 | 18:13 |
| weshay | mwhahaha, The nodeset "single-fedora-28-node" was not found. | 18:18 |
| weshay | wahhh wahhh waaaaaaaaaaaaaaaaa | 18:18 |
| mwhahaha | weshay: yea it's like the rdo version of zuul isn't properly picking up the node set from the same change. I wonder if we have to do two changes to make it happy | 18:18 |
| Tengu | EvilienM: not sure it's a bad thing - having to declare things is always better... | 18:18 |
| weshay | nhicher, ? | 18:19 |
| Tengu | so that we ensure rights, ownership, and there isn't any black magic behind ;). | 18:19 |
| nhicher | mwhahaha, weshay: yes we can try 2 changes | 18:20 |
| weshay | nhicher, so start w/ the node change, get it merged | 18:20 |
| weshay | then add the standalone? | 18:20 |
| nhicher | or nodeset and depends on for the second patch | 18:20 |
| *** gfidente has quit IRC | 18:21 | |
| mwhahaha | probably not depends-on, just stack the changes, though not sure how that's any different | 18:22 |
| nhicher | but, that's really strange, I proposed a change few weeks ago with the same thing https://review.rdoproject.org/r/#/c/15895/ | 18:22 |
| mwhahaha | silly zuul | 18:22 |
| mwhahaha | oh | 18:22 |
| nhicher | but the label was present on sf zuul | 18:22 |
| nhicher | that's the main difference | 18:22 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: udpate reproducer to install required deps https://review.openstack.org/600836 | 18:22 |
| mwhahaha | that's it | 18:22 |
| mwhahaha | we need to propose the single-fedora-28-node to rdo-jobs | 18:23 |
| mwhahaha | if you look on that change, there is already a single-centos-7-node | 18:23 |
| mwhahaha | so the rdo zuul isn't picking up nodesets.yaml from in the project repo | 18:23 |
| nhicher | mwhahaha: but centos-7 label doesn't exist on sf https://review.rdoproject.org/zuul/labels.html | 18:24 |
| mwhahaha | no but it's in zuul.d/nodesets.yaml | 18:24 |
| mwhahaha | nhicher: https://review.rdoproject.org/r/#/c/15895/6/zuul.d/nodesets.yaml@80 | 18:24 |
| mwhahaha | the issue isn't the label, it's the nodeset itself | 18:24 |
| weshay | mwhahaha, to rdo-jobs? | 18:25 |
| weshay | well there too | 18:25 |
| weshay | but let's run it upstream no? | 18:25 |
| mwhahaha | right but that breaks rdo 3rd party | 18:26 |
| mwhahaha | so we have to add the node set into rdo | 18:26 |
| mwhahaha | pesky zuuls | 18:26 |
| * weshay has an errand | 18:28 | |
| weshay | bbiab | 18:28 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 18:28 |
| mwhahaha | and then -^ | 18:29 |
| mwhahaha | i think that might work | 18:29 |
| *** andreaf has joined #tripleo | 18:29 | |
| *** geneliu1 has joined #tripleo | 18:33 | |
| *** dtrainor_ has joined #tripleo | 18:33 | |
| *** dtrainor has quit IRC | 18:35 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: Add template to push telemetry data into amqp1.0 bus https://review.openstack.org/600843 | 18:44 |
| *** ykarel|away has joined #tripleo | 18:55 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add a standalone role for creating the zuul-hosts file https://review.openstack.org/579154 | 18:59 |
| *** ykarel|away has quit IRC | 19:00 | |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Use generic names for container platform https://review.openstack.org/600847 | 19:02 |
| *** morazi has joined #tripleo | 19:05 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 19:08 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: WIP - podman: create, enable and start containers with systemd https://review.openstack.org/600849 | 19:09 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 19:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790987 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 19:10 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 19:10 |
| openstack | Launchpad bug 1790987 in tripleo "rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 19:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 19:10 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 19:17 |
| *** rbrady has quit IRC | 19:27 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition https://review.openstack.org/595374 | 19:29 |
| *** beekneemech has quit IRC | 19:29 | |
| *** mjturek has quit IRC | 19:30 | |
| *** mjturek has joined #tripleo | 19:32 | |
| nhicher | weshay: that's strange, the zuul job failed https://review.rdoproject.org/zuul/stream.html?uuid=991b56f8a3b940588fe8904fe94652f9&logfile=console.log, but I'm on the undercloud node and the stack status is 'CREATE_IN_PROGRESS' | 19:32 |
| *** EvilienM is now known as EmilienM | 19:33 | |
| *** mjturek has quit IRC | 19:35 | |
| *** jbadiapa has quit IRC | 19:36 | |
| openstackgerrit | Merged openstack/tripleo-common master: Reintroduce empty_container for tripleoclient https://review.openstack.org/600467 | 19:40 |
| mwhahaha | hey it finally landed | 19:40 |
| *** mjturek has joined #tripleo | 19:41 | |
| *** spartakos has quit IRC | 19:49 | |
| *** dprince has quit IRC | 19:53 | |
| *** radeks__ has joined #tripleo | 19:53 | |
| *** radeks_ has quit IRC | 19:54 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 20:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 20:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 20:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 20:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 20:10 |
| *** med_ has joined #tripleo | 20:12 | |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Use generic names for container platform https://review.openstack.org/600847 | 20:13 |
| *** agopi is now known as agopi|off | 20:14 | |
| weshay | bah.. | 20:15 |
| weshay | gate reset? | 20:15 |
| *** agopi|off has quit IRC | 20:19 | |
| openstackgerrit | Sam Doran proposed openstack/ansible-role-openstack-operations master: Make role support multiple container runtimes https://review.openstack.org/600859 | 20:20 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Modify reproducer to work with /tmp/deployed_nodes.yml https://review.openstack.org/600860 | 20:21 |
| *** mjturek has quit IRC | 20:23 | |
| openstackgerrit | Sam Doran proposed openstack/tripleo-common master: Use known_hosts module for tripleo-ssh-known-hosts role https://review.openstack.org/550278 | 20:24 |
| *** khyr0n has quit IRC | 20:28 | |
| *** med_ has quit IRC | 20:31 | |
| *** trown is now known as trown|outtypewww | 20:32 | |
| *** boazel has quit IRC | 20:33 | |
| *** mjturek has joined #tripleo | 20:35 | |
| *** jtomasek has quit IRC | 20:36 | |
| weshay | mwhahaha, that is some whacky zuul shit | 20:40 |
| mwhahaha | pretty much | 20:40 |
| mwhahaha | it's since they are both v3 now, the configs get shared to a certain extent | 20:40 |
| weshay | mwhahaha, I guess they are peers | 20:41 |
| weshay | and not parent/child | 20:41 |
| weshay | meh | 20:41 |
| weshay | wow rocky promoted | 20:41 |
| mwhahaha | frenemies | 20:41 |
| weshay | mwhahaha, fyi.. master should promote any minute | 20:41 |
| mwhahaha | A LIKELY STORY | 20:41 |
| *** ansmith has quit IRC | 20:42 | |
| mwhahaha | so i guess we should probably cut rc2 then | 20:42 |
| weshay | mwhahaha, you need better promotion jokes | 20:42 |
| mwhahaha | i need to get with folks next week about what's left and if we have any major blockers | 20:42 |
| mwhahaha | i took a look at the bugs and there didn't seem to be any other than that random nova deploy thing | 20:42 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-common master: Disable external plugins in rabbitmq container https://review.openstack.org/600426 | 20:43 |
| mwhahaha | do we still know if https://bugs.launchpad.net/tripleo/+bug/1782317 is happening? | 20:43 |
| openstack | Launchpad bug 1782317 in tripleo "[master] scenario008 multinode job failing at undercloud giving Invalid local_interface specified. br-ex is not available." [Critical,Triaged] | 20:43 |
| *** zaneb has quit IRC | 20:46 | |
| *** mjturek has quit IRC | 20:51 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Modify reproducer to work with /tmp/deployed_nodes.yml https://review.openstack.org/600860 | 20:52 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Honor timeout with config-download https://review.openstack.org/586358 | 20:56 |
| *** jtomasek has joined #tripleo | 21:00 | |
| *** spartakos has joined #tripleo | 21:01 | |
| *** rh-jelabarre has quit IRC | 21:02 | |
| *** spartakos has quit IRC | 21:02 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Modify reproducer to work with /tmp/deployed_nodes.yml https://review.openstack.org/600860 | 21:04 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 21:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 21:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 21:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 21:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 21:10 |
| *** holser_ has joined #tripleo | 21:17 | |
| *** ssbarnea|bkp2 has quit IRC | 21:19 | |
| *** takamatsu has joined #tripleo | 21:28 | |
| *** holser_ has quit IRC | 21:35 | |
| *** tosky has quit IRC | 21:40 | |
| *** rdopiera has joined #tripleo | 21:40 | |
| *** radeks__ has quit IRC | 21:41 | |
| *** tzumainn has quit IRC | 21:44 | |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: DNM: Add python3 fedora standalone https://review.openstack.org/600871 | 21:44 |
| *** med_ has joined #tripleo | 21:45 | |
| *** rdopiera has quit IRC | 21:46 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: udpate reproducer to install required deps https://review.openstack.org/600836 | 21:46 |
| *** myoung|dr is now known as myoung | 21:49 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: the virt resize seems to fail when executed by ansible https://review.openstack.org/600872 | 21:51 |
| *** agopi|off has joined #tripleo | 21:52 | |
| *** gbarros has quit IRC | 21:53 | |
| *** spartakos has joined #tripleo | 21:56 | |
| *** jtomasek has quit IRC | 21:58 | |
| *** Chaserjim has quit IRC | 21:58 | |
| *** abishop has quit IRC | 21:58 | |
| *** Chaserjim has joined #tripleo | 21:59 | |
| *** abishop has joined #tripleo | 22:00 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 22:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 22:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 22:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 22:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 22:10 |
| *** mcornea has quit IRC | 22:11 | |
| *** spartakos has quit IRC | 22:14 | |
| *** rfolco has quit IRC | 22:15 | |
| *** spartakos has joined #tripleo | 22:15 | |
| *** saneax has joined #tripleo | 22:17 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: udpate reproducer to install required deps https://review.openstack.org/600836 | 22:18 |
| *** saneax has quit IRC | 22:22 | |
| *** spartakos has quit IRC | 22:23 | |
| *** spartakos has joined #tripleo | 22:28 | |
| *** lblanchard has joined #tripleo | 22:54 | |
| *** rlandy has quit IRC | 22:55 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 23:10 |
| openstack | Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1790489 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791169 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1791332 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 23:10 |
| openstack | Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] | 23:10 |
| openstack | Launchpad bug 1791169 in tripleo "zuul post-failure: collect logs timing out" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 23:10 |
| openstack | Launchpad bug 1791332 in tripleo "[CI] New image prepare workflow is incompatible with updates and upgrades" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 23:10 |
| *** ansmith has joined #tripleo | 23:16 | |
| *** spartakos has quit IRC | 23:17 | |
| *** zaneb has joined #tripleo | 23:25 | |
| *** sri_ has quit IRC | 23:27 | |
| *** sri_ has joined #tripleo | 23:28 | |
| *** lblanchard has quit IRC | 23:42 | |
| *** sri_ has quit IRC | 23:46 | |
| *** sri_ has joined #tripleo | 23:46 | |
| *** sri_ has quit IRC | 23:51 | |
| *** sri_ has joined #tripleo | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
