*** ooolpbot has joined #tripleo | 00:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
---|---|---|
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 00:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 00:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 00:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 00:10 |
*** ooolpbot has quit IRC | 00:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 00:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 00:10 |
*** pmannidi has quit IRC | 00:11 | |
*** artom has joined #tripleo | 00:13 | |
*** rlandy has quit IRC | 00:17 | |
*** bfournie has joined #tripleo | 00:20 | |
*** bfournie has quit IRC | 00:22 | |
*** bfournie has joined #tripleo | 00:25 | |
openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Add pre-selected Control Plane IP support. https://review.openstack.org/413278 | 00:29 |
*** pmannidi has joined #tripleo | 00:31 | |
*** itlinux has joined #tripleo | 00:44 | |
*** jbadiapa has quit IRC | 00:52 | |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 00:54 |
*** itlinux has quit IRC | 00:55 | |
*** itlinux has joined #tripleo | 00:56 | |
*** itlinux has quit IRC | 00:56 | |
*** rhallisey has quit IRC | 00:58 | |
*** jbadiapa has joined #tripleo | 00:59 | |
*** mdnadeem has joined #tripleo | 01:05 | |
*** japestinho has joined #tripleo | 01:06 | |
*** ooolpbot has joined #tripleo | 01:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 01:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 01:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 01:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 01:10 |
*** ooolpbot has quit IRC | 01:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 01:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 01:10 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: PPC bootloader; install to root partition https://review.openstack.org/471525 | 01:11 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: PPC bootloader; install to boot partition https://review.openstack.org/471525 | 01:16 |
*** trozet has quit IRC | 01:17 | |
*** tzumainn has quit IRC | 01:17 | |
*** trozet has joined #tripleo | 01:17 | |
*** tzumainn has joined #tripleo | 01:17 | |
openstackgerrit | Victoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila Share service https://review.openstack.org/471527 | 01:18 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: PPC bootloader; install to boot partition https://review.openstack.org/471525 | 01:24 |
*** gvrangan_ has quit IRC | 01:29 | |
*** fzdarsky_ has joined #tripleo | 01:30 | |
*** fzdarsky has quit IRC | 01:33 | |
*** atoth has quit IRC | 01:34 | |
*** itlinux has joined #tripleo | 01:39 | |
*** itlinux has quit IRC | 01:45 | |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add a keep-output flag for functional tests https://review.openstack.org/471532 | 01:48 |
openstackgerrit | Vu Cong Tuan proposed openstack/tripleo-common master: Using assertIsNone(xxx) instead of assertEqual(None, xxx) https://review.openstack.org/470610 | 01:50 |
*** chlong has quit IRC | 01:50 | |
openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Make overcloud_containers.yaml template driven https://review.openstack.org/471537 | 01:55 |
*** itlinux has joined #tripleo | 01:57 | |
*** yamahata has quit IRC | 01:59 | |
*** gbarros has joined #tripleo | 02:04 | |
*** itlinux has quit IRC | 02:05 | |
*** ooolpbot has joined #tripleo | 02:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 02:10 |
*** ooolpbot has quit IRC | 02:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 02:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 02:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 02:10 |
*** mdnadeem has quit IRC | 02:11 | |
*** gbarros has quit IRC | 02:21 | |
*** mdnadeem has joined #tripleo | 02:22 | |
*** cdearborn_ has quit IRC | 02:50 | |
*** links has joined #tripleo | 02:52 | |
*** egafford has joined #tripleo | 03:00 | |
*** egafford has quit IRC | 03:04 | |
*** ooolpbot has joined #tripleo | 03:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 03:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 03:10 |
*** ooolpbot has quit IRC | 03:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 03:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 03:10 |
*** tzumainn has quit IRC | 03:12 | |
*** thrash is now known as thrash|g0ne | 03:17 | |
*** dbecker has quit IRC | 03:19 | |
*** jmelvin has quit IRC | 03:21 | |
*** itlinux has joined #tripleo | 03:22 | |
*** owalsh_ has joined #tripleo | 03:24 | |
*** psahoo has joined #tripleo | 03:27 | |
*** owalsh has quit IRC | 03:27 | |
openstackgerrit | zhongshengping proposed openstack/puppet-tripleo master: Fix the default port for Panko API https://review.openstack.org/471567 | 03:29 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update tracing in block_device_create_config_file https://review.openstack.org/471163 | 03:31 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move ppc block-device default to right $ARCH https://review.openstack.org/471170 | 03:31 |
*** dbecker has joined #tripleo | 03:33 | |
openstackgerrit | Vu Cong Tuan proposed openstack/python-tripleoclient master: Remove support for py34 https://review.openstack.org/471572 | 03:42 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Pass all blockdevices to bootloader https://review.openstack.org/471183 | 03:49 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: PPC bootloader; install to boot partition https://review.openstack.org/471525 | 03:49 |
openstackgerrit | Vu Cong Tuan proposed openstack/tripleo-common master: Remove support for py34 https://review.openstack.org/471574 | 03:50 |
*** atheurer has quit IRC | 03:54 | |
*** japestinho has quit IRC | 03:55 | |
*** yprokule has joined #tripleo | 03:57 | |
*** ykarel_ has joined #tripleo | 03:59 | |
*** ramishra has quit IRC | 03:59 | |
openstackgerrit | zhongshengping proposed openstack/instack-undercloud master: Fix the port for Panko API https://review.openstack.org/471577 | 04:02 |
openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart master: Add virt power to root auth keys when qemu system https://review.openstack.org/471578 | 04:03 |
*** atheurer has joined #tripleo | 04:04 | |
*** ooolpbot has joined #tripleo | 04:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 04:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 04:10 |
*** ooolpbot has quit IRC | 04:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 04:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 04:10 |
*** pmannidi has quit IRC | 04:12 | |
*** skramaja has joined #tripleo | 04:13 | |
openstackgerrit | Merged openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/470451 | 04:15 |
openstackgerrit | Merged openstack/diskimage-builder master: Use https in docs links https://review.openstack.org/470791 | 04:16 |
*** pmannidi has joined #tripleo | 04:29 | |
*** janki has joined #tripleo | 04:31 | |
openstackgerrit | zhongshengping proposed openstack/puppet-tripleo master: Fix the port for Panko API https://review.openstack.org/471567 | 04:34 |
*** dparkes has quit IRC | 04:40 | |
*** Aakash has quit IRC | 04:44 | |
*** atheurer has quit IRC | 04:46 | |
*** itlinux has quit IRC | 04:49 | |
*** florianf has joined #tripleo | 04:52 | |
*** pmannidi has quit IRC | 04:54 | |
*** pcaruana has joined #tripleo | 04:56 | |
*** ooolpbot has joined #tripleo | 05:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 05:10 |
*** ooolpbot has quit IRC | 05:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 05:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 05:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 05:10 |
*** pmannidi has joined #tripleo | 05:12 | |
*** pgadiya has joined #tripleo | 05:12 | |
*** dsariel_ has joined #tripleo | 05:13 | |
*** pcaruana has quit IRC | 05:14 | |
*** bkopilov has joined #tripleo | 05:14 | |
*** prateek has joined #tripleo | 05:15 | |
*** ykarel__ has joined #tripleo | 05:16 | |
*** ykarel_ has quit IRC | 05:18 | |
*** nyechiel has joined #tripleo | 05:20 | |
*** pgadiya has quit IRC | 05:26 | |
openstackgerrit | Saravanan KR proposed openstack/tripleo-common master: Move passwords from mistral environment to swift environment https://review.openstack.org/467909 | 05:27 |
*** mdnadeem has quit IRC | 05:28 | |
yolanda | thrash|g0ne, bnemec , that is true, we pass the repos in an env var. I did on https://review.openstack.org/471481. But without delorean-repo element, these vars are not used anywhere, it'd delorean-repo eelment that uses them and installs the repo on the image | 05:34 |
*** paramite has joined #tripleo | 05:35 | |
*** aufi has joined #tripleo | 05:35 | |
*** numan_ has joined #tripleo | 05:36 | |
yolanda | bnemec i have been testing on a vm, the combination of 471479 and 471481 produced a successful image | 05:39 |
openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Enable DPDK on boot using PreNetworkConfig https://review.openstack.org/467490 | 05:42 |
*** lmiccini has joined #tripleo | 05:44 | |
*** mdnadeem has joined #tripleo | 05:45 | |
*** yprokule has quit IRC | 05:47 | |
*** nicky has joined #tripleo | 05:48 | |
*** agurenko has joined #tripleo | 05:51 | |
openstackgerrit | yolanda.robla proposed openstack/tripleo-common master: Add delorean-repo element to centos https://review.openstack.org/471479 | 05:52 |
*** dparkes has joined #tripleo | 05:53 | |
*** ratailor has joined #tripleo | 05:55 | |
*** noslzzp has joined #tripleo | 05:55 | |
*** mdnadeem has quit IRC | 06:05 | |
openstackgerrit | David Sariel proposed openstack/tripleo-heat-templates master: Increase default for CinderLVMLoopDeviceSize https://review.openstack.org/470745 | 06:06 |
openstackgerrit | David Sariel proposed openstack/tripleo-heat-templates master: Increase default for CinderLVMLoopDeviceSize https://review.openstack.org/470745 | 06:08 |
*** ooolpbot has joined #tripleo | 06:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 06:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 06:10 |
*** ooolpbot has quit IRC | 06:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 06:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 06:10 |
*** jaosorior_away is now known as jaosorior | 06:11 | |
*** bkopilov has quit IRC | 06:17 | |
openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: Configuration for containerized MySQL clients https://review.openstack.org/466531 | 06:18 |
*** udesale has joined #tripleo | 06:19 | |
*** jprovazn has joined #tripleo | 06:19 | |
*** mdnadeem has joined #tripleo | 06:20 | |
*** rcernin has joined #tripleo | 06:21 | |
*** mcornea has joined #tripleo | 06:23 | |
*** aufi has quit IRC | 06:25 | |
*** radeks has joined #tripleo | 06:26 | |
yolanda | bnemec, so i updated the change with comments, please take a look. My only concern is that i'd really like to include delorean-repo conditionally, as it was done before. But i don't see how to do it with yaml | 06:31 |
*** anshulbehl has joined #tripleo | 06:31 | |
*** anshulbehl has left #tripleo | 06:32 | |
*** noslzzp has quit IRC | 06:38 | |
*** hjensas has quit IRC | 06:42 | |
*** b00tcat_ has joined #tripleo | 06:48 | |
*** b00tcat has quit IRC | 06:48 | |
openstackgerrit | Merged openstack/diskimage-builder master: Add a keep-output flag for functional tests https://review.openstack.org/471532 | 06:50 |
*** athomas has joined #tripleo | 06:53 | |
*** ffiore has joined #tripleo | 06:53 | |
*** bogdando has joined #tripleo | 07:00 | |
*** afazekas has quit IRC | 07:01 | |
*** afazekas has joined #tripleo | 07:01 | |
*** pcaruana has joined #tripleo | 07:03 | |
*** ooolpbot has joined #tripleo | 07:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 07:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 07:10 |
*** ooolpbot has quit IRC | 07:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 07:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 07:10 |
*** limao has quit IRC | 07:10 | |
*** tesseract has joined #tripleo | 07:11 | |
*** limao has joined #tripleo | 07:11 | |
*** mdnadeem has quit IRC | 07:11 | |
*** jlinkes has joined #tripleo | 07:12 | |
*** mdnadeem has joined #tripleo | 07:12 | |
*** flepied has quit IRC | 07:15 | |
*** jaganathan has joined #tripleo | 07:16 | |
*** sid1 has joined #tripleo | 07:18 | |
*** amoralej|off is now known as amoralej | 07:18 | |
*** bkopilov has joined #tripleo | 07:20 | |
*** shardy has joined #tripleo | 07:21 | |
*** aufi has joined #tripleo | 07:23 | |
openstackgerrit | Christian Schwede proposed openstack/puppet-tripleo master: Fix Swift ring management in container deploymnents https://review.openstack.org/468790 | 07:24 |
openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: WIP Fix pacemaker remote resource creation with pcs 0.9.158 https://review.openstack.org/471607 | 07:26 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix upgrade tasks to use correct service name https://review.openstack.org/471421 | 07:27 |
openstackgerrit | Merged openstack/tripleo-ui master: Fix a few forgotten PropTypes imports https://review.openstack.org/465146 | 07:28 |
openstackgerrit | Alfredo Moralejo proposed openstack-infra/tripleo-ci master: DNM: testing something for gate https://review.openstack.org/471361 | 07:28 |
*** limao has quit IRC | 07:28 | |
*** noslzzp has joined #tripleo | 07:29 | |
*** limao has joined #tripleo | 07:29 | |
*** ccamacho has quit IRC | 07:30 | |
*** fragatina has joined #tripleo | 07:32 | |
*** ccamacho has joined #tripleo | 07:33 | |
*** hjensas has joined #tripleo | 07:33 | |
*** oidgar has joined #tripleo | 07:33 | |
*** jpena|off is now known as jpena | 07:34 | |
*** numan_ has quit IRC | 07:35 | |
*** iranzo has quit IRC | 07:39 | |
*** iranzo has joined #tripleo | 07:39 | |
*** iranzo has joined #tripleo | 07:39 | |
openstackgerrit | yolanda.robla proposed openstack/python-tripleoclient master: DNM: Testing experimental https://review.openstack.org/470211 | 07:42 |
*** egonzalez has joined #tripleo | 07:45 | |
*** aufi has quit IRC | 07:45 | |
*** flepied has joined #tripleo | 07:51 | |
*** rcernin has quit IRC | 07:52 | |
*** rcernin has joined #tripleo | 07:52 | |
*** jpich has joined #tripleo | 07:53 | |
openstackgerrit | Christian Schwede proposed openstack/puppet-tripleo master: Fix Swift ring management in container deployments https://review.openstack.org/468790 | 07:56 |
*** zzzeek has quit IRC | 08:00 | |
*** zzzeek has joined #tripleo | 08:00 | |
*** fragatina has quit IRC | 08:00 | |
*** fragatina has joined #tripleo | 08:01 | |
*** pmannidi has quit IRC | 08:02 | |
*** fragatina has quit IRC | 08:06 | |
bogdando | folks, could we merge custom undercloud patch? it seems passing extended CI well https://review.openstack.org/#/c/466322/ | 08:08 |
bogdando | jistr could you +1 ? you was asking for checking upgrades job for it | 08:09 |
bogdando | https://review.openstack.org/#/c/450792/ ^^ | 08:09 |
bogdando | weshay: ^^ | 08:09 |
*** ooolpbot has joined #tripleo | 08:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 08:10 |
*** ooolpbot has quit IRC | 08:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 08:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 08:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 08:10 |
bogdando | shardy, mandre: PTAL, I'd like to have someone's +1/2 from containers DFG please | 08:10 |
shardy | bogdando: I'm not a quickstart expert but lgtm | 08:14 |
*** japestinho has joined #tripleo | 08:15 | |
bogdando | shardy: the main thing for our dfg is README with an example guide I tried to provide, perhaps. And the install script | 08:15 |
bogdando | shardy: thanks! | 08:15 |
bogdando | note, the script is based on the doit.sh the team uses, AFAIK | 08:16 |
*** ramishra has joined #tripleo | 08:16 | |
*** lucas-afk is now known as lucasagomes | 08:16 | |
openstackgerrit | Merged openstack/diskimage-builder master: Update tracing in block_device_create_config_file https://review.openstack.org/471163 | 08:18 |
openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Derive Params (part 1): Initial workflow to start https://review.openstack.org/456469 | 08:20 |
*** numans has joined #tripleo | 08:22 | |
openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Only set the stonith property on the pacemaker_master node https://review.openstack.org/471630 | 08:24 |
yolanda | bnemec, around? | 08:26 |
*** ffiore_ has joined #tripleo | 08:26 | |
yolanda | or thrash|g0ne ? | 08:27 |
*** ffiore has quit IRC | 08:27 | |
*** zoli|gone is now known as zoli | 08:27 | |
*** aufi has joined #tripleo | 08:28 | |
openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Only set the stonith property on the pacemaker_master node https://review.openstack.org/471630 | 08:29 |
EmilienM | yolanda: they are US based, I hope they are sleeping now | 08:30 |
yolanda | EmilienM, i'm having this problem with image builds that are broken | 08:31 |
EmilienM | where? | 08:31 |
yolanda | see https://review.openstack.org/470211 | 08:31 |
yolanda | yesterday we switched from using the legacy builds from tripleoclient to the new ones based on yaml files, on the ci | 08:32 |
*** oneswig has joined #tripleo | 08:32 | |
yolanda | this uncovered a problem, that the delorean-repo, that was added by code on the legacy build, is not added on the images | 08:32 |
yolanda | so they fail | 08:32 |
yolanda | i tested and https://review.openstack.org/471479 solves it, but it also means adding delorean-repo everywhere, not optionally with the flag, as was done before | 08:32 |
yolanda | so i wanted to discuss the best way to do it, i was thinking on update delorean-repo element, to only do any action depending on the old USE_DELOREAN_TRUNK flag, or something similar | 08:33 |
*** iranzo has quit IRC | 08:33 | |
EmilienM | yolanda: does it affect all tripleo CI now? | 08:34 |
yolanda | yes | 08:34 |
EmilienM | yolanda: what jobs are failing now? | 08:34 |
yolanda | well, the buildimae jos, that are on tripleoclient and diskimagebuilder | 08:34 |
EmilienM | just gate-tripleo-buildimage-overcloud-full-centos-7 ? | 08:34 |
yolanda | yes | 08:34 |
EmilienM | ok, not the ovb jobs, right? | 08:34 |
yolanda | no | 08:34 |
EmilienM | I'm asking because we are in the middle of a release | 08:34 |
*** florianf has quit IRC | 08:34 | |
EmilienM | and the last thing I want is a broken CI | 08:34 |
yolanda | i know... | 08:34 |
EmilienM | I see this: http://logs.openstack.org/11/470211/2/check/gate-tripleo-buildimage-overcloud-full-centos-7/f434bba/console.html#_2017-06-07_08_08_16_545898 | 08:35 |
EmilienM | not sure we want delorean-repo element everywhere | 08:35 |
yolanda | but that is the want that configures repo for images, and without that, the build fails, as cannot install the packages: http://logs.openstack.org/11/470211/2/check/gate-tripleo-buildimage-overcloud-full-centos-7/f434bba/console.html#_2017-06-07_08_06_06_759864 | 08:36 |
EmilienM | it sounds like the element doesn't work with old environment, iiuc, which is not backward compatible :( | 08:36 |
EmilienM | maybe updating the element | 08:36 |
yolanda | that's why i was thinking, we could include it, but update delorean-repo element, to only configure things if USE_DELOREAN_TRUNK=1 | 08:36 |
*** iranzo has joined #tripleo | 08:37 | |
*** derekh has joined #tripleo | 08:37 | |
*** florianf has joined #tripleo | 08:38 | |
yolanda | EmilienM, or can you think on another alternative? | 08:38 |
openstackgerrit | Merged openstack/tripleo-docs master: Document hardware types for the undercloud and virtualbmc https://review.openstack.org/468910 | 08:38 |
shardy | I think in CI we've typically used DIB_YUM_REPO_CONF to point at the repos on the host? | 08:39 |
shardy | https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh#L654 | 08:39 |
* shardy hasn't looked at how oooqs does it but I assume similar | 08:39 | |
*** tosky has joined #tripleo | 08:40 | |
yolanda | so i could update the buildimage job, to configure the repo inside the host, then point to it with DIB_YUM_REPO_CONF ? the delorean element is deprecated? | 08:41 |
shardy | yolanda: yeah that may be more consistent with the other ci jobs I think, but perhaps check with sshnaidm|afk or adarazs re what quickstart does for image building | 08:42 |
yolanda | they use DIB_YUM_REPO_CONF, yes | 08:43 |
*** ckyriakidou has joined #tripleo | 08:43 | |
yolanda | actually at some point we could change this buildimage job, and use quickstart on it | 08:43 |
yolanda | anyway, going to fix the buildimage job to reflect it | 08:43 |
shardy | yolanda: sounds good, thanks! | 08:44 |
*** yprokule has joined #tripleo | 08:45 | |
*** limao has quit IRC | 08:45 | |
yolanda | thx for the pointers | 08:45 |
*** limao has joined #tripleo | 08:45 | |
jistr | bogdando: unfortunately it looks like it does break the upgrade :( https://review.openstack.org/#/c/450792/35/roles/undercloud-deploy/tasks/create-scripts.yml | 08:46 |
*** oneswig has quit IRC | 08:46 | |
bogdando | hm, jistr but CI has passed it seems https://review.openstack.org/#/c/466322/, wasn't that case covered by CI? | 08:46 |
jistr | bogdando: it didn't pass, in the link above you pasted, see gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv | 08:47 |
bogdando | jistr, oh, right | 08:47 |
EmilienM | yolanda: yeah, I don't have any other alternative in mind, please go ahead with this proposal | 08:48 |
jistr | bogdando: that permission denied looks a bit as if the jenkins user was trying to delete the t-h-t dir, but it was maybe owned by root or something. Not sure if this is what happened and how, but it looks a bit like it. | 08:48 |
bogdando | jistr: is it possible to retrigger only that case? | 08:48 |
*** oneswig has joined #tripleo | 08:49 | |
jistr | bogdando: i don't think so, we can retrigger everything if we want, but the error doesn't look like an intermittent one... | 08:49 |
openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: [WIP] Add docker templates for octavia services https://review.openstack.org/467254 | 08:49 |
bogdando | jistr: strange, I see nothing that includes the role with become http://codesearch.openstack.org/?q=undercloud-deploy&i=nope&files=&repos= | 08:50 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-image-elements master: Release 7.0.0.0b2 https://review.openstack.org/471642 | 08:53 |
bogdando | jistr: this https://review.openstack.org/#/c/450792/35/roles/overcloud-prep-config/tasks/custom_tht.yml@13 breaks... | 08:54 |
jistr | bogdando: that means it'll run as root, right? | 08:55 |
bogdando | yup | 08:55 |
bogdando | I think I will remove sudo. But that restricts the overcloud_templates_path for home dirs | 08:56 |
bogdando | but it always have been so, as it appears | 08:56 |
jistr | right, the original doesn't have the become https://review.openstack.org/#/c/450792/35/roles/overcloud-prep-config/tasks/main.yml | 08:56 |
bogdando | I added ti cuz I was testing with paths like /usr | 08:56 |
bogdando | but I think it is ok to use only home, /tmp, /var/tmp other places with sticky bits | 08:56 |
*** salmankhan has joined #tripleo | 08:57 | |
bogdando | I will update docs then | 08:57 |
jistr | bogdando: yea i hope it shouldn't matter. The whole play runs as some deployment user (should be the owner of the directory where the t-h-t will be written). So if the deployment user was root (instead of stack or jenkins), it would have root permissions even without `become`. | 08:57 |
*** dparkes has quit IRC | 08:57 | |
*** akrivoka has joined #tripleo | 08:57 | |
bogdando | jistr: no, the idea was to allow non root users to install custom tht by any paths, with sudo | 08:58 |
bogdando | but that doesn't play well with ci | 08:58 |
*** oneswig has left #tripleo | 08:58 | |
jistr | bogdando: ok.. i hope not having the right to write into any paths wouldn't be a big issue... The user which does the deployment and wants to clone the t-h-t, should always be able to clone into their home dir, right? I mean, i don't see the benefit in cloning it somewhere else. | 08:59 |
bogdando | jistr: yes, should not be an issue, given that that's the current restriction anyway | 08:59 |
openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: DO NOT MERGE Testing containerized tacker service https://review.openstack.org/471646 | 09:00 |
jistr | bogdando: i have one more question at https://review.openstack.org/#/c/450792/35/roles/undercloud-deploy/templates/undercloud-deploy.sh.j2 | 09:02 |
jistr | bogdando: oh i see it in the readme now, i'll read it :) | 09:03 |
* jistr didn't scroll to the bottom there | 09:03 | |
openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 09:06 |
openstackgerrit | Merged openstack/tripleo-common master: Parse roles from roles_data.yaml directly https://review.openstack.org/467678 | 09:07 |
openstackgerrit | Merged openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/470147 | 09:08 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/470565 | 09:08 |
*** sshnaidm|afk is now known as sshnaidm | 09:09 | |
bogdando | jistr: clarified the case with required vs update packages | 09:09 |
*** ooolpbot has joined #tripleo | 09:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 09:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 09:10 |
*** ooolpbot has quit IRC | 09:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 09:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 09:10 |
*** florianf has quit IRC | 09:11 | |
openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow https://review.openstack.org/463765 | 09:12 |
*** dparkes has joined #tripleo | 09:15 | |
skramaja | jtomasek: hey. i had to drop off yesterday in the middle of the dicussion, is there anything still open, which we should discuss now? | 09:15 |
jistr | bogdando: ack i see, thanks for the explanation. Maybe the installs/updates of packages could at some point be a separate script from `undercloud-deploy.sh`, but now i see how it makes sense as a feature. | 09:18 |
jistr | bogdando: i'm ready to +2 when https://review.openstack.org/#/c/466322/ is green | 09:19 |
bogdando | jistr, thanks! | 09:19 |
openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli https://review.openstack.org/463728 | 09:20 |
openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow https://review.openstack.org/463765 | 09:22 |
yolanda | EmilienM, shardy : https://review.openstack.org/#/c/471647/ | 09:22 |
*** florianf has joined #tripleo | 09:22 | |
openstackgerrit | Lingyong Xu proposed openstack/instack master: Drop py34 target in tox.ini https://review.openstack.org/471663 | 09:23 |
EmilienM | yolanda: +1 | 09:27 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Ability to enable/disable debug mode per OpenStack service https://review.openstack.org/467698 | 09:27 |
EmilienM | shardy: upgrade jobs are voting, I wonder if we want them voting | 09:27 |
EmilienM | err | 09:27 |
EmilienM | shardy: upgrade jobs are passing (except one scenario), I wonder if we want them voting | 09:27 |
openstackgerrit | Merged openstack/instack-undercloud master: Remove neutron ml2 plugin linuxbridge mechanism_driver https://review.openstack.org/471152 | 09:28 |
shardy | EmilienM: yes I think that would be good | 09:28 |
ccamacho | EmilienM they should, right? | 09:29 |
ccamacho | Is there a reason why not? | 09:29 |
*** limao has quit IRC | 09:29 | |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: When enrolling Dell nodes, use drac_address instead of deprecated drac_host https://review.openstack.org/469918 | 09:29 |
openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: DO NOT MERGE Testing containerized congress service https://review.openstack.org/471674 | 09:30 |
*** dparkes has quit IRC | 09:30 | |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Add workflow for IPMI nodes discovery https://review.openstack.org/462577 | 09:30 |
EmilienM | ccamacho, shardy: there are reasons to NOT make them voting. Like the fact they are not part of the promotion pipeline | 09:30 |
EmilienM | which means they can break tomorrow | 09:31 |
EmilienM | because they don't vote in promotion pipeline | 09:31 |
EmilienM | so if a bad patch merges in Nova, our gate will be broken | 09:31 |
EmilienM | because upgrade jobs will fail and be voting | 09:31 |
EmilienM | and it will happen for sure | 09:32 |
shardy | EmilienM: IMO we should have upgrade coverage in the promotion pipeline | 09:33 |
shardy | everything in our (voting) gate should also run for periodic promotion? | 09:34 |
*** ebarrera has joined #tripleo | 09:34 | |
EmilienM | shardy: it's not so easy | 09:34 |
EmilienM | shardy: sshnaidm is working on having multinode jobs in the promotion pipeline | 09:34 |
EmilienM | shardy: but it's not so easy because multinode jobs are run in infra clouds and they need access to the promotion server | 09:34 |
shardy | EmilienM: Ok, I guess then in the short term we'll have to deal with the regressions when they happen | 09:35 |
EmilienM | shardy: so for security reasons we can't open ports from all IPs | 09:35 |
shardy | EmilienM: can't we have a job on the promotion servers which polls for the status of some infra jobs? | 09:35 |
EmilienM | shardy: I think sshnaidm can tell more about what he's preparing | 09:35 |
jtomasek | skramaja: so there are 2 things with the current workflow. It will probably be better if I comment on the review | 09:36 |
sshnaidm | shardy, we'll have script on mirror-server that will pull results from openstack-health about status of multinode jobs | 09:36 |
shardy | sshnaidm: ah, yeah that's the sort of thing I imagined, sounds good | 09:36 |
EmilienM | sshnaidm: which means we need to run tempest on the multinode jobs, right? | 09:36 |
EmilienM | openstack-health only works with tempest AFIK no? | 09:37 |
shardy | hmm | 09:37 |
EmilienM | unless we fake the subunit file with fake results | 09:37 |
EmilienM | we could do that in the meantime | 09:37 |
sshnaidm | EmilienM, no, but with subunit. I created subunit results for pingtest too :) | 09:37 |
EmilienM | fake a subunit file if job didn't fail | 09:37 |
EmilienM | ahah you rocks | 09:37 |
* EmilienM waves on sshnaidm | 09:37 | |
shardy | sounds good :) | 09:37 |
*** mdnadeem has quit IRC | 09:38 | |
sshnaidm | http://status.openstack.org/openstack-health/#/g/project/openstack-infra~2Ftripleo-ci | 09:38 |
sshnaidm | first jobs were registered today as I see | 09:38 |
EmilienM | that's really cool | 09:38 |
EmilienM | so now we can subscribe to RSS when it fails :D | 09:39 |
EmilienM | openstack-health is really cool for these who don't know it, I suggest to have a look | 09:39 |
EmilienM | sshnaidm, shardy, ccamacho: so in conclusion. Once we have promotion jobs working with multinode (sounds super close) - let's add upgrade jobs | 09:40 |
EmilienM | and also all scenarios | 09:40 |
EmilienM | and then I'm ok to make them voting | 09:40 |
EmilienM | shardy: so everytime something breaks in upgrades, we can escalate to the right team while our CI keeps working | 09:40 |
EmilienM | I prefer to not have a promotion during 5 days instead of having our CI full red during 5 days | 09:41 |
EmilienM | what do you think? | 09:41 |
*** limao has joined #tripleo | 09:43 | |
shardy | EmilienM: yes I agree | 09:43 |
*** owalsh_ is now known as owalsh | 09:44 | |
EmilienM | cool | 09:45 |
openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Pre-upgrade: Check disk space https://review.openstack.org/471686 | 09:45 |
*** karimb has joined #tripleo | 09:46 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/471689 | 09:47 |
*** limao has quit IRC | 09:47 | |
EmilienM | jtomasek: how are we doing with ui today? :) | 09:49 |
owalsh | bogdando, jaosorior: FYI I raised a bug for the /etc/ssh/ssh_known_hosts issue on the undercloud - https://bugs.launchpad.net/tripleo/+bug/1696283 | 09:50 |
openstack | Launchpad bug 1696283 in tripleo "Containerized undercloud fails to mount /etc/ssh/ssh_known_hosts when /etc/ is mounted readonly" [Medium,Triaged] - Assigned to Oliver Walsh (owalsh) | 09:50 |
jtomasek | EmilienM: RDO should be up to date now, I think we're ready to release | 09:50 |
EmilienM | yeah? | 09:50 |
openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add pre-upgrade group to RAM/disk space validations https://review.openstack.org/471691 | 09:50 |
owalsh | bogdando, jaosorior: I'm not missing anything obvious, am I? This looks like a docker bug - maybe specific to devicemapper. | 09:51 |
jtomasek | I'll give you final confirmation when I get honza's and florianf's confirmation | 09:51 |
jtomasek | EmilienM: ^ | 09:51 |
EmilienM | ok ok ok | 09:51 |
EmilienM | jtomasek: also, please patch tripleo-ui with tag bump if not done already | 09:52 |
bogdando | owalsh: it seems that undercloud is configured by CI scripts to use overlay2. While the overlcoud nodes' overlay2 is ensured by t-h-t docker service template | 09:52 |
EmilienM | jtomasek: don't forget to update releasenotes/source/conf.py also | 09:52 |
bogdando | owalsh: but yeah, by default undercloud deployed with devicemapper, by quickstart | 09:52 |
jtomasek | EmilienM: ok | 09:53 |
*** fzdarsky_ is now known as fzdarsky|lunch | 09:58 | |
*** fragatina has joined #tripleo | 10:00 | |
openstackgerrit | Merged openstack/diskimage-builder master: Move ppc block-device default to right $ARCH https://review.openstack.org/471170 | 10:02 |
ebarrera | EmilienM, can you take a quick look to https://bugzilla.redhat.com/show_bug.cgi?id=1455503 ? | 10:07 |
openstack | bugzilla.redhat.com bug 1455503 in openstack-tripleo-heat-templates "external ip not crrectly set for apache on 1+1 plus isolated ctlplane + external" [Unspecified,New] - Assigned to emacchi | 10:07 |
*** ffiore_ has quit IRC | 10:08 | |
ebarrera | EmilienM, I'm not sure if I'm doing something wrong or it's really a bug.. some help would be nice to move the case forward | 10:08 |
openstackgerrit | yolanda.robla proposed openstack/python-tripleoclient master: DNM: Testing experimental https://review.openstack.org/470211 | 10:09 |
*** ooolpbot has joined #tripleo | 10:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 10:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 10:10 |
*** ooolpbot has quit IRC | 10:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 10:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 10:10 |
owalsh | bogdando, jaosorior: reproduced https://bugs.launchpad.net/tripleo/+bug/1696283 using overlay2 | 10:13 |
openstack | Launchpad bug 1696283 in tripleo "Containerized undercloud fails to mount /etc/ssh/ssh_known_hosts when /etc/ is mounted readonly" [Medium,Triaged] - Assigned to Oliver Walsh (owalsh) | 10:13 |
*** lucasagomes has quit IRC | 10:15 | |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Stop/disable l3 agent in docker service upgrade_tasks https://review.openstack.org/469924 | 10:16 |
numans | EmilienM, shardy Hi, can you please add this to your review queue - https://review.openstack.org/#/c/436767/ | 10:16 |
*** ffiore_ has joined #tripleo | 10:17 | |
openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: quickstart transition: add promotion logic https://review.openstack.org/445467 | 10:18 |
*** lucasagomes has joined #tripleo | 10:19 | |
EmilienM | numans: we still don't have testing for ovn, I see your patch still in progress | 10:22 |
numans | EmilienM, sorry for the wrong link - https://review.openstack.org/#/c/436767/ | 10:23 |
shardy | numans: does OVN really need it's own VIP vs just routing traffic via one of the per-network ones? | 10:23 |
openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: quickstart transition: add promotion logic https://review.openstack.org/445467 | 10:23 |
shardy | we've got one for Redis, but that was a special case and not something we really want to replicate | 10:23 |
EmilienM | numans: I think the links was good. My point is that we still don't have tests for OVN in our CI and you're asking me to review something that we haven't test | 10:25 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Add host logging for redis service template https://review.openstack.org/471709 | 10:26 |
numans | EmilienM, From what i spoke with the rdo team, they want to have a update job in CI to test from ovs 2.6 to ovs 2.7 before using ovs 2.7 | 10:26 |
numans | EmilienM, i will follow up with them. | 10:27 |
*** sai has quit IRC | 10:27 | |
numans | shardy, I think it needs its own VIP. However I will think more on it and see if the existing VIP can be used and comeback on it | 10:28 |
openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test ci patch https://review.openstack.org/471710 | 10:28 |
shardy | numans: ack, it'd be better to avoid it if possible | 10:29 |
*** rook has quit IRC | 10:29 | |
shardy | numans: context is I'm working composable networking atm, and special-cases like this are kind of a hassle in that model | 10:29 |
openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart master: Create configuration for periodic jobs in featureset002 https://review.openstack.org/463739 | 10:30 |
*** rook has joined #tripleo | 10:30 | |
shardy | vs just mapping a service to a vip via ServiceNetMapDefaults | 10:30 |
*** sai has joined #tripleo | 10:30 | |
numans | shardy, ok. | 10:30 |
*** rook is now known as Guest52590 | 10:30 | |
*** agurenko_ has joined #tripleo | 10:30 | |
numans | shardy, this is the puppet tripelo patch for OVN HA. Basically i want ovn server to be started on the vip node. | 10:31 |
ccamacho | EmilienM shardy **sorry was in a meeting, yeahp I agree with having it voting after having working the promotion jobs with multinodes. thanks | 10:31 |
numans | shardy, i guess in the case where existing vip is used, I need to make sure that VIP is created before ovn::ocf resource is created. | 10:32 |
shardy | numans: we create all the vips before any services/nodes | 10:32 |
openstackgerrit | Bogdan Dobrelya proposed openstack-infra/tripleo-ci master: DO NOT MERGE testing 450792 https://review.openstack.org/466322 | 10:33 |
numans | shardy, ok. cool. in that case, creating ovn::ocf resource and setting the proper colocation constraints would do. i will do the testing | 10:33 |
openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add pre-upgrade group to undercloud RAM validation https://review.openstack.org/471691 | 10:33 |
numans | shardy, thanks for the comments. | 10:33 |
*** agurenko has quit IRC | 10:34 | |
shardy | numans: thanks, I added a comment on the review but we can revisit if there's a specific reason ServiceNetMapDefaults won't work in this case | 10:34 |
*** ubijtsa is now known as assassin | 10:37 | |
openstackgerrit | Merged openstack/tripleo-quickstart-extras master: collect-logs: Remove collection of docker etc/ dirs https://review.openstack.org/447176 | 10:38 |
numans | shardy, for ovn there is already a map in the ServiceNetMapDefaults - https://github.com/openstack/tripleo-heat-templates/blob/master/network/service_net_map.j2.yaml#L80 | 10:39 |
shardy | EmilienM: hey I wanted to draw your attention to https://review.openstack.org/#/c/468292/ - sounds like we may have to change the release model for os-*-config | 10:40 |
shardy | If you're OK with that I can update the review | 10:40 |
EmilienM | shardy: big +1 | 10:42 |
*** ffiore_ has quit IRC | 10:43 | |
*** japestinho has quit IRC | 10:45 | |
*** dsariel_ has quit IRC | 10:45 | |
*** ffiore_ has joined #tripleo | 10:47 | |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Ensure /etc/ssh/ssh_known_hosts exist in docker config-data. https://review.openstack.org/471475 | 10:47 |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Map /etc/ssh/ssh_known_hosts to all containers https://review.openstack.org/468461 | 10:48 |
*** jprovazn has quit IRC | 10:49 | |
*** nicky has quit IRC | 10:49 | |
owalsh | mandre, bogdando, jaosorior: FYI ^^^ | 10:49 |
owalsh | it only occurs when the files does not already exist in config-data | 10:50 |
owalsh | seems you can mount a file over an existing ro file, but can't mount a new file into an existing ro directory | 10:53 |
bogdando | owalsh: an interesting finding! | 10:55 |
*** rhallisey has joined #tripleo | 10:56 | |
owalsh | bogdando: yea. I've not hit this one before, but then I wouldn't be using ro mounts very often | 10:56 |
bogdando | owalsh: ro mounts are safe for concurrent access | 10:57 |
bogdando | we want this for the most of the config data cases | 10:57 |
bogdando | actually for all cases... | 10:58 |
shardy | bandini, dciabrin: Hey, yesterday we discussed getting HA coverage in the container CI, any idea if there are patches up already to enable that, or do we need to start it? | 10:58 |
bandini | shardy: I believe we would need to start it | 10:59 |
owalsh | bogdando: ack. It's only an issue if /etc is mounted. Usually we mount /etc/<foo> | 10:59 |
shardy | jistr: ^^ just in case it's something you already looked at | 10:59 |
shardy | bandini: ack, it'd be good to get that started if the pieces we need have landed, are we still blocked on some RDO things? | 11:00 |
owalsh | bogdando: I guess it makes sense, creating a new file in /etc/ssh would need to modify the directory entry in the /etc/:ro mount | 11:00 |
janki | marios, hey | 11:01 |
bandini | shardy: yeah we are still waiting on the rdo pcmk stuff (should happen rather soon now, today or tomorrow) | 11:01 |
shardy | bandini: Ok, thanks, pls let me know when it's ready and we can sync again re CI :) | 11:02 |
jistr | shardy, bandini, dciabrin: haven't looked into it yet, but assuming we don't need to do anything else than including the right services, it should be easy from the CI side. We just need to switch the composable_scenario. Do you have a job in mind where we could do this? Maybe the multinode job? It runs in 1.5 hours so it has some space for experiments. | 11:02 |
bandini | shardy: yess! thanks :) | 11:02 |
bandini | jistr: ideally yes a multinode job with 3 controllers | 11:02 |
marios | o/ janki | 11:03 |
*** agurenko_ is now known as agurenko | 11:03 | |
shardy | jistr: yeah I was thinking the multinode job, then later we need pacemaker plus minor updates which has the same requirement | 11:03 |
janki | marios, I am still stuck at the odl_user error same as last time. I checked all puppet manifests for odl_username and it is mentioned properly | 11:03 |
shardy | e.g 3 controllers to test a rolling minor update w/pacemaker | 11:03 |
janki | marios, on second note, could this be because of firewall. pasting logs on pastebin | 11:04 |
jistr | shardy, bandini: right. I hope we could start with 1 all-in-one undercloud w/ "pseudo HA" and then expand the number of nodes and add updates. | 11:04 |
*** lucasagomes is now known as lucas-hungry | 11:04 | |
openstackgerrit | Beth Elwell proposed openstack/tripleo-heat-templates master: TEST ONLY: Check horizon container upgrades https://review.openstack.org/471721 | 11:05 |
shardy | jistr: sure we can start with 1 as a first step | 11:05 |
jistr | shardy, bandini: err s/undercloud/overcloud/ | 11:05 |
marios | janki: so, i don't recall the details of what you were doing (deploying with ODL)... I seem to recall there was a missing parameter and we were digging into the puppet module and found where it was defaulted | 11:05 |
*** shardy is now known as shardy_lunch | 11:06 | |
*** agurenko is now known as agurenko_ | 11:06 | |
janki | marios, https://pastebin.com/MawgVuQt. Check the firewall rules. all ipv4 and ipv6 drop | 11:06 |
janki | marios, correct. that's what we were doing. I understand that parameter (username) mentioend correctly at all required positions | 11:07 |
janki | marios, the failure could be because of ip rules | 11:07 |
marios | janki: so it could be that, but the error in the paste (has failures) may just be because an earlier dependency failed (I'd need more info to check) log all ipv4]: Skipping because of failed dependencies for example | 11:09 |
janki | marios, ohk. Where would I find more logs, like it says (truncated, view all with --long) | 11:10 |
marios | Error: Could not find a suitable provider for odl_user still seems to be the root from that paste | 11:10 |
*** ooolpbot has joined #tripleo | 11:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 11:10 |
*** ooolpbot has quit IRC | 11:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 11:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 11:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 11:10 |
marios | janki: so that is during the puppet run... looks like control1? | 11:10 |
*** jkilpatr has joined #tripleo | 11:10 | |
janki | marios, are these puppet files fetched at run time? because a patch was added to puppet-opendaylight about username yesterday | 11:11 |
marios | janki: so look on that box for more info on the puppet error and dig at specific service logs if you think that is the problem. but again, there still seems from that paste at least something to do with Error: Could not find a suitable provider for odl_user are you sure it is being passed correctly tripleo heat templates <--> puppet-module | 11:11 |
marios | janki: no they will be whatever is on your nodes when you deployed them (the pupet modules i mean, unless you manually update them ) | 11:11 |
janki | marios, I deployed it today right from undercloud | 11:12 |
*** pkovar has joined #tripleo | 11:12 | |
marios | janki: heh well whatever is on your overcloud-full image anyway. | 11:12 |
* marios biab coffee | 11:12 | |
janki | marios, how to add log files in puppet and get those logs effective while deploying? | 11:12 |
*** agurenko_ is now known as agurenko | 11:12 | |
yolanda | so, green light on buildimage: https://review.openstack.org/#/c/470211/ | 11:13 |
skramaja | jtomasek: ack. i will wait for your comments on the review. | 11:15 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart master: Switch containers multinode CI to deploy HA https://review.openstack.org/471724 | 11:15 |
bandini | jistr: ack works for me as a first pass | 11:16 |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Ensure /etc/ssh/ssh_known_hosts exist in docker config-data. https://review.openstack.org/471475 | 11:16 |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Map /etc/ssh/ssh_known_hosts to all containers https://review.openstack.org/468461 | 11:17 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart master: Switch containers multinode CI to deploy HA https://review.openstack.org/471724 | 11:17 |
jistr | bandini, shardy_lunch: on CI side i'd expect something like this -- https://review.openstack.org/471724/ -- though we may need to tweak how environments/docker.yaml and environments/puppet-pacemaker.yaml interact. Maybe we need environments/docker-pacemaker.yaml instead? Anyway, i think we fortunately don't have many problems anymore that would be specific only to CI or using nodepool. So once we can get | 11:20 |
jistr | docker+pacemaker to deploy, and then get it to deploy via OOOQ, the CI should just click into place. | 11:20 |
yolanda | hi thrash|g0ne , so i seem to be building my jobs correctly now, but i see that experimental jobs for ramdisk are broken now: http://logs.openstack.org/11/470211/3/experimental/gate-tripleo-buildimage-agent-ramdisk-centos-7-nv/cf3ab6b/console.html | 11:21 |
yolanda | it tries to create the image name agent-ramdisk, but that's not defined on the overcloud yamls | 11:22 |
*** dsariel_ has joined #tripleo | 11:22 | |
yolanda | actually there is no ramdisk definition there, just overcloud | 11:22 |
bandini | jistr: I have a review dor docker-ha.yaml up somewhere | 11:22 |
bandini | jistr: I need to clean it up though | 11:22 |
jistr | bandini: ah nice, so yea i think we'd need to tweak OOOQ a bit to be able to deploy with that | 11:23 |
marios | janki: so the puppet aply logs will be on the node itself, i'd probably start with the journal or even vi /var/log/messages | 11:23 |
bandini | jistr: https://review.openstack.org/#/c/471384/ right now am duplicating docker.yaml so the operator does not have to do -e docker and -e ha-docker, but maybe we just prefer not duplicating the services and use both environments in the right order | 11:23 |
janki | marios, will do | 11:23 |
bandini | jistr: dciabrin just finished convincing me to not duplicate stuff in the yaml file and have the op specify -e docker *and* -e ha-docker, so I will do that later today | 11:24 |
*** pkovar has quit IRC | 11:26 | |
*** b00tcat_ has quit IRC | 11:26 | |
jistr | bandini: yea sounds a bit better to me too... Eventually, i assume we'd switch the entries in the default resource registry anyway, so docker.yaml would essentially go away. So yea i'd optimize for the scenario which includes both. | 11:26 |
jistr | and maybe at some point we'd also make docker-ha.yaml to be the default too, and both would go away | 11:27 |
bandini | jistr: 2 against 1. I will totally change it now :D | 11:27 |
*** jaganathan has quit IRC | 11:27 | |
bandini | jistr: agreed, makes sense | 11:27 |
*** athomas has quit IRC | 11:33 | |
dciabrin | bandini, oh that's great :) | 11:34 |
*** athomas has joined #tripleo | 11:34 | |
janki | marios, thanks :) | 11:37 |
*** fragatina has quit IRC | 11:38 | |
*** fzdarsky|lunch is now known as fzdarsky | 11:38 | |
*** zoli is now known as zoli|brb | 11:39 | |
*** oanson has quit IRC | 11:40 | |
*** Guest52590 is now known as rook | 11:40 | |
*** ffiore_ has quit IRC | 11:41 | |
rook | shardy_lunch zaneb -- os-collect-config - when i see ErrorResponse on the overcloud nodes, does that mean there was a issue reading from Swift? | 11:41 |
*** milan has joined #tripleo | 11:41 | |
*** oanson has joined #tripleo | 11:42 | |
*** jpena is now known as jpena|lunch | 11:42 | |
sshnaidm | arxcruz, have you tried multinode with tempest? | 11:42 |
*** shardy_lunch is now known as shardy | 11:43 | |
shardy | rook: depends on the error but possibly - what did it say? | 11:43 |
shardy | rook: my normal test is to grab the request URL from the config file and download it with curl | 11:44 |
*** thrash|g0ne is now known as thrash | 11:44 | |
rook | <ErrorResponse><Error><Message>The request processing has failed due to an internal error:Timed out waiting for a reply to message ID | 11:44 |
*** karimb has quit IRC | 11:44 | |
rook | I think I remember hitting something like this, and it was due to Swift | 11:44 |
shardy | rook: yeah, sounds like something isn't working on the undercloud - do swift CLI commands work OK there? | 11:44 |
thrash | yolanda: I'm pretty sure that job can just go away | 11:44 |
rook | shardy: this isn't a env that I have access to, just one I am trying to help remotely debug. | 11:45 |
*** karimb has joined #tripleo | 11:45 | |
*** dparkes has joined #tripleo | 11:45 | |
thrash | openstackgerrit: or, it needs to be renamed ironic-python-agent | 11:45 |
rook | shardy the operator is having issue scaling up - always fails ~50 nodes (newton). | 11:45 |
thrash | instead of ramdisk-agent | 11:45 |
shardy | rook: ack, I'd start by checking the status of the swift services on the undercloud | 11:45 |
rook | shardy I wonder if there is a issue with how swift is configured... Maybe this was a earlier release. | 11:45 |
rook | I know we hit swift issues early, but got them fixed. | 11:46 |
shardy | rook: not running out of resources perhaps as the deploy gets bigger, and having something get killed on the undercloud? | 11:46 |
*** agurenko is now known as agurenko_ | 11:46 | |
*** karimb has quit IRC | 11:46 | |
thrash | yolanda: so s/agent-ramdisk/python-ironic-agent/ | 11:46 |
openstackgerrit | Merged openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/471689 | 11:47 |
thrash | yolanda: and deploy-ramdisk just needs to go away | 11:48 |
thrash | yolanda: sending up a patch now | 11:48 |
rook | shardy: thanks! | 11:48 |
*** karimb has joined #tripleo | 11:48 | |
*** jprovazn has joined #tripleo | 11:49 | |
janki | marios, think found the root Error: /Stage[main]/Opendaylight::Install/File_line[java_options_systemd]: Could not evaluate: No such file or directory - /usr/lib/systemd/system/opendaylight.service | 11:50 |
*** atoth has joined #tripleo | 11:50 | |
openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: Don't create networks if neutron is not enabled https://review.openstack.org/471736 | 11:53 |
*** abishop has joined #tripleo | 11:54 | |
marios | janki: cool... so looks like the service isn't in the image? You may just manually install it for now as workaround? | 11:54 |
janki | marios, yes. will do that. BUt i did give correct environment argument to overcloud-deploy | 11:55 |
janki | marios, where else to enforce that the service gets included in the image | 11:56 |
*** florianf has quit IRC | 11:56 | |
marios | janki: so you may have included the env files/the service manifest (assume you have the service in roles data etc etc but if it was never installed on the image then you can't use it) | 11:57 |
janki | marios, how to get it installed in the image? | 11:58 |
marios | janki: so grep tells me it is already there | 11:58 |
marios | janki: it needs to go to tripleo-puppet-modules sec finding link for you | 11:59 |
janki | marios, where and what did you grep (sorry bugging you) | 11:59 |
*** sid1 has quit IRC | 11:59 | |
adarazs | EmilienM: do you know which bugs are still in the way of a successful promotion? | 12:00 |
marios | janki: unless you used tripleoquickstart stable/ocata some older version that doesn't have this@ https://github.com/openstack/tripleo-puppet-elements/blob/master/elements/overcloud-controller/install.d/package-installs-overcloud-controller#L66 | 12:00 |
*** dparkes has quit IRC | 12:00 | |
janki | marios, I used master-tripleo-ci. checking the link | 12:01 |
*** pkovar has joined #tripleo | 12:01 | |
EmilienM | adarazs: https://bugs.launchpad.net/tripleo/+bug/1696094 | 12:01 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 12:01 |
adarazs | EmilienM: thanks! | 12:01 |
janki | marios, I am assuming these packages come from RDO | 12:01 |
ansiwen | shardy: the stable/ocata backport is still failing (since three weeks), and actually now I don't find the problem in the logs. https://review.openstack.org/#/c/462542/ | 12:01 |
*** chlong has joined #tripleo | 12:02 | |
honza | jtomasek: so, i shouldn't try and get a few last minute patches in today? i thought releases were usually cut on thursdays? | 12:04 |
*** noslzzp has quit IRC | 12:04 | |
openstackgerrit | Merged openstack/tripleo-validations master: Updated from global requirements https://review.openstack.org/470148 | 12:05 |
openstackgerrit | Sven Anderson proposed openstack/tripleo-common master: Add EC2-API container image https://review.openstack.org/469486 | 12:05 |
*** sid1 has joined #tripleo | 12:05 | |
*** karimb has quit IRC | 12:06 | |
oidgar | hi, seems like my undercloud heat engine is stuck, i got the following message: | 12:06 |
oidgar | 2017-06-07 12:00:32.425 8061 INFO heat.engine.stack [req-8dacc439-326a-41bc-8f32-c60d682c9bd4 - admin - - -] Stack DELETE FAILED (overcloud): Resource DELETE failed: resources.AllNodesDeploySteps: Engine went do | 12:06 |
oidgar | wn during stack DELETE | 12:06 |
*** ccamacho is now known as ccamacho|lunch | 12:06 | |
oidgar | anyone knows how to fix such issue? | 12:06 |
shardy | ansiwen: it looks like the pingtest stack timed out - EmilienM hey you mentioned scenario002 was still broken, do you have a bug reference? | 12:06 |
*** morazi has joined #tripleo | 12:06 | |
owalsh | ratailor: did the os_traits packing happen? | 12:06 |
EmilienM | shardy: this one ? https://bugs.launchpad.net/tripleo/+bug/1695904 | 12:07 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 12:07 |
ansiwen | mandre: can you +2 this? https://review.openstack.org/#/c/469494/ | 12:07 |
jtomasek | honza: we definitely can release tomorrow | 12:07 |
shardy | EmilienM: ah, thanks | 12:07 |
owalsh | ratailor: FYI http://logs.openstack.org/61/468461/7/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/34ba4af/logs/undercloud/var/log/nova/nova-manage.log.txt.gz#_2017-06-07_11_53_43_882 | 12:07 |
EmilienM | shardy: it's for ocata. We have another bug on master, affecting HA jobs: https://bugs.launchpad.net/tripleo/+bug/1680195 (randomly AFIK) | 12:07 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 12:07 |
*** ebarrera has quit IRC | 12:07 | |
ansiwen | shardy or EmilienM, maybe you as well? It's a trivial image addition. https://review.openstack.org/#/c/469494/ | 12:08 |
*** ebarrera has joined #tripleo | 12:08 | |
*** lucas-hungry is now known as lucasagomes | 12:09 | |
*** ooolpbot has joined #tripleo | 12:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 12:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 12:10 |
*** ooolpbot has quit IRC | 12:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 12:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 12:10 |
ansiwen | shardy: thanks for +2! | 12:11 |
*** karimb has joined #tripleo | 12:11 | |
openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Update code to React Router v4 API https://review.openstack.org/466746 | 12:11 |
EmilienM | ansiwen: why gate-tripleo-ci-centos-7-ovb-containers-oooq-nv is red? | 12:12 |
*** ratailor has quit IRC | 12:12 | |
owalsh | ratailor: seems it was just added yesterday https://github.com/rdo-packages/nova-distgit/commit/361e542183d21e71695a7e9f8ad7dde488f8e827 | 12:12 |
*** fragatina has joined #tripleo | 12:12 | |
owalsh | EmilienM: ^^^ will this cause promotion to fail? | 12:13 |
*** trown|outtypewww is now known as trown | 12:13 | |
EmilienM | owalsh: I don't know, do we have python-os-traits in rdo? | 12:14 |
*** liverpooler has quit IRC | 12:14 | |
owalsh | EmilienM: yea - https://review.rdoproject.org/r/#/c/6870/ | 12:15 |
*** florianf has joined #tripleo | 12:16 | |
owalsh | EmilienM: I think it's just the docker images in containers-multinode-upgrades that are affected | 12:16 |
*** jaganathan has joined #tripleo | 12:17 | |
*** sid2 has joined #tripleo | 12:19 | |
ansiwen | EmilienM: i have no idea, but it's always red for all jobs I ever saw | 12:19 |
EmilienM | ansiwen: because it's always red doesn't mean we want to ignore it | 12:20 |
*** sid1 has quit IRC | 12:20 | |
EmilienM | ansiwen: I won't +2 it until we know why it's red | 12:20 |
ansiwen | EmilienM: even here: https://review.openstack.org/#/c/470147/ | 12:21 |
shardy | http://tripleo.org/cistatus.html shows it was green for some time, but the last few jobs have failed | 12:21 |
ansiwen | EmilienM: it's non-voting for a reason, no? | 12:21 |
ansiwen | EmilienM: all the recent merges had it red | 12:21 |
* shardy sighs | 12:21 | |
EmilienM | non voting doesn't mean we don't care | 12:22 |
dciabrin | mandre, hey, got some time for some kolla rebuild question? It seems when I rebuild my images locally package puppet is not installed anymore in the images :/ | 12:22 |
ansiwen | EmilienM: no, but merging anyway does? I don't say we shouldn't look what is going on there, I just say it's not related to my change | 12:23 |
openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Show Node Introspection Data https://review.openstack.org/471318 | 12:24 |
openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Add Introspection status to Nodes https://review.openstack.org/467872 | 12:24 |
EmilienM | ansiwen: the problem is that everyone wants their patches merged to get their work done, but a very few folks actually care of the CI | 12:24 |
EmilienM | ansiwen: if more folks doing features would do little help to maintain CI jobs, jobs would be more stable for sure | 12:24 |
EmilienM | instead of ignoring and approving patches even when jobs merge | 12:25 |
mandre | dciabrin: can you show me how you build the images? | 12:25 |
EmilienM | mandre: do you know why the ovb-containers update is failing now? once it's green again, we should make it voting | 12:25 |
*** ratailor has joined #tripleo | 12:26 | |
owalsh | EmilienM: timeout? | 12:26 |
ebarrera | EmilienM, can you take a quick look to https://bugzilla.redhat.com/show_bug.cgi?id=1455503 ? | 12:26 |
openstack | bugzilla.redhat.com bug 1455503 in openstack-tripleo-heat-templates "external ip not crrectly set for apache on 1+1 plus isolated ctlplane + external" [Unspecified,New] - Assigned to emacchi | 12:26 |
shardy | No they're all failing after about 30mins | 12:26 |
shardy | looking at logs.. | 12:26 |
*** rlandy has joined #tripleo | 12:26 | |
shardy | the undercloud install is broken | 12:27 |
dciabrin | mandre, http://paste.openstack.org/show/611692/ using the tripleo_kolla_template_overrides from tripleo-common | 12:27 |
EmilienM | ebarrera: I'm assigned because I'm the default assignee for this projects - but I'm not actually looking at this bug now, it's out of my scope for now. | 12:27 |
shardy | http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-06-07_09_57_38 | 12:27 |
ebarrera | EmilienM, ack, do you know someone able to help ? | 12:28 |
shardy | owalsh: seems to be nova-manage cellsv2 stuff | 12:28 |
ansiwen | EmilienM: it's not a regression for sure, because master was failing as well. And I'm happy if it is blocked in general, and not only on my changes. :-) | 12:28 |
shardy | did anything change in that recently? | 12:28 |
owalsh | shardy: missing the os-traits dependency | 12:28 |
mandre | EmilienM: another issue we've seen recently, https://bugs.launchpad.net/tripleo/+bug/1694709 but I don't think that's it | 12:28 |
openstack | Launchpad bug 1694709 in tripleo "Reaching disk space capacity for containers job" [High,Triaged] - Assigned to Dan Prince (dan-prince) | 12:28 |
owalsh | shardy: I was just talking about that :-) | 12:28 |
shardy | owalsh: aha | 12:28 |
shardy | I assumed it'd just block promotion | 12:28 |
owalsh | shardy: are we building docker images from source? Nova change that requires this landed yesterday AFAIK | 12:29 |
shardy | owalsh: we're building them from packages but the push to dockerhub is till manual atm | 12:30 |
shardy | this is the undercloud install though, so not containerized | 12:30 |
mandre | dciabrin: maybe it's better for you to work from a kolla checkout rather than rpm install if you're rebuilding frequently | 12:31 |
shardy | owalsh: perhaps we're using an old version, they're logged here: | 12:31 |
shardy | http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/var/log/messages.txt.gz | 12:32 |
*** zoli|brb is now known as zoli | 12:32 | |
shardy | openstack-nova-common-16.0.0-0.20170607093107.0fc64f0.el7.centos.noarch | 12:32 |
*** jprovazn has quit IRC | 12:33 | |
ansiwen | EmilienM: you should make it voting then, because now you are blocking the wrong changes and not the causing ones. | 12:33 |
owalsh | shardy: this adds the requires python-nova-16.0.0-0.20170607093107.0fc64f0.el7.centos.noarch | 12:34 |
owalsh | EmilienM, shardy: so that just got promoted? | 12:34 |
shardy | owalsh: yeah it seems so | 12:34 |
owalsh | hrmm, with nova api db_sync is failing? | 12:35 |
ratailor | owalsh, this dependency has been added in nova.spec https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L451 | 12:36 |
sshnaidm | bnemec, why not to add OVB code to openstack projects? | 12:36 |
owalsh | ratailor: yea, but it's a bit late | 12:36 |
openstackgerrit | Christian Schwede proposed openstack/puppet-tripleo master: Fix Swift ring management in container deployments https://review.openstack.org/468790 | 12:36 |
EmilienM | owalsh: https://dashboards.rdoproject.org/rdo-dev - we haven't had a promotion in 17 days | 12:37 |
EmilienM | ansiwen: I'm trying to get some help, but obviously it's hard to get some. | 12:37 |
arxcruz | sshnaidm: not in a long time | 12:37 |
ratailor | owalsh, couldn't get time last week because of tripleo bootcamp. it got merged today only. https://review.rdoproject.org/r/#/c/6974/2 | 12:38 |
jtomasek | honza: what patches do you want me to review primarily? | 12:40 |
shardy | EmilienM: ah, yeah I was looking at the ocata pin | 12:40 |
shardy | why are we installing latest nova then? Hmm.. | 12:41 |
EmilienM | shardy: in upgrade jobs? | 12:42 |
shardy | http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/etc/yum.repos.d/delorean.repo.txt.gz | 12:42 |
shardy | that repo is wrong | 12:42 |
shardy | it's pointing to current for delorean and delorean-current | 12:42 |
openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy kubernetes using TripleO on the overcloud https://review.openstack.org/471759 | 12:42 |
shardy | jistr: ^^ any idea where we set those repos? | 12:42 |
ansiwen | EmilienM: it's not that I don't want to help, I'd love to actually, I just don't have the ability. I am overwhelmed by the logs and don't see what's going on. :-( | 12:42 |
thrash | yolanda: https://review.openstack.org/471758 | 12:42 |
*** jaganathan has quit IRC | 12:43 | |
flaper87 | shardy: EmilienM fultonj therve https://review.openstack.org/471759 no need to review, it's WIP but just a heads up | 12:43 |
*** karimb has quit IRC | 12:43 | |
social | oh hi, can I get someone to review this https://review.openstack.org/#/c/468960 I'm quite blocked on this one | 12:43 |
*** agurenko_ is now known as agurenko | 12:44 | |
EmilienM | shardy: looks like a quickstart thing, since this job is not managed by tripleo-ci scripts | 12:45 |
jistr | https://github.com/openstack-infra/tripleo-ci/blob/master/toci-quickstart/playbooks/multinode.yml#L54-L62 | 12:45 |
jistr | https://github.com/openstack/tripleo-quickstart/tree/master/roles/repo-setup | 12:45 |
jistr | https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml | 12:45 |
* EmilienM invokes jistr :D | 12:45 | |
jistr | shardy: ^ that's how i *think* it should work. There could be some bad switch along the way... | 12:46 |
shardy | https://trunk.rdoproject.org/centos7/current-tripleo/ points to different nova version, so we're somehow writing or overwriting the wrong repo | 12:47 |
jistr | oh wait we're talking about upgrade job | 12:48 |
EmilienM | yeah | 12:48 |
*** jpena|lunch is now known as jpena | 12:48 | |
EmilienM | I was looking at the code which upgrade repos | 12:48 |
shardy | yeah, the undercloud install of the upgrade job | 12:48 |
EmilienM | shardy: the nova version in tripleo repo is consistent between openstack-nova-* and python-nova, I don't see the problem | 12:49 |
shardy | EmilienM: we're installing nova from -current not the -current-tripleo pin | 12:49 |
EmilienM | what | 12:50 |
shardy | because the repo files for -current and -current-tripleo both point to the same repo hash | 12:50 |
shardy | http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/etc/yum.repos.d/delorean.repo.txt.gz | 12:50 |
EmilienM | indeed | 12:50 |
shardy | http://mirror.regionone.osic-cloud1.openstack.org:8080/rdo/centos7/0f/c6/0fc64f0ab663edb2564cba5868113ad6529d078b_6cc62d70 | 12:50 |
EmilienM | yeah | 12:50 |
shardy | http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/etc/yum.repos.d/delorean-current.repo.txt.gz | 12:51 |
*** jmelvin has joined #tripleo | 12:51 | |
* EmilienM digs in quickstart code | 12:51 | |
* owalsh biab, gotta grab some lunch | 12:52 | |
*** atheurer has joined #tripleo | 12:52 | |
jistr | i don't see anything that would do repo-setup on undercloud in multinode.yml | 12:52 |
*** karimb has joined #tripleo | 12:52 | |
jistr | there's https://github.com/openstack-infra/tripleo-ci/blob/master/toci-quickstart/playbooks/multinode.yml#L2 | 12:52 |
jistr | which points to https://github.com/openstack/tripleo-quickstart/blob/master/playbooks/quickstart.yml | 12:53 |
jistr | but i don't see any UC repo-setup either there | 12:53 |
mandre | ansiwen: do you have the link to the patch using the nova-consoleauth and nova-novncproxy images handy? | 12:54 |
yolanda | thrash, just flipped words | 12:55 |
mandre | ansiwen: aha, found it https://review.openstack.org/#/c/469576/, sorry for the noise | 12:55 |
thrash | yolanda: oops. :D | 12:55 |
EmilienM | flaper87: ack | 12:56 |
*** jaganathan has joined #tripleo | 12:56 | |
*** liverpooler has joined #tripleo | 12:56 | |
ansiwen | mandre: yes, and https://review.openstack.org/#/c/469541/ | 12:56 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 12:57 |
*** ccamacho|lunch is now known as ccamacho | 12:57 | |
*** dparkes has joined #tripleo | 12:57 | |
*** trown is now known as trown|brb | 12:58 | |
*** pkovar has quit IRC | 12:58 | |
thrash | yolanda: updated. Good catch. | 12:58 |
ansiwen | mandre: thanks for +2 | 12:59 |
*** zoli is now known as zoli|afk | 13:00 | |
*** agopi has joined #tripleo | 13:00 | |
*** ffiore_ has joined #tripleo | 13:00 | |
*** agurenko is now known as agurenko_ | 13:01 | |
openstackgerrit | Brent Eagles proposed openstack/tripleo-docs master: Post deployment installation instructions for Octavia https://review.openstack.org/447496 | 13:02 |
shardy | https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/defaults/main.yml#L24 | 13:02 |
shardy | shouldn't that be delorean-current.repo? | 13:03 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add support for external network bridges https://review.openstack.org/406421 | 13:03 |
*** pkovar has joined #tripleo | 13:04 | |
shardy | http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/home/jenkins/repo_setup.log.txt.gz | 13:04 |
shardy | that shows we download the right files, but somehow copy the wrong delorean.repo file from repo_role_tmp | 13:05 |
openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow https://review.openstack.org/463765 | 13:05 |
jistr | shardy: possibly it should be delorean-current.repo, but i think the `repos` variable is just completely overriden by this https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml | 13:06 |
jistr | shardy: so it shouldn't be a problem | 13:06 |
jistr | shardy: i still don't see how we run a repo-setup on undercloud though, i'm starting to think we don't... | 13:07 |
shardy | Ah, I was trying to match the log output with repo_setup.sh.j2 and that perhaps explains why it doesn't match except for delorean-deps | 13:07 |
shardy | jistr: doesn't that log output show it happening? | 13:08 |
jistr | shardy: ah right it does | 13:08 |
* jistr looks at console.html | 13:08 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Fix the disable expirer to remove crontab https://review.openstack.org/470357 | 13:08 |
shardy | the repo_setup.log shows downloading repo files, and it shows the wrong delorean.repo at the end | 13:08 |
shardy | I just haven't seen exactly where/why | 13:09 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 13:09 |
jistr | shardy: found it https://github.com/openstack/tripleo-quickstart-extras/blob/a7b34d30656b9d9bb3030954419aeb8ce75eef7b/roles/undercloud-setup/tasks/main.yml#L13-L15 | 13:09 |
*** ooolpbot has joined #tripleo | 13:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 13:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 13:10 |
*** ooolpbot has quit IRC | 13:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 13:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 13:10 |
openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow https://review.openstack.org/463765 | 13:10 |
*** ffiore has joined #tripleo | 13:11 | |
*** ffiore_ has quit IRC | 13:11 | |
EmilienM | shardy: just a quick fyi on upgrade jobs that are all passing (except the container one): https://review.openstack.org/#/c/467698 | 13:11 |
EmilienM | it would be nice to keep it going :D | 13:11 |
EmilienM | making them voting and having them in promotion pipeline will certainly help | 13:12 |
shardy | EmilienM: Nice! So much green! :) | 13:12 |
*** toure|biab is now known as toure | 13:13 | |
*** trown|brb is now known as trown | 13:13 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/python-tripleoclient master: Updated from global requirements https://review.openstack.org/470555 | 13:13 |
jistr | shardy: can you explain what do you mean by 'copy wrong delorean.repo'? | 13:14 |
jistr | shardy: i'm looking at the script and it looks correct to me... http://logs.openstack.org/74/471674/1/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/6a36fb5/logs/undercloud/home/jenkins/repo_setup.sh.txt.gz | 13:14 |
EmilienM | yolanda: i self approved the t-i-e patch, please update the releases patch when merged | 13:14 |
openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient master: [WIP] Add 'overcloud node discover' command https://review.openstack.org/471767 | 13:14 |
*** tzumainn has joined #tripleo | 13:15 | |
* jistr double checking the logs | 13:16 | |
shardy | jistr: search for "repo: delorean" and "repo: delorean-deps" in the repo_setup log | 13:17 |
shardy | they are the same | 13:17 |
shardy | both pointing to -current | 13:17 |
shardy | 0fc64... | 13:17 |
bogdando | jistr: bad luck for the patch http://logs.openstack.org/22/466322/4/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/f44b8aa/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-06-07_10_58_58 | 13:17 |
shardy | earlier in the log we download current-tripleo/delorean.repo | 13:17 |
shardy | 38/47/38471d4ccf3914805fafaa56b21db2cc83755e95_5d6d179f | 13:18 |
shardy | but for some reason, we copy current/delorean.repo to both repo files under /etc | 13:18 |
bogdando | EmilienM, jistr: oh, the same bug as well | 13:19 |
bogdando | > 15:11:47 - EmilienM: shardy: just a quick fyi on upgrade jobs that are all passing (except the container one): https://review.openstack.org/#/c/467698 | 13:19 |
*** gbarros has joined #tripleo | 13:19 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 13:20 |
jistr | shardy: oh i see it, fix coming right up. It's the mirror URL replacement logic, there's copypasta mistake. | 13:20 |
mandre | jistr: looks like https://review.openstack.org/#/c/462211/ is mergeable | 13:20 |
shardy | jistr: aha, well spotted! | 13:21 |
*** ebarrera has quit IRC | 13:21 | |
shardy | mandre: yeah would be good to land that one I think, I'm re-testing the md5sum one locally now so can rebase after that lands | 13:22 |
*** ebarrera has joined #tripleo | 13:22 | |
openstackgerrit | Dan Trainor proposed openstack/instack-undercloud master: Add proxy and config endpoints of ironic-inspector https://review.openstack.org/470058 | 13:22 |
openstackgerrit | Brent Eagles proposed openstack/tripleo-docs master: Post deployment installation instructions for Octavia https://review.openstack.org/447496 | 13:23 |
openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart master: qemu://system fixes https://review.openstack.org/471578 | 13:24 |
mandre | shardy: cool! I've blocked some time for code reviews, anything special you need me to look at? | 13:25 |
jistr | shardy: is there a bug reported yet that i can tag in the commit message? | 13:25 |
jistr | if not i could perhaps just send it as-is | 13:25 |
shardy | jistr: no I've not raised one yet | 13:26 |
shardy | mandre: not yet, the md5sum one is failing ci and I'm still testing some follow-up patches locally, thanks though :) | 13:26 |
shardy | will be good to land the ansible one as it's been around for a while | 13:26 |
*** pradk has quit IRC | 13:27 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 13:28 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart master: Fix repo mirror logic for tripleo-ci/master-undercloud-ocata-overcloud https://review.openstack.org/471774 | 13:28 |
*** jcoufal has joined #tripleo | 13:29 | |
jistr | shardy, EmilienM ^^ | 13:29 |
*** egafford has joined #tripleo | 13:29 | |
karthiks | shardy, can you please review https://review.openstack.org/#/c/466693/ and https://review.openstack.org/#/c/469164/ | 13:30 |
EmilienM | jistr: nice shot | 13:30 |
EmilienM | jistr: thanks | 13:30 |
*** links has quit IRC | 13:30 | |
*** skramaja has quit IRC | 13:31 | |
*** jprovazn has joined #tripleo | 13:31 | |
*** amoralej is now known as amoralej|lunch | 13:31 | |
jistr | mandre: wow so green | 13:31 |
* jistr reviews | 13:31 | |
*** jaganathan has quit IRC | 13:32 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: enable the rhsm/cdn yum repos in repo-setup https://review.openstack.org/470150 | 13:32 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: add a config for building images https://review.openstack.org/456373 | 13:33 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: add steps to enable tripleo_ui https://review.openstack.org/459061 | 13:33 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add ssh tunneling for TripleO-UI https://review.openstack.org/407733 | 13:33 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: baseos-undercloud release configuration for all supported releases. https://review.openstack.org/464033 | 13:33 |
*** ansmith has joined #tripleo | 13:34 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 13:37 |
openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Make container names consistent https://review.openstack.org/471778 | 13:38 |
*** Goneri has joined #tripleo | 13:39 | |
*** psahoo has quit IRC | 13:39 | |
*** jmelvin is now known as jmelvin|brb | 13:40 | |
*** florianf has quit IRC | 13:41 | |
openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient master: [WIP] Add 'overcloud node discover' command https://review.openstack.org/471767 | 13:42 |
shardy | karthiks: done! | 13:42 |
karthiks | shardy, Thanks | 13:43 |
*** chlong has quit IRC | 13:43 | |
*** ratailor has quit IRC | 13:43 | |
*** jaganathan has joined #tripleo | 13:44 | |
*** noslzzp has joined #tripleo | 13:44 | |
mandre | jistr: I know! make sure to put your sunglasses on | 13:44 |
jistr | :D | 13:44 |
openstackgerrit | Christian Schwede proposed openstack/puppet-tripleo master: Fix Swift ring management in container deployments https://review.openstack.org/468790 | 13:45 |
*** jmelvin|brb has quit IRC | 13:45 | |
trown | lol... if you review with rose colored glasses, you can ignore red CI | 13:46 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 13:47 |
*** rcrit has quit IRC | 13:48 | |
jistr | shardy++ nice work @ https://review.openstack.org/462211 | 13:51 |
jistr | this is going to be easier to explain on the deep dive. The ordering of actions is much more apparent in the playbook | 13:51 |
*** gkadam has joined #tripleo | 13:52 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-validations master: Add undercloud service status check before update/upgrade https://review.openstack.org/462918 | 13:53 |
openstackgerrit | Michele Baldessari proposed openstack/tripleo-common master: Do not use multiple overrides with the same name https://review.openstack.org/471781 | 13:54 |
*** pchavva has joined #tripleo | 13:57 | |
openstackgerrit | Michele Baldessari proposed openstack/tripleo-common master: Do not use multiple overrides with the same name https://review.openstack.org/471781 | 13:57 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 13:57 |
*** florianf has joined #tripleo | 13:58 | |
bogdando | folks, updated comments for https://bugs.launchpad.net/tripleo/+bug/1694709 | 14:00 |
openstack | Launchpad bug 1694709 in tripleo "Reaching disk space capacity for containers job" [High,Triaged] - Assigned to Dan Prince (dan-prince) | 14:00 |
bogdando | resluts are frustrating | 14:00 |
openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli https://review.openstack.org/463728 | 14:01 |
shardy | jistr: thanks, yeah that was what I was aiming for | 14:01 |
florianf | bnemec: Hi, I put up a patch for a pre-upgrade validation you suggested (disk space): https://review.openstack.org/#/c/471686/ | 14:01 |
shardy | jistr: and it's much easier to run the entire deploy sequence on each node (for minor updates) | 14:01 |
florianf | bnemec: Just wanted to make sure I picked up the 10GB requirement right. | 14:01 |
matbu | shardy: hey, can you check my comment in this review, when you have a minute https://review.openstack.org/#/c/444224 | 14:02 |
*** nyechiel has quit IRC | 14:02 | |
jistr | shardy: nice | 14:03 |
*** dparkes has quit IRC | 14:05 | |
*** amoralej|lunch is now known as amoralej | 14:06 | |
shardy | jistr: replied to your comment - it does depend on the host puppet step1 | 14:06 |
shardy | thanks for the review :) | 14:06 |
*** hjensas has quit IRC | 14:06 | |
*** agurenko has joined #tripleo | 14:06 | |
jistr | bogdando, mandre: this might have been made worse by me pushing the keystone container image (to fix the critical CI issue earlier this week) separately from the other container images. I didn't want to rebuild them all because 1) i have no idea if my env is correct for it, so i wanted to make the smallest possible change. 2) dprince mentioned at some point that he baked in some special HA ingredients from | 14:06 |
jistr | bandini/dciabrin to make testing with pacemaker possible, and i wasn't sure if that's still the case, so i didn't want to undo the changes 3) it was getting late in the evening and i wanted to go home | 14:06 |
jistr | bogdando: that's re the issue that we're running out of space for images | 14:07 |
*** bfournie has quit IRC | 14:07 | |
shardy | matbu: +1 on starting with the CLI approach | 14:07 |
shardy | matbu: FYI I started a SwiftObject plugin for heat here: https://review.openstack.org/#/c/469039/ | 14:07 |
shardy | I'll try to finish it this week | 14:07 |
jistr | bogdando, mandre: having a separately built keystone image means it doesn't share the layers below with the other images | 14:07 |
bogdando | jistr, mandre: hm, that's actually a good point. The way we rebuild images and reuse a common base | 14:08 |
mandre | jistr: it's ok, I'll push a new batch of container images very soon | 14:08 |
jistr | mandre: great, thanks a lot! i think that could mitigate the space issue | 14:08 |
*** dparkes has joined #tripleo | 14:08 | |
mandre | jistr: it depends on how you build the images, there is actually a way to reuse existing layers in kolla but you need to have them in your docker cache | 14:08 |
bogdando | jistr: right, and we really should double-check all of the images use the common centos base and properly layered | 14:09 |
bogdando | all the time we push rebuild | 14:09 |
*** agurenko_ has quit IRC | 14:09 | |
tdasilva | trown: hello, have you seen this error before: http://paste.openstack.org/show/611718/ ? | 14:09 |
*** ykarel__ has quit IRC | 14:09 | |
*** ooolpbot has joined #tripleo | 14:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 14:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 14:10 |
*** ooolpbot has quit IRC | 14:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 14:10 |
jistr | mandre: nice, i didn't know we could do that | 14:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 14:10 |
*** jmelvin has joined #tripleo | 14:12 | |
trown | tdasilva: I have not, but that seems like you have an old checkout of quickstart https://github.com/openstack/tripleo-quickstart/blob/master/roles/parts/ovs/tasks/main.yml | 14:12 |
trown | tdasilva: could you delete the .quickstart dir and run again? or run with --clean (which does the same thing) | 14:12 |
*** gkadam has quit IRC | 14:13 | |
tdasilva | trown: that's the weird part, i always delete .quickstart AND run with --clean | 14:13 |
openstackgerrit | Martin André proposed openstack/tripleo-common master: Add workaround for missing dep in octavia image https://review.openstack.org/471788 | 14:13 |
*** bfournie has joined #tripleo | 14:14 | |
matbu | shardy: ack cool thx :) | 14:14 |
bogdando | jistr, mandre: so let's fix that then :) we know the problem now | 14:14 |
*** noslzzp has quit IRC | 14:14 | |
trown | tdasilva: but from a local git checkout with --no-clone? The code in the paste does not match the code in the upstream repo | 14:15 |
matbu | shardy: do you think its doable to add it in pike M2 ? (i mena the cli patch) | 14:15 |
matbu | s/mena/mean/ | 14:15 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 14:15 |
tdasilva | trown: been running like this: ./quickstart.sh --requirements quickstart-extras-requirements.txt --playbook quickstart-extras.yml --config ~/.quickstart/config/general_config/containers_minimal.yml --teardown all --tags all --clean --release master 127.0.0.2 | 14:16 |
*** janki has quit IRC | 14:16 | |
shardy | matbu: the first step is probably to fix the tests etc, the unit test failures are real | 14:17 |
trown | tdasilva: ah I see it is just bad error message from ansible... code is at https://github.com/openstack/tripleo-quickstart/blob/master/roles/environment/setup/tasks/main.yml#L2 | 14:17 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 14:17 |
shardy | m2 is now, so I wouldn't expect it to necessarily land in time for that | 14:17 |
trown | tdasilva: odd thing is that has not changed in a couple months... something else must be happening in your environment | 14:18 |
matbu | shardy: ack, i can fix pep8 and UT asap | 14:18 |
tdasilva | trown: correct, i noticed that too, that code was last introduced back in dec, i think | 14:18 |
tdasilva | and i've run oooq many times since ;) | 14:19 |
mandre | jistr, shardy: can you do a very quick review before I start re-building the images? https://review.openstack.org/#/c/471781/ and https://review.openstack.org/#/c/471788/ | 14:19 |
openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli https://review.openstack.org/463728 | 14:19 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 14:19 |
trown | tdasilva: can you try without adding --requirements? I think that would be overriding all of the requirements files, and we did recently change quickstart to use everything from the venv, so you might be missing stuff | 14:20 |
shardy | matbu: also I wonder if we might consider something more generic and not upgrade specific | 14:20 |
shardy | matbu: like openstack overcloud config download ? | 14:20 |
shardy | that could output all the config data from role_data and be useful for more than just upgrade | 14:20 |
matbu | shardy: hmm yep | 14:21 |
tdasilva | trown: running now | 14:21 |
*** zoli|afk is now known as zoli|wfh | 14:21 | |
*** prateek has quit IRC | 14:22 | |
matbu | shardy: so you mean starting a new overcloud_config.py, where we can manage all those config related stuffs? | 14:22 |
shardy | matbu: yes, just something which can download the various configs from the overcloud and write them to disk | 14:23 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 14:23 |
shardy | e.g not only upgrade_tasks | 14:23 |
jpich | akrivoka: What do you think about bringing up the "stop using mistral env" patches at the next meeting? It has so many implications including for upgrades, I think it'd be better to merge earlier rather than later | 14:23 |
shardy | but the step_config puppet manifest, config_settings variables, etc | 14:23 |
jpich | If we want it in Pike (I kinda should have thought about this yesterday, sorry!) | 14:24 |
shardy | all of role_data for each role basically, but perhaps reformatted into a useful format on disk | 14:24 |
akrivoka | jpich: sure, makes sense | 14:24 |
*** cdearborn has joined #tripleo | 14:24 | |
*** gbarros has quit IRC | 14:24 | |
matbu | shardy: ack i'll rework that | 14:24 |
shardy | as it's already possible to dump it as yaml/json directly with openstack stack output show | 14:24 |
*** pradk has joined #tripleo | 14:24 | |
shardy | matbu: thanks - that would mean we could use the same approach for manual minor updates potentially | 14:25 |
jpich | akrivoka: Ok! Adding it now so I don't forget :) | 14:25 |
akrivoka | jpich: thank you :) | 14:25 |
jistr | mandre: is python-cotyledon packages? I see it mentioned in ocata RDO somewhere, but i don't see it in http://trunk.rdoproject.org/centos7/current-tripleo/ | 14:26 |
jistr | s/packages/packaged/ | 14:26 |
*** ebarrera_ has joined #tripleo | 14:26 | |
jistr | ah i forgot to check delorean-deps | 14:27 |
jpich | akrivoka: Gah I just noticed there's yet another merge conflict as well, I'll look into it as well | 14:27 |
jistr | mandre: yea it's there, sorry for false alarm +2'd both | 14:27 |
*** pradk has quit IRC | 14:28 | |
mandre | jistr: ack, thanks | 14:28 |
akrivoka | jpich: seems like gerrit doesn't send email notifications on merge conflicts? :\ | 14:29 |
jpich | akrivoka: This is weird, I see a merge conflict on the dashboard but not when I look at the patch directly... O_o | 14:29 |
mandre | FWIW I think we can just land them, because these files are not tested in CI yet | 14:29 |
*** gbarros has joined #tripleo | 14:29 | |
sshnaidm | hrybacki, jaosorior https://trello.com/c/Irj8mWDK | 14:29 |
akrivoka | jpich: I'm seeing a red "Cannot Merge" on the patch | 14:30 |
jaosorior | sshnaidm: makes sense to me | 14:30 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 14:30 |
jpich | akrivoka: Huh. doesn't show for me even after refresh. ¯\_(ツ)_/¯ | 14:30 |
*** pradk has joined #tripleo | 14:31 | |
akrivoka | jpich: I can resolve the conflict if that's ok with you? | 14:31 |
openstackgerrit | Dimitri Savineau proposed openstack/puppet-tripleo master: Cinder: Add NexentaStor Support https://review.openstack.org/370956 | 14:31 |
*** jaosorior is now known as jaosorior_away | 14:31 | |
jpich | akrivoka: Oh sure, be my guest! Doesn't look to be a too mean one | 14:32 |
*** karimb has quit IRC | 14:32 | |
*** limao has joined #tripleo | 14:36 | |
hrybacki | sshnaidm: ack, thank you! | 14:36 |
akrivoka | jpich: ah the conflict seems due to another one of your commits :) https://github.com/openstack/tripleo-common/commit/c824b58a22b6824eb4f4f1d3bc50ebc443e440f9 | 14:37 |
jpich | akrivoka: Sorryyyy | 14:37 |
akrivoka | jpich: lol no worries :) | 14:37 |
jpich | :) | 14:37 |
*** ebarrera_ has quit IRC | 14:37 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 14:38 |
openstackgerrit | Yedidyah Bar-David proposed openstack/tripleo-quickstart master: Do not log virt_power private key https://review.openstack.org/471802 | 14:38 |
*** limao_ has joined #tripleo | 14:40 | |
*** limao has quit IRC | 14:40 | |
*** sri_ has joined #tripleo | 14:42 | |
*** didib has joined #tripleo | 14:42 | |
*** pkovar has quit IRC | 14:43 | |
*** pkovar has joined #tripleo | 14:44 | |
*** karimb has joined #tripleo | 14:46 | |
ansiwen | mandre: why did you +2 this: https://review.openstack.org/#/c/469576/ but not this: https://review.openstack.org/#/c/469541/ ? | 14:47 |
panda | Slower: by default kolla installs packages from the host repository ? | 14:47 |
ansiwen | mandre: I locally deployed both and actually got a authorized VNC connection to a booted instance. so it is working, that much I know :-) | 14:47 |
ansiwen | mandre: I'm just trying to understand what you are looking at, because I'm still lost in all the CI logs | 14:48 |
mandre | ansiwen: I'm waiting to see the containers be up in CI, the job for the nova-consoleauth container timed out and didn't complete log collection | 14:51 |
openstackgerrit | Ana Krivokapic proposed openstack/tripleo-common master: Use Swift to store Plan environment https://review.openstack.org/452291 | 14:52 |
akrivoka | jpich: ^ | 14:52 |
*** lmiccini has quit IRC | 14:53 | |
jpich | akrivoka: Yay, thanks! | 14:53 |
pradk | mandre, so regarding the tacker errors.. this seems like something new we picked up in the recent builds | 14:54 |
ansiwen | mandre: which job exactly? there are many, and for the other one that you +2 also | 14:58 |
*** didib has quit IRC | 15:00 | |
mandre | pradk: ok, this will likely be fixed with newer container image | 15:00 |
*** karimb has quit IRC | 15:01 | |
*** mdnadeem has joined #tripleo | 15:01 | |
mandre | pradk: I'll recheck when I'm done rebuilding | 15:01 |
*** isq has quit IRC | 15:01 | |
mandre | ansiwen: I'm looking at gate-tripleo-ci-centos-7-ovb-containers-oooq-nv result | 15:02 |
*** karimb has joined #tripleo | 15:02 | |
mandre | ansiwen: it seems we do not enable nova-consoleauth in the multinode container job | 15:02 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 15:04 |
ansiwen | mandre: what about all the other container jobs? I can see 4 | 15:05 |
*** lmiccini has joined #tripleo | 15:05 | |
*** hjensas has joined #tripleo | 15:06 | |
*** hjensas has quit IRC | 15:06 | |
*** hjensas has joined #tripleo | 15:06 | |
mandre | ansiwen: they don't test your container as far as I can tell | 15:07 |
ansiwen | mandre: and that's a bug or a feature? ;-) do I have to add it somwhere else beside docker.yaml? | 15:07 |
mandre | ansiwen: it is tested by gate-tripleo-ci-centos-7-ovb-containers-oooq-nv, the one I'm waiting the result for | 15:09 |
mandre | ansiwen: :) | 15:09 |
*** ooolpbot has joined #tripleo | 15:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 15:10 |
*** ooolpbot has quit IRC | 15:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 15:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 15:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 15:10 |
ansiwen | mandre: yes, but my question is, where is it configured what is tested? because as far as I can tell it should be in the defailt services for the controller role. so if the other jobs don't deploy it, I am wondering why. | 15:10 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Stop/disable l3 agent in docker service upgrade_tasks https://review.openstack.org/469924 | 15:11 |
*** gvrangan has joined #tripleo | 15:11 | |
*** SlickNik- has joined #tripleo | 15:11 | |
*** rcernin has quit IRC | 15:11 | |
*** SlickNik has quit IRC | 15:12 | |
openstackgerrit | Merged openstack/tripleo-image-elements master: Release 7.0.0.0b2 https://review.openstack.org/471642 | 15:12 |
*** eggmaster has joined #tripleo | 15:15 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup https://review.openstack.org/466213 | 15:16 |
mandre | jistr: did you know heat-api is doa on the multinode-upgrade job? http://logs.openstack.org/41/469541/3/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/1a720f8/logs/subnode-2/var/log/extra/docker/containers/heat_api/stdout.log.txt.gz | 15:19 |
openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Ensure liberty support for images preparation role. https://review.openstack.org/471824 | 15:19 |
mandre | ansiwen: I'm not too familiar with it, but I think the multinode jobs only enable a subset of services, and ultimately everything is tested via the different scenarios | 15:20 |
owalsh | ansiwen: controller services are overridden for multinode e.g https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/multinode-containers.yaml#L18 | 15:21 |
mandre | ansiwen: see for example https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario002-multinode-containers.yaml | 15:21 |
ansiwen | owalsh: thanks, that's exactly the file that I was now finally looking at :-) | 15:22 |
*** noslzzp has joined #tripleo | 15:22 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-validations master: Add undercloud service status check before update/upgrade https://review.openstack.org/462918 | 15:23 |
jistr | mandre: no i didn't. We should probably add pingtest after upgrade, though some of the jobs complete around 2.5h mark, so i'm a bit worried that we'd start hitting timeouts soon... Might be still worth a shot though. | 15:24 |
sshnaidm | bnemec, does OVB support creating swap on the host? | 15:25 |
sshnaidm | rlandy, the question raised because if we want to use separate node for undercloud, we need enough memory | 15:25 |
sshnaidm | rlandy, right now we have 8G swap in addition to 8G of host | 15:26 |
openstackgerrit | Merged openstack/tripleo-quickstart master: include tqe changes in basic.sh gate jobs https://review.openstack.org/471444 | 15:26 |
*** chlong has joined #tripleo | 15:26 | |
sshnaidm | rlandy, so I'm not sure ci.m1.large will be enough.. | 15:26 |
sshnaidm | rlandy, how much memory do you have in your jobs? | 15:27 |
rlandy | sshnaidm, ci.m1.xlarge? | 15:27 |
rlandy | would that work? | 15:27 |
sshnaidm | rlandy, yeah, but it's crazy waste of disk space | 15:27 |
*** dsneddon has joined #tripleo | 15:27 | |
sshnaidm | 160GB of disk, omg | 15:28 |
rlandy | sshnaidm: problem is we don't have control over flavors | 15:28 |
sshnaidm | rlandy, yeah, and I'm not sure we don't exceed quotas quickly using xlarge | 15:28 |
sshnaidm | swap seems a not bad workaround for that | 15:28 |
rlandy | we waste some form of quota meeting some other requirement all the time with fixed flavors | 15:29 |
rlandy | I'd agree | 15:29 |
sshnaidm | rlandy, but maybe it improved - how much memory do you have in jobs on undercloud? | 15:29 |
bnemec | I believe swap today is created as a file on the undercloud host. | 15:30 |
bnemec | But we really don't want to be using swap if we can help it. | 15:30 |
*** gvrangan has quit IRC | 15:30 | |
*** dparkes has quit IRC | 15:31 | |
*** itlinux has joined #tripleo | 15:34 | |
*** karimb has quit IRC | 15:35 | |
ansiwen | mandre: I always thought, the scenarios are for the opt-in services, wasn't aware that we also remove default services in CI runs | 15:36 |
*** udesale has quit IRC | 15:36 | |
*** karimb has joined #tripleo | 15:36 | |
openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Migrate quickstart-extras.yml --> TQE https://review.openstack.org/469614 | 15:37 |
openstackgerrit | Merged openstack/tripleo-quickstart master: Migrate quickstart-extras.yml --> TQE https://review.openstack.org/469611 | 15:37 |
*** agurenko has quit IRC | 15:38 | |
*** itlinux has quit IRC | 15:42 | |
*** atheurer has quit IRC | 15:42 | |
*** yprokule has quit IRC | 15:44 | |
sshnaidm | panda, ping | 15:44 |
*** panda is now known as panda|bbl | 15:45 | |
panda|bbl | sshnaidm: I'll be back in ~2 hours | 15:45 |
panda|bbl | sshnaidm: really have to run now | 15:46 |
panda|bbl | sorry | 15:46 |
abishop | hey experts, I need help with an upgrade issue when cinder-volume is controlled by pacemaker | 15:46 |
abishop | problem is after upgrade it's being restarted *before* puppet restarts the other cinder services | 15:46 |
abishop | I'm hoping there's a way for pacemaker to restart cinder-volume after puppet | 15:46 |
sshnaidm | panda|bbl, np, I will write in review | 15:46 |
*** ckyriakidou has quit IRC | 15:46 | |
*** ckyriakidou has joined #tripleo | 15:47 | |
*** fragatina has quit IRC | 15:48 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-validations master: Add undercloud service status check before update/upgrade https://review.openstack.org/462918 | 15:48 |
openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates stable/newton: DO NOT MERGE: Testing a fix https://review.openstack.org/471381 | 15:49 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates stable/ocata: Expose metric delay processing metric https://review.openstack.org/469217 | 15:50 |
*** oidgar has quit IRC | 15:50 | |
openstackgerrit | Martin André proposed openstack/tripleo-quickstart-extras master: Allow deleting all of docker cache https://review.openstack.org/471832 | 15:51 |
radez | dtantsur: ping, making some progress on the ironic and arm stuff have a question if you're around? | 15:53 |
dtantsur | radez: I am, hi! | 15:54 |
radez | hey dtantsur, so here's where I'm at... I've proivisioned my overcloud nodes and trying to get them to boot | 15:54 |
radez | if I direct boot via libvirt with overcloud and kernel/rd I can boot ok, but if i boot with ironic I get avcs | 15:54 |
radez | I tried disabling selinux by hand and then I can boot | 15:55 |
openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: Fix 1ctlr deployment. https://review.openstack.org/471835 | 15:55 |
openstackgerrit | Martin André proposed openstack-infra/tripleo-ci master: Workaround for limited disk space on undercloud https://review.openstack.org/471836 | 15:55 |
radez | is there a way as a temp workaround to pass enforcing=0 to all my nodes so I can boot and investigate the avcs? | 15:55 |
openstackgerrit | Brent Eagles proposed openstack/puppet-tripleo master: Remove unnecessary references to neutron core plugin hiera https://review.openstack.org/471837 | 15:56 |
dtantsur | radez: you were changing some files in /tftpboot, right? I think you may need to update its attributes | 15:56 |
openstackgerrit | Petr Blaho proposed openstack/tripleo-quickstart-extras master: Adds 'exit 1' to validate-overcloud-ipmi-connection template https://review.openstack.org/471838 | 15:56 |
dtantsur | radez: hmm, or are the denials on the nodes themselves? is it in the under or overcloud? | 15:56 |
radez | dtantsur: in the nodes themselves in the overcloud, I'm able to pxe and provision successfully now | 15:56 |
radez | and the overcloud nodes boot ok | 15:56 |
*** iranzo has quit IRC | 15:56 | |
radez | by boot I mean the kernel loads and the drives mount properly | 15:57 |
dtantsur | so at which stage to you get the denial? | 15:57 |
radez | but in the systemd process I get the avcs on the overcloud nodes | 15:57 |
mandre | bogdando: let's see if https://review.openstack.org/471832 and https://review.openstack.org/471836 help with the disk space issue | 15:57 |
radez | dtantsur: lemme get a log | 15:57 |
mandre | bogdando: it's a bit brutal but it should work :) | 15:57 |
dtantsur | radez: otherwise you can pass selinux=0 to the kernel command line | 15:58 |
radez | dtantsur: that was my short term workaround idea, is there a way to set that as a property on the ironic nodes? | 15:58 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: enable the rhsm/cdn yum repos in repo-setup https://review.openstack.org/470150 | 15:58 |
trown | dtantsur: we were able to do that manually... how do we tell ironic to do that? | 15:58 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: enable the rhsm/cdn yum repos in repo-setup https://review.openstack.org/470150 | 15:59 |
dtantsur | radez, trown, if you're using local boot (the default in ooo), it's not so easy | 15:59 |
openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK https://review.openstack.org/430277 | 16:00 |
*** atheurer has joined #tripleo | 16:00 | |
radez | dtantsur: :( | 16:00 |
radez | dtantsur: happen to have any docs that could help me through it? | 16:00 |
trown | dtantsur: ya localboot, we have to do it in the configdrive? was looking for where we do that? | 16:00 |
dtantsur | radez, trown, you can use whole-disk images (ask yolanda how), then you're in control of grub arguments | 16:00 |
trown | dtantsur: ah k.. configdrive is too late? | 16:01 |
dtantsur | trown: yep. it's read after the kernel is booted | 16:01 |
radez | dtantsur: maybe I'll try and just catch the nodes manually to get the deploy to run and then investigate? | 16:01 |
*** fzdarsky is now known as fzdarsky|afk | 16:01 | |
*** mcornea has quit IRC | 16:01 | |
dtantsur | radez: yeah, maybe this issue is easy to fix, and you won't need to mess with kernel arguments | 16:01 |
dtantsur | I suspect we need to whitelist something somewhere in the end | 16:02 |
*** jlinkes has quit IRC | 16:02 | |
radez | dtantsur: is there a place I could just hack into the ironic code selinux=0 to get it up and running? | 16:02 |
trown | dtantsur: I am suspicious that config drive is actually causing the avcs... do you know where we create that? | 16:02 |
dtantsur | radez: it's on the ramdisk, it may be quite hard to hack.. | 16:02 |
dtantsur | trown: this is possible, though it's strange that it only happens on ARM | 16:03 |
radez | dtantsur: ah, kk | 16:03 |
trown | dtantsur: ya... maybe something very low level has bad packaging though | 16:03 |
beagles | shardy: did Or ping you about missing neutron /etc/neutron/conf.d subdirectories? | 16:03 |
dtantsur | trown: maybe worth checking openstack-selinux package | 16:03 |
beagles | shardy: was just wondering what the problem/solution was - running into it myself right now | 16:04 |
radez | dtantsur: k, lemme see if I can work around it manually and get into the machines to look at the avcs more in depth | 16:04 |
dtantsur | radez, trown, you can temporary disable local boot (change boot_option capability to netboot on both flavor on nodes) | 16:04 |
dtantsur | radez, trown, in this case the nodes will always PXE boot | 16:04 |
trown | radez: ^ seems like a good immediate workaround | 16:04 |
*** gkadam has joined #tripleo | 16:04 | |
dtantsur | and you can hack this option: https://github.com/openstack/ironic/blob/master/etc/ironic/ironic.conf.sample#L3282 | 16:04 |
radez | and when they pxe they will just fall back to disk because they're already provisioned? | 16:05 |
trown | radez: at least would allow you to move forward and see if that is the only issue | 16:05 |
dtantsur | radez: no, ironic will provide kernel and ramdisk, then they'll continue booting from disk | 16:05 |
radez | ack | 16:05 |
shardy | beagles: hey, I think mandre has been taking a look and helping with the octavia patches | 16:06 |
radez | kk, I'll put that in my pocket too, I think I have a couple options to help me move forward here, thx for the help dtantsur trown I'll circle back after I try them | 16:06 |
shardy | beagles: sorry been a bit busy so not yet looked into it myself | 16:06 |
beagles | shardy: ah cool | 16:06 |
beagles | shardy: no problem | 16:06 |
*** noslzzp has quit IRC | 16:07 | |
*** ooolpbot has joined #tripleo | 16:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 16:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 16:10 |
*** ooolpbot has quit IRC | 16:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 16:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 16:10 |
*** ffiore has quit IRC | 16:11 | |
*** ebarrera has quit IRC | 16:12 | |
*** aufi has quit IRC | 16:13 | |
bogdando | mandre: interesting, I'm not sure that works, never tried before to run a container from an image then just remove the ground from beneath its legs so brutally :D | 16:13 |
bogdando | mandre: I thought that docker run-time COW layer is based on top of the stored image | 16:14 |
bogdando | and if stopped docker, removed the image and started, the container will never fly again | 16:14 |
*** cylopez has joined #tripleo | 16:15 | |
*** karimb has quit IRC | 16:16 | |
*** lmiccini has quit IRC | 16:17 | |
*** florianf has quit IRC | 16:17 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 16:22 |
*** ffiore has joined #tripleo | 16:28 | |
*** trown is now known as trown|lunch | 16:30 | |
*** limao_ has quit IRC | 16:32 | |
openstackgerrit | Cyril Lopez proposed openstack/tripleo-docs master: Explain how to use firewall in tripleo https://review.openstack.org/466252 | 16:32 |
*** tesseract has quit IRC | 16:34 | |
*** jpich has quit IRC | 16:36 | |
*** jkilpatr has quit IRC | 16:36 | |
*** milan has quit IRC | 16:36 | |
*** lucasagomes is now known as lucas-afk | 16:38 | |
*** bogdando has quit IRC | 16:39 | |
*** milan has joined #tripleo | 16:40 | |
*** jkilpatr has joined #tripleo | 16:40 | |
*** ramishra has quit IRC | 16:41 | |
*** dparkes has joined #tripleo | 16:42 | |
ansiwen | mandre: still around? this time it got deployed correctly (although the job is failing): http://logs.openstack.org/41/469541/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/cdcb4e0/logs/oooq/overcloud-controller-0/var/log/containers/nova/nova-consoleauth.log.txt.gz | 16:45 |
*** ffiore has quit IRC | 16:45 | |
ansiwen | mandre: http://logs.openstack.org/41/469541/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/cdcb4e0/logs/oooq/overcloud-controller-0/var/log/extra/docker/containers/nova_consoleauth/docker_info.log.txt.gz | 16:46 |
*** atheurer has quit IRC | 16:47 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Tacker Services https://review.openstack.org/466306 | 16:48 |
*** flepied has quit IRC | 16:48 | |
*** salmankhan has quit IRC | 16:49 | |
*** itlinux has joined #tripleo | 16:50 | |
*** ckyriakidou has quit IRC | 16:50 | |
*** milan has quit IRC | 16:51 | |
*** tvignaud has quit IRC | 16:53 | |
openstackgerrit | Alan Bishop proposed openstack/tripleo-heat-templates master: WIP: Experiment upgrading cinder-volume under pacemaker https://review.openstack.org/471447 | 16:55 |
*** egonzalez has quit IRC | 16:55 | |
mandre | ansiwen: great, +2 | 16:55 |
*** derekh has quit IRC | 16:56 | |
ansiwen | mandre: the problem seems to be a normal timeout, 145 minutes are just not enough, it's killed while validating | 16:57 |
ansiwen | mandre: this ansible step takes almost an hour: http://logs.openstack.org/41/469541/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/cdcb4e0/console.html#_2017-06-07_14_38_36_300199 | 16:58 |
mandre | ansiwen: correct, by using the multinode CI we can speed things up a lot, problem is we don't yet have parity | 17:00 |
mandre | ansiwen: there is a lot of work going on on the containers CI, mostly by jistr | 17:01 |
ansiwen | mandre: but don't we have to increase the timeout then for now? | 17:01 |
mandre | ansiwen: that's the time infra gives us to run out tests (we can't have more), we need to speed up our CI jobs | 17:02 |
*** sri_ has quit IRC | 17:03 | |
ansiwen | mandre: ok, so we can't make that job voting as EmilienM wants to, as long as we can't figure that out | 17:03 |
* EmilienM dancing | 17:05 | |
ansiwen | EmilienM: are you still around? so it's a simple timeout, could you remove the -1 then? | 17:05 |
ansiwen | the service is actually deployed | 17:05 |
EmilienM | I'll look at it in a few. I'm afk now | 17:05 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 17:05 |
*** tvignaud has joined #tripleo | 17:06 | |
ansiwen | EmilienM: afk, I see, you and your fancy speech-recognition IRC client... ;-) | 17:06 |
EmilienM | Yeah I'm on my phone | 17:06 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 17:07 |
ansiwen | EmilienM: ha, you are as crazy as me, I also have an IRC client on my phone... :-) | 17:08 |
*** cylopez has quit IRC | 17:08 | |
EmilienM | Probably my biggest mistake | 17:09 |
*** atheurer has joined #tripleo | 17:09 | |
*** cylopez has joined #tripleo | 17:10 | |
*** ooolpbot has joined #tripleo | 17:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 17:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 17:10 |
*** ooolpbot has quit IRC | 17:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 17:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 17:10 |
ansiwen | mandre: same here, btw. https://review.openstack.org/#/c/469486/ | 17:10 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: DO NOT MERGE Testing containerized tacker service https://review.openstack.org/471646 | 17:10 |
*** gbarros has quit IRC | 17:12 | |
openstackgerrit | Sven Anderson proposed openstack/tripleo-heat-templates master: DONT MERGE - Test for Ib96040c2e27ad76b1fa6ecb9468bb9d97b3c4518 https://review.openstack.org/469201 | 17:13 |
openstackgerrit | Sven Anderson proposed openstack/tripleo-heat-templates master: Add Ec2Api to container based deployment https://review.openstack.org/469200 | 17:13 |
tdasilva | trown|lunch: just wanted to give you a feedback that removing the --requirements flag worked for me, thanks! | 17:14 |
*** agopi has quit IRC | 17:16 | |
owalsh | mwhahaha: https://review.openstack.org/471319 ... this is essentially just a workaround until I can containerize polkitd | 17:16 |
owalsh | mwhahaha: trying to land a big puppet-tripleo refactor to get live-migraiton working, so ^^^ can wait till pike-3 | 17:16 |
*** shardy has quit IRC | 17:18 | |
mwhahaha | owalsh: ok just wondering if we have to have that. Let me know because the code isn't terrible if it's a short term fix but I just want to make sure we're not carrying the tech debt longer than we have to | 17:20 |
*** fragatina has joined #tripleo | 17:20 | |
owalsh | mwhahaha: alternative is to override the kolla nova uid so that it matches what we use on the rpm | 17:20 |
owalsh | mwhahaha: but dprince and I agreed on the polkit/puppet-tripleo option | 17:20 |
owalsh | mwhahaha: ultimately we are dependant on the host polkitd, so I think it's a must fix for pike-3 | 17:21 |
*** jpena is now known as jpena|off | 17:21 | |
*** itlinux has quit IRC | 17:24 | |
openstackgerrit | Merged openstack/tripleo-quickstart master: Container featuresets for scenarios 001-004 https://review.openstack.org/466696 | 17:27 |
*** agopi has joined #tripleo | 17:28 | |
openstackgerrit | Merged openstack/tripleo-quickstart master: Container upgrade featuresets for scenarios 001-004 https://review.openstack.org/465947 | 17:31 |
*** rcernin has joined #tripleo | 17:31 | |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Convert puppet and docker steps to ansible https://review.openstack.org/462211 | 17:31 |
openstackgerrit | Merged openstack/tripleo-common master: Use the mistral-lib actions https://review.openstack.org/471268 | 17:31 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Congress API service https://review.openstack.org/465734 | 17:32 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: DO NOT MERGE Testing containerized congress service https://review.openstack.org/471674 | 17:32 |
mwhahaha | owalsh: right but can we just deliver that in a different way than via puppet (ie package)? | 17:34 |
mwhahaha | owalsh: because what is there isn't really something that needs puppet | 17:34 |
*** fragatina has quit IRC | 17:35 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Congress API service https://review.openstack.org/465734 | 17:37 |
*** trown|lunch is now known as trown | 17:37 | |
trown | tdasilva: nice glad to hear it | 17:37 |
EmilienM | ansiwen: back on computer | 17:46 |
EmilienM | ansiwen: recheck | 17:46 |
EmilienM | ansiwen: if ci pass I'll approve | 17:46 |
EmilienM | ansiwen: thanks for your patience | 17:46 |
*** dtantsur is now known as dtantsur|afk | 17:50 | |
*** agopi has quit IRC | 17:51 | |
*** itlinux has joined #tripleo | 17:54 | |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Ability to enable/disable debug mode per OpenStack service https://review.openstack.org/467698 | 17:56 |
openstackgerrit | Merged openstack/tripleo-quickstart master: Fix repo mirror logic for tripleo-ci/master-undercloud-ocata-overcloud https://review.openstack.org/471774 | 17:56 |
openstackgerrit | Merged openstack/puppet-tripleo stable/newton: Restrict nova migration ssh tunnel https://review.openstack.org/471433 | 17:57 |
openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Send mail tool https://review.openstack.org/423340 | 17:57 |
*** zoli|wfh is now known as zoli|gone | 17:57 | |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Ensure /etc/ssh/ssh_known_hosts exist in docker config-data. https://review.openstack.org/471475 | 17:57 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Map /etc/ssh/ssh_known_hosts to all containers https://review.openstack.org/468461 | 17:57 |
*** gkadam has quit IRC | 17:59 | |
openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Add pre-selected Control Plane IP support. https://review.openstack.org/413278 | 18:03 |
*** lblanchard has joined #tripleo | 18:03 | |
*** lblanchard has quit IRC | 18:04 | |
*** agopi has joined #tripleo | 18:04 | |
*** lblanchard has joined #tripleo | 18:04 | |
*** itlinux has quit IRC | 18:05 | |
*** sid2 has quit IRC | 18:06 | |
openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Use Deployment actions for blacklist https://review.openstack.org/471876 | 18:06 |
*** gbarros has joined #tripleo | 18:09 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 18:09 |
*** ooolpbot has joined #tripleo | 18:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 18:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 18:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 18:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 18:10 |
*** ooolpbot has quit IRC | 18:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 18:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 18:10 |
*** salmankhan has joined #tripleo | 18:11 | |
*** itlinux has joined #tripleo | 18:11 | |
openstackgerrit | Merged openstack/tripleo-common master: Do not use multiple overrides with the same name https://review.openstack.org/471781 | 18:14 |
*** iranzo has joined #tripleo | 18:14 | |
*** iranzo has joined #tripleo | 18:14 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 18:15 |
*** salmankhan has quit IRC | 18:15 | |
*** lblanchard has quit IRC | 18:16 | |
*** lblanchard has joined #tripleo | 18:16 | |
*** lblanchard1 has joined #tripleo | 18:16 | |
*** lblanchard has quit IRC | 18:16 | |
*** lblanchard has joined #tripleo | 18:18 | |
*** lblanchard1 has quit IRC | 18:18 | |
*** pkovar has quit IRC | 18:19 | |
openstackgerrit | Jon Bernard proposed openstack/tripleo-heat-templates master: Move iscsid to a container https://review.openstack.org/462538 | 18:24 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 18:24 |
*** mdnadeem has quit IRC | 18:25 | |
*** pchavva has quit IRC | 18:29 | |
*** jaganathan has quit IRC | 18:29 | |
*** pchavva has joined #tripleo | 18:33 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 18:38 |
*** trown is now known as trown|mtg | 18:41 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 18:41 |
*** gbarros has quit IRC | 18:44 | |
*** gbarros has joined #tripleo | 18:44 | |
openstackgerrit | John Fulton proposed openstack/tripleo-common master: Add Mistral workbook to call ceph-ansible to install Ceph https://review.openstack.org/469644 | 18:46 |
openstackgerrit | John Fulton proposed openstack/tripleo-common master: Add Mistral workbook to call ceph-ansible to install Ceph https://review.openstack.org/469644 | 18:48 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP, adjust ansible ssh settings for performance https://review.openstack.org/471886 | 18:48 |
*** amoralej is now known as amoralej|off | 18:54 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Add support to configure Num of Storage sacks https://review.openstack.org/471334 | 18:54 |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 18:54 |
openstackgerrit | Merged openstack/tripleo-common stable/newton: Add MigrationSshKey to generated passwords https://review.openstack.org/466402 | 18:57 |
openstackgerrit | Merged openstack/tripleo-puppet-elements stable/newton: Include openstack-nova-migration in the overcloud images https://review.openstack.org/466405 | 18:58 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP, adjust ansible ssh settings for performance https://review.openstack.org/471886 | 18:58 |
*** itlinux has quit IRC | 19:00 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Fix the disable expirer to remove crontab https://review.openstack.org/470357 | 19:00 |
*** itlinux has joined #tripleo | 19:00 | |
*** gbarros has quit IRC | 19:00 | |
*** gbarros has joined #tripleo | 19:01 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 19:02 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 19:02 |
openstackgerrit | Merged openstack/tripleo-heat-templates stable/newton: Add migration SSH tunneling support https://review.openstack.org/471478 | 19:04 |
openstackgerrit | Merged openstack/tripleo-heat-templates stable/newton: Restrict nova migration ssh tunnel https://review.openstack.org/471435 | 19:04 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: only updates local_workspace if checkouts exist https://review.openstack.org/471878 | 19:05 |
openstackgerrit | Jon Bernard proposed openstack/tripleo-common master: Add missing cinder container images https://review.openstack.org/469401 | 19:06 |
*** lblanchard has quit IRC | 19:09 | |
*** ebarrera has joined #tripleo | 19:09 | |
*** ooolpbot has joined #tripleo | 19:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 19:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 19:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 19:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 19:10 |
*** ooolpbot has quit IRC | 19:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 19:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 19:10 |
*** dsariel_ has quit IRC | 19:10 | |
*** paramite has quit IRC | 19:11 | |
*** rcernin has quit IRC | 19:17 | |
*** dsneddon has quit IRC | 19:18 | |
*** rcrit has joined #tripleo | 19:19 | |
*** lblanchard has joined #tripleo | 19:21 | |
*** dsneddon has joined #tripleo | 19:22 | |
*** lblanchard has quit IRC | 19:22 | |
*** lblanchard1 has joined #tripleo | 19:22 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 19:24 |
rcrit | I've forgotten a lot of my ruby setup stuff, can someone help me run the puppet-tripleo tests? | 19:25 |
rcrit | I did a bundle install to get puppet-openstack_spec_helper but rspec can't find it | 19:26 |
* rcrit notes that it would be nice if the package had a testing section | 19:26 | |
openstackgerrit | Ryan Brady proposed openstack/tripleo-common master: Adds ansible actions https://review.openstack.org/470021 | 19:27 |
openstackgerrit | Lee Yarwood proposed openstack/tripleo-quickstart master: devmode: Add release switch https://review.openstack.org/471893 | 19:27 |
*** trown|mtg is now known as trown | 19:28 | |
*** fultonj has quit IRC | 19:28 | |
*** flepied has joined #tripleo | 19:30 | |
*** fultonj has joined #tripleo | 19:35 | |
*** rcernin has joined #tripleo | 19:35 | |
*** panda|bbl is now known as panda | 19:37 | |
*** ebarrera_ has joined #tripleo | 19:38 | |
rook | I haven't played with the container work, or kept up with it, but i am curious, if we have a worker count set to > 1 do we launch a container per worker, or a single container with multiple processes (i know that is a no-no in the container world, right?) | 19:42 |
*** trown is now known as trown|brb | 19:42 | |
rook | EmilienM ^ | 19:42 |
*** gbarros has quit IRC | 19:42 | |
bnemec | rcrit: I use https://docs.openstack.org/developer/puppet-openstack-guide/testing.html | 19:43 |
mwhahaha | rcrit: bundle install; bundle exec rake spec | 19:43 |
rcrit | thanks, I'll take a look | 19:43 |
rcrit | might be nice to mention it in the project code I guess | 19:43 |
*** flepied has quit IRC | 19:43 | |
rcrit | I just want to contribute a test and can't get anything running at all | 19:44 |
mwhahaha | rcrit: to be fair, there doesn't seem to be an documentation on running any tests python or puppet :D | 19:47 |
rcrit | but the python stuff I was able to discover pretty quicky, the ruby docs had me doing all sorts of stupid things | 19:50 |
*** iranzo has quit IRC | 19:50 | |
EmilienM | rook: puppet runs inside a container during the deployment, so facter will look for /proc/cpuinfo | 19:51 |
mwhahaha | rcrit: eh i've had equally infuriating experiances with python tests. but the puppet-openstack docs cover how to run the tests | 19:51 |
rcrit | I guess, I'm looking at puppet-tripleo so didn't think to look elsewhere for docs | 19:52 |
*** dsariel_ has joined #tripleo | 19:52 | |
EmilienM | rook: so if we mount /proc (which I think is the case), I guess facter will take the values from there and configure openstack services | 19:52 |
*** trown|brb is now known as trown | 19:55 | |
*** jcoufal has quit IRC | 19:56 | |
*** agopi has quit IRC | 19:59 | |
*** agopi has joined #tripleo | 20:00 | |
*** gbarros has joined #tripleo | 20:00 | |
sshnaidm | bnemec, around? | 20:02 |
sshnaidm | bnemec, do you know what is min-ready parameter in cloud node definition? http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n274 | 20:02 |
sshnaidm | rlandy, ^^ | 20:02 |
*** lblanchard1 has quit IRC | 20:03 | |
rlandy | sshnaidm: panda put the answer on #oooq | 20:03 |
EmilienM | yolanda: you still around? | 20:03 |
EmilienM | yolanda: promotion jobs fail on http://logs.openstack.org/15/359215/107/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-ha/14b94c4/console.html#_2017-06-07_13_59_14_021247 | 20:04 |
*** gbarros has quit IRC | 20:04 | |
*** lblanchard has joined #tripleo | 20:05 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: add steps to enable tripleo_ui https://review.openstack.org/459061 | 20:06 |
*** lblanchard has quit IRC | 20:06 | |
*** rbowen has quit IRC | 20:06 | |
*** rbowen has joined #tripleo | 20:06 | |
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org is being restarted now to clear some excessive connection counts while we debug the intermittent request failures reported over the past few minutes | 20:07 | |
*** paramite has joined #tripleo | 20:08 | |
pradk | huh whats with all the merge conflict failures .. i guess related to the above message? | 20:10 |
*** ooolpbot has joined #tripleo | 20:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 20:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 20:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 20:10 |
*** ooolpbot has quit IRC | 20:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 20:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 20:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 20:10 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Tacker Services https://review.openstack.org/466306 | 20:11 |
*** akrivoka has quit IRC | 20:12 | |
*** jprovazn has quit IRC | 20:13 | |
openstackgerrit | Victoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila API service https://review.openstack.org/449292 | 20:14 |
trown | EmilienM: do you know what the neutron fix is so I can try it? | 20:15 |
EmilienM | it didn't help AFIK | 20:15 |
EmilienM | but let me find the link | 20:15 |
trown | EmilienM: ah k... I tried earlier today and still got failure to ping in the pingtest (though stack deployed fine) | 20:16 |
EmilienM | trown: ihar can confirm but https://review.openstack.org/#/c/471446/ is the patch | 20:16 |
trown | pingtest stack deployed fine that is | 20:16 |
EmilienM | pingtest was fine? | 20:16 |
EmilienM | wow | 20:16 |
EmilienM | it should fail to create the network | 20:16 |
EmilienM | neutron-server should fail | 20:16 |
trown | hmm in my test the stack created fine, but ping failed | 20:17 |
EmilienM | trown: but could you talk to neutron? | 20:18 |
EmilienM | like creating net/subnet? | 20:18 |
*** cylopez has left #tripleo | 20:18 | |
trown | ya, have to do that in order to create the stack... maybe my reproducer was not valid | 20:18 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Add host logging for redis service template https://review.openstack.org/471709 | 20:19 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Mount /var/run/libvirt on ceilo agent compute https://review.openstack.org/469935 | 20:19 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 20:20 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Fix IronicInspectorAdmin to be https https://review.openstack.org/466432 | 20:20 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/newton: Addition of firewall rules for Nuage https://review.openstack.org/462286 | 20:20 |
*** Goneri has quit IRC | 20:20 | |
*** gbarros has joined #tripleo | 20:21 | |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Dell SC: Add exclude_domain_ip option https://review.openstack.org/455413 | 20:21 |
*** flepied has joined #tripleo | 20:21 | |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Don't create networks if neutron is not enabled https://review.openstack.org/471736 | 20:22 |
rook | EmilienM: so that means, each container will have multiple Eventlet/WSGI processes | 20:22 |
openstackgerrit | Merged openstack-infra/tripleo-ci master: Use bool with ansible boolean variables https://review.openstack.org/465111 | 20:22 |
trown | EmilienM: ah I think the config I tried to use is not updating overcloud images... so indeed my reproducer is not valid | 20:22 |
rook | https://review.openstack.org/#/c/471889/ | 20:23 |
EmilienM | rook: probably. Of course we might need to change that | 20:23 |
rook | sorry wrong channel ^^ | 20:23 |
rook | EmilienM I think it is OK, but Container people might bark/throwup | 20:23 |
EmilienM | trown: now promotion jobs fail on packaging: http://logs.openstack.org/15/359215/107/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-ha/14b94c4/console.html#_2017-06-07_13_59_14_021247 | 20:23 |
*** dsariel_ has quit IRC | 20:23 | |
EmilienM | rook: I don't think we want them to be sick | 20:24 |
EmilienM | trown: nevermind, they fixed it. see #rdo | 20:24 |
EmilienM | trown: (talking about the packaging thing) | 20:24 |
rook | EmilienM: Roger - then we might need a way to launch N containers per N service workers. | 20:24 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 20:24 |
*** gbarros has quit IRC | 20:24 | |
EmilienM | rook: I agree | 20:24 |
rook | EmilienM: looking forward to trying it! | 20:25 |
rook | Trying Pike that is. | 20:25 |
EmilienM | rook: keep in mind there is always room for improvment, specially in tripleo | 20:25 |
EmilienM | rook: I would ask you to maybe file a bug and triage it, so container team can look at it | 20:25 |
EmilienM | so we can track this discussion | 20:25 |
*** agopi has quit IRC | 20:29 | |
EmilienM | jrist: any blocker before I release tripleo-ui? | 20:33 |
EmilienM | jrist: can you submit the patch in tripleo-ui that update reno / release bump? | 20:34 |
*** dougbtv_ has joined #tripleo | 20:34 | |
EmilienM | rook: can you ack and confirm you'll create a bug report? just making sure it happens before I leave computer | 20:35 |
*** liverpooler has quit IRC | 20:37 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: add steps to enable tripleo_ui https://review.openstack.org/459061 | 20:37 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: DO NOT MERGE Testing containerized congress service https://review.openstack.org/471674 | 20:38 |
*** dougbtv has quit IRC | 20:38 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates stable/newton: Expose metric delay processing metric https://review.openstack.org/469226 | 20:39 |
*** agopi has joined #tripleo | 20:39 | |
*** atoth has quit IRC | 20:44 | |
*** lblanchard has joined #tripleo | 20:45 | |
*** ansmith has quit IRC | 20:46 | |
*** thrash is now known as thrash|bbl | 20:51 | |
*** pchavva has quit IRC | 20:51 | |
*** trown is now known as trown|outtypewww | 20:52 | |
*** pcaruana has quit IRC | 20:53 | |
*** pcaruana has joined #tripleo | 20:53 | |
weshay | rlandy, I suspect packaging | 21:02 |
rlandy | weshay: yep - can't be ovb-specific at that point - so we must see it elsewhere | 21:02 |
*** ebarrera_ has quit IRC | 21:06 | |
*** morazi has quit IRC | 21:06 | |
*** sid1 has joined #tripleo | 21:06 | |
openstackgerrit | Carlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud restore https://review.openstack.org/471021 | 21:06 |
*** lblanchard has quit IRC | 21:09 | |
weshay | rlandy, http://logstash.openstack.org/#dashboard/file/logstash.json?query=%5C%22%2Fetc%2Fgnocchi%2Fgnocchi.conf%20%20returned%201%20instead%20of%20one%20of%20%5B0%5D%5C%22 | 21:09 |
*** ooolpbot has joined #tripleo | 21:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 21:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 21:10 |
*** ooolpbot has quit IRC | 21:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 21:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 21:10 |
weshay | meh.. bad url | 21:12 |
*** agopi has quit IRC | 21:15 | |
*** pcaruana has quit IRC | 21:16 | |
*** sid1 has quit IRC | 21:17 | |
*** egafford has quit IRC | 21:18 | |
*** abishop has quit IRC | 21:22 | |
*** funzo_ has joined #tripleo | 21:22 | |
*** fultonj has quit IRC | 21:22 | |
*** fragatina has joined #tripleo | 21:22 | |
*** spredzy_ has joined #tripleo | 21:23 | |
*** bfournie1 has joined #tripleo | 21:23 | |
*** Tyrantelf_ has joined #tripleo | 21:25 | |
*** Hazelesque_ has joined #tripleo | 21:26 | |
*** Baggypan1s12000 has joined #tripleo | 21:26 | |
*** bfournie has quit IRC | 21:29 | |
*** funzo has quit IRC | 21:29 | |
*** spredzy has quit IRC | 21:29 | |
*** Baggypants12000 has quit IRC | 21:29 | |
*** Hazelesque has quit IRC | 21:29 | |
*** Tyrantelf has quit IRC | 21:29 | |
*** spredzy_ is now known as spredzy | 21:29 | |
*** SlickNik- has quit IRC | 21:32 | |
*** SlickNik has joined #tripleo | 21:34 | |
*** fultonj has joined #tripleo | 21:34 | |
*** SlickNik has left #tripleo | 21:35 | |
*** SlickNik has joined #tripleo | 21:36 | |
*** SlickNik has left #tripleo | 21:36 | |
*** ebarrera has quit IRC | 21:39 | |
*** ansmith has joined #tripleo | 21:40 | |
*** radeks has quit IRC | 21:41 | |
*** rcernin has quit IRC | 21:44 | |
*** panda is now known as panda|off | 21:47 | |
*** SlickNik has joined #tripleo | 21:48 | |
*** flepied has quit IRC | 21:48 | |
*** yamahata has joined #tripleo | 21:49 | |
*** cdearborn has quit IRC | 21:50 | |
*** dtrainor has quit IRC | 21:53 | |
*** dtrainor has joined #tripleo | 21:53 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: DO NOT REVIEW test ovb-ha against RDO latest builds https://review.openstack.org/471485 | 21:59 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: DO NOT REVIEW test ovb-ha against RDO latest builds https://review.openstack.org/471485 | 22:02 |
openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Make overcloud_containers.yaml template driven https://review.openstack.org/471537 | 22:05 |
*** ecerquei has quit IRC | 22:05 | |
openstackgerrit | Luigi Toscano proposed openstack/tripleo-heat-templates master: Containerize Sahara https://review.openstack.org/462681 | 22:08 |
*** ooolpbot has joined #tripleo | 22:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 22:10 |
*** ooolpbot has quit IRC | 22:10 | |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 22:10 |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 22:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 22:10 |
*** agopi has joined #tripleo | 22:15 | |
*** tosky has quit IRC | 22:19 | |
*** jmelvin has quit IRC | 22:20 | |
*** pradk has quit IRC | 22:20 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates stable/ocata: Add ignore_projects to filter gnocchi events https://review.openstack.org/469202 | 22:25 |
*** sid1 has joined #tripleo | 22:26 | |
*** itlinux has quit IRC | 22:31 | |
*** chlong has quit IRC | 22:32 | |
*** jkilpatr has quit IRC | 22:34 | |
*** rlandy has quit IRC | 22:35 | |
*** sid1 has quit IRC | 22:36 | |
openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Refactor nova migration config into client & target profiles https://review.openstack.org/471950 | 22:41 |
*** itlinux has joined #tripleo | 22:41 | |
*** sid1 has joined #tripleo | 22:42 | |
*** dsneddon has quit IRC | 22:42 | |
openstackgerrit | Merged openstack/tripleo-quickstart master: devmode: Add release switch https://review.openstack.org/471893 | 22:43 |
*** itlinux has quit IRC | 22:54 | |
*** sid1 has quit IRC | 22:55 | |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Run the nova-compute container as the nova user https://review.openstack.org/468466 | 22:55 |
*** sid1 has joined #tripleo | 22:56 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: DO NOT REVIEW test ovb-ha against RDO latest builds https://review.openstack.org/471485 | 23:05 |
openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Refactor nova migration config into client & target profiles https://review.openstack.org/471950 | 23:07 |
*** fragatina has quit IRC | 23:09 | |
*** ooolpbot has joined #tripleo | 23:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 23:10 |
openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] - Assigned to John Trowbridge (trown) | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695904 | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1696094 | 23:10 |
*** ooolpbot has quit IRC | 23:10 | |
openstack | Launchpad bug 1695904 in tripleo "stable/ocata scenario002 broken due to volume went to status error" [Critical,Triaged] | 23:10 |
openstack | Launchpad bug 1696094 in tripleo "CI: ovb-ha promotion job fails with 504 gateway timeout, neutron-server create-subnet timing out" [Critical,Triaged] | 23:10 |
*** openstack has joined #tripleo | 23:13 | |
*** sid1 has quit IRC | 23:17 | |
openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Make overcloud_containers.yaml template driven https://review.openstack.org/471537 | 23:18 |
*** sid1 has joined #tripleo | 23:18 | |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Add support for nova live/cold-migration with containers https://review.openstack.org/471956 | 23:19 |
openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Refactor nova migration config into client & target profiles https://review.openstack.org/471950 | 23:23 |
*** sid1 has quit IRC | 23:24 | |
*** thrash|bbl is now known as thrash | 23:27 | |
*** pmannidi has joined #tripleo | 23:30 | |
openstackgerrit | John Fulton proposed openstack/tripleo-common master: Add Mistral workbook to call ceph-ansible to install Ceph https://review.openstack.org/469644 | 23:40 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use class as super() argument https://review.openstack.org/471961 | 23:44 |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Add support for nova live/cold-migration with containers https://review.openstack.org/471956 | 23:47 |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Add support for nova live/cold-migration with containers https://review.openstack.org/471956 | 23:50 |
openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Modify libvirtd container command line when TLS is enabled https://review.openstack.org/471963 | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!