Friday, 2018-03-30

*** zshi has joined #tripleo00:00
*** rajinir_ has joined #tripleo00:04
openstackgerritOliver Walsh proposed openstack/instack-undercloud master: DNM: test with 15 wsgi threads  https://review.openstack.org/55785300:05
*** timothyb89_ has joined #tripleo00:06
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: DNM: test with 15 wsgi threads  https://review.openstack.org/55782900:06
*** moisa has joined #tripleo00:07
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755600:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]00:10
*** zoliXXL has joined #tripleo00:10
*** mgagne_ has joined #tripleo00:10
*** fzdarsky has joined #tripleo00:11
*** moguimar has quit IRC00:12
*** rajinir has quit IRC00:12
*** ajo has quit IRC00:12
*** Sl0w has quit IRC00:12
*** zoli has quit IRC00:12
*** eric-young has quit IRC00:12
*** mgagne has quit IRC00:12
*** timothyb89 has quit IRC00:12
*** homegrown has quit IRC00:12
*** atoth has quit IRC00:12
*** social has quit IRC00:12
*** yolanda has quit IRC00:12
*** tdasilva has quit IRC00:12
*** jjoyce has quit IRC00:12
*** dsneddon has quit IRC00:12
*** sdake has quit IRC00:12
*** rickflare has quit IRC00:12
*** toure|biab has quit IRC00:12
*** jaganathan_ has quit IRC00:12
*** jlabarre has quit IRC00:12
*** ukalifon has quit IRC00:12
*** thrash|g0ne has quit IRC00:12
*** slagle has quit IRC00:12
*** owalsh has quit IRC00:12
*** noslzzp has quit IRC00:12
*** michchap has quit IRC00:12
*** arxcruz|rover has quit IRC00:12
*** jokke_ has quit IRC00:12
*** pabelanger has quit IRC00:12
*** chkumar246 has quit IRC00:12
*** tbonds has quit IRC00:12
*** SlickNik has quit IRC00:12
*** gunix has quit IRC00:12
*** EmilienM has quit IRC00:12
*** kambiz has quit IRC00:12
*** migi has quit IRC00:12
*** rfolco|off has quit IRC00:12
*** ChanServ has quit IRC00:12
*** rajinir_ is now known as rajinir00:12
*** ajo has joined #tripleo00:12
*** Sl0w has joined #tripleo00:13
*** chkumar246 has joined #tripleo00:13
*** jaganathan_ has joined #tripleo00:13
*** tbonds has joined #tripleo00:13
*** jlabarre has joined #tripleo00:13
*** ukalifon has joined #tripleo00:13
*** SlickNik has joined #tripleo00:13
*** pabelanger has joined #tripleo00:13
*** gunix has joined #tripleo00:13
*** EmilienM has joined #tripleo00:13
*** thrash|g0ne has joined #tripleo00:13
*** slagle has joined #tripleo00:13
*** owalsh has joined #tripleo00:13
*** kambiz has joined #tripleo00:13
*** migi has joined #tripleo00:13
*** rfolco|off has joined #tripleo00:13
*** noslzzp has joined #tripleo00:13
*** michchap has joined #tripleo00:13
*** ChanServ has joined #tripleo00:13
*** jokke_ has joined #tripleo00:13
*** arxcruz|rover has joined #tripleo00:13
*** barjavel.freenode.net sets mode: +vo EmilienM ChanServ00:13
*** bandini has quit IRC00:14
*** ericyoung has joined #tripleo00:15
*** bandini has joined #tripleo00:16
openstackgerritMerged openstack/tripleo-quickstart-extras master: remove virt-bmc setup for tqe  https://review.openstack.org/55437600:21
openstackgerritMerged openstack/tripleo-quickstart master: setup the virt-bmc in tq  https://review.openstack.org/55437700:21
*** csmart has quit IRC00:23
*** csmart has joined #tripleo00:33
*** psahoo has joined #tripleo00:36
*** khrystoph has joined #tripleo00:38
*** jcoufal has quit IRC00:40
*** rwsu has quit IRC00:41
*** khrystoph has quit IRC00:43
*** homegrown has joined #tripleo00:44
*** psahoo has quit IRC00:44
*** sanjayu has quit IRC00:45
*** psahoo has joined #tripleo00:58
*** khrystoph has joined #tripleo01:00
*** fragatina has quit IRC01:00
*** cshastri has joined #tripleo01:02
*** fragatina has joined #tripleo01:03
*** khrystoph has quit IRC01:04
*** fragatina has quit IRC01:07
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755601:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]01:10
*** fzdarsky_ has joined #tripleo01:23
*** fzdarsky has quit IRC01:27
*** fzdarsky_ has quit IRC01:28
*** atoth has joined #tripleo01:28
*** social has joined #tripleo01:28
*** yolanda has joined #tripleo01:28
*** tdasilva has joined #tripleo01:28
*** jjoyce has joined #tripleo01:28
*** dsneddon has joined #tripleo01:28
*** sdake has joined #tripleo01:28
*** rickflare has joined #tripleo01:28
*** toure|biab has joined #tripleo01:28
*** hamzy has joined #tripleo01:28
*** itlinux has joined #tripleo01:30
*** bandini has quit IRC01:30
*** fzdarsky_ has joined #tripleo01:31
*** csmart has quit IRC01:32
*** bandini has joined #tripleo01:33
*** weshay has quit IRC01:35
*** weshay has joined #tripleo01:36
*** myoung has quit IRC01:38
*** myoung has joined #tripleo01:43
openstackgerritMerged openstack/tripleo-docs master: Document toggling debug for containerized services  https://review.openstack.org/55730501:47
openstackgerritzenghui.shi proposed openstack/tripleo-heat-templates stable/newton: Ensure openvswitch user exist before package update  https://review.openstack.org/55716401:50
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: Allow installing glean from package  https://review.openstack.org/55786701:50
*** itlinux has quit IRC01:54
openstackgerritzenghui.shi proposed openstack/tripleo-heat-templates master: Ensure openvswitch user exist before package update  https://review.openstack.org/55715501:55
*** csmart has joined #tripleo01:55
*** fragatina has joined #tripleo02:01
*** fragatina has quit IRC02:05
*** moshele has joined #tripleo02:07
*** itlinux has joined #tripleo02:09
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755602:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]02:10
*** fragatina has joined #tripleo02:12
*** fragatina has quit IRC02:13
*** fragatina has joined #tripleo02:14
*** itlinux has quit IRC02:16
*** moshele has quit IRC02:25
*** itlinux has joined #tripleo02:29
*** itlinux has quit IRC02:30
*** daidv has quit IRC02:34
*** daidv has joined #tripleo02:34
*** shrjoshi has joined #tripleo02:59
EmilienMalee_afk: I reproduced the bug perfectly in my env03:01
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755603:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]03:10
*** cshastri has quit IRC03:11
openstackgerritSai Ram Peesapati proposed openstack/tripleo-heat-templates master: Adding parameters to Enable or Disbale Nuage VRS and Metadata  https://review.openstack.org/55787703:25
openstackgerritSai Ram Peesapati proposed openstack/tripleo-heat-templates master: Adding parameters to Enable or Disbale Nuage VRS and Metadata  https://review.openstack.org/55787703:26
*** cshastri has joined #tripleo03:31
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Add Mistral to the provided controller roles  https://review.openstack.org/54641103:36
*** daidv has quit IRC03:51
*** daidv has joined #tripleo03:51
openstackgerritHoang Trung Hieu proposed openstack/diskimage-builder master: Add iscsi-boot element  https://review.openstack.org/54270804:01
*** fragatina has quit IRC04:02
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: Fix tag for rolling upgrade compte task  https://review.openstack.org/55788004:03
*** cshastri_ has joined #tripleo04:03
*** chandankumar has joined #tripleo04:03
*** chkumar246 has quit IRC04:04
*** cshastri has quit IRC04:07
*** cshastri_ has quit IRC04:08
*** chandankumar has quit IRC04:08
*** cshastri_ has joined #tripleo04:08
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755604:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]04:10
*** chandankumar has joined #tripleo04:20
*** shrjoshi has quit IRC04:22
*** tcw has quit IRC04:39
*** gkadam has joined #tripleo04:41
*** athomas has quit IRC04:46
*** ykarel|away has joined #tripleo04:48
*** fragatina has joined #tripleo04:50
*** daidv has quit IRC04:55
*** psachin` has joined #tripleo05:09
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755605:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]05:10
*** ooolpbot has quit IRC05:10
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: DNM - containerized undercloud upgrade  https://review.openstack.org/55362905:12
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: DNM - test containerized undercloud upgrades  https://review.openstack.org/55363305:12
EmilienMhjensas: nice! https://review.openstack.org/#/c/547326/ is working fine, see https://review.openstack.org/#/c/542556/05:13
EmilienMhjensas: https://logs.rdoproject.org/56/542556/88/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z28f6ca71f2864d2faffb749a6f25bd41/console.txt.gz#_2018-03-30_02_09_20_48505:14
EmilienMhttps://logs.rdoproject.org/56/542556/88/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z28f6ca71f2864d2faffb749a6f25bd41/undercloud/var/log/host_info.txt.gz05:15
EmilienMhjensas: grep for BOOTSTACK_MASQ05:15
EmilienMhjensas: really nice work, thanks for your help05:15
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud  https://review.openstack.org/54255605:17
EmilienMhjensas: last adjustment and we're ready ^05:17
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Revert "undercloud: workaround for masquerading network in CI/OVB"  https://review.openstack.org/55785105:17
*** fpan has joined #tripleo05:30
*** tzumainn has quit IRC05:31
*** zzzeek has quit IRC05:39
*** gkadam has quit IRC05:43
*** zzzeek has joined #tripleo05:50
*** zzzeek has quit IRC05:51
*** zzzeek has joined #tripleo05:52
*** matbu has joined #tripleo05:59
*** chkumar246 has joined #tripleo06:03
*** cshastri has joined #tripleo06:04
*** chandankumar has quit IRC06:04
*** cshastri_ has quit IRC06:07
*** chkumar246 has quit IRC06:08
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755606:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]06:10
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: FFU: Add support for panko service upgrade  https://review.openstack.org/55767506:18
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Remove the lacp bond mode constraints  https://review.openstack.org/55759706:18
openstackgerritMerged openstack/puppet-tripleo stable/queens: Add configuration for the Nova proxy endpoint  https://review.openstack.org/55639906:18
openstackgerritMerged openstack/tripleo-common master: Move health_manager cfg section to octavia_worker  https://review.openstack.org/55748306:18
openstackgerritMerged openstack/tripleo-validations master: add lower-constraints job  https://review.openstack.org/55626706:18
*** chkumar246 has joined #tripleo06:20
*** etingof has quit IRC06:23
*** marios has joined #tripleo06:25
*** dbecker has quit IRC06:32
*** moshele has joined #tripleo06:33
openstackgerritzenghui.shi proposed openstack/puppet-tripleo master: Add interface mapping support for sriov configuration  https://review.openstack.org/49944806:36
*** moshele has quit IRC06:38
*** yamahata has joined #tripleo06:39
*** dbecker has joined #tripleo06:45
*** cylopez has joined #tripleo06:56
*** ykarel|away has quit IRC07:05
*** ykarel|away has joined #tripleo07:05
*** zshi has quit IRC07:06
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755607:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]07:10
*** zshi has joined #tripleo07:10
*** psachin` has quit IRC07:16
*** yamahata has quit IRC07:16
*** tosky__ has joined #tripleo07:16
*** tesseract has joined #tripleo07:17
*** bogdando has joined #tripleo07:24
*** dtrainor has quit IRC07:33
*** dtrainor has joined #tripleo07:37
*** cylopez has left #tripleo07:39
*** dtrainor has quit IRC07:44
*** dtrainor has joined #tripleo07:47
*** ykarel|away has quit IRC07:49
*** ykarel|away has joined #tripleo07:50
*** dtrainor has quit IRC07:53
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Produce nicer stdout for quickstart.sh  https://review.openstack.org/55777507:53
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260707:55
*** dtrainor has joined #tripleo07:57
*** cshastri_ has joined #tripleo08:03
*** chandankumar has joined #tripleo08:04
*** chkumar246 has quit IRC08:04
*** cshastri has quit IRC08:07
*** chandankumar has quit IRC08:08
*** cshastri_ has quit IRC08:08
*** suuuper has joined #tripleo08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755608:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]08:10
*** cshastri_ has joined #tripleo08:20
*** chandankumar has joined #tripleo08:20
*** ykarel_ has joined #tripleo08:20
*** ykarel|away has quit IRC08:21
*** gkadam has joined #tripleo08:46
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Produce nicer stdout for quickstart.sh  https://review.openstack.org/55777508:46
*** avivgt has quit IRC08:47
*** moshele has joined #tripleo08:48
*** fzdarsky_ has quit IRC08:49
openstackgerritBogdan Dobrelya proposed openstack-infra/tripleo-ci master: Collect quickstart.sh execution logs  https://review.openstack.org/55791008:49
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Collect quickstart.sh execution logs  https://review.openstack.org/55791108:49
bogdandoo/ PTAL https://review.openstack.org/#/c/556516/09:02
bogdandodtantsur|afk: ^^ perchance please :)09:02
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755609:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]09:10
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: set libvirt root system config as default  https://review.openstack.org/54926809:17
*** tosky__ is now known as tosky09:20
*** matbu has quit IRC09:22
openstackgerritHoang Trung Hieu proposed openstack/diskimage-builder master: Add iscsi-boot element  https://review.openstack.org/54270809:36
*** waleedm has joined #tripleo09:42
*** moshele has quit IRC09:45
*** waleedm has quit IRC09:47
*** ykarel_ has quit IRC09:50
openstackgerritCarlos Goncalves proposed openstack/tripleo-common stable/queens: Move health_manager cfg section to octavia_worker  https://review.openstack.org/55791810:01
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Remove the os-net-config element config.json template  https://review.openstack.org/55773910:05
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755610:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]10:10
*** liverpooler has joined #tripleo10:24
*** dciabrin_ has quit IRC10:31
*** dciabrin has joined #tripleo10:32
*** cshastri_ has quit IRC10:38
*** ykarel_ has joined #tripleo10:38
*** gfidente has joined #tripleo10:51
*** asbishop has joined #tripleo10:53
*** salmankhan has joined #tripleo10:54
*** matbu has joined #tripleo11:03
*** liverpooler has quit IRC11:05
*** salmankhan has quit IRC11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755611:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]11:10
*** moshele has joined #tripleo11:12
*** salmankhan has joined #tripleo11:13
*** moshele has quit IRC11:20
*** ykarel_ is now known as ykarel11:23
*** salmankhan has quit IRC11:32
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Migrate Ceph deployment to external_deploy_task  https://review.openstack.org/54696611:35
*** waleedm has joined #tripleo11:39
*** jaganathan_ has quit IRC11:42
*** cshastri_ has joined #tripleo11:48
*** rfolco|off is now known as rfolco|ruck11:50
openstackgerritMerged openstack/tripleo-heat-templates master: Add ctlplane networking for routed networks  https://review.openstack.org/54732612:01
openstackgerritGiulio Fidente proposed openstack/tripleo-common master: Force ANSIBLE_CALLBACK_WHITELIST to empty list for collect_nodes_uuid  https://review.openstack.org/55263612:07
openstackgerritGiulio Fidente proposed openstack/tripleo-common master: Force ANSIBLE_CALLBACK_WHITELIST to empty list for collect_nodes_uuid  https://review.openstack.org/55263612:08
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755612:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]12:10
Tenguo______O ok. that one, I never ever saw it before: 2018-03-30 12:09:01Z [overcloud.AllNodesDeploySteps]: UPDATE_FAILED  [Errno 13] Permission denied: '/var/log/heat/heat-engine.log'12:11
Tenguunbelievable. I must be cursed with my migration, I have no other explanation.12:11
openstackgerritGiulio Fidente proposed openstack/tripleo-quickstart master: Add a new nodes configuration and featureset049 for Ceph  https://review.openstack.org/55337512:13
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade master: Add debug during stack output creation.  https://review.openstack.org/54926912:18
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade master: Make validation-errors-nonfatal option available for FFU.  https://review.openstack.org/55658312:18
*** cshastri_ has quit IRC12:20
*** chem has quit IRC12:22
*** chem has joined #tripleo12:23
*** mmethot has quit IRC12:23
openstackgerritMerged openstack/tripleo-common master: Check mode support for Server Deployments  https://review.openstack.org/54906612:26
*** agopi has quit IRC12:30
*** bfournie has joined #tripleo12:31
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade stable/queens: Correct var name tripleo_upgrade_validations_non_fatal.  https://review.openstack.org/55792612:48
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade stable/queens: Include new CLI changes for overcloud update.  https://review.openstack.org/55792812:50
*** mmethot has joined #tripleo12:50
*** rfolco has joined #tripleo13:04
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755613:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]13:10
*** jcoufal has joined #tripleo13:14
*** waleedm has quit IRC13:24
*** zshi has quit IRC13:24
openstackgerritRafael Folco proposed openstack/tripleo-quickstart-extras master: Collect dstat on overcloud nodes  https://review.openstack.org/55745513:28
*** mmethot has quit IRC13:32
*** ykarel has quit IRC13:46
*** moshele has joined #tripleo13:47
*** moshele has quit IRC13:49
*** suuuper has quit IRC13:51
*** mmethot has joined #tripleo14:06
*** ykarel has joined #tripleo14:07
*** dsariel has joined #tripleo14:07
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755614:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]14:10
openstackgerritMarios Andreou proposed openstack/tripleo-common master: WIP Update package_update action + workbook for ffu upgrade cli  https://review.openstack.org/55793614:25
openstackgerritMarios Andreou proposed openstack/python-tripleoclient master: WIP - Add Fast Forward Upgrade CLI  https://review.openstack.org/55793714:25
*** weshay is now known as weshay_mod14:37
openstackgerritMerged openstack/tripleo-common stable/queens: Check if plan_env has resource_registry key  https://review.openstack.org/55770314:45
EmilienMbogdando: hey14:50
openstackgerritRafael Folco proposed openstack/tripleo-quickstart-extras master: Collect dstat on overcloud nodes  https://review.openstack.org/55745514:51
EmilienMbogdando: thx for the reviews, nice progress14:51
weshay_modrfolco, lolz.. and tempest timed out on master14:51
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates stable/pike: Remove the os-net-config element config.json template  https://review.openstack.org/55793914:51
EmilienMbogdando: the ovb is still unstable :-(14:52
EmilienMweshay_mod: hey14:52
weshay_modrfolco, twice in a row14:52
weshay_modEmilienM, howdy14:52
weshay_modEmilienM, if you have a sec.. https://review.openstack.org/#/c/557455/14:53
EmilienMweshay_mod: sure14:53
*** suuuper has joined #tripleo14:53
weshay_modwe'll need this for first steps in debugging perf14:53
EmilienMweshay_mod: I already +2 it last night14:53
weshay_modword.. I'm merging14:54
EmilienMwait14:54
EmilienMhe pushed a new pathcset...14:54
weshay_modEmilienM, new commit msg14:55
EmilienMyeah14:55
EmilienMI just saw it14:55
EmilienM+214:55
EmilienMweshay_mod, bogdando : I have a CI job that upgrade the undercloud to rocky, containerized14:55
*** tosky__ has joined #tripleo14:55
EmilienMit's a prototype, with workarounds (just one or 2, nothing critical, since we need a new semver)14:55
weshay_modEmilienM, point me at what we want to cover today14:55
EmilienMbut it works14:55
*** dsariel has quit IRC14:55
EmilienMsee http://logs.openstack.org/33/553633/14/check/tripleo-ci-centos-7-undercloud-upgrades/8422fc5/logs/undercloud/home/zuul/undercloud_upgrade.log.txt.gz14:56
EmilienMit's upgrading the undercloud from Queens (non containerized) to Rocky (containerized14:56
*** tosky has quit IRC14:56
*** tesseract has quit IRC14:57
*** tesseract has joined #tripleo14:57
weshay_modEmilienM, be u ti ful14:58
EmilienMlol14:59
weshay_modEmilienM, give me a few14:59
weshay_modto go through it14:59
EmilienMweshay_mod: we do the push ups on bluejeans?15:01
weshay_modEmilienM, lolz15:01
weshay_modI want to finish my morning coffee15:01
* weshay_mod looks into http://logs.openstack.org/33/553633/14/check/tripleo-ci-centos-7-undercloud-upgrades/8422fc5/job-output.txt.gz#_2018-03-30_06_10_02_35823815:02
openstackgerritMerged openstack/tripleo-quickstart master: Add overcloud related variables in master upgrade release file.  https://review.openstack.org/55770515:03
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Correct var name tripleo_upgrade_validations_non_fatal.  https://review.openstack.org/55792615:03
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Include new CLI changes for overcloud update.  https://review.openstack.org/55792815:03
weshay_modci merge party15:03
*** dsariel has joined #tripleo15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755615:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]15:10
*** ooolpbot has quit IRC15:10
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Add fs050: containerized undercloud upgrade  https://review.openstack.org/55794115:11
EmilienMweshay_mod: what's progress on https://bugs.launchpad.net/tripleo/+bug/1757556 ?15:15
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]15:15
weshay_modchem, the shelf life of an ungated workflow is roughly 24-48 hours15:16
chemweshay_mod: very hard (impossible) to gate ffu from master, but queens's ffu jobs should be added to tripleo sooner than later so that we know before the merge15:18
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: DNM - containerized undercloud upgrade  https://review.openstack.org/55362915:18
chemweshay_mod: or else I'm going to learn to really enjoy those 24-48h every week :)15:18
weshay_modchem, the ffu job was queens right?15:19
chemweshay_mod: yeap15:20
weshay_modwe had a promotion.. so my guys will be getting that job in the promotion pipeline15:20
weshay_modwe should have caught it there15:20
chemweshay_mod: but ffu patch comes from master15:20
chemweshay_mod: then queens ... I guess no way around this15:20
weshay_modthe ci should be one branch15:20
weshay_modthat's not a prob15:20
weshay_modnot to get into the branched repo thang atm..15:21
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Introduce tripleo-ci-centos-7-containerized-undercloud-upgrades  https://review.openstack.org/55363315:21
EmilienMweshay_mod: ^ boom15:21
weshay_modbut gating queens, pike etc.. w/ ci from master is not a prob is what I'm saing15:21
weshay_modya.. /me continues my review..15:21
chemweshay_mod: enjoy :) (root cause found ... maybe a bit late for a proper patch though)15:22
*** neo_ has joined #tripleo15:22
weshay_modchem, aye15:22
weshay_modchem, the ffu upgrade scripts are getting injected into non-ffu jobs15:23
* weshay_mod needs to look into that15:23
weshay_modmaybe it's fine15:24
chemweshay_mod: ¿ didn't get that sentence, do you have an example ?15:24
weshay_modEmilienM, bogdando had a patch that would remove skipped tasks.. we should revive that patch15:24
EmilienMweshay_mod: link?15:24
weshay_modchem, I think it's fine.. it's all skipped in the run I'm looking at15:25
weshay_modsorry15:25
weshay_modEmilienM, I'll go look for it15:25
chemweshay_mod: yeah, rigth that's a bit confusing15:25
weshay_modjust noisy15:26
weshay_modfor everything, not just upgrades15:26
weshay_modchem, OH.. here it is http://logs.openstack.org/33/553633/14/check/tripleo-ci-centos-7-undercloud-upgrades/8422fc5/logs/undercloud/home/zuul/tripleo-QrUBKd-config/15:26
weshay_modthat's what I saw the other day15:26
weshay_modand was wondering about15:27
EmilienMbogdando: https://review.openstack.org/#/c/557851/ I'm debugging this one now, it's a good test to see if the Masquerade code actually works15:27
bogdandoEmilienM: hey. and nice progress with instack upgrades! :)15:28
chemweshay_mod: it's the whole downloaded ansible output that get saved in the logs ... it's expected15:28
chemweshay_mod: and super useful15:28
EmilienMbogdando: yeah good! it's actually without instack ;-)15:28
bogdandoweshay_mod: yeah, let me pick it... skippy plugin or so15:28
weshay_modbogdando, right15:28
EmilienMbogdando: you know what would be fun, is to remove instack-undercloud rpm and see how things work15:28
* weshay_mod waves at bogdando :)15:29
*** ayoung has joined #tripleo15:29
bogdandoweshay_mod: and folks, btw, did you see https://review.openstack.org/#/q/topic:nice+(status:open+OR+status:merged) ? it's NICE :D15:29
bogdandoat least for me15:29
EmilienMbogdando: show us an example?15:30
EmilienMwhat file should we look at?15:30
bogdandohttps://review.openstack.org/#/c/477569/ weshay_mod15:30
bogdandoEmilienM: it is in the comments, like https://www.diffchecker.com/xTIIxwRv15:31
*** cgoncalves has joined #tripleo15:33
bogdandoit might be not yet fully shown in CI console logs, need to check the remaining patches in the group. Anyway, the quickstart.sh itself, when just run locally, produces outputs like that example EmilienM weshay_mod15:33
bogdandoI'm not sure JSON parsing tools would do better, as ansible dumps not really right jsons15:33
bogdandoI wish it had a pure json outputs mode, like journalctl15:34
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: move virtbmc var to common  https://review.openstack.org/55794415:34
*** agopi has joined #tripleo15:34
cgoncalvesis my understanding correct that several project horizon dashboards are being enabled unconditionally? https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/horizon.yaml#L148-L17015:36
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Use skippy stdout callback for ansible.cfg  https://review.openstack.org/47756915:36
cgoncalvesif so, doesn't make much sense to enable them if their respective services are not enabled15:36
weshay_modEmilienM, so generally speaking.. it would awesome if we could print the version from and to in the logs.. like in http://logs.openstack.org/33/553633/14/check/tripleo-ci-centos-7-undercloud-upgrades/8422fc5/logs/undercloud/home/zuul/undercloud_upgrade.sh.txt.gz15:39
weshay_modEmilienM, for another patch..15:39
*** itlinux has joined #tripleo15:40
weshay_modEmilienM, oh snap we hard coded the build_id https://review.openstack.org/#/c/553629/10/templates/undercloud_upgrade.sh.j215:43
EmilienMweshay_mod: yes, like I said it's WIP15:44
EmilienMweshay_mod: sbaker is working on a workflow15:44
weshay_modk15:44
EmilienMweshay_mod: so my stuff will disappear15:44
*** ykarel is now known as ykarel|away15:45
weshay_modEmilienM, ok.. so here's how that will work in the future for $branch dlrn_hash -> $branch dlrn_hash15:45
weshay_modEmilienM, note the current-tripleo and previous-current-tripleo https://images.rdoproject.org/master/rdo_trunk/15:45
*** dsariel has quit IRC15:45
weshay_modEmilienM, we need to update our promoter script to create that for the dlrn repos and I guess containers15:46
*** neo_ has quit IRC15:46
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001/035: enable MasqueradeNetworks service  https://review.openstack.org/55795015:46
*** dhill_ has joined #tripleo15:47
cgoncalvesEmilienM: could you help me clarify my question ^? the way I see it is I'd need to add ENABLE_OCTAVIA=yes and that would be unconditional :/15:48
weshay_modhttps://trello.com/c/l5eo528k/654-create-previous-symlinks-via-dlrnapi-for-upgrade-jobs15:48
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud  https://review.openstack.org/54255615:48
*** bogdando has quit IRC15:53
*** etingof has joined #tripleo15:53
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Revert "FFU: Add support for panko service upgrade"  https://review.openstack.org/55795315:59
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/queens: Revert "FFU: Add support for panko service upgrade"  https://review.openstack.org/55795415:59
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Remove nat/masquerading config workarounds.  https://review.openstack.org/55785115:59
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud  https://review.openstack.org/54255616:00
EmilienMcgoncalves: this code is weird, I just found out how it was done16:01
EmilienMcgoncalves: I'm not sure what to suggest, tbh16:02
*** tesseract has quit IRC16:02
*** itlinux has quit IRC16:03
*** gkadam_ has joined #tripleo16:03
*** chkumar246 has joined #tripleo16:04
*** chandankumar has quit IRC16:04
cgoncalvesEmilienM: FYI how horizon plugins enablement is done in kolla-ansible: https://review.openstack.org/#/c/557940/16:05
*** gkadam__ has joined #tripleo16:05
EmilienMcgoncalves: cool but how in tripleo we conditionally enable the plugins if the service is actually deployed?16:06
EmilienMthats my concern16:06
cgoncalvesEmilienM: right, we share the same concern lol16:06
chemweshay_mod: by the way we need that to have a chance to get better upgrade https://review.openstack.org/#/c/557035/16:07
*** gkadam has quit IRC16:07
EmilienMcgoncalves: let me look at the code, I have an idea16:07
weshay_modchem, k16:07
* weshay_mod looks16:07
chemweshay_mod: currently they fail super early because rdo docker registry is != from upstream16:07
chemweshay_mod: thanks16:08
*** chkumar246 has quit IRC16:09
*** gkadam_ has quit IRC16:09
*** marios has quit IRC16:09
weshay_modchem, so I think sagi is referring to https://review.openstack.org/#/c/557444/16:09
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755616:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]16:10
chemweshay_mod: well ... hard to tell from his comment :)16:10
EmilienMcgoncalves: going to try something with service_config_settings16:10
weshay_modya.. he should have specified16:10
weshay_modchem, his patch is cleaner though16:11
weshay_modchem, so.. my thought is this...16:11
weshay_modany upgrade job that is not gated in tqe/tu and has a promotion job that is currently working will break in 3-4 days16:12
weshay_modthe ci team's focus should be pushing as much as possible for those checks, gates, and promotion jobs16:13
weshay_modand *hopefully* deliver them at the end of this sprint16:13
weshay_modif not we continue16:13
weshay_modon the same topic16:13
weshay_moduntil it's done16:13
weshay_modthen I'll worry about fixing any given breakage in the upgrade jobs16:13
weshay_modbut we're atm trying to patch something that is not defended properly..16:14
weshay_modand that is very hard16:14
weshay_moddoes that make sense?16:14
EmilienMcgoncalves: what is weird is the yes/no thing I wish it could be a boolean16:15
EmilienMcgoncalves: I could easily fix the problem16:15
cgoncalvesEmilienM: please, be my guest :)16:18
weshay_modchem, if you don't think so.. just holla16:19
EmilienMcgoncalves: I'll investigate later, but this isn't an easy one16:19
chemweshay_mod: oki, but that shouldn't prevent actual fix to be merged though16:19
weshay_modchem, sure.. does sagi's patch not fix the issue?16:20
*** chkumar246 has joined #tripleo16:20
*** ffiore has joined #tripleo16:20
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Introduce tripleo-ci-centos-7-containerized-undercloud-upgrades  https://review.openstack.org/55363316:20
chemweshay_mod: that's what I was trying to understand from the log and the patch16:20
cgoncalvesEmilienM: cool, much appreciated. please add me as reviewer16:21
*** agopi is now known as agopi|brb16:26
chemweshay_mod: well it doesn't seem to work properly yet "docker pull failed: Error: image tripleopike/centos-binary-neutron-openvswitch-agent:fc8ded09c37a2e258dac7f2103b5b9406be5502e_758ddb1e not found"16:28
weshay_modchem, ya.. I think for it to work the containers need to be pushed again16:28
weshay_modlet's line up the naming convention vs.. adding debt on top of it16:29
chemweshay_mod: is all the work around the docker registry changing has been done ?16:29
chemweshay_mod: there is nothing that point to that (the depends on is unrelated it seems)16:29
weshay_modchem, hrm.. you are referring to which work specifically?16:29
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: Fix tag for rolling upgrade compte task  https://review.openstack.org/55788016:30
chemweshay_mod: the one sagi is talking about in there https://review.openstack.org/#/c/557035/3/toci-quickstart/config/testenv/multinode-rdocloud.yml16:30
chemweshay_mod: " hope to resolve it by creating the same namespace in trunk.registry.rdoproject.org,"16:30
weshay_modchem, right16:31
weshay_modit's just a change to the commands we use to push/update_tags in the registries16:31
weshay_modlet's just get that prioritized and have the names lined up properly between the two registries16:32
chemweshay_mod: so back to square 1 :)16:33
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: Fix tag for rolling upgrade compute task  https://review.openstack.org/55788016:33
weshay_modchem, if that is where it needs to be fixed.. than ya16:33
weshay_modthat is fairly trivial16:33
chemweshay_mod: while this get prioritized and merged and test and done could we merged https://review.openstack.org/#/c/557035/316:33
weshay_modafaict16:33
chemweshay_mod: when the other stuff is done this will make the jobs fail and then we push sagi's patch16:34
weshay_modchem, will you personally help review and fix the root cause?16:34
chemweshay_mod: well it will be easy, it will fail in 40min instead of 1h4016:35
chemweshay_mod: and for the moment ... I think we have help debug all upgrade failure, no ?16:35
weshay_modya..16:35
weshay_modwe do16:35
chemweshay_mod: or we can just sync :)16:35
weshay_modsync the names?16:36
chemweshay_mod: when it's going to be done we know we have to adjust the namespace16:36
chemweshay_mod: sync the changes16:36
chemweshay_mod: my point is that this fix/tech-debt in Ronelle patch cannot survive the change in infra16:37
weshay_modright16:37
* weshay_mod checks something16:38
*** suuuper has quit IRC16:38
chemweshay_mod: and for the moment we (in upgrade squad) are bearing the cost looking at non-failure, making depends-on all over the place and letting a lot of patches through that just break us.16:38
weshay_modchem, this patch has very little to do w/ that issue16:39
weshay_modbut that is exactly what I'm hoping to fix asap16:40
chemweshay_mod: which patch ?16:40
weshay_modthe cost of looking at broken glass16:40
weshay_modbroken jobs..etc16:40
weshay_modchaising tails16:40
chemweshay_mod: well it fixes an issue :)16:41
weshay_modbut not the root cause16:42
weshay_modwe need to chip away at the root cause, not the symptoms16:42
chemweshay_mod: oki, but it's nice to help the patient while the real cure is coming16:44
weshay_modI understand.. yes.. to use your metaphor. It's ok to give the patient a tranfusion of blood as long as long as there are people sowing up the wounds.  If nobody is sowing up the wounds, transfusions of blood are a wast of time16:46
*** ffiore has quit IRC16:46
*** ffiore has joined #tripleo16:48
chemweshay_mod: not sure I get the "nobody is showing up", but back to work :)16:50
chemweshay_mod: I may hove spoted why sagi patch is not working during upgrade anyway, I'll comment there and meanwhile will continue to use crazy depends-on all over the place16:50
weshay_modchem, that's not what I'm saying16:50
weshay_modchem, people are working hard and showing up16:50
weshay_modchem, but w/o the check/gate jobs and promotion jobs in place16:51
weshay_modwe will spin wheels on this16:51
*** ffiore has quit IRC16:53
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: remove dstat from the virthost  https://review.openstack.org/55796716:53
chemweshay_mod: oki, please let us know how we can help to speed thing up, at least add some people from upgrade squad to the relevent review (or ping us)16:59
weshay_modchem, ya.. and for what it's worth I of very mixed mind on this.. and if others think I'm saying the wrong thing it's just a -117:00
weshay_modif others want the patch to land.. I won't fight it17:01
chemweshay_mod: np, I've got your point, like "we're bumping into a lot of legacy stuff, so let's fix them along the way" it's the sane thing to do17:01
chemweshay_mod: we've got our fair share of -2 for the week :)17:02
weshay_modk.. there are so many things to do and so many fires it's important to me to focus on the meaningful ones17:02
*** gkadam__ has quit IRC17:03
*** itlinux has joined #tripleo17:06
*** psahoo has quit IRC17:07
*** yamahata has joined #tripleo17:10
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755617:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]17:10
chemweshay_mod: do you have a timeframe for https://review.rdoproject.org/r/#/c/13128/ (this looks like a major change in infra)17:12
weshay_modchem, afaict.. that just needs review.. and then we'd need a promotion on each branch we're working with17:13
weshay_modchem, or we could manually repush17:14
weshay_modchem, added to https://trello.com/c/Ble66eRY/431-list-of-upstream-reviews17:16
weshay_modwhich is reviewed at every stand up17:16
chemweshay_mod: ack, thanks.17:16
*** gkadam__ has joined #tripleo17:18
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: undercloud: load masquerade-networks service  https://review.openstack.org/55797117:24
*** srini_ has quit IRC17:24
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud  https://review.openstack.org/54255617:28
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud  https://review.openstack.org/54255617:28
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001/035: enable MasqueradeNetworks service  https://review.openstack.org/55795017:28
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001: enable masquerading via THT  https://review.openstack.org/55797217:35
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Revert "FFU: Add support for panko service upgrade"  https://review.openstack.org/55795317:35
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001: enable masquerading via THT  https://review.openstack.org/55797217:36
*** dhill_ has quit IRC17:38
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Remove nat/masquerading config workarounds.  https://review.openstack.org/55785117:39
*** itlinux has quit IRC17:41
*** tosky__ is now known as tosky17:43
*** fragatina has quit IRC17:49
weshay_modEmilienM, rfolco do you know if pike should have ovn configured?17:52
weshay_modhttps://logs.rdoproject.org/openstack-periodic-24hr/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike/d9e1506/overcloud-controller-0/var/log/config-data/neutron/etc/neutron/plugins/ml2/ml2_conf.ini.txt.gz17:52
*** chem is now known as chem|eow17:53
weshay_modEmilienM, nvrmind17:55
*** EmilienM has quit IRC17:58
weshay_modrfolco, https://trunk.rdoproject.org/centos7-ocata/18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755618:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]18:10
*** fragatina has joined #tripleo18:10
weshay_modrfolco, https://review.openstack.org/#/q/Ic13901b92cac05e8ddf1092b66aa5bcc5623fa8a18:14
weshay_modbah.18:14
weshay_modnvrmind18:14
weshay_modrfolco, https://logs.rdoproject.org/openstack-periodic-24hr/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike/d9e1506/undercloud/home/jenkins/overcloud_validate.log.txt.gz#_2018-03-27_06_44_5518:15
weshay_modhttps://logs.rdoproject.org/openstack-periodic-24hr/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike/d9e1506/overcloud-novacompute-0/var/log/extra/errors.txt.gz#_2018-03-27_06_42_50_63018:16
*** gfidente has quit IRC18:28
*** itlinux has joined #tripleo18:31
openstackgerritHarald JensÃ¥s proposed openstack/python-tripleoclient master: undercloud: load masquerade-networks service  https://review.openstack.org/55797118:35
*** sai_p has joined #tripleo18:36
openstackgerritCarlos Goncalves proposed openstack/tripleo-heat-templates master: Enable Octavia dashboard on horizon container  https://review.openstack.org/55798118:37
*** ykarel|away has quit IRC18:42
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: undercloud: load masquerade-networks service  https://review.openstack.org/55797119:06
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755619:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]19:10
*** ooolpbot has quit IRC19:10
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: undercloud: load masquerade-networks service  https://review.openstack.org/55797119:15
*** itlinux has quit IRC19:17
*** Stickersvg has joined #tripleo19:25
Stickersvg.-.            .-.19:25
Stickersvg.-.            .-.19:25
Stickersvg.-.            .-.19:25
Stickersvg.-.            .-.19:25
Stickersvg/   \          /   \19:25
Stickersvg/   \          /   \19:25
Stickersvg/   \          /   \19:25
Stickersvg/   \          /   \19:25
Stickersvg|   _ \        / _   |19:25
Stickersvg|   _ \        / _   |19:25
Stickersvg|   _ \        / _   |19:25
Stickersvg|   _ \        / _   |19:25
Stickersvg;  | \ \      / / |  ;19:25
Stickersvg;  | \ \      / / |  ;19:25
Stickersvg;  | \ \      / / |  ;19:25
Stickersvg;  | \ \      / / |  ;19:25
Stickersvg\  \ \ \_.._/ / /  /19:25
Stickersvg\  \ \ \_.._/ / /  /19:25
Stickersvg\  \ \ \_.._/ / /  /19:25
Stickersvg\  \ \ \_.._/ / /  /19:25
Stickersvg'. '.;'    ';,' .'19:25
Stickersvg'. '.;'    ';,' .'19:25
Stickersvg'. '.;'    ';,' .'19:25
Stickersvg'. '.;'    ';,' .'19:25
Stickersvg'./ _    _ \.'19:25
Stickersvg'./ _    _ \.'19:25
Stickersvg'./ _    _ \.'19:25
Stickersvg'./ _    _ \.'19:25
Stickersvg.'  a __ a  '.19:25
Stickersvg.'  a __ a  '.19:25
Stickersvg.'  a __ a  '.19:25
Stickersvg.'  a __ a  '.19:25
Stickersvg'--./ _,   \/   ,_ \.--'19:25
Stickersvg'--./ _,   \/   ,_ \.--'19:25
Stickersvg'--./ _,   \/   ,_ \.--'19:25
Stickersvg'--./ _,   \/   ,_ \.--'19:25
Stickersvg----|   \   /\   /   |----19:25
Stickersvg----|   \   /\   /   |----19:25
Stickersvg----|   \   /\   /   |----19:25
Stickersvg----|   \   /\   /   |----19:25
Stickersvg.--'\   '-'  '-'    /'--.19:25
Stickersvg.--'\   '-'  '-'    /'--.19:25
Stickersvg.--'\   '-'  '-'    /'--.19:25
Stickersvg.--'\   '-'  '-'    /'--.19:25
Stickersvg_>.__  -- _.-  `;19:25
Stickersvg_>.__  -- _.-  `;19:25
Stickersvg_>.__  -- _.-  `;19:25
Stickersvg_>.__  -- _.-  `;19:25
Stickersvg.' _     __/     _/19:25
Stickersvg.' _     __/     _/19:25
Stickersvg.' _     __/     _/19:25
Stickersvg.' _     __/     _/19:25
Stickersvg/    '.,:".-\    /:,19:25
Stickersvg/    '.,:".-\    /:,19:25
Stickersvg/    '.,:".-\    /:,19:25
Stickersvg/    '.,:".-\    /:,19:25
Stickersvg|      \.'   `""`'.\\19:26
Stickersvg|      \.'   `""`'.\\19:26
Stickersvg|      \.'   `""`'.\\19:26
Stickersvg|      \.'   `""`'.\\19:26
Stickersvg'-,.__/  _   .-.  ;|_19:26
Stickersvg'-,.__/  _   .-.  ;|_19:26
Stickersvg'-,.__/  _   .-.  ;|_19:26
Stickersvg'-,.__/  _   .-.  ;|_19:26
Stickersvg/` `|| _/ `\/_  \_|| `\19:26
Stickersvg/` `|| _/ `\/_  \_|| `\19:26
Stickersvg/` `|| _/ `\/_  \_|| `\19:26
Stickersvg/` `|| _/ `\/_  \_|| `\19:26
Stickersvg|    ||/ \-./` \ / ||   |19:26
Stickersvg|    ||/ \-./` \ / ||   |19:26
Stickersvg|    ||/ \-./` \ / ||   |19:26
Stickersvg|    ||/ \-./` \ / ||   |19:26
Stickersvg\   ||__/__|___|__||  /19:26
Stickersvg\   ||__/__|___|__||  /19:26
Stickersvg\   ||__/__|___|__||  /19:26
Stickersvg\   ||__/__|___|__||  /19:26
Stickersvg\_ |_Happy Easter_| /19:26
Stickersvg\_ |_Happy Easter_| /19:26
Stickersvg\_ |_Happy Easter_| /19:26
Stickersvg\_ |_Happy Easter_| /19:26
Stickersvgjgs .'  \ =  _= _ = _= /`\19:26
Stickersvgjgs .'  \ =  _= _ = _= /`\19:26
Stickersvgjgs .'  \ =  _= _ = _= /`\19:26
Stickersvgjgs .'  \ =  _= _ = _= /`\19:26
Stickersvg/     `-;----=--;--'   \19:26
Stickersvg/     `-;----=--;--'   \19:26
Stickersvg/     `-;----=--;--'   \19:26
Stickersvg/     `-;----=--;--'   \19:26
*** jcoufal has quit IRC19:26
Stickersvg\    _.-'        '.    /19:26
Stickersvg\    _.-'        '.    /19:26
Stickersvg\    _.-'        '.    /19:26
Stickersvg\    _.-'        '.    /19:26
Stickersvg`""`              `""`19:26
Stickersvg`""`              `""`19:26
Stickersvg`""`              `""`19:26
Stickersvg`""`              `""`19:26
StickersvgL0DE AND CHRON FROM #LRH & L0DE RADIO HOUR (IRC.EFNET.ORG) WANTED TO SAY HAPPY EASTER!!19:26
StickersvgL0DE AND CHRON FROM #LRH & L0DE RADIO HOUR (IRC.EFNET.ORG) WANTED TO SAY HAPPY EASTER!!19:26
StickersvgL0DE AND CHRON FROM #LRH & L0DE RADIO HOUR (IRC.EFNET.ORG) WANTED TO SAY HAPPY EASTER!!19:26
StickersvgL0DE AND CHRON FROM #LRH & L0DE RADIO HOUR (IRC.EFNET.ORG) WANTED TO SAY HAPPY EASTER!!19:26
Stickersvgs a i _ p   f r a g a t i n a   g k a d a m _ _   y a m a h a t a   c h k u m a r 2 4 6   e t i n g o f   a g o p i | b r b   c g o n c a l v e s   a y o u n g   t o s k y   m m e t h o t   j c o u f a l   r f o l c o   b f o u r n i e   c h e m | e o w   m a t b u   a s b i s h o p   d c i a b r i n   d t r a i n o r   d b e c k e r   z z z e e k   f p a n   c s m a r t   m y o u n g   w e s h a y _ m o d   b a n d i n i   h a m z y   t o u r e | b i19:26
Stickersvgsai_p fragatina gkadam__ yamahata chkumar246 etingof agopi|brb cgoncalves ayoung tosky mmethot jcoufal rfolco bfournie chem|eow matbu asbishop dciabrin dtrainor dbecker zzzeek fpan csmart myoung weshay_mod bandini hamzy toure|biab rickflare sdake dsneddon jjoyce tdasilva yolanda social atoth homegrown ericyoung arxcruz|rover jokke_ michchap noslzzp rfolco|ruck migi kambiz owalsh slagle thrash|g0ne gunix pabelanger SlickNik ukalifon jlabarre tbonds Sl0w19:26
*** Stickersvg has quit IRC19:26
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: undercloud: allow to configure ctlplane_masquerade in undercloud.conf  https://review.openstack.org/55799119:28
*** agopi|brb is now known as agopi19:30
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001: enable masquerading via THT  https://review.openstack.org/55797219:30
*** gkadam__ has quit IRC19:31
*** eggmaster has quit IRC19:40
*** tzumainn has joined #tripleo19:41
*** fragatina has quit IRC19:43
*** itlinux has joined #tripleo19:55
*** emilien has joined #tripleo19:59
emilienhey19:59
*** emilien is now known as EmilienM19:59
*** EmilienM has quit IRC20:02
*** EmilienM has joined #tripleo20:02
*** ChanServ sets mode: +v EmilienM20:02
EmilienMweshay_mod: back20:02
*** itlinux has quit IRC20:02
weshay_modnice20:03
EmilienMweshay_mod: I need help on this one: http://logs.openstack.org/33/553633/16/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/ccb9155/job-output.txt.gz#_2018-03-30_17_10_35_76670420:08
EmilienMweshay_mod: I updated my patches for containerized undercloud upgrade20:09
EmilienMweshay_mod: and now it runs overcloud validate, it's maybe a question of tags or something20:09
EmilienMbut we shouldn't run these tasks20:09
EmilienM"validate the overcloud" shouldn't be executed20:09
weshay_modright.. they should be turn off in the env20:09
weshay_modsec20:09
* weshay_mod looks20:09
EmilienMthx20:09
weshay_modvalidate is overloaded20:09
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755620:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]20:10
EmilienMhjensas: the maquerade-networks iptables rules aren't working20:13
EmilienMhttps://logs.rdoproject.org/72/557972/3/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Zd136a14ca48047be87c75fb408366aae/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2018-03-30_19_54_0420:13
EmilienMlet me pull the hieradata20:13
EmilienMhttps://logs.rdoproject.org/72/557972/3/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Zd136a14ca48047be87c75fb408366aae/undercloud/home/jenkins/tripleo-dBylha-config/Undercloud/merged_config_settings.yaml.txt.gz20:14
EmilienMtripleo.masquerade_networks.firewall_rules:20:14
EmilienM- "'137 routed_network return_0':\n  table: 'nat'\n  source: '192.168.24.0/24'\n \20:14
EmilienM  \ destination:\n     - 192.168.24.0/24\n  jump: 'RETURN'\n'138 routed_network masquerade_0':\n\20:14
EmilienM  \  table: 'nat'\n  source: '192.168.24.0/24'\n  jump: 'MASQUERADE'\n"20:14
weshay_modEmilienM, https://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/multinode-validate.yml line 1020:14
weshay_modjust set test_ping = false20:14
weshay_modin the fs20:15
EmilienMweshay_mod: well I used the same as regular undercloud -upgrade20:15
EmilienMlet me check20:15
EmilienMweshay_mod: see config/general_config/featureset047.yml, I took the same20:15
EmilienMweshay_mod: config/general_config/featureset047.yml is undercloud-upgrades -the existing one that works-20:16
EmilienMweshay_mod: ah, I missed https://review.openstack.org/#/c/553850/20:17
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Remove unnecessary parameters from featureset047.  https://review.openstack.org/55385020:17
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Add fs050: containerized undercloud upgrade  https://review.openstack.org/55794120:18
hjensasEmilienM: -- I'll be back after live shader qulification are done.  https://www.twitch.tv/revisionparty20:18
*** hjensas is now known as hjensas|away20:19
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Ignore empty values for dlrn hashes  https://review.openstack.org/55488220:19
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Remove unnecessary parameters from featureset047.  https://review.openstack.org/55385020:19
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Add fs050: containerized undercloud upgrade  https://review.openstack.org/55794120:19
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Add fs050: containerized undercloud upgrade  https://review.openstack.org/55794120:20
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Run playbooks with custom args  https://review.openstack.org/55347420:20
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Use tripleo-upgrade role in undercloud upgrades job.  https://review.openstack.org/54897420:20
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Introduce tripleo-ci-centos-7-containerized-undercloud-upgrades  https://review.openstack.org/55363320:20
EmilienMweshay_mod: ok, problem solved20:20
EmilienMweshay_mod: thx20:20
EmilienMhjensas|away: ok no prob, we'll probably catchup on monday then20:24
openstackgerritMerged openstack/tripleo-quickstart-extras master: Collect dstat on overcloud nodes  https://review.openstack.org/55745520:30
openstackgerritMerged openstack/instack-undercloud stable/queens: Fix help string for subnets option  https://review.openstack.org/55752120:30
openstackgerritMerged openstack/tripleo-quickstart master: move virtbmc var to common  https://review.openstack.org/55794420:30
EmilienMhjensas|away: https://bugs.launchpad.net/tripleo/+bug/176021120:34
openstackLaunchpad bug 1760211 in tripleo "masquerading rules format isn't correct for hiera" [Medium,Triaged]20:34
*** hjensas|away is now known as hjensas20:39
hjensasEmilienM: So, we are passing in a list, while it should be a hash to the puppet module?20:43
sai_pIs it possible to add "if" condition in "step_config" to include puppet classes. If so can you give me the syntax for it?20:44
sai_pThank you20:44
*** waleedm has joined #tripleo20:44
EmilienMhjensas: let me check20:57
hjensasEmilienM: hmm, this: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/masquerade-networks.yaml#L45-L66 - it iterates and that generates a list.20:57
EmilienMhjensas: we should generate an hash20:57
EmilienMI updated the bug report20:58
EmilienMhjensas: I wonder if for_each generates a list20:59
hjensasEmilienM: yes, I guess the repeat does array.append() not hash.update() ...20:59
EmilienMThe for_each argument is a dictionary that defines how to generate the repetitions of the template and perform substitutions. In this dictionary the keys are the placeholder names that will be replaced in the template, and the values are the lists to iterate on.21:00
EmilienMfrom https://docs.openstack.org/heat/latest/template_guide/hot_spec.html21:00
*** avivgt has joined #tripleo21:03
*** chandankumar has joined #tripleo21:03
*** chkumar246 has quit IRC21:04
hjensasEmilienM: The result of this function is a new list, where the elements are set to the template, rendered for each list item.21:04
EmilienMhjensas: correct21:06
hjensasEmilienM: wonder if we can do something in puppet to make it support a list? https://github.com/openstack/puppet-tripleo/blob/master/manifests/firewall/service_rules.pp#L3821:06
EmilienMgood question21:07
EmilienMlet me dig21:07
*** chandankumar has quit IRC21:08
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755621:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]21:10
EmilienMhjensas: maybe something with https://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/functions/hash.rb but i'm not even sure21:11
hjensasEmilienM: or we just add a map_merge in the service?21:11
hjensashttps://docs.openstack.org/heat/latest/template_guide/hot_spec.html#map-merge21:11
EmilienMyeah21:11
EmilienMI wonder how the map merge will interpret an array21:12
EmilienMif it returns an hash with one array21:12
EmilienMlet me check heat code21:12
*** rfolco|ruck is now known as rfolco|off21:13
EmilienMhjensas: mhh, we can try21:14
EmilienMhjensas: see the output21:14
hjensasEmilienM: I'll have a patch in a minute.21:14
EmilienMok21:14
EmilienMI'll need to disappear soon, taking the road for the weekend21:15
*** avivgt has quit IRC21:17
openstackgerritHarald JensÃ¥s proposed openstack/tripleo-heat-templates master: Add map_merge to masqurade network service  https://review.openstack.org/55800021:18
hjensasEmilienM: ack, I'm out on PTO next week. Hopefully this works.21:18
EmilienMok21:19
EmilienMhjensas: i'm going to recheck my patches to actually test your patch21:19
hjensasEmilienM: which one should I look at to see results?21:20
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001: enable masquerading via THT  https://review.openstack.org/55797221:20
*** yamahata has quit IRC21:20
hjensasoh, that one. :)21:20
EmilienMhjensas: ^ this one21:20
EmilienMenjoy your PTO :)21:20
*** yamahata has joined #tripleo21:20
*** chandankumar has joined #tripleo21:21
*** itlinux has joined #tripleo21:26
*** itlinux has quit IRC21:27
*** rfolco has quit IRC21:29
*** rfolco|off has quit IRC21:29
*** paramite has joined #tripleo21:33
*** jlabarre has quit IRC21:35
*** social has quit IRC21:37
*** paramite has quit IRC21:41
*** rfolco has joined #tripleo21:57
*** rfolco has quit IRC21:57
*** waleedm has quit IRC22:00
*** waleedm has joined #tripleo22:00
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755622:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]22:10
*** rfolco|off has joined #tripleo22:14
*** fragatina has joined #tripleo22:15
*** fragatina has quit IRC22:15
*** fragatina has joined #tripleo22:16
*** dsariel has joined #tripleo22:26
*** ledo_ has joined #tripleo23:02
*** waleedm has quit IRC23:05
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/175755623:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged]23:10
*** tosky has quit IRC23:30
*** mmethot has quit IRC23:31
*** zshi has joined #tripleo23:47
*** orange_julius has joined #tripleo23:49
orange_juliusWe've been having some trouble creating custom VLANs using tripleo. Does anyone have a working example of how to use bonds+vlans in tripleo pike? We can get the bond to create correctly, however our VLAN configurations are ignored and the deployment appears to be using the deployment IP subnet for all interfaces / vlans23:49
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260723:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!