Monday, 2018-08-27

*** lblanchard has joined #tripleo00:08
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676400:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791000:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890700:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)00:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)00:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)00:10
*** lblanchard has quit IRC00:19
openstackgerritMerged openstack/python-tripleoclient master: undercloud: support noop management interface for snmp and ipmi  https://review.openstack.org/58913401:05
*** gbarros has joined #tripleo01:07
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676401:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791001:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890701:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)01:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)01:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)01:10
*** tristanC has joined #tripleo02:08
*** ramishra has joined #tripleo02:09
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676402:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791002:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890702:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)02:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)02:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)02:10
*** gbarros has quit IRC02:15
*** mmethot has quit IRC02:20
*** adrianreza has joined #tripleo02:30
ramishrastevebaker: Hi around?02:39
*** psachin has joined #tripleo02:47
*** skramaja has joined #tripleo02:51
stevebakerramishra: hey!03:00
*** ramishra has quit IRC03:00
*** ramishra has joined #tripleo03:08
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP POC export image data to crane format  https://review.openstack.org/59601703:09
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676403:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791003:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890703:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)03:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)03:10
*** dxiri has joined #tripleo03:11
*** udesale has joined #tripleo03:27
*** dmacpher has joined #tripleo03:29
*** gkadam has joined #tripleo03:41
*** dxiri has quit IRC03:50
*** Petersingh has joined #tripleo03:53
*** dxiri has joined #tripleo03:54
*** Petersingh is now known as Petersingh|afk03:54
*** chem has quit IRC04:05
*** dxiri has quit IRC04:05
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676404:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791004:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890704:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)04:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)04:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)04:10
*** dtrainor has quit IRC04:22
*** jaganathan has joined #tripleo04:28
*** Petersingh|afk is now known as Petersingh04:37
*** dxiri has joined #tripleo04:46
Tenguhello there :)04:50
*** ykarel has joined #tripleo04:54
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676405:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791005:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890705:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)05:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)05:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)05:10
*** numans has joined #tripleo05:29
*** dxiri has quit IRC05:30
*** dxiri has joined #tripleo05:31
*** Petersingh is now known as Petersingh|afk05:31
*** honza has joined #tripleo05:37
*** dxiri has quit IRC05:42
*** chem has joined #tripleo05:47
*** Petersingh|afk is now known as Petersingh05:50
*** jfrancoa has joined #tripleo05:50
*** aufi has joined #tripleo05:57
*** yprokule has joined #tripleo05:59
*** radeks__ has joined #tripleo06:00
*** agurenko has joined #tripleo06:01
*** tbonds has quit IRC06:01
*** tbonds has joined #tripleo06:03
*** janki has joined #tripleo06:06
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676406:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791006:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890706:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)06:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)06:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)06:10
*** jtomasek has joined #tripleo06:17
openstackgerritMerged openstack/tripleo-ui stable/queens: Add general error message to NodesAssignmentForm  https://review.openstack.org/59552806:35
*** chandankumar is now known as chkumar|pto06:35
*** apetrich has joined #tripleo06:38
*** rdopiera has joined #tripleo06:45
*** kopecmartin has joined #tripleo06:48
*** pcaruana has joined #tripleo06:49
*** ccamacho has joined #tripleo06:50
*** links has joined #tripleo06:59
*** dmellado has quit IRC07:00
*** saneax has joined #tripleo07:01
*** dmellado has joined #tripleo07:02
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676407:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791007:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890707:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)07:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)07:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)07:10
*** Petersingh is now known as Petersingh|afk07:14
*** ksambor has joined #tripleo07:15
*** Petersingh|afk is now known as Petersingh07:16
*** skramaja has quit IRC07:18
*** jpich has joined #tripleo07:19
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Keep plan on update and store user env_files into swift  https://review.openstack.org/58314507:21
*** dtantsur|afk is now known as dtantsur07:26
*** slaweq has quit IRC07:30
*** slaweq has joined #tripleo07:31
*** rcernin has quit IRC07:31
*** tosky has joined #tripleo07:36
*** shadower has joined #tripleo07:41
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade stable/pike: Don't invoke multiple migration requests for single vm.  https://review.openstack.org/58847907:42
*** Petersingh is now known as Petersingh|lunch07:45
*** jpena|off is now known as jpena07:45
ksamborze nikt nie lubi*07:46
*** d0ugal has joined #tripleo07:54
*** dmacpher_ has joined #tripleo07:54
*** florianf has joined #tripleo07:56
*** dmacpher has quit IRC07:56
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Set container_images_file for containerized undercloud.  https://review.openstack.org/58428607:58
*** ykarel is now known as ykarel|lunch07:59
*** dolly has quit IRC08:04
*** skramaja has joined #tripleo08:08
openstackgerritMartin AndrĂ© proposed openstack/tripleo-quickstart master: Revert "Revert "Move to openshift-ansible 3.10""  https://review.openstack.org/58946908:08
*** dbecker has joined #tripleo08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676408:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791008:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890708:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)08:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)08:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)08:10
*** jtomasek has quit IRC08:15
*** jtomasek has joined #tripleo08:16
*** Petersingh|lunch is now known as Petersingh08:34
*** flaper87 has joined #tripleo08:38
*** ykarel|lunch is now known as ykarel08:41
*** radeks has joined #tripleo08:41
*** radeks__ has quit IRC08:43
*** radeks_ has joined #tripleo08:44
*** radeks has quit IRC08:47
*** yprokule has quit IRC08:50
*** skramaja has quit IRC08:54
*** skramaja has joined #tripleo08:56
*** skramaja has quit IRC08:57
*** skramaja has joined #tripleo08:58
*** owalsh_ is now known as owalsh09:04
*** holser_ has joined #tripleo09:07
*** skramaja has quit IRC09:09
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676409:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791009:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890709:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)09:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)09:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)09:10
*** paramite has joined #tripleo09:10
*** sri_ has joined #tripleo09:12
*** aufi has quit IRC09:29
openstackgerritCarlos Camacho proposed openstack/python-tripleoclient master: Updating python-tripleoclient requires to upgrade the containers images repo  https://review.openstack.org/59667809:32
*** dmacpher_ has quit IRC09:34
ccamachoHey EmilienM09:44
ccamachoquick question to fix this BZ https://bugzilla.redhat.com/show_bug.cgi?id=160742109:44
ccamachowhen we update the python-tripleoclient package openstack-tripleo-common-containers is not updated09:44
ccamachoSo, so you think this missing bits on the tripleo-common version might be the issue?09:44
ccamachohttps://review.openstack.org/#/c/596678/09:44
ccamachohttps://review.rdoproject.org/r/#/c/15936/09:44
openstackbugzilla.redhat.com bug 1607421 in python-tripleoclient "[UPGRADES][14] Multiple services ain't picked up during image generation" [High,On_dev] - Assigned to yprokule09:44
*** dtantsur is now known as dtantsur|brb09:44
*** lennyb has joined #tripleo09:47
honzai'm getting this error using reproducer, any ideas?  Could not find or access '/tmp/.quickstart/playbooks/overcloud-validate-ha.yml'09:49
honza(installing undercloud on ovb)09:49
sshnaidmd0ugal, apetrich hi, guys, I have problem with my patch again :) After fixing d0ugals issues it started to work (thanks!), but 2 days later stopped.. I didn't find something related in changes, so maybe you can look in your time please: https://review.openstack.org/#/c/58379910:01
sshnaidmd0ugal, apetrich maybe you know how to debug such issues, because no idea even where it could be broken..10:06
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676410:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791010:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890710:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)10:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)10:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)10:10
*** dolly has joined #tripleo10:13
cgoncalvesstevebaker, hi. re: "Property DockerCinderConfigImage not assigned", still blocked by RDO promotion?10:14
openstackgerritCarlos Camacho proposed openstack/python-tripleoclient master: Updating python-tripleoclient requires to upgrade the containers images repo  https://review.openstack.org/59667810:15
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations stable/queens: Revert "Fail if the Overcloud inventory is empty"  https://review.openstack.org/59668410:23
*** jaosorior has joined #tripleo10:25
*** pcaruana has quit IRC10:32
*** pcaruana has joined #tripleo10:32
*** dtantsur|brb is now known as dtantsur10:41
*** slagle has quit IRC10:42
*** slagle has joined #tripleo10:44
*** Petersingh is now known as Petersingh|afk10:45
*** lvdombrkr has joined #tripleo10:51
lvdombrkrhello folks10:51
lvdombrkris the tenant network and tunneling network are the same?10:51
*** slagle has quit IRC10:55
EmilienMccamacho: yeah you got it11:03
*** apetrich has quit IRC11:04
*** moguimar has joined #tripleo11:06
*** Petersingh|afk is now known as Petersingh11:07
*** two_oes has joined #tripleo11:08
openstackgerritSorin Sbarnea proposed openstack/tripleo-upgrade master: limit duration of setup HEAT outputs via update prepare task  https://review.openstack.org/59638111:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676411:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791011:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890711:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)11:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)11:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)11:10
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Assure copied image is owned by current user  https://review.openstack.org/59642811:12
*** apetrich has joined #tripleo11:13
apetrichsshnaidm, d0ugal is in pto today. I will look at it in a minute11:15
two_oesHi all,  I am trying to deploy split controller deployment and I am Hitting a MySQL connection failed. when I deploy it without the split everything is working find11:15
two_oeshas anybody encounter that ?11:16
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Make sure OVB reproducer has all requirements  https://review.openstack.org/59668911:19
sshnaidmhonza, ykarel ^^11:19
sshnaidmapetrich, thanks!11:19
ykarelsshnaidm, ack11:19
*** jpena is now known as jpena|lunch11:22
ykareltwo_oes, u mean split stack deployment: https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/split_stack.html? if yes i remember once pdeore tried it and faced some issue, the far i checked it's broken after config-download introduced11:23
*** gkadam has quit IRC11:23
*** gkadam has joined #tripleo11:23
ykareltwo_oes, and it worked with queens release11:23
EmilienMmandre: https://review.openstack.org/#/c/596536/ if you're fine with it11:23
*** rh-jelabarre has joined #tripleo11:25
*** Petersingh is now known as Petersingh|afk11:25
*** rdopiera has quit IRC11:28
*** lblanchard has joined #tripleo11:37
*** udesale has quit IRC11:38
*** ykarel is now known as ykarel|away11:42
*** two_oes has quit IRC11:43
*** two_oes has joined #tripleo11:43
*** jroll has quit IRC11:43
*** aufi has joined #tripleo11:43
*** jroll has joined #tripleo11:44
*** ykarel|away has quit IRC11:46
two_oesykarel, no I am working with Red Hat Director for OpenStack 1311:47
two_oesykarel, this what I am trying to deploy : https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/advanced_overcloud_customization/roles#Arch-Split_Controller11:48
*** ksambor_ has joined #tripleo11:57
*** morazi has joined #tripleo11:57
*** agurenko has quit IRC11:57
*** ksambor has quit IRC11:57
*** raildo has joined #tripleo11:59
*** agurenko has joined #tripleo12:02
*** yprokule has joined #tripleo12:04
*** dtrainor has joined #tripleo12:05
apetrichsshnaidm, I think I've found it12:07
sshnaidmapetrich, what is it?12:07
apetrichsshnaidm, typo.12:07
apetrich:)12:07
sshnaidmapetrich, O_O12:07
*** Petersingh|afk is now known as Petersingh12:09
openstackgerritSagi Shnaidman proposed openstack/python-tripleoclient master: Support ARA report tracking from command line  https://review.openstack.org/58379912:10
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676412:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791012:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890712:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)12:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)12:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)12:10
sshnaidmapetrich, thanks!!12:10
dtantsurfolks, how do I fix Message: Minimum required for /: 25G - volume free space: 22.0, assuming that it's a re-installation and it hopefully won't need 25G anew?12:11
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Collect overcloud statistics with ARA  https://review.openstack.org/57846212:11
dtantsurotherwise we need to bump quickstart undercloud defaults from 50G to, I don't know, TiB?12:11
sshnaidmapetrich, hope it is the error, no idea how it worked before :)12:12
openstackgerritwes hayutin proposed openstack/tripleo-heat-templates stable/rocky: Pass in rndc key to Designate deployment  https://review.openstack.org/59669612:12
openstackgerritwes hayutin proposed openstack/puppet-tripleo stable/rocky: add scenario003 tests when designate changes  https://review.openstack.org/59669712:13
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates stable/rocky: Pass in rndc key to Designate deployment  https://review.openstack.org/59669612:13
dtantsurhttps://github.com/openstack/tripleo-validations/commit/f911acd54b555579cfb71663cdabd6b582614aae is the wrong commit12:14
dtantsurthough no, it was there before it12:14
mandretrown|outtypewww: hi! did you get a chance to upload the etcd image to the dockerhub?12:14
dtantsuraha, florianf may be to blame: https://github.com/openstack/tripleo-validations/commit/0bc2b05fc5dcb07ba0bbe41636afc29bd172670a :)12:14
apetrichsshnaidm, me two. if it's not let me know and I'll have a harder look, but I don't see anything wrong otherwise12:16
florianfdtantsur: what did I do now? ;)12:16
*** thrash|g0ne is now known as thrash12:16
sshnaidmapetrich, a general question - can I see somewhere that these parameters were passed to playbooks? maybe it's saved somewhere?12:16
dtantsurflorianf: initially - the free disk space calculation that does not account for whether the undercloud is already installed12:16
sshnaidmapetrich, just curious how can I debug it and detect at least in which step it doesn't work..12:17
dtantsuralternatively, we can bump https://github.com/openstack/tripleo-quickstart/blob/5d1b169c593a32a7245120b0adf13e102565b4a6/roles/common/defaults/main.yml#L55 to at least 7512:17
dtantsurbut I dunno what oooq folks think about it12:17
florianfdtantsur: the validation may not even be run without the undercloud installed (at least for now).12:18
dtantsurflorianf: then I don't quite get how it works.. so, I have 50G. undercloud took 25G. why do I need 25 more free?12:18
florianfdtantsur: tbh, I'm not sure how the exact numbers were determined. too long ago12:19
apetrichsshnaidm, usually mistral action-execution-get-input does what I need but it does not show the playbooks. probably in the logs12:19
honzasshnaidm: awesome! i'll give it a go12:19
florianfdtantsur: maybe to make sure folks have a little room can add additional images or other space-consuming stuff? logs?12:19
dtantsurflorianf: a little room is 22G in my case :)12:20
* dtantsur rebuilds with 100G undercloud12:21
florianfdtantsur: better safe than sorry :)12:22
dtantsuryeah, maybe it makes sense :)12:23
dtantsurwhen you mess with ironic images a lot, the free space goes away quickly. now the same is with containers.12:23
*** jpena|lunch is now known as jpena12:24
numansEmilienM, Hi, can you please take a look at this patch - https://review.openstack.org/#/c/596594/ when you get time12:26
*** lblanchard has quit IRC12:26
EmilienMnumans: quickly commented12:28
EmilienMbut the patch isn't good for me12:28
Aeliacan somebody well versed in the ceph part of tripleo help me on this issue https://bugs.launchpad.net/tripleo/+bug/1786434 ?12:28
openstackLaunchpad bug 1786434 in tripleo "openstack overlay deploy failed in heat resource_type OS::Mistral::ExternalResource" [Medium,Triaged] - Assigned to CĂ©dric Jeanneret (cjeanner)12:28
*** rfolco has joined #tripleo12:28
*** rlandy has joined #tripleo12:30
Tengufultonj: -^^  maybe ?12:31
*** ykarel|away has joined #tripleo12:31
*** ykarel|away is now known as ykarel12:32
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient master: Bump the timeout for plan creation workflow to 20 minutes  https://review.openstack.org/59670012:35
*** Petersingh has quit IRC12:36
*** dprince has joined #tripleo12:36
*** ykarel_ has joined #tripleo12:36
*** jcoufal has joined #tripleo12:37
*** ykarel has quit IRC12:39
numansEmilienM, ack. environmetns/docker-ha.yaml enables the cinder service - OS::TripleO::Services::CinderVolume. so probably scenario007 should set it back to None then12:40
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Include minimal Browbeat playbook in baremetal playbook  https://review.openstack.org/58148812:40
EmilienMnumans: yes12:40
numansEmilienM, on to it. thanks for the review12:40
EmilienMnp12:41
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Include minimal Browbeat playbook in baremetal playbook  https://review.openstack.org/58148812:41
*** trown|outtypewww is now known as trown12:44
*** tzumainn has joined #tripleo12:46
*** ramishra has quit IRC12:46
trownmandre: no, but doing it right now12:47
*** pliu has quit IRC12:48
fultonjAelia Tengu /me looking12:48
Aeliafultonj: thank you12:49
weshay|roverEmilienM, look ok to you sir? https://review.openstack.org/#/c/596697/12:50
Tengufultonj: thanks :).12:50
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: Fix the scenario007 CI job failure  https://review.openstack.org/59659412:51
*** gkadam has quit IRC12:51
trownmandre: I am guessing we actually need the fedora one for some reason? There is already a centos etcd container in the rdo mirror: https://hub.docker.com/r/tripleomaster/centos-binary-etcd/12:51
weshay|rovermwhahaha, EmilienM rocky needs this too right? https://review.openstack.org/#/c/596696/12:51
*** rbrady has joined #tripleo12:53
mandretrown: yeah, there is a reason for the one coming from fedora... the centos-binary-etcd doesn't work for openshift, I don't recall the reason exactly but I can find it12:53
trownmandre: ok I uploaded the fedora one, putting up a patch to switch to that12:53
*** ykarel_ is now known as ykarel12:53
mandretrown: here is the reason https://review.openstack.org/#/c/576497/12:53
fultonjAelia: what type of node are you trying to scale in your Pike overcloud? a controller?12:53
Aeliafultonj: a compute12:54
*** moshele has joined #tripleo12:54
mandretrown: centos-binary-etcd doesn't work because the default user is etcd and not root12:55
fultonjAelia: did you deploy originally with MDS?12:55
*** janki has quit IRC12:55
mandretrown: we'll need https://review.openstack.org/#/c/576837/ to set the right etcd image to use for CI12:56
*** links has quit IRC12:57
Aeliafultonj: I did not perform the initial deployment, but before this upscale mds is present.12:57
trownmandre: hmm... how is CI currently working? We must just use the defaults from tripleo-common12:57
*** myoung|training is now known as myou7ng12:57
*** myou7ng is now known as myoung12:57
Aeliafultonj: from what I know the initial deployment was not containerized so we had to perform a baremetal -> container, change in pike12:58
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Enable extra playbooks to run post-deployment  https://review.openstack.org/59458312:58
mandretrown: i'd prefer we keep the default to the fedora registry and just tweak the docker_openshift_etcd_* for CI in featureset03312:59
*** pliu has joined #tripleo13:00
trownmandre: ah ok, that works I suppose13:00
Aeliafultonj: it did not go well, and we still have a few issues with some parts, like the neutron-l3-agent and a few things. but they are not blocking, and I have documented workarounds. This one is blocking since the production cluster is at capacity and we need to add nodes in it (and I don't want to touch the production before managing to perform the action on the lab).13:00
openstackgerritAakarsh proposed openstack/tripleo-quickstart master: DNM: Testing browbeat's sysmetric collection  https://review.openstack.org/59535113:01
fultonjAelia: on the controller node, what does `ls /var/run/ceph/` return?13:01
fultonj(not in the container but on the container host)13:01
Aeliafultonj: empty directory13:02
*** ykarel has quit IRC13:03
*** ykarel has joined #tripleo13:03
fultonjAelia: so the ceph-mons are not running either13:06
Aeliadocker.io/ceph/daemon:v3.0.1-stable-3.0-jewel-centos-7-x86_64 ceph-mon-lab-controller-213:07
Aeliarunning 6 days13:08
Aeliain the container I have /var/run/ceph/ceph-mon.lab-controller-2.asok13:09
EmilienMweshay|rover: ask bnemec ?13:09
Aeliafultonj: ^13:09
*** gbarros has joined #tripleo13:09
Aeliafultonj: missing mount bind ?13:09
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676413:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791013:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890713:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)13:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)13:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)13:10
*** mjturek has joined #tripleo13:11
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Use mirrored etcd container for openshift CI  https://review.openstack.org/59671313:11
Aeliafultonj: the volumes binds I have are /var/lib/ceph:/var/lib/ceph /etc/ceph:/etc/ceph /etc/localtime:/etc/localtime:ro and volumes are : "/etc/ceph": {}  "/etc/ganesha": {} "/var/lib/ceph": {}13:13
Aelia( as reported by docker inspect on the running container )13:14
fultonjAelia: docker ps | grep ceph13:15
Aelia3 containers , ceph-rgw-lab-controller-2 ceph-mds-lab-controller-2 ceph-mon-lab-controller-213:16
fultonjAelia: how long have they been up?13:16
Aeliafultonj: all Up 6 days13:16
fultonjAelia: ok, good13:16
fultonjAelia: systemd unit files for them have restart always13:17
fultonjwhat's what keeps bringing them back as per your bug report13:18
fultonjthat's what keeps bringing them back as per your bug report13:18
Aeliaok that explains it.13:18
*** ansmith has joined #tripleo13:18
fultonjAelia: but it doesn't sounds like they're flapping if they're up for 6 days13:19
fultonjAelia: 'rpm -q ceph-ansible' on your undercloud13:20
Aeliafultonj: ceph-ansible-3.0.27-1.el7.noarch13:20
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Move toci BASH variables to ansible  https://review.openstack.org/58944813:24
Tenguflorianf: hello! do you have some more courage in order to get a (hopefully) last look on the validation framework review? Had to add some more component and rephrase some parts in it :).13:31
florianfTengu: yup, happy to have another look13:32
Tenguflorianf: great, thanks!13:33
*** moshele has quit IRC13:40
*** moshele has joined #tripleo13:41
Aeliafultonj: still alive ?13:45
fultonjAelia: yes13:45
openstackgerritMartin AndrĂ© proposed openstack/tripleo-quickstart master: Use mirrored etcd container for openshift CI  https://review.openstack.org/59671313:48
Aeliafultonj: do you need more info about my setup ?13:48
openstackgerritMartin AndrĂ© proposed openstack/tripleo-quickstart master: Revert "Revert "Move to openshift-ansible 3.10""  https://review.openstack.org/58946913:49
*** bdodd has joined #tripleo13:49
*** jaganathan has quit IRC13:49
fultonjAelia: not at the moment, i am balancing priorities13:49
Aeliaok13:49
*** psachin has quit IRC13:50
fultonjAelia: it confuses me that /var/run/ceph is empty yet 'docker ps | grep ceph' shows your containers13:51
*** ade_lee has joined #tripleo13:51
fultonji wondering if it's something about jewel containers vs lumionus contianers, which is what i'm using13:51
fultonjon the newer stuff13:51
Aeliathere is no bind for /var/run/ceph in the containers, checked by looking into the /etc/systemd/system/ceph-{mon,mds}@.service13:52
Aeliafor mds I have -> -v /var/lib/ceph:/var/lib/ceph -v /etc/ceph:/etc/ceph -v /etc/localtime:/etc/localtime:ro13:53
Aeliaif there is an issue with /var/run/ceph empty, there is no link between jewel vs. luminous13:54
fultonjAelia: in the bug you reported you saw  /var/run/ceph/ceph-mds.lab-controller-2.asok13:55
Aeliain the container13:55
Aeliait exists, but not outside the container13:55
Aeliaand the command to check the existance of the asok, seem to be executing the check with a " docker exec -it " IF I believe the variable is correctly set13:56
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643213:56
mwhahahaEmilienM: so rather than disable 2.7 jobs for paunch i think we need to mock the test like i did for tripleo-client in https://review.openstack.org/#/c/596471/13:57
EmilienMmwhahaha: ok I'm find with it13:57
EmilienMfine13:57
EmilienMI can do that today13:57
fultonjAelia: https://github.com/ceph/ceph-ansible/blob/v3.0.27/roles/ceph-defaults/defaults/main.yml#L391-L39213:59
fultonjAelia: basically the playbook has a handler that bounces the container in question and it retries and then waits the number of times in ^ for the service to come back before it gives up14:00
Aeliafultonj: ok so it explains why I see it retrying a lot of times14:01
*** saneax has quit IRC14:01
fultonjAelia: it determines if service is up by looking for that file14:02
Aeliahttps://github.com/ceph/ceph-ansible/blob/v3.0.27/roles/ceph-defaults/templates/restart_mds_daemon.sh.j2#L17 <-- this is why I think it should be checked inside the container14:02
fultonji have seen, with the use_fqdn flag that it's looking for a longer version of the name, foo instead of foo.bar.com to cause issues yere14:02
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642214:02
*** raildo_ has joined #tripleo14:03
*** raildo has quit IRC14:03
fultonjAelia: i agree14:04
Aeliaand this is why I am confused, because in the container the asok exists with the name it tries to check ...14:05
*** dmacpher_ has joined #tripleo14:05
openstackgerritMartin AndrĂ© proposed openstack/tripleo-heat-templates master: Force absolute path for CONFIG_VOLUME_PREFIX  https://review.openstack.org/59674414:06
Aeliafultonj: for example -> [root@lab-controller-2]# docker exec -it ceph-mds-lab-controller-2 ls /var/run/ceph/ceph-mds.lab-controller-2.asok -> /var/run/ceph/ceph-mds.lab-controller-2.asok14:06
fultonjor it didn't exist within the time to search14:06
fultonjso you could give it more time14:06
*** rmascena__ has joined #tripleo14:06
*** dxiri has joined #tripleo14:07
fultonjso it's nice it exists now, but when ceph-ansible was looking for it it didn't find it14:07
fultonjperhaps tell ceph-ansible to be patient14:07
fultonjby increasing retires before it gives up14:07
*** hamdyk has joined #tripleo14:08
weshay|roverbnemec, please ping me when you have a sec to chat about rocky and designate14:08
*** raildo_ has quit IRC14:09
openstackgerritwes hayutin proposed openstack/tripleo-heat-templates stable/rocky: Pass in rndc key to Designate deployment  https://review.openstack.org/59669614:09
fultonjAelia: and monitor the other controller nodes when you re-run ceph-ansible with a larger retry time to see if they restart ok14:09
Aeliafultonj: I need to have a bit more info here about the meaning of the variables then14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676414:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791014:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890714:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)14:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)14:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to yatin (yatinkarel)14:10
fultonjhttps://github.com/ceph/ceph-ansible/blob/v3.0.27/roles/ceph-defaults/templates/restart_mds_daemon.sh.j2#L18-L1914:10
fultonjAelia: ^14:10
fultonjsleep and retry should be clear in that script14:10
Aeliafultonj: I tried to stop manually the container, let systemd restart it, and it took about 10 seconds for the asok to come back14:10
*** vkapalav has joined #tripleo14:12
Aeliaso the 50 seconds it is waiting now *should* be enough14:12
Aeliabut I can try to increase the time and try to run the playbook, however I am not sure how to run the playbook correctly...14:13
Aelia(or even how to override these options at all)14:14
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: Make sure that stonith state is enforced before attempting a scaleup  https://review.openstack.org/58252114:15
fultonjAelia: did it restart ok on the other nodes?14:16
*** tzumainn has quit IRC14:17
Aeliaon all 3 controllers I have the containers up 6 days with the asok existing14:18
Aeliafultonj: ^14:18
fultonjmds containers up 6 days14:19
fultonjon all three, good14:19
Aelia(well since I restarted manually on a controller-2 it has less uptime now)14:19
fultonjAelia: two ways to re-run14:20
fultonjAelia: https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ceph_config.html#customizing-ceph-conf-with-ceph-ansible14:20
fultonjif you want to override parameters14:21
fultonj  CephAnsibleExtraConfig:14:21
fultonjAelia: which is not additive, use it once14:21
fultonjgrep CephAnsibleExtraConfig *.yaml | wc -l --> 114:21
fultonjAelia: have you seen this issue repeatedly14:22
fultonj?14:22
fultonjAelia: you can re-run by rerunning 'openstack overcloud deploy...'14:22
fultonjAelia: you might be able to get mistral to-rerun the failed task if you don't want to override params which is worth a shot14:22
fultonjAelia: if you restart them all manually and observe they restart without issue, then you could just do the heat stack updatre14:23
openstackgerritMerged openstack/tripleo-ui master: Add ability to create plan using default templates  https://review.openstack.org/58781314:23
Aeliafultonj: as I said I can restore a previous state (before trying to perform the openstack overcloud deploy step) and each time I try I have this issue.14:23
AeliaI already tried 3 times ...14:24
Aeliais it is not really a "random" occurence14:24
*** thrash is now known as thrash|biab14:26
openstackgerritMerged openstack/tripleo-ui master: Add ability to import plan from Git repository  https://review.openstack.org/58817914:27
fultonjAelia: so you always restore from a snapshot or something before re-running14:27
Aeliafultonj: yes14:27
*** two_oes has quit IRC14:28
fultonjAelia: restore, manually restart and see if you hit it indepdently of ceph-ansible14:28
Aeliafultonj: because I want to try to simulate what would happend in the production setup, currently not broken14:28
fultonji dn't see how manually restarting them to try to reproduce the issue prior to running ceph-ansible would prevent you from simulating what would happen in production set up14:29
fultonjyou could open a bug w/ ceph-ansible regarding this too. they'd probably want you to reproduce w/ latest version14:30
*** ramishra has joined #tripleo14:30
fultonjAelia: however, if you restore and see that simply bouncing MDS services fails regardless of what bounces it, then it might give you an idea of what's happening14:30
Aeliaright now the lab setup is broken (I left it as-is after last try), so I tried to restart the container manually to see what happens.14:31
Aeliaso the heat stack status is UPDATE_FAILED14:31
fultonjif the heat stack is update_failed you can still run openstck overcloud deploy to attempt to update heat stack, but i'm not suggesting that yet14:33
Aeliabut stopping the containers and letting them be restarted has always worked until now14:33
fultonji don't know what nodes your snapshot restore is for14:34
AeliaI have had a complete power outage a few weeks back and everything went back online without issue even on prod14:34
Aeliafultonj: I have the lab, and a production cluster, I want to test adding a node to the cluster first on the lab, and then apply the same procedure on the prod14:34
fultonji understand all that14:35
fultonjbut right now aren't we trying to find out what exactly is causing this handler to fail?14:35
fultonjso we're foucssed on the restart of the MDS container14:35
fultonjwhy does the MDS container not simply restart when ceph-ansible tries to make it do it?14:36
fultonji think that's the question14:36
Aeliarestarting the containers is no issue even on prod, the cluster has had a power outage and went back up without much issues.14:36
fultonjwe're talking about your dev environment14:37
fultonjand looking for a procdure for it14:37
Aeliawhat I don't understand is why ceph-ansible THINKS something is wrong when it tries to restart the ceph containers14:37
Aeliabecause they are up14:37
fultonji don't either14:37
fultonjso you focus on that particular issue and experiement in the environment where it's happening14:37
openstackgerritChuck Short proposed openstack/instack-undercloud master: Replace assertRaisesRegexp with assertRaisesRegex  https://review.openstack.org/59677214:38
AeliaI do not touch the prod as long as I cannot reproduce the operation confidently on the lab.14:38
bnemecweshay|rover: Oh, ick.  So I screwed up two of those patches. :-(14:38
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643214:38
fultonjthat's good14:38
fultonji agree with that philosophy14:38
*** hamdyk has quit IRC14:38
Aeliawhich is also why I did not update the undercloud yet everything is versioned using Satellite14:38
AeliaI can update the lab without touching the prod14:39
Aeliaand be sure the packages, images etc are exactly the version that was tested before changing the prod14:39
*** dxiri has quit IRC14:40
*** ksambor_ has quit IRC14:40
openstackgerritChuck Short proposed openstack/diskimage-builder master: Replace assertRaisesRegexp with assertRaisesRegex  https://review.openstack.org/59677414:40
*** dxiri has joined #tripleo14:40
Aeliaso now, the only way to retry on the lab is to A. restart the whole "openstack overcloud deploy" with check timeout change B. restart only the mistral task for ceph-ansible without check timeout change14:42
Aeliasince B. was somehow tested 3 times already, I think I will try A.14:42
Aeliaso my yaml override will be -> parameter_defaults.CephAnsibleExtraConfig.handler_health_mds_check_retries: 100 ?14:44
*** jcoufal has quit IRC14:44
*** jcoufal has joined #tripleo14:45
mwhahahabnemec: so designate stuff, where are we at with the reverts to fix scenario003? which patches do we need to land14:46
openstackgerritLuigi Toscano proposed openstack/tripleo-quickstart-extras master: tempestmail: update the notification list for cinder  https://review.openstack.org/59677914:46
openstackgerritJason E. Rist proposed openstack/tripleo-ui master: New regex for the FQDN that handles many subdomains  - also case insensitive.  https://review.openstack.org/59584714:47
jristhonza: ^14:47
fultonjAelia: that override is fine, but...14:47
fultonjif you restore your controller nodes from a clean snapshot14:48
fultonjssh into them and restart the MDSs14:48
fultonjisn't that different from attempting the upgrade before you restart them?14:48
fultonjit sounds to me that you're only reattempting restart after ceph-ansible fails to restart them14:49
bnemecmwhahaha: Apparently two of those patches had the same problem, so the single revert wasn't enough.14:49
*** jcoufal_ has joined #tripleo14:49
*** jcoufal has quit IRC14:50
Aeliafultonj: I can try. not convinced it will change anything. But since it doesn't take much time I can do it.14:50
fultonjAelia: i'm hoping you'll see some sort of insight as to what's going on through a different appraoch which is less complicated14:50
fultonjand as you say, it doesn't take much time14:50
*** tzumainn has joined #tripleo14:51
mwhahahabnemec: so do we need to land something specific or do we need to squash all your THT patches together to get it working?14:51
fultonjAelia: if simply runnig the same command on the fresh restore works, why wouldn't it work with ceph-ansible?14:51
fultonjOR if running the same command fails, then you can ignore ceph-ansible and focus on the ceph environment14:52
bnemecmwhahaha: At this point I'm tempted to just squash all the puppet fixes into one patch and land that.  Those are much smaller changes than the full tht series.14:52
*** jcoufal_ has quit IRC14:52
mwhahahabnemec: k let me know if you need help/reviews14:52
fultonjAelia: then you'll know if you need to talk to ceph team or ceph-ansible team at least14:52
bnemecmwhahaha: Will do.  I'm just getting everything rebased correctly so I should have something for ci to chew on shortly.14:53
Aeliamy instinctive answer here would be, the ceph container works as expected. because it works without issues on the prod. but then maybe some weird side effect makes it work on the prod but not on the lab.14:54
fultonj"maybe some weird side effect makes it work on the prod but not on the lab" <-- yes14:55
fultonjand you could get closer to findintg out by restoring fresh and attempting a simple restart14:55
*** thrash|biab is now known as thrash14:56
Aeliaok restored the lab, applied workarounds for neutron-l3-agent, waiting for everything to settle down (since the restore is a cold restore, from shutdown state, I need to wait for all the services to be brought up)14:59
openstackgerritBen Nemec proposed openstack/puppet-tripleo master: Conditionalize rndc changes  https://review.openstack.org/59646715:00
bnemecmwhahaha: ^ will hopefully unblock scenario 3.15:01
mwhahahak15:01
openstackgerritRaoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Add ironic-pyton-agent image download option  https://review.openstack.org/59678915:01
mwhahahaweshay|rover: i noticed legacy-DLRN-rpmbuild jobs running (and failing), what are those15:08
ykarelbnemec, looks like it will still fail without tht due to https://review.openstack.org/#/c/596467/3/manifests/profile/base/designate/worker.pp@3215:08
mwhahahaalso anyone around to approve https://review.openstack.org/#/c/596471 to unblock the py27 tests for triploe-common15:08
*** jcoufal has joined #tripleo15:09
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676415:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791015:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890715:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)15:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)15:10
bnemecykarel: If tht doesn't set the hieradata then that will be a noop like it was before.15:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)15:10
mwhahahabnemec: no that'll fail if it's not defined15:10
mwhahahabnemec: that should have a fallback to undef or false or soemthign15:11
mwhahahahiera('designate_rndc_key', false) (to really be safe)15:11
*** bdodd_ has joined #tripleo15:11
bnemecmwhahaha: Oh, right.  I'll get that fixed.15:11
openstackgerritJiri Stransky proposed openstack/python-tripleoclient master: Use tripleo-admin user for minor updates  https://review.openstack.org/59679715:12
Aeliafultonj: ok 50 seconds might be a bit short15:12
AeliaI do a stop of all 3 ceph containers at the same time, and wait for mds (the last one stopped) to spawn the asok, it took 62 seconds15:13
*** bdodd has quit IRC15:14
fultonjAelia: ok, then increasing timeout might help15:14
Aelia66 seconds the second time15:14
fultonjAelia: also, ceph-ansible does them sequentially not concurrently15:14
Aeliamds only takes about the same time15:16
Aelia57 seconds there15:16
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: Support for ARA report for ansible playbooks in deploy  https://review.openstack.org/56507715:16
openstackgerritSagi Shnaidman proposed openstack/python-tripleoclient master: Support ARA report tracking from command line  https://review.openstack.org/58379915:16
weshay|rovermwhahaha, alfredo owns those15:18
weshay|roverbnemec, do we need backports to rocky for designate?15:19
weshay|roverbnemec, like https://review.openstack.org/#/c/596697/15:19
bnemecweshay|rover: We will, after the master ones land.15:20
Aeliafultonj: I changed the timeout, and preparing the environment for a openstack overcloud deploy that will actually perform something useful.15:21
weshay|roverbnemec, can you please spell out what steps are required to unblock master/rocky in this case, here https://trello.com/c/8VC1IusK/711-cixlp1788907tripleociproa-failed-running-docker-puppetpy-for-designate15:22
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: TEST featureset override  https://review.openstack.org/59680915:24
*** rmascena__ is now known as raildo15:26
*** pcaruana has quit IRC15:29
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643215:30
Aeliahmm... my virtual baremetal-nodes kept going into maintenance mode... vbmc was doing nasty things...15:35
openstackgerritEmilien Macchi proposed openstack/paunch master: Adjust tenacity mock  https://review.openstack.org/59681615:36
*** gbarros has quit IRC15:36
*** gbarros has joined #tripleo15:38
lvdombrkrhello folks15:40
openstackgerritMartin AndrĂ© proposed openstack/tripleo-common master: Switch to openshift 3.10  https://review.openstack.org/59682015:41
*** ykarel has quit IRC15:42
Aeliafultonj: ok command started, I will let everything run all night. it is starting to be late here. Thank you for your help, I will update the issue tomorrow morning.15:42
fultonjAelia: ok15:43
*** ykarel has joined #tripleo15:43
*** agurenko has quit IRC15:43
lvdombrkrwho can help little bit with networking15:49
lvdombrkrhow vm packet flow (inbound/outbound) differs with and without external network15:49
lvdombrkr?15:49
mwhahahathat sounds like a question for #openstack-neutron15:51
lvdombrkrmwhahaha : thanks i will ask15:53
*** aufi has quit IRC15:56
openstackgerritEmilien Macchi proposed openstack/paunch master: Adjust tenacity mock  https://review.openstack.org/59681616:00
openstackgerritEmilien Macchi proposed openstack/paunch master: Refactor compose1.py to welcome new builders  https://review.openstack.org/58931216:01
openstackgerritEmilien Macchi proposed openstack/paunch master: Refactor the runner to support more than docker  https://review.openstack.org/58999616:01
openstackgerritEmilien Macchi proposed openstack/paunch master: Introduce --default-runtime parameter  https://review.openstack.org/59006716:01
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Support ARA statistics in InfluxDB for longest tasks  https://review.openstack.org/58023816:02
openstackgerritBen Nemec proposed openstack/puppet-tripleo master: Conditionalize rndc changes  https://review.openstack.org/59646716:04
bnemecmwhahaha: weshay|rover: ^16:04
bnemecSorry for the delay.  The Oslo meeting is also Monday morning. :-/16:04
mwhahahapesky meetings16:04
weshay|roverbnemec, thanks16:05
bnemecweshay|rover: I believe that will need to be backported to rocky once it's working and that should unblock things.16:05
*** jpena is now known as jpena|off16:09
*** jfrancoa has quit IRC16:09
*** ccamacho is now known as ccamacho|brb16:10
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676416:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791016:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890716:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)16:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)16:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)16:10
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements master: Port items from instack-undercloud  https://review.openstack.org/59683316:13
*** yprokule has quit IRC16:15
openstackgerritMerged openstack/tripleo-validations stable/queens: Revert "Fail if the Overcloud inventory is empty"  https://review.openstack.org/59668416:16
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Remove instack-undercloud from image building  https://review.openstack.org/59683516:17
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements master: Port items from instack-undercloud  https://review.openstack.org/59683316:19
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643216:21
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Remove instack-undercloud and --use-heat  https://review.openstack.org/58860616:22
openstackgerritEmilien Macchi proposed openstack/paunch master: Adjust tenacity mock  https://review.openstack.org/59681616:24
openstackgerritEmilien Macchi proposed openstack/paunch master: Refactor compose1.py to welcome new builders  https://review.openstack.org/58931216:25
openstackgerritEmilien Macchi proposed openstack/paunch master: Refactor the runner to support more than docker  https://review.openstack.org/58999616:25
openstackgerritEmilien Macchi proposed openstack/paunch master: Introduce --default-runtime parameter  https://review.openstack.org/59006716:25
*** ykarel is now known as ykarel|dinner16:26
*** radeks_ has quit IRC16:26
*** pcaruana has joined #tripleo16:26
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642216:27
*** itlinux has joined #tripleo16:29
openstackgerritEmilien Macchi proposed openstack/paunch master: WIP - First support for Podman (runner + builder)  https://review.openstack.org/59049116:29
*** radeks has joined #tripleo16:29
*** trown is now known as trown|lunch16:29
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643216:30
*** raildo_ has joined #tripleo16:31
*** jpich has quit IRC16:32
*** raildo has quit IRC16:32
openstackgerritEmilien Macchi proposed openstack/paunch master: WIP - First support for Podman (runner + builder)  https://review.openstack.org/59049116:32
*** ykarel|dinner has quit IRC16:33
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643216:36
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643216:38
*** raildo_ is now known as raildo16:41
*** kopecmartin has quit IRC16:41
*** raildo has quit IRC16:48
*** raildo has joined #tripleo16:48
*** dxiri has quit IRC16:49
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642216:56
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: remove older and slower portions of collect-logs  https://review.openstack.org/59684116:58
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Enable featureset override  https://review.openstack.org/59451116:58
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: TEST featureset override  https://review.openstack.org/59680916:58
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: TEST featureset override  https://review.openstack.org/59680917:03
*** raildo_ has joined #tripleo17:03
*** openstackgerrit has quit IRC17:04
*** raildo has quit IRC17:05
*** openstackgerrit has joined #tripleo17:09
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Enable featureset override  https://review.openstack.org/59451117:09
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: TEST featureset override  https://review.openstack.org/59680917:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676417:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791017:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890717:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)17:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)17:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)17:10
*** vkapalav has quit IRC17:14
*** rmascena__ has joined #tripleo17:16
*** gbarros has quit IRC17:17
*** raildo has joined #tripleo17:18
*** raildo_ has quit IRC17:18
*** ykarel has joined #tripleo17:20
*** rmascena__ has quit IRC17:21
*** raildo_ has joined #tripleo17:22
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642217:22
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: DNM: add vexxhost env for validation  https://review.openstack.org/59643217:23
*** raildo has quit IRC17:23
*** dtantsur is now known as dtantsur|afk17:24
*** trown|lunch is now known as trown17:26
*** lvdombrkr has quit IRC17:28
*** raildo_ has quit IRC17:28
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Collect overcloud statistics with ARA  https://review.openstack.org/59684717:28
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Use mirrored etcd container for openshift CI  https://review.openstack.org/59671317:28
*** ykarel_ has joined #tripleo17:29
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: TEST featureset override  https://review.openstack.org/59680917:30
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Enable featureset override  https://review.openstack.org/59451117:31
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: TEST featureset override  https://review.openstack.org/59680917:31
*** ykarel has quit IRC17:31
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642217:33
openstackgerritRafael Folco proposed openstack/tripleo-docs master: Add featureset_override instructions  https://review.openstack.org/59684817:34
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642217:34
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: Support for ARA report for ansible playbooks in deploy  https://review.openstack.org/56507717:36
*** raildo has joined #tripleo17:40
*** radeks has quit IRC17:41
trownmandre: I dont think depends-on will work for https://review.openstack.org/#/c/589469/ -> https://review.openstack.org/#/c/596713/ since they are both on tripleo-quickstart17:42
trownmandre: I can rebase if we want to try 3.10 on top of the etcd fixes17:42
mandretrown: I think it should work, IIUC depends-on also works in the same repo17:44
trownmandre: ok, I will wait to rebase then... that definitely did not work in the past, but maybe it does now17:45
mandretrown: if not, then by all means please rebase the patches :)17:45
trownmandre: ya we should be able to see by the time I leave today, so I can make sure to rebase them if needed17:46
mandretrown: FYI I'm addressing shardy's comment for https://review.openstack.org/#/c/582495/, on a local branch for now17:46
trownmandre: awesome17:47
mandretrown: sounds good, so I can check the result in my morning17:47
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642217:47
*** sri_ has quit IRC17:48
*** sri_ has joined #tripleo17:48
*** assassin has quit IRC17:53
openstackgerritRaoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Add ironic-pyton-agent image download option  https://review.openstack.org/59678917:53
*** sri_ has quit IRC17:54
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Include minimal Browbeat playbook in baremetal playbook  https://review.openstack.org/58148817:56
*** holser_ has quit IRC17:56
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642218:00
*** dsneddon has joined #tripleo18:02
openstackgerritChuck Short proposed openstack/paunch master: add python 3.6 unit test job  https://review.openstack.org/59604318:05
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Create missing directories before mounting them  https://review.openstack.org/59583118:05
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Create missing directories before mounting them  https://review.openstack.org/59583118:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Create missing directories before mounting them  https://review.openstack.org/59583118:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Initial support for Podman in docker-puppet  https://review.openstack.org/58865518:07
*** trozet has joined #tripleo18:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker-puppet: don't force FACTER_hostname  https://review.openstack.org/59527418:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test podman  https://review.openstack.org/59058718:09
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676418:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791018:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890718:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)18:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)18:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)18:10
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Remove inclusion of legacy playbook in tripleo-ci-base  https://review.openstack.org/59642218:10
*** morazi has quit IRC18:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test podman  https://review.openstack.org/59058718:11
*** gbarros has joined #tripleo18:12
*** dtrainor has quit IRC18:14
*** florianf has quit IRC18:14
*** bdodd_ has quit IRC18:22
*** bdodd has joined #tripleo18:23
*** ykarel_ is now known as ykarel18:25
*** ykarel is now known as ykarel|away18:38
*** toure has joined #tripleo18:40
*** dsneddon has quit IRC18:43
*** dtrainor has joined #tripleo18:44
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Enable support for running refstack tests in TQE  https://review.openstack.org/57071918:46
*** vinaykns has joined #tripleo18:48
openstackgerritIan Main proposed openstack/tripleo-common master: WIP: Use process-templates.py and cache templates to speed up plan ops.  https://review.openstack.org/58115318:50
openstackgerritIan Main proposed openstack/python-tripleoclient master: WIP: Speed up plan operations by using a tarball in swift.  https://review.openstack.org/58114118:51
*** itlinux has quit IRC18:54
*** radeks has joined #tripleo18:55
openstackgerritBen Nemec proposed openstack/puppet-tripleo master: Conditionalize rndc changes  https://review.openstack.org/59646718:56
bnemec****!  I was on stable/rocky.  .gitreview must not be correct?18:58
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Revert "Revert "Move to openshift-ansible 3.10""  https://review.openstack.org/58946918:59
bnemecOh good, and now it won't let me re-push the correct one because "no new changes".19:03
openstackgerritBen Nemec proposed openstack/puppet-tripleo master: Conditionalize rndc changes  https://review.openstack.org/59646719:04
mwhahahawe haven't landed the .gitreview yet19:04
mwhahahai don't think19:04
bnemecI see that now. :-/19:05
openstackgerritBen Nemec proposed openstack/puppet-tripleo stable/rocky: Conditionalize rndc changes  https://review.openstack.org/59686419:07
bnemecAlright, well that patch on master had passed scenario003 before I killed it.19:07
mwhahaha:D19:08
bnemecSo once those land both master and rocky should be unblocked.19:08
*** dprince has quit IRC19:09
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676419:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791019:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890719:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)19:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)19:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)19:10
*** sl0w has joined #tripleo19:11
*** itlinux has joined #tripleo19:11
*** pcaruana has quit IRC19:13
*** itlinux has quit IRC19:16
*** morazi has joined #tripleo19:19
*** ykarel|away has quit IRC19:20
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: devmode.sh has been upgraded  https://review.openstack.org/58409719:22
openstackgerritTim Rozet proposed openstack/tripleo-common master: Fixes OVN docker image prep with Neutron  https://review.openstack.org/59687119:22
*** paramite has quit IRC19:34
*** ramishra has quit IRC19:38
openstackgerritRaoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Fetch images again inside BM undercloud playbook  https://review.openstack.org/59687719:40
rascasofthey folks, ERROR: Property error: : resources.Controller<nested_stack>.resources.0<https://192.168.24.2:13808/v1/AUTH_9590821899f948d286884f41741525e7/overcloud/puppet/controller-role.yaml>.resources.NetworkConfig.properties: : Unknown Property ControlPlaneDefaultRoute... Did we deprecated the ControlPlaneDefaultRoute parameter?19:42
*** dprince has joined #tripleo19:42
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Collect overcloud statistics with ARA  https://review.openstack.org/59684719:48
openstackgerritRaoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Fetch images again inside BM undercloud playbook  https://review.openstack.org/59687719:48
*** holser_ has joined #tripleo19:50
rascasoftmwhahaha, hey man! any idea about the ControlPlaneDefaultRoute Unknown Property above?19:54
mwhahaharascasoft: sounds like bad netconfigs19:55
mwhahaha rascasoft i think they made it optional? i'd have to defer to hjenas dsneddon or bfournie19:56
rascasoftmwhahaha, the net env conf file I'm using is the same in ages, so it should not be my case19:56
rascasoftmwhahaha, ok thanks, let's see if they have ideas19:57
bfournierascasoft:  is it defined in your network-environment.yaml?19:57
bfournierascasoft: ControlPlaneDefaultRoute: 10.8.146.119:58
bfourniefor example19:58
bfournierascasoft: we'd have to see which set of env and nic confof files you are using19:59
*** moshele has quit IRC20:00
*** dtrainor_ has joined #tripleo20:05
*** dtrainor has quit IRC20:06
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676420:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791020:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890720:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)20:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)20:10
*** khyr0n has joined #tripleo20:10
*** khyr0n has quit IRC20:12
*** khyr0n has joined #tripleo20:12
openstackgerritRafael Folco proposed openstack/tripleo-quickstart master: WIP: Stop relying on old legacy sub_nodes_private  https://review.openstack.org/59688720:26
*** lblanchard has joined #tripleo20:31
*** rlandy is now known as rlandy|brb20:31
*** raildo has quit IRC20:32
*** holser_ has quit IRC20:36
*** dprince has quit IRC20:37
*** ansmith has quit IRC20:37
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Create missing directories before mounting them  https://review.openstack.org/59583120:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Initial support for Podman in docker-puppet  https://review.openstack.org/58865520:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker-puppet: don't force FACTER_hostname  https://review.openstack.org/59527420:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test podman  https://review.openstack.org/59058720:41
fultonji have a question about tripleo::firewall::firewall_rules http://paste.openstack.org/show/72887420:41
EmilienMfultonj: probably a custom rule no?20:42
EmilienMfultonj: sorry wrong answer20:42
EmilienMcan I see your templates?20:42
fultonjEmilienM: yes20:42
EmilienM https://github.com/fultonj/cephmetrics-osp/blob/cephmetrics-network/tht/cephmetrics.yaml20:42
EmilienMlooking20:43
fultonjexactly20:43
fultonjhttps://github.com/fultonj/cephmetrics-osp/tree/cephmetrics-network/tht20:43
*** trown is now known as trown|outtypewww20:43
fultonjgiven the syntax that i passed would you expect it to work or am i using it wrong?20:44
EmilienMweird20:44
*** mjturek has quit IRC20:45
fultonjthis is with queens20:45
EmilienMoh20:45
EmilienMyeah20:45
EmilienMso20:45
EmilienMremove L920:45
EmilienMfirewall_rules is an hash20:46
fultonjEmilienM: thanks20:46
EmilienMthe 301 overrided the 30020:46
EmilienMremove L9 and that's good20:46
fultonjEmilienM: also, 301 on the Ceph nodes didn't work20:46
EmilienMfultonj: be careful about CephStorageConfig20:46
EmilienMI think it's CephStorageExtraConfig20:47
EmilienMhttps://git.openstack.org/cgit/openstack/tripleo-docs/tree/doc/source/install/advanced_deployment/node_config.rst#n3920:47
fultonjdoh!20:47
fultonjyeah, that's it20:47
EmilienMcool20:47
fultonjthanks EmilienM20:47
EmilienMno prob!20:48
*** rlandy|brb is now known as rlandy20:53
*** raildo has joined #tripleo20:54
*** raildo has quit IRC20:54
stevebakermwhahaha: hey, I'd really like to get this blueprint landed. https://review.openstack.org/#/c/581919/20:54
mwhahahastevebaker: unfortunately scenario003 is broken until we land the fix20:55
* mwhahaha is aware however approving that patch will solve nothing20:55
stevebakerok20:55
stevebakermwhahaha: can it be backported for rc2?20:56
mwhahahastevebaker: it's likely we won't have an rc220:56
mwhahahaunless we reaaaaally need this for rc220:57
stevebakermwhahaha: are you saying this blueprint won't be delivered for rocky? this is upsetting20:57
mwhahahastevebaker: well that's what i'm trying to understand, is it just this patch or are there others?20:58
stevebakermwhahaha: its this one patch, and the oooq patch to switch over to the new prepare https://review.openstack.org/#/c/573476/20:59
stevebakerreviews have been sloow20:59
mwhahahastevebaker: so if it's just that one i guess we can have an rc2 with it20:59
mwhahahai'm trying to fixup CI20:59
mwhahahabut reviews have also been slow20:59
mwhahahaso if people could stop breaking things and review outstanding patches that would be nice21:00
stevebakermwhahaha: feel free to add me to any urgent review21:03
EmilienMstevebaker, mwhahaha : I'm not against backporting this code to stable/rocky without rc221:07
mwhahahai mean we might be able to land it this evening21:08
EmilienMstevebaker: when you have time this week, please create a buildah RFE, thanks21:08
EmilienMyeah21:08
mwhahahabnemec had a patch for designate21:08
mwhahahalet me go check on that21:08
stevebakerEmilienM: so landing it so it will be part of x.0.0?21:08
bnemecmwhahaha: ci is still running21:08
stevebakerEmilienM: do we have an RFE for docker-distribution replacement?21:09
EmilienMstevebaker: not sure about tag21:09
EmilienMstevebaker: we produce tags every 2 weeks on stable branches21:09
EmilienMroughly21:09
mwhahahaEmilienM, stevebaker: https://review.openstack.org/#/c/596467/ is the patch to fix scenario00321:10
EmilienMnot sure for the RFE21:10
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676421:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791021:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890721:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)21:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)21:10
*** mjturek has joined #tripleo21:17
openstackgerritGoutham Pacha Ravi proposed openstack/tripleo-heat-templates master: Add/fix manila environments in the capabilities-map  https://review.openstack.org/59690421:32
*** radeks has quit IRC21:36
openstackgerritRafael Folco proposed openstack/tripleo-quickstart-extras master: WIP: Stop relying on old legacy sub_nodes_private  https://review.openstack.org/59690621:49
*** rcernin has joined #tripleo21:51
*** slaweq has quit IRC21:55
*** gbarros has quit IRC22:04
*** apetrich has quit IRC22:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676422:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791022:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890722:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)22:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)22:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)22:10
*** apetrich has joined #tripleo22:14
*** dsneddon has joined #tripleo22:16
*** dsneddon has quit IRC22:19
*** dsneddon has joined #tripleo22:19
*** mjturek has quit IRC22:23
*** owalsh has quit IRC22:26
*** gbarros has joined #tripleo22:29
*** owalsh has joined #tripleo22:29
*** slaweq has joined #tripleo22:32
*** vinaykns has quit IRC22:35
*** slaweq has quit IRC22:37
*** khyr0n has quit IRC22:39
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Fix all-nodes.sh for python3  https://review.openstack.org/59691722:43
openstackgerritMerged openstack/paunch master: Adjust tenacity mock  https://review.openstack.org/59681622:47
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: add rocky release file for centos  https://review.openstack.org/59691922:48
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Collect local ansible config  https://review.openstack.org/59692022:49
*** bnemec has quit IRC23:00
*** bnemec has joined #tripleo23:01
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements master: Add python3 heat agent for Fedora  https://review.openstack.org/59649823:04
*** tosky has quit IRC23:08
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676423:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178791023:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178890723:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)23:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)23:10
openstackLaunchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Critical,In progress] - Assigned to Ben Nemec (bnemec)23:10
*** jcoufal has quit IRC23:16
mwhahahastevebaker: so i +A'd the change to add the image prepare service, when it lands we should backport it to rocky and have EmilienM cut an rc2 with the THT/puppet-tripleo bits later in the week. We'll need to make sure the designate bits properly land in rocky too23:19
stevebakermwhahaha: ok, thanks23:19
*** gbarros has quit IRC23:25
*** Jeffrey4l has quit IRC23:26
*** Jeffrey4l has joined #tripleo23:28
openstackgerritMerged openstack/tripleo-common master: Adjust tenacity mock  https://review.openstack.org/59647123:35
openstackgerritMerged openstack/tripleo-common master: Do not require username/password when enrolling Redfish nodes  https://review.openstack.org/59177723:35
*** gbarros has joined #tripleo23:41
*** gbarros has quit IRC23:45
stevebakermwhahaha: let me see if I can come up with a better tenacity sleep mocking pattern https://github.com/jd/tenacity/issues/10623:46
mwhahahastevebaker: k I kinda think we shouldn't be testing the wait functionality in the unit tests and let that coverage live in tenacity itself. But I patched to fix ci23:52
stevebakermwhahaha: the intent is to test the failure paths of the functions, and to ensure the right exceptions get reraised23:53
mwhahahastevebaker: yea it's just a really awkward mock23:55
openstackgerritMerged openstack/paunch master: Refactor compose1.py to welcome new builders  https://review.openstack.org/58931223:56
openstackgerritMerged openstack/paunch master: Refactor the runner to support more than docker  https://review.openstack.org/58999623:56
stevebakermwhahaha: yeah, just need a decent pattern for mocking out sleep and advancing time, so failure tests involving tenacity don't take many seconds23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!