Tuesday, 2018-11-06

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722400:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044500:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429300:00
*** ansmith has joined #tripleo00:00
mwhahahaall hail chrony as our new time keeping overlord00:03
jillr\o/00:05
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176900:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]00:10
*** artom has joined #tripleo00:11
*** lblanchard has quit IRC00:20
openstackgerritAlex Schultz proposed openstack/tripleo-repos master: Fix pep8 errors  https://review.openstack.org/61570000:44
*** phuongnh has joined #tripleo00:44
openstackgerritAlex Schultz proposed openstack/tripleo-repos master: Add fedora support to tripleo-repos  https://review.openstack.org/61569300:51
openstackgerritAlex Schultz proposed openstack-infra/tripleo-ci master: TripleO Container build CI job  https://review.openstack.org/61439200:53
dpeacockmwhahaha: I for one welcome our new time keeping overlord00:59
*** dsneddon has quit IRC01:08
*** mcornea has quit IRC01:08
*** rlandy has quit IRC01:09
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176901:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]01:10
*** mjturek has quit IRC01:24
*** zaneb has joined #tripleo01:31
*** zaneb has quit IRC01:31
*** zaneb has joined #tripleo01:31
*** rh-jelabarre has joined #tripleo01:37
*** tongl has quit IRC01:40
*** phuongnh has quit IRC01:57
*** phuongnh has joined #tripleo01:58
openstackgerrit98k proposed openstack/tripleo-repos master: Don't quote {posargs} in tox.ini  https://review.openstack.org/60915302:00
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176902:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]02:10
openstackgerrit98k proposed openstack/tripleo-common master: fix tox python3 overrides  https://review.openstack.org/60823702:28
openstackgerritMerged openstack/tripleo-quickstart master: Remove openshift-ansible repos  https://review.openstack.org/60899402:29
*** abishop has quit IRC02:42
*** mrsoul has joined #tripleo02:46
*** agopi_ has joined #tripleo03:03
*** agopi has quit IRC03:06
openstackgerritEmilien Macchi proposed openstack/tripleo-common stable/rocky: Delete old tarball from config container on download  https://review.openstack.org/61571303:08
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176903:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]03:10
*** ooolpbot has quit IRC03:10
openstackgerritMerged openstack/tripleo-quickstart master: Switch to OpenShift 3.11  https://review.openstack.org/61067403:22
openstackgerritMerged openstack/tripleo-heat-templates master: Update to OpenShift 3.11 release  https://review.openstack.org/61068203:22
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add vars for setting gluster images for openshift  https://review.openstack.org/61264703:22
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add vars for setting images for openshift 3.11  https://review.openstack.org/61264803:22
openstackgerritNguyen Hai Truong proposed openstack/tripleo-heat-templates master: Switch to stestr  https://review.openstack.org/61571703:44
*** tzumainn has quit IRC03:56
tonybDo I actually need a +1 from 'RDO Third party CI' I have a couple of patches that would be great to land on master (and rocky) this week04:08
mwhahahatonyb: not necessarily04:09
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176904:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]04:10
mwhahahatonyb: if you pass me some links i'll review tomorrow04:12
tonybmwhahaha: Okay that's good to know.  It looks like the latest recheck got a -1 from zuul so I'll work out what that means04:13
tonybmwhahaha: https://review.openstack.org/#/c/580829/ and https://review.openstack.org/#/c/612873/04:13
mwhahahatonyb: i don't think 580829 is related to your patch. that's the second time i've seen that error, probably need to file a bug tomorrow on it. i rechecked and will keep an eye on it04:15
tonybmwhahaha: Thanks.04:15
* mwhahaha wanders off04:16
tonybmwhahaha: I didn't really think it was my bad but I do like to check before I 'recheck'04:16
tonybmwhahaha: have a good night04:16
tonybmwhahaha: Thanks04:16
*** ykarel has joined #tripleo04:20
*** janki has joined #tripleo04:28
*** udesale has joined #tripleo04:55
*** udesale has quit IRC04:58
*** udesale has joined #tripleo04:58
*** udesale has quit IRC04:58
openstackgerritMerged openstack/tripleo-heat-templates master: Set `TraceEnable` directive for apache to 'Off'  https://review.openstack.org/61502805:04
*** udesale has joined #tripleo05:05
openstackgerritMerged openstack/tripleo-common master: Run Mistral workflow to make temporary Swift URLs on upgrade  https://review.openstack.org/61337305:06
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176905:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]05:10
*** ratailor has joined #tripleo05:12
ykarelweshay, jistr EmilienM ^^ is failing post https://review.openstack.org/#/c/611601/2/roles/extras-common/defaults/main.yml05:12
ykarelperiodic jobs are also affected ^^, adding promotion-blocker tag05:12
openstackgerritMerged openstack/tripleo-heat-templates master: Use container_cli for post_upgrade_tasks & external_upgrade_tasks  https://review.openstack.org/61560105:13
*** jtomasek has joined #tripleo05:14
openstackgerrit98k proposed openstack/tripleo-heat-templates master: fix tox python3 overrides  https://review.openstack.org/60823905:19
openstackgerrit98k proposed openstack/tripleo-validations master: fix tox python3 overrides  https://review.openstack.org/60824105:20
*** agopi_ has quit IRC05:22
*** ramishra has joined #tripleo05:29
*** rascasoft has quit IRC05:32
*** rh-jelabarre has quit IRC05:32
jaosoriorgood morning!05:32
*** nhicher has quit IRC05:33
*** rascasoft has joined #tripleo05:35
*** udesale has quit IRC05:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Flatten Keystone service configuration  https://review.openstack.org/61555205:43
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Revert "Smarter containerization defaults"  https://review.openstack.org/61572605:48
*** udesale has joined #tripleo05:53
ykareljaosorior, may be can just fix hostname.yml against ^^05:55
ykarelfix can be append undercloud_cloud_domain if hostnamectl --static doesn't contain "."05:57
ykarelmmm but undercloud_cloud_domain has no default05:59
jaosoriorykarel: is undercloud_cloud_domain defined by default?05:59
jaosoriorykarel: that's what I was looking for05:59
jaosoriorouhm...05:59
ykarelhmm just checked has no default05:59
jaosoriorI'll add it05:59
ykarelokk06:00
ykarelmay be default is not set intentionally06:00
jaosoriorykarel: apparently it is intentional06:03
ykarelokk then instead of fixing in hurry, revert seems better now until correct fix is in place06:04
jaosoriorykarel: still working on the fix though06:04
ykarelcool06:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Set undercloud hostname from ansible variables  https://review.openstack.org/61573006:10
jaosoriorykarel: lets see if this does the trick ^^06:10
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176906:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]06:10
*** ooolpbot has quit IRC06:10
ykareljaosorior, ack06:13
ykareljaosorior, good to test the failing job with the patch06:17
ykareljaosorior, also ansible_domain: "" so ansible_domain not defined condition would not work06:20
ykareli am referring variables from a job: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-rocky-containers-build/4e679dc/logs/undercloud/var/log/extra/dump_variables_vars.json.txt.gz06:22
openstackgerritFilippo Inzaghi proposed openstack-infra/tripleo-ci master: fix tox python3 overrides  https://review.openstack.org/60561506:24
Tenguhello there06:25
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Add podman support to validate-tempest role  https://review.openstack.org/60535606:33
Tengucan anyone point me the doc for the instackenv.json content? Can't manage to find it... dtantsur|afk ?06:33
openstackgerritRabi Mishra proposed openstack/tripleo-heat-templates stable/rocky: Set `TraceEnable` directive for apache to 'Off'  https://review.openstack.org/61573306:34
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Set undercloud hostname from ansible variables  https://review.openstack.org/61573006:35
jaosoriorykarel: what about this? ^^06:35
ykarellooking06:35
*** jtomasek has quit IRC06:39
ykareljaosorior, looks good now, can u also cover other cleanup comments06:39
*** agurenko has joined #tripleo06:40
ykarelmove common condition to block and ansible_fqdn --> ansible_hostname when ansible_domain == ""06:40
ykareland fire up a test patch to test the failing job06:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Set undercloud hostname from ansible variables  https://review.openstack.org/61573006:44
jaosoriorykarel: done ^^06:44
jaosoriorcan you do the test patch?06:44
*** ksambor has joined #tripleo06:46
ykareljaosorior, okk will do, but just realized first condition should go in last06:46
jaosoriorykarel: which? and why?06:46
ykarelso if undercloud_cloud_domain is defined then 3 conditional also goes to true06:46
jaosoriorykarel: oh, you're right06:47
*** saneax has joined #tripleo06:47
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Set undercloud hostname from ansible variables  https://review.openstack.org/61573006:47
jaosoriorykarel: fixed ^^06:47
ykarelexcellent06:48
*** quiquell|off is now known as quiquell06:49
*** phuongnh has quit IRC06:51
*** jfrancoa has joined #tripleo06:51
*** phuongnh has joined #tripleo06:52
*** udesale has quit IRC06:54
openstackgerrityatin proposed openstack/tripleo-common stable/rocky: [DNM] Testing https://review.openstack.org/#/c/615730/  https://review.openstack.org/61574206:56
ykareljaosorior, testing ^^06:56
ykarelrunning only minimal set of jobs, mainly checking tripleo-ci-centos-7-undercloud-oooq which is reported in bug06:57
*** huynq has joined #tripleo07:04
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176907:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]07:10
*** ooolpbot has quit IRC07:10
*** mschuppert has joined #tripleo07:19
*** iurygregory|home is now known as iurygregory07:19
openstackgerritCédric Jeanneret proposed openstack/tripleo-quickstart master: Move standalone job to podman instead of docker  https://review.openstack.org/61397507:22
openstackgerritCédric Jeanneret proposed openstack/tripleo-quickstart master: Move standalone job to podman instead of docker  https://review.openstack.org/61397507:22
*** yprokule has joined #tripleo07:24
*** aufi has joined #tripleo07:25
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add MTU to neutron networks and nic-config templates  https://review.openstack.org/59959307:30
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Use mtu of Tenant network to control neutron global_physnet_mtu  https://review.openstack.org/61253307:30
*** udesale has joined #tripleo07:35
*** pcaruana has joined #tripleo07:36
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514007:41
*** paramite has joined #tripleo07:45
*** gkadam_ has joined #tripleo07:45
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Add support for running tempest tests serially  https://review.openstack.org/61576407:49
*** quiquell is now known as quiquell|brb07:50
*** ade_lee has quit IRC07:52
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514007:55
jaosoriorykarel: seems the undercloud job failed http://logs.openstack.org/42/615742/1/check/tripleo-ci-centos-7-undercloud-oooq/c2c9924/07:56
jaosoriorseems the hostname was set with a domain: undercloud_hostname = centos-7-vexxhost-sjc1-0000358513.tripleodomain.example.com07:57
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: [WIP] testing tempest run serially  https://review.openstack.org/60604607:57
jaosoriorand the undercloud install passed07:57
jaosoriorthe problem was tempest07:57
ykarelyes, undercloud install passed and it failed at tempest07:57
jaosoriorhttp://logs.openstack.org/42/615742/1/check/tripleo-ci-centos-7-undercloud-oooq/c2c9924/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-11-06_07_47_5507:58
jaosoriorwell... that's trange IOError: [Errno 13] Permission denied: '/var/log/tempest/tempest.log'07:58
ykarelhttp://logs.openstack.org/42/615742/1/check/tripleo-ci-centos-7-undercloud-oooq/c2c9924/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-11-06_07_47_5507:58
ykarelyes07:58
ykareldamn07:58
ykarelthis was seen till now in some ovb jobs, but now here07:58
chandankumarykarel: one min07:58
jaosoriorthat seems unrelated to this issue though07:58
ykarelchandankumar, ^07:59
ykareljaosorior, yup07:59
chandankumarykarel: which fs it is?07:59
*** florianf has joined #tripleo08:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044508:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429808:00
ykarelchandankumar, 00308:00
ykarelchandankumar, good to see why it's occurring rather than fixing featureset by featureset, we might miss something else08:00
ykareli see there are different patches for different featuresets08:01
*** iurygregory is now known as iurygregory|mtg08:01
chandankumarykarel: it is a mess08:01
ykarelseeing https://bugs.launchpad.net/tripleo/+bug/180068208:02
openstackLaunchpad bug 1800682 in tripleo "tempest: import errors on master" [Critical,Fix released] - Assigned to wes hayutin (weshayutin)08:02
jaosoriorykarel, chandankumar: so... what';s going on there?08:02
*** ccamacho has joined #tripleo08:03
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Run tempest from package in non_containerized env  https://review.openstack.org/61576608:04
chandankumarjaosorior: ykarel ^^08:04
ykareljaosorior, /me don't know may be bug has sufficient info08:04
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Run tempest from package in non_containerized env  https://review.openstack.org/61576608:04
chandankumarykarel: jaosorior I am thinking of adding validation on fs changes08:05
chandankumarit is much needed08:05
jaosoriorchandankumar: what validations do you have in mind?08:05
*** cylopez has joined #tripleo08:06
*** kopecmartin|off is now known as kopecmartin08:06
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade stable/rocky: Set containerized_undercloud_upgrade to true  https://review.openstack.org/61551108:06
chandankumarjaosorior: for tempest, we can add a validation if there is a containerized undercloud then undercloud_enable_tempest should be true and tempest format should be container08:06
chandankumarlet me put some code for the same08:07
openstackgerritMartin Schuppert proposed openstack/tripleo-common master: Fix nova-api healthcheck  https://review.openstack.org/61397108:08
jaosoriorchandankumar: that sounds like a good idea08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176908:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]08:10
*** paramite has quit IRC08:10
therved0ugal: Hi, I have a question about the send message workflow when you're around08:13
ykarelchandankumar, so only fs003 is non containerized as per https://review.openstack.org/#/c/615766?08:14
*** quiquell|brb is now known as quiquell08:14
jaosoriorhjensas: seen this before?08:14
jaosorior2018-11-05 16:07:15.574 8821 WARNING tripleoclient.v1.tripleo_deploy.Deploy [  ]         "Error: /Stage[main]/Ironic::Pxe/Ironic::Pxe::Tftpboot_file[pxelinux.0]/File[/var/lib/ironic/tftpboot/pxelinux.0]: Could not evaluate: Could not retrieve information from environment production source(s) file:/usr/share/syslinux/pxelinux.0",08:14
jaosorior2018-11-05 16:07:15.574 8821 WARNING tripleoclient.v1.tripleo_deploy.Deploy [  ]         "Error: /Stage[main]/Ironic::Pxe/Ironic::Pxe::Tftpboot_file[chain.c32]/File[/var/lib/ironic/tftpboot/chain.c32]: Could not evaluate: Could not retrieve information from environment production source(s) file:/usr/share/syslinux/chain.c32",08:14
chandankumarykarel: that brings another question containerized_undercloud becomes true there?08:14
jaosoriordtantsur|afk: ^^08:14
ykarelchandankumar, https://review.openstack.org/#/c/611601/2/roles/extras-common/defaults/main.yml made it08:15
hjensasjaosorior: no, is syslinux package missing from the container?08:15
*** ramishra has quit IRC08:15
chandankumarykarel: I need to push one more fix containerized_undercloud: to false08:16
chandankumarotherwise we will again got screwed08:16
ykarelchandankumar, so it was identified fs050 : https://review.openstack.org/#/c/615518/1/config/general_config/featureset050.yml08:16
ykarelit was changed only for 50, don't know why other fs were not considered08:16
jaosoriorhjensas: any idea what image I should check?08:16
chandankumarykarel: 47 is also needed changes08:16
ykarelmay be jistr knows ^^08:16
jaosoriorhjensas: trying to install the undercloud and it fails with that... so there is no running container available. But I could check the image.08:17
chandankumaras 50 is inherited from 4708:17
ykarelchandankumar, yup all should go together, there should be mechanism to find that08:17
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Run tempest from package in non_containerized env  https://review.openstack.org/61576608:18
hjensasjaosorior: I think it would be - centos-binary-ironic-pxe08:18
chandankumarykarel: ^^ let's fix this then I need to grep where is containerized adn non-containerized and due to mass updae08:18
*** ykarel_ has joined #tripleo08:19
jaosoriorhjensas: is that the name of the package? syslinux?08:20
jaosoriorhjensas: syslinux-tftpboot is installed in the iamge08:21
hjensasjaosorior: those files are in syslinux-4.05-13.el7.x86_64 - syslinux-tftpboot package installs the files under /var/lib/tftpboot08:22
*** ykarel has quit IRC08:22
jaosoriorhjensas: yep, syslinux is not installed there, just syslinux-tftpboot08:23
*** ramishra has joined #tripleo08:24
openstackgerritMartin André proposed openstack/tripleo-common stable/rocky: Store prepare defaults in packaged yaml  https://review.openstack.org/61577508:31
openstackgerritMartin André proposed openstack/tripleo-common stable/rocky: Add container images for openshift 3.11  https://review.openstack.org/61577608:31
jaosoriorykarel_: so, do you think then that (after the CI run) https://review.openstack.org/#/c/615730/ is ready to go>08:35
jaosorior?08:35
ykarel_jaosorior, to me it looks good, but until the tempest issue is clear we are blocked08:36
openstackgerritMartin André proposed openstack/tripleo-heat-templates stable/rocky: Update to OpenShift 3.11 release  https://review.openstack.org/61577808:36
ykarel_but promotion job might be clear as there fs003 is not running, not sure other featureset are affected or not08:37
*** jtomasek has joined #tripleo08:41
Tengufor those interested in the small sync call today about the validation framework, it's in ~3 minutes here: https://bluejeans.com/328497573  ( florianf gchamoul - apparently that will be just the three of us ^^')08:42
gchamoulTengu: already in08:42
Tengu\o/08:43
*** amoralej|off is now known as amoralej08:44
*** cylopez has left #tripleo08:46
*** jpena|off is now known as jpena08:47
*** akrivoka has joined #tripleo08:50
*** bogdando has joined #tripleo08:56
*** jpich has joined #tripleo08:57
evrardjpo/08:58
*** betherly has joined #tripleo09:01
*** paramite has joined #tripleo09:04
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514009:05
quiquellmarios: does it look better ? ^09:06
*** ratailor_ has joined #tripleo09:08
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514009:09
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514009:09
jaosoriorhjensas: https://review.openstack.org/#/c/615709/09:09
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Use correct package manager at DLRN  https://review.openstack.org/61552809:09
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176909:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]09:10
*** ykarel__ has joined #tripleo09:11
*** ratailor has quit IRC09:11
*** ykarel_ has quit IRC09:12
hjensasjaosorior: did we have that dowstream already?09:12
jaosoriorhjensas: unsure09:13
jaosoriorhjensas: I mean, I think we don't have that downstream and that's why it's failing.09:13
*** itlinux has joined #tripleo09:14
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Use correct package manager at DLRN  https://review.openstack.org/61552809:14
openstackgerritMerged openstack/puppet-tripleo master: Make sure that the _member_ role is assigned to admin.  https://review.openstack.org/61191909:17
openstackgerritMerged openstack/puppet-tripleo master: Fix wrapper containers for podman w/o sockets  https://review.openstack.org/60609509:17
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add time for overcloud deletion  https://review.openstack.org/61553309:17
hjensasjaosorior: hmm, I don't think so. We should check with bfournier later?09:17
hjensasjaosorior: however, if we do that in puppet-ironic - we probably need to update https://github.com/openstack/puppet-ironic/blob/master/manifests/params.pp#L67-L68 as well.09:17
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Use correct package manager at DLRN  https://review.openstack.org/61552809:18
*** ykarel__ is now known as ykarel09:19
hjensasjaosorior: oops, yeah so that patch does change the path as well.09:20
*** gfidente has joined #tripleo09:20
mariosquiquell: ack09:22
*** huynq has quit IRC09:25
*** huynq has joined #tripleo09:25
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547909:29
*** aedc has joined #tripleo09:33
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Collect dnf logs and config  https://review.openstack.org/61580009:34
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Collect dnf logs and config  https://review.openstack.org/61580009:34
*** derekh has joined #tripleo09:39
ykarelquiquell, can u try my comment https://review.openstack.org/#/c/615235/109:42
ykarelto get standalone issue for puppet-designate resolved09:42
ykarelalso u need to fire a similar patch for saharaclient if not already done09:42
quiquellykarel: Weird thing f28 is still failing at puppet-designate09:42
*** chem has joined #tripleo09:43
naturalbluequick query if anyone has a moment.09:45
*** sshnaidm|afk is now known as sshnaidm|rover09:45
ykarelquiquell, yes i commented the fix for it,, puppet-designate has issue09:46
ykarelin ::designate::client class09:46
quiquellykarel: ack, let me fix it09:46
sshnaidm|rovermarios, panda please review https://review.openstack.org/#/c/615515/09:47
naturalbluein the undercloud.conf for rocky. If I have undercloud_public_host = <theshotname> I get an error of "error configuring mysql_init_tasks" but if I use a IP address it builds undercloud fine. I had believed I needed to have the undercloud_public_host as a hostname name if I wanted to generate a certificate and use tripleo UI09:48
quiquellykarel: something like this will do ? https://paste.fedoraproject.org/paste/IoOFafZeEa0d~Zgd8-BMYg09:52
ykarelquiquell, nope09:53
ykarelquiquell, u need to fix https://github.com/openstack/puppet-designate/blob/master/manifests/client.pp#L1709:55
quiquellI was changing the spec that are the testing09:56
ykarelquiquell, nope no need to change those spec atlest for this09:57
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Fail fast if local clone is missing on include-gate-changes.sh  https://review.openstack.org/61580509:57
quiquellykarel: yep I know09:57
quiquellykarel: so this ? https://paste.fedoraproject.org/paste/dk6fg5-esJKa4nVVl345dw09:58
jistrykarel, chandankumar: i think if everything works as it should, no FS should need changing when we landed the smart defaults. The fact that fs50 needed change is a hint that fs50 didn't work as expected. And it still doesn't -- we say `containerized_undercloud: false` in fs50 but it *does* deploy containerized undercloud.09:58
jistris there a problem with another featureset?09:59
ykarelquiquell, yes that is okk to try but that would replace the override feature for designate::client09:59
ykarelquiquell, u can try https://review.rdoproject.org/paste/show/164/09:59
ykareljistr, atleast fs003 got some issue today and fs02310:00
ykarelmay be others also affected but don't how about that10:01
quiquellykarel: puppet question, why do we need to inheritence since line 9 is already reading the value from params ?10:01
ykarelquiquell, /me don't know much but that's required if u are trying to use class from other variables when setting default10:02
ykarelin class body it works fine but parameters it's required10:02
quiquellykarel: Ahh ok is the default value of an function argument10:03
quiquellok ok10:03
*** arxcruz has quit IRC10:03
quiquellykarel: parameter will override the value from inherintance ?10:03
quiquellykarel: ok adding a ps to mwhahaha review10:04
ykarelso if nothing passed to designate::client use default ::designate::params::client_package_name, if passed override the default10:04
ykarelquiquell, ack good to try10:05
*** aedc has quit IRC10:05
ykarelquiquell, also fix the commit message10:05
*** aedc has joined #tripleo10:05
chandankumarjistr: https://review.openstack.org/#/q/topic:bug/1800682+(status:open+OR+status:merged)10:06
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547910:06
quiquellykarel: testing it here ^10:06
ykarelquiquell, ack10:06
ykarelatleast this should pass ahead of designateclient10:07
ykarelquiquell, u also need to fix sahara as seen in the logs10:07
quiquellykarel: Similar stuff I suppose10:07
quiquellykarel: let's do that maybe we can have a working hash not that lon g10:08
ykarelquiquell, yes similar changes are required10:08
chandankumarjistr: sorry you mean to say setting `containerized_undercloud: false` in fs50 still deploys containerized undercloud?10:08
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176910:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]10:10
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Add podman support to validate-tempest role  https://review.openstack.org/60535610:12
jistrchandankumar: yes, there is no way to deploy non-containerized in Rocky and further AFAIK. So if i understood correctly, with `containerized_undercloud: false` we deploy containerized, and with `containerized_undercloud: true` the job breaks on some error that "hostname isn't fully qualified" -- we didn't investigate that yet.10:12
*** paramite has quit IRC10:12
*** ykarel is now known as ykarel|lunch10:12
chandankumarjistr: yes in some cases, https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset003.yml#L33 because of this10:13
sshnaidm|rovermarios, panda and this one please: https://review.openstack.org/#/c/615357/10:17
chandankumarsshnaidm|rover: one more issue related to tempest container https://review.openstack.org/61576610:18
quiquellykarel|lunch: https://review.openstack.org/61581210:19
chandankumarjistr: nope, in FS if `containerized_undercloud: false` is not passed, then the jobs were failing10:20
*** iurygregory|mtg is now known as iurygregory10:20
ykarel|lunchquiquell, inherit thing not required in ^^10:20
quiquellykarel|lunch: Ups, right there is no argument there10:21
quiquelland it's already using params10:21
quiquellfor the name10:21
quiquellok thank s10:21
*** gfidente has quit IRC10:21
openstackgerritThomas Herve proposed openstack/tripleo-common master: Fix config-download timeout  https://review.openstack.org/61565310:22
jistrchandankumar: yea exactly. So i think we should set temporarily `containerized_undercloud: false` in the specific FS that are failing, and figure out how we can make the variable match reality.10:22
jistrchandankumar: setting `containerized_undercloud: false` should only be a stop gap solution10:23
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547910:23
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Revert "Revert "Convert *tasks from bootstrap_nodeid to short_bootstrap_node_name""  https://review.openstack.org/61180010:27
*** salmankhan has joined #tripleo10:32
*** ykarel|lunch is now known as ykarel10:33
*** arxcruz has joined #tripleo10:36
*** iurygregory is now known as iury|lunch10:41
*** phuongnh has quit IRC10:42
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo stable/rocky: Make sure that the _member_ role is assigned to admin.  https://review.openstack.org/61582110:45
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: add zuul.projects and executed playbooks into reproducer  https://review.openstack.org/61357210:54
openstackgerritNatal Ngétal proposed openstack/tripleo-common stable/pike: [Pike only] Fix docker registry url.  https://review.openstack.org/61159610:56
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154011:00
*** huynq has quit IRC11:03
openstackgerritNatal Ngétal proposed openstack/tripleo-common stable/pike: [Queens only] Fix syntax error.  https://review.openstack.org/61582711:03
openstackgerritNatal Ngétal proposed openstack/tripleo-common stable/pike: [Pike only] Fix syntax error.  https://review.openstack.org/61582711:05
*** florianf is now known as florianf|lunch11:06
Hobbestigrouhi11:07
Hobbestigrouin fact, the syntax error was also present in pike, i have cherry-pick the commmit and update the commit message https://review.openstack.org/#/c/615827/11:07
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176911:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]11:10
openstackgerritJiri Stransky proposed openstack/python-tripleoclient stable/rocky: Always run upgrades/updates as tripleo-admin  https://review.openstack.org/61582911:11
*** iury|lunch is now known as iurygregory11:11
ykarelquiquell, looks like standalone job failed early: http://logs.openstack.org/79/615479/11/check/tripleo-ci-fedora-28-standalone/1a5a66a/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz11:13
*** aedc has quit IRC11:13
quiquellykarel: I have update hashes (to get some fixes) so maybe we have new issues11:13
ykarelquiquell, ack11:14
quiquellykarel: have to talk with panda about it, what to do with hashes11:14
ykarelquiquell, ack11:14
quiquellykarel: btw    "NameError: name \'basestring\' is not defined",' is python3 issue11:16
quiquellykarel: at py3 we are upose to use six or str11:16
ykarelhmm looks like11:16
ykarelbut what changed, heat-agents?11:17
quiquellykarel: yep heat-templates is ok but heat-agents not11:18
quiquellhttp://codesearch.openstack.org/?q=hot_input&i=nope&files=&repos=11:18
ykarelyup https://github.com/openstack/heat-agents/blob/235e1ae262c873cbe87e893363b988aa73bf0195/heat-config/os-refresh-config/configure.d/55-heat-config#L9611:18
quiquellykarel: new commit ?11:18
*** gfidente has joined #tripleo11:18
ykarelquiquell, heat-agent recently fixed python2 to python311:19
ykareliirc u only pushed the patch11:19
ykarelin spec11:19
quiquellykarel: yep, i have change hashes to use it, that' swhy :-)11:19
quiquellykarel: so it's 'legit'11:19
ykarelyup11:19
quiquelllet's fix it11:19
ykarelso need to fix heat-agents11:19
ykarelyes11:19
ykarelgo for it11:19
quiquellsure11:19
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Fix multiple-nic sample template  https://review.openstack.org/61368311:20
shardyHobbestigrou: Hi, can you clarify why wasn't that a backport of https://github.com/openstack/tripleo-common/commit/cd2532fd8e9ed7dcc99fd82efe30b2fc3abf0ca6#diff-9c5f38d2dfc069a3d9db879b5d84d67f ?11:20
shardyjistr, chem, ccamacho: ^^11:21
quiquellykarel: https://review.openstack.org/#/c/615831/11:23
chemshardy: well I guess we overlooked the fact that it was fixed in master11:23
shardyOk, Not a big deal but I get nervous whenever I see e.g Queens only patches :)11:23
shardythe first thing we do should be to check if it's fixed in master IMO11:23
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547911:23
quiquellykarel: ^ let's see where we are now11:24
chemshardy: yeah, right, didn't pay enough attention as the fix was obvious, but that's definitively a good advice, thanks.11:24
quiquellchem, jistr: we can workflow this https://review.openstack.org/#/c/611842/11:25
ykarelquiquell, ack11:25
chemquiquell: done11:26
quiquellchem: thanks11:27
quiquellchem: Do you need some reviewing on some of the upgrade thing ?11:27
*** brault has joined #tripleo11:28
chemquiquell: well, I just realized I forgot to hit the reply button on your questions for https://review.openstack.org/#/c/604736/11:28
quiquellchem: jeje ok11:29
*** ramishra has quit IRC11:30
*** dtantsur|afk is now known as dtantsur11:33
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Collect dnf logs and config  https://review.openstack.org/61580011:34
dtantsurjaosorior: sorry, what's the question?11:35
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547911:35
dtantsurTengu: http://tripleo.org/install/environments/baremetal.html#instackenv-json11:35
chemquiquell: I've answered there https://review.openstack.org/#/c/604736/, hope you like it :)11:35
openstackgerrityatin proposed openstack/tripleo-common stable/rocky: [DNM] Testing https://review.openstack.org/#/c/615730/  https://review.openstack.org/61574211:36
quiquellchem: so maybe we can remove the standalone_virt_type all together ?11:37
pandahashes to hashes, dust to dust11:38
chemquiquell: well the indirection may prove useful if somebody want to use the role and override that value, beside ...11:38
*** bfournie has joined #tripleo11:39
chemquiquell: I would like to keep the diff -ruN between standalone and standalone-upgrade to a minimum11:39
quiquellchem: ok fair enough, all good then11:39
chemquiquell: \o/, getting my bottle of champagne ready for friday!11:40
quiquellchem: Don't add reviews then :-)11:40
chemquiquell: well if we could merged that one https://review.openstack.org/#/c/612979/, it's a very easy one11:42
Tengudtantsur: hmm, I'm pretty sure there were another one, more complete.11:42
dtantsurI don't think so11:43
chemquiquell: that would be worth a bottle of beer11:43
Tengudtantsur: I'm fighting with multi-disk VMs, apparently there are some issues setting the root_device properly :(11:43
ccamachoshardy sorry for that11:46
ccamachoyou are right there11:46
*** gkadam_ has quit IRC11:46
*** gkadam has joined #tripleo11:46
*** dprince has joined #tripleo11:49
quiquellmarios: I think this is compatible with refactoring of the reproducer https://review.openstack.org/#/c/61297911:53
quiquellchem: ^11:53
*** raildo has joined #tripleo11:54
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Remove obsoleted code  https://review.openstack.org/61093711:55
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Remove obsoleted code  https://review.openstack.org/61093711:55
*** eck` is now known as eck`gone11:56
openstackgerritRadoslaw Smigielski proposed openstack/tripleo-upgrade master: Simplify some scripts and remove some unnecessary jq usage  https://review.openstack.org/61584411:56
*** hkominos has joined #tripleo11:57
hkominosHi all. Is this the right channel to ask about tripleo-common ?11:58
*** paramite has joined #tripleo11:59
*** janki has quit IRC12:00
*** amoralej is now known as amoralej|lunch12:01
*** paramite has quit IRC12:05
*** paramite|clone has joined #tripleo12:06
*** akrivoka has quit IRC12:07
*** florianf|lunch is now known as florianf12:08
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Collect dnf logs and config  https://review.openstack.org/61580012:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176912:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]12:10
*** ooolpbot has quit IRC12:10
*** paramite|clone has quit IRC12:15
*** paramite has joined #tripleo12:16
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514012:21
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547912:21
quiquellykarel: Humm heat-agents failing :-/ will check tox12:23
jistrshardy, chem: yup sorry, i didn't pay enough attention to where the fix was going12:23
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: add zuul.projects and executed playbooks into reproducer  https://review.openstack.org/61357212:23
ykarelquiquell, ack12:24
*** paramite|clone has joined #tripleo12:24
*** paramite has quit IRC12:24
ykarelquiquell, so earlier patches on heat-agents are also failing12:25
ykarelso not specific to ur patch, but good to fix12:25
*** gfidente has quit IRC12:26
naturalblueIf I was trying to have the overcloud environment machine have a routable/accessible public address as well as the provision/ctrplane address, would I used something like https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/routed_spine_leaf_network.html or is there just a simple setting in undercloud.conf to setup NIC1 as mgmt/public and NIC2 as ctrplane/provision.12:26
naturalblueI have a section in undercloud.conf called [ctlplane-subnet] , can i setup a [mgmt-subnet] section12:27
*** ratailor_ has quit IRC12:29
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Move UndercloudPostDeployment to python  https://review.openstack.org/61454012:29
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud  https://review.openstack.org/61585312:30
*** paramite has joined #tripleo12:30
EmilienMchandankumar: hey, do you have any update on tempest container for podman? Do you need help?12:30
chandankumarEmilienM: I think it is ready https://review.openstack.org/605356 and http://logs.openstack.org/56/605356/55/check/tripleo-ci-centos-7-undercloud-containers/3ca2118/logs/tempest.html.gz12:32
EmilienMchandankumar: good news then I'll wait for CI results :D12:33
EmilienMchandankumar: nice work!12:33
*** bcafarel has quit IRC12:33
honzajrist: with the reproducer script in rdo cloud, as per the bug; no zuul changes, dlrn hash set to current-tripleo12:34
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud  https://review.openstack.org/61585312:34
*** paramite|clone has quit IRC12:34
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Update fedora28 release file current dlrn hash  https://review.openstack.org/61514012:35
*** paramite|clone has joined #tripleo12:35
*** paramite has quit IRC12:35
*** jpena is now known as jpena|lunch12:38
*** ansmith has quit IRC12:39
EmilienMjaosorior: please review https://review.openstack.org/#/c/615200/ when you have time12:40
EmilienMjaosorior: nevermind I did a typo in doc12:40
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Implement tripleo-docker-rm role  https://review.openstack.org/61520012:42
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Implement tripleo-docker-rm role  https://review.openstack.org/61520012:42
*** snecklifter has quit IRC12:42
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - upgrade: remove Docker containers now managed by Podman  https://review.openstack.org/61520912:42
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - upgrade: remove Docker containers now managed by Podman  https://review.openstack.org/61520912:42
*** gfidente has joined #tripleo12:45
jristhonza: I thought you had to do it against a particular review12:50
honzajrist: nope, you can set it to an empty string12:50
*** bcafarel has joined #tripleo12:51
jristok12:52
jristI wonder if someone from keystone could help, honza12:53
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart master: Remove developer tools from tripleo .gitignore  https://review.openstack.org/61558912:53
openstackgerritMarios Andreou proposed openstack/python-tripleoclient master: Remove developer tools from tripleo .gitignore  https://review.openstack.org/61585712:53
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Remove developer tools from tripleo .gitignore  https://review.openstack.org/61559012:53
*** aedc has joined #tripleo12:53
*** akrivoka has joined #tripleo12:56
*** abishop has joined #tripleo12:57
*** akrivoka has quit IRC12:58
*** akrivoka has joined #tripleo12:58
*** rlandy has joined #tripleo13:02
*** cdearborn has joined #tripleo13:06
*** ooolpbot has joined #tripleo13:10
honzachandankumar: hey, nice work on the git patch, looks like it's on its way to be merged; who else with +2 could we ping to push this over the edge?  https://review.openstack.org/#/c/612377/13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176913:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]13:10
*** ooolpbot has quit IRC13:10
*** ykarel_ has joined #tripleo13:11
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Add option to install chrome when running tempest  https://review.openstack.org/60732313:11
honza'git patch' /me needs moar coffee13:11
chandankumarhonza: ^^ we need this one to pass13:12
chandankumarmy patch is dependent on that13:12
chandankumarhonza: happy helping :-)13:12
quiquelljoin #openstack-heat13:12
quiquellups13:12
*** eck`gone is now known as eck`13:13
*** ykarel has quit IRC13:14
honzachandankumar: oh!  thanks for clarifying that, i'm clearly not awake yet13:14
chandankumarhonza: from tomorrow I am on pto till this week, feel free to poke arxcruz kopecmartin they are happy to help on this :-)13:15
*** quiquell is now known as quiquell|lunch13:15
honzachandankumar: perfect13:15
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/queens: Increase the deploy_plan timeout in tripleoclient  https://review.openstack.org/61586613:16
*** quiquell|lunch is now known as quiquell13:17
quiquelljaosorior: Do you know stuff about openstack/heat-agents ?13:17
quiquelljaosorior: I think tox is broken there13:17
jaosoriornot really13:17
jaosoriortherve: ^13:18
quiquelltherve: I have some changes at openstack/heat-agents failing, but it's non related, testing a noop now13:18
quiquelltherve: I think mocking HEAT_DOCKER_CMD is not working13:19
*** amoralej|lunch is now known as amoralej13:19
*** thrash|g0ne is now known as thrash13:19
thervequiquell: OK, looking. Do you have a trace?13:20
quiquelltherve: https://review.openstack.org/#/c/615831/13:21
quiquelltherve: Also has a 'noop' review to test tox13:21
quiquelltherve: https://review.openstack.org/#/c/615865/13:21
*** ykarel_ is now known as ykarel13:21
*** jcoufal has joined #tripleo13:23
jristhonza: asked lbragstad in keystone https://bugs.launchpad.net/tripleo/+bug/1801778/comments/113:23
openstackLaunchpad bug 1801778 in tripleo "Keystone circular reference on OPTIONS" [High,Triaged]13:23
honzajrist: thanks13:24
thervequiquell: paunch breakage it seems13:24
quiquelltherve: damn13:24
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Create a reproducer-quickstart dir  https://review.openstack.org/61563513:24
quiquelltherve: How did you found it ?13:24
thervequiquell: print stderr before the test failures13:24
therveThat's EmilienM's fault13:25
therveEmilienM: Don't break compatibility dude13:25
quiquellEmilienM: Is a cowboy13:25
*** quiquell is now known as quiquell|lunch13:26
EmilienMuh13:26
therveEmilienM: https://review.openstack.org/#/c/601290/13:27
therveYou renamed parameters13:27
EmilienMright13:27
EmilienMtherve: I didn't know it would affect heat-agents13:28
*** jpena|lunch is now known as jpena13:28
therveEmilienM: It affects any user of the library?13:28
therveGranted, there aren't many13:28
EmilienMtherve: I didn't know heat-agents was a user of this library13:28
EmilienMsorry :(13:28
therveThat's ok I still like you13:29
EmilienMdamn :(13:29
quiquell|lunchtherve: Then I can fix heat-agents in the same review ?13:29
thervequiquell|lunch: No let me propose a patch13:30
quiquell|lunchtherve: ack13:30
quiquell|lunchwill do lunch now13:30
*** aufi has quit IRC13:31
quiquell|lunchtherve, EmilienM: Does it make sense to add some heat-agents tests at paunch ?13:33
*** kforde has joined #tripleo13:33
EmilienMquiquell|lunch: yes probably13:34
cdearbornhey guys, I'm running into the request stacking issue that was theoretically addressed by https://review.openstack.org/#/c/59149313:34
cdearbornthat fix works on most 9 node clusters, but i have 1 9 node cluster where request stacking is still happening13:34
cdearborni tried hacking /etc/httpd/conf.d/10-ironic_wsgi.conf, and changing the number of threads to 30 and that seems to solve the problem13:34
cdearbornanyone know if there is a way to configure the number of threads in tripleo, as opposed to hacking the conf file post install?13:34
kfordequestion about overcloud update if someone can clarify for me: when running overcloud update the servers are updated one-by-one, with breakpoints set for each server. With 50+ servers this is incredibly long process. Is it possible that I just remove the breakpoints for the computes, allowing them to update in parallel and keep breakpoints for the controllers. Version = Ocata.13:35
cdearbornIncidentally, this is a pretty severe regression in tripleo functionality.  Prior to switching to wsgi, we had success deploying with 2 racks of equipment, and now deployment seems to work with only 9 overcloud nodes (most of the time)13:37
*** quiquell|lunch is now known as quiquell13:37
jristhonza: can you poke lbragstad on #openstack-keystone ?13:39
honzajrist: yes, we've been chatting13:40
jristsweet13:40
honzajrist: thanks for making the connection13:40
jristyou're welcome13:40
owalshcdearborn: might help - https://review.openstack.org/557825 & https://review.openstack.org/55782913:40
*** radeks has joined #tripleo13:41
*** iurygregory has quit IRC13:41
openstackgerritJohn Fulton proposed openstack/tripleo-common stable/rocky: Run Mistral workflow to make temporary Swift URLs on upgrade  https://review.openstack.org/61480113:41
cdearbornowalsh: thanks for the pointer13:41
*** ykarel has quit IRC13:42
owalshcdearborn: ironic seems to be more affected though, due to sync resuests. Could set ironic::wsgi::apache::threads via ExtraConfig parameter13:42
*** iurygregory has joined #tripleo13:43
owalshcdearborn: https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/node_config.html#node-config13:43
cdearbornowalsh: that was exactly what i was looking for.  Thank you very much!  I really think something more needs to be done for ironic.  between everything that's going on during deployment and power state polls, the ironic api threads are being overwhelmed with only 9 overcloud nodes.  i would think that tripleo would want to support more than a 9 node cluster by default13:45
quiquelltherve: We are missing some linting it should have capture docker_cmd not existing13:45
thervequiquell: You can't do that in python13:46
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Avoids sort: write failed: standard output: Broken pipe with du  https://review.openstack.org/60941313:50
*** paramite|clone has quit IRC13:53
EmilienMchandankumar: +2 on the tempest/podman, thx for your work13:56
*** mcornea has joined #tripleo13:56
*** saneax has quit IRC14:00
*** lblanchard has joined #tripleo14:00
jristguys is tripleo meeting right now or 1 hour?14:01
beaglesdaylight savings ftw14:01
mariosi thought it was now... ?14:02
beaglessame14:02
arxcruzsame14:02
owalshcdearborn: np, have you created an LP for this issue?14:02
thervequiquell: https://review.openstack.org/61588614:03
ssbarneai tought the same as it was saved as GMT in my calendar.14:03
rlandyalso thought it was now14:03
*** aufi has joined #tripleo14:03
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: add zuul.projects and executed playbooks into reproducer  https://review.openstack.org/61357214:03
*** ykarel has joined #tripleo14:03
mariosrlandy: for the love of ... http://logs.openstack.org/72/613572/20/check/tripleo-ci-centos-7-standalone/dcbd11b/logs/quickstart_collect_logs.log i think it might be because i was setting them as fact "AnsibleUndefinedVariable: 'str object' has no attribute 'canonical_name" just posted 21 ^^14:04
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Fix runtime Ansible warnings  https://review.openstack.org/61184414:04
rlandymarios: looking14:04
mariosrlandy: just ranting about the ongoing console playbooks thing (the console json is written fine but my last run failed on that str object issue)14:05
rlandymarios: why should anything be easy????14:05
mariosrlandy: :)14:06
*** tzumainn has joined #tripleo14:07
mwhahahajaosorior: Soo meeting?14:07
fultonjjaosorior: tripleo meeting?14:07
jaosoriormwhahaha: isn't it in one hour?14:07
mwhahahaNot any more14:07
fultonjas per https://wiki.openstack.org/wiki/Meetings/TripleO it's at 14:00 UTC14:07
mwhahaha1400 utc14:07
*** ansmith has joined #tripleo14:07
jaosoriorcrap!14:07
fultonjit's currently 14:07 UTC14:07
mwhahahaWelcome to daylight savings time :D14:07
*** yolanda has joined #tripleo14:08
jaosorior#startmeeting tripleo14:08
openstackMeeting started Tue Nov  6 14:08:09 2018 UTC and is due to finish in 60 minutes.  The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot.14:08
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:08
jaosorior#topic agenda14:08
*** openstack changes topic to " (Meeting topic: tripleo)"14:08
openstackThe meeting name has been set to 'tripleo'14:08
jaosorior* Review past action items14:08
*** openstack changes topic to "agenda (Meeting topic: tripleo)"14:08
jaosorior* One off agenda items14:08
jaosorior* Squad status14:08
jaosorior* Bugs & Blueprints14:08
jaosorior* Projects releases or stable backports14:08
jaosorior* Specs14:08
jaosorior* open discussion14:08
jaosoriorAnyone can use the #link, #action and #info commands, not just the moderatorǃ14:08
jristo/14:08
weshay\014:08
fultonjo/14:08
d0ugalo/14:08
*** agopi_ has joined #tripleo14:09
rlandyo/14:09
ksamboro/14:09
arxcruzo/14:09
shardyo/14:09
hjensaso/14:09
mwhahahahi2u14:09
marioso/14:09
*** mwhahaha sets mode: +o jaosorior14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176914:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]14:10
jaosorior#topic review past action items14:11
*** openstack changes topic to "review past action items (Meeting topic: tripleo)"14:11
jaosoriorThere was one:14:11
jaosorior    jaosorior to poke the squads and check status. Based on this we might need to update the list of active squads.14:11
jaosoriorI didn poke all the squads, so, I'll leave this action active until I do.14:12
jaosoriorat least we did get some updates on some of them14:12
jaosorior#topic one off agenda items14:12
*** openstack changes topic to "one off agenda items (Meeting topic: tripleo)"14:12
jtomaseko/14:12
jaosoriorFirst of them: (sshnaidm) New workflow for having master containers updated with tripleo packages from delorean-current (consistent) repo14:13
quiquello/14:13
sshnaidm|rovero/14:13
*** matbu has joined #tripleo14:13
dpeacockoh crap this is at 09:00 now14:13
honzao/14:13
dpeacocko/14:13
matbuo/14:13
chemo/14:13
*** ramishra has joined #tripleo14:13
jristjaosorior: jtomasek or I will update squad for UI. still active, but busy with releasing14:13
jaosoriorjrist: thanks!14:13
trozetweshay: is there anyway you can restore that queens undercloud.qcow2? our opnfv release is Friday14:14
sshnaidm|roverjaosorior, so we have delorean-current repos for tripleo-* packages that we use in our CI jobs, it's the newest packages. And I suppose we need to have the same in containers to avoid all these long updates14:14
gfidenteo/14:14
trozetweshay: its fine to not have it for rocky and later, but since undercloud isnt containerized for our queens deployments we kind of rely on that image14:15
owalsho/14:15
rfolco|rucko/14:15
sshnaidm|roverjaosorior, so after a time period or after every tripleo-* repo package is built we need to update all our containers with it and to publish on docker.14:15
*** nhicher has joined #tripleo14:16
mwhahahasshnaidm|rover: we might be able to leverage the container CI job I'm working on to do a periodic update14:16
sshnaidm|rovermwhahaha, periodic update to all containers?14:17
mwhahaha#link https://review.openstack.org/#/c/614392/14:17
weshaytrozet, see pm #tripleo meeting atm man14:18
sshnaidm|rovermwhahaha, as we saw in jobs the most time is wasted on containers updates from delorean-current repo, it's about 40 min - 1 hour, which causes timeouts14:18
mwhahahasshnaidm|rover: yea14:18
sshnaidm|rovermwhahaha, so if it's periodic update of all containers - it's actually what I propose :)14:18
mwhahahasshnaidm|rover: it currently isnt but could be adopted to support that14:19
mwhahahasshnaidm|rover: btw container builds are broken due to grafana14:19
mwhahahaIt looks like14:19
* mwhahaha afk for a bit14:19
sshnaidm|rovermwhahaha, not sure I understand14:19
sshnaidm|rovermwhahaha, ok, we can take it offline and discuss later14:19
sshnaidm|roverthat's it for this point from me, if anyone has comments..14:21
jaosoriorsshnaidm|rover: is there any work you hve done to enable this? or were you looking for input first?14:21
sshnaidm|roverjaosorior, not done yet, in planning stage14:21
sshnaidm|roverand looking for input, yes14:22
hjensashow much time do we spend on updating containers when packages are up-to-date in the containers?14:22
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: (Partial) Revert "Use container_cli for post_upgrade_tasks & external_upgrade_tasks"  https://review.openstack.org/61589114:22
hkominosfultonj: ping14:22
sshnaidm|roverhjensas, it will take time only to install a built package from patch14:22
sshnaidm|roverhjensas, I think overall it sholdn't take more than 10 mins14:22
fultonjhkominos: there's a tripleo meeting happening now; can you ping me after?14:23
hjensassshnaidm|rover: ok, thanks.14:23
sshnaidm|roverhjensas, we can see it right after a promotion when all packages are up to date14:23
hkominosfultonj: sure :) take care14:24
jaosoriorsshnaidm|rover: I think it makes sense, though I guess first you wanna sync with mwhahaha to get more input, right?14:24
sshnaidm|roverjaosorior, yep14:24
sshnaidm|roverjaosorior, need to sync if he's working on this too14:24
jaosoriorgot it14:25
*** aedc has quit IRC14:25
jaosoriorso, the next topic is:     (sshnaidm) OVB check jobs turning off in Wed14:25
*** aedc has joined #tripleo14:25
sshnaidm|roverok, so we actually decided to turn them off even today14:25
sshnaidm|rover#link https://review.rdoproject.org/r/#/c/17272/14:25
sshnaidm|roverwe turn off check jobs for patches14:26
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: DNM: Squash for fedora28 job  https://review.openstack.org/61553214:26
sshnaidm|roverperiodic jobs will still run14:26
dpeacock:q!14:26
sshnaidm|roverand if you want to run ovb jobs on your patch, you can run them by comment "check-rdo"14:26
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547914:26
sshnaidm|roverbut nothing will run automatically14:26
jaosoriorsshnaidm|rover: what's the background on that?14:26
weshayjaosorior, the background reason why we need to shut off 3rd party check?14:27
sshnaidm|roverjaosorior, RDO cloud is not stable for last months14:27
jaosoriorright14:27
weshayjaosorior, the pass rate is roughly 15%, lots of broken glass not providing value. The root cause seems to be networking and the load on rdo-cloud atm14:28
jaosoriorthought the instability would be addressed, and not that we would be moving away from testing our patches there.14:28
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman  https://review.openstack.org/60697514:28
weshayso we're hoping to at least get full promotions running, when 50 red / broken check jobs are not currently running14:28
weshayjaosorior, yes.. I also thought this would be addressed by now14:29
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman  https://review.openstack.org/60697514:29
jaosoriorwhen/if these issues in RDO cloud get addressed, is the plan to turn the checks back on?14:29
weshayjaosorior, yes, of course..14:29
weshayand it could be we use another cloud for some time.. not sure14:29
weshayjaosorior, I am trying coordinate a cross team response to the issue here14:30
jaosorioralright14:30
jaosoriorweshay: any way I could help there?14:30
weshayhttps://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https14:30
weshay://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tr14:30
weshayee.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.ta14:30
weshayiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.i14:30
weshayo/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/pro14:30
weshayject/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/14:30
weshaytripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripl14:30
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547914:31
weshayeo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci14:31
weshay-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-boar14:31
jaosoriorweshay: none of the links worked for me :/14:31
weshayd/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298https://tree.taiga.io/project/tripleo-ci-board/epic/298vvvvvvvvvvv14:31
weshayoh wow14:31
weshaylolz.. my laptop just had a brain fart14:31
weshayhttps://tree.taiga.io/project/tripleo-ci-board/epic/29814:31
weshayjaosorior, ur on the epic14:31
jaosoriorohoo, THAT epic :D14:31
weshayalso escalated https://trello.com/c/2k9DDzpX/778-cixlp1797526tripleociproa-bring-ovb-jobs-back-to-passing-at-7514:31
weshayjaosorior, the communication on this has been very odd.. folks were trying to move that issue to preventative action14:32
sshnaidm|roverjaosorior, just how looks rdo cloud ci: http://dashboard-ci.tripleo.org/dashboard/snapshot/spgOhTI6f9QG3CyyRu38rzfpIAg48LZE?orgId=1&from=now-90d&to=now14:32
jaosoriorweshay: lets catch up on this after this meeting14:32
weshayjaosorior, mwhahaha and I should coordinate prior to a prod-chain infra meeting14:32
weshayjaosorior, k14:33
jaosoriorthanks for the info14:33
jaosoriorsshnaidm|rover: ouch14:33
sshnaidm|roveryep14:33
jaosoriorSo, the next topic is mine14:34
jaosorior* No meeting next week14:34
jaosoriorSo, a bunch of folks will be in Berlin in the Summit next week, so will I14:34
jaosoriorso, we won't have a meeting next week14:34
jaosoriorso, if you're going, hope to see you folks there :D14:35
jaosoriorthat's all for the one-off agenda items14:35
*** aedc has quit IRC14:35
jaosorior#topic Squad status14:36
*** openstack changes topic to "Squad status (Meeting topic: tripleo)"14:36
jaosoriorci14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting14:36
jaosoriorupgrade14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status14:36
jaosoriorcontainers14:36
jaosorior#link https://trello.com/b/S8TmOU0u/tripleo-podman14:36
jaosoriorintegration14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-integration-squad-status14:36
jaosoriorui/cli14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status14:36
jaosoriorvalidations14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-validations-squad-status14:36
jaosoriornetworking14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-networking-squad-status14:36
jaosoriorsecurity14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-security-squad14:36
jaosorioredge14:36
jaosorior#link https://etherpad.openstack.org/p/tripleo-edge-squad-status14:37
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman  https://review.openstack.org/60697514:37
jaosoriorAny squad wanting to bring up their status, or a topic for the general public?14:37
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Allow the environment to enable Podman on the overcloud  https://review.openstack.org/61252214:38
openstackgerritMerged openstack-infra/tripleo-ci master: turn on tcpdump debug by default for ovb jobs  https://review.openstack.org/61007214:38
openstackgerritMerged openstack-infra/tripleo-ci master: Collect atop statistics  https://review.openstack.org/61551514:38
EmilienMo/14:39
jaosorior\o14:39
jaosorior#topic bugs & blueprints14:39
*** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)"14:39
jaosorior#link https://launchpad.net/tripleo/+milestone/stein-114:39
jaosoriorFor Stein we currently have 28 (for stein-1) and 4 (for stein-2) blueprints open in Launchpad.14:39
jaosoriorBugs: 757 (-4) stein-2, 1 stein-3.  102 (-1) open Storyboard bugs.14:40
openstackbug 757 in Launchpad itself "Assignee should be CC'd when assigneed to a private bug" [Medium,Invalid] https://launchpad.net/bugs/75714:40
jaosorior#link https://storyboard.openstack.org/#!/project_group/7614:40
*** ade_lee has joined #tripleo14:40
EmilienMlol the bot14:40
jaosorior#topic projects releases or stable backports14:41
*** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)"14:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Set setype for log and persistant data directories  https://review.openstack.org/61180114:42
jaosoriornothing here on my side14:42
jaosorior#topic specs14:42
*** openstack changes topic to "specs (Meeting topic: tripleo)"14:42
jaosorior#link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open14:43
jaosoriorAny specs that folks would like to bring up here?14:43
hjensas#link https://review.openstack.org/45158414:44
jaosoriorI'll check it out, thanks hjensas !14:44
hjensasMost of the code to implement is up for review ... so would be good to get that done. :)14:44
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Reimplement HA docker tagging  https://review.openstack.org/61482714:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Support HA container tagging with podman  https://review.openstack.org/61483014:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman  https://review.openstack.org/60697514:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Set setype for log and persistant data directories  https://review.openstack.org/61180114:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Allow the environment to enable Podman on the overcloud  https://review.openstack.org/61252214:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Reimplement HA docker tagging  https://review.openstack.org/61482714:46
jaosorior#topic open discussion14:46
*** openstack changes topic to "open discussion (Meeting topic: tripleo)"14:46
jaosoriorAnything else that folks want to bring up to the meeting?14:46
honza#link https://review.openstack.org/#/c/607323/14:47
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Drop ssh-user from 'upgrade run' template.  https://review.openstack.org/61589714:47
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP / fs010: enable Podman on the overcloud  https://review.openstack.org/61252614:47
honzaNeed some eyes/help on that patch14:47
*** agopi_ is now known as agopi14:47
jaosoriorack, added to my queue14:48
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP / fs010: enable Podman on the overcloud  https://review.openstack.org/61252614:48
honza<314:49
jaosorioralright! thanks for joining folks!14:50
jaosorior#endmeeting14:50
*** openstack changes topic to "Welcome to Stein | CI Status: A Citrus Fruit (ORANGE). 3rd party is dead to me | https://docs.openstack.org/tripleo-docs/latest/"14:50
rfolco|ruckCI Community Meeting starts now at https://bluejeans.com/5878458097 - agenda: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting ~L3214:50
openstackMeeting ended Tue Nov  6 14:50:31 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-11-06-14.08.html14:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-11-06-14.08.txt14:50
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-11-06-14.08.log.html14:50
weshayCI Community Meeting starts immediately upon this meeting closing in #oooq. This is our team's weekly "open office hours." All are welcome!  Ask/discuss anything, we don't bite. Agenda (add items freely)14:51
derekhNew Ironic CI job is still looking for reviews https://review.openstack.org/#/c/582294/ and https://review.openstack.org/#/c/509728/ if anybody has a few minutes14:51
weshayfultonj, ^14:51
fultonjweshay: ack14:51
*** toure is now known as toure|biab14:51
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs001/035/053: switch undercloud to podman  https://review.openstack.org/61402314:53
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade stable/rocky: Drop ssh-user from 'upgrade run' template.  https://review.openstack.org/61590114:54
chandankumarweshay: EmilienM couple of reviews14:57
chandankumarhttps://review.openstack.org/605356 https://review.openstack.org/61237714:58
weshaychandankumar, join my blue14:58
*** rh-jelabarre has joined #tripleo14:59
weshayrfolco|ruck, panda which bluejeans are you in15:01
weshay?15:01
weshayfultonj, https://bluejeans.com/u/whayutin/15:01
fultonjweshay: i am on https://bluejeans.com/5878458097 with the others15:01
rfolco|ruckoh I see what happened15:01
rfolco|rucksorry weshay i paste the invite to tripleo mtg using my bj15:02
openstackgerritNatal Ngétal proposed openstack/tripleo-common stable/pike: Ensure that GetHostCpusListAction.run() returns a deterministic result  https://review.openstack.org/61590615:06
ssbarneaEmilienM mwhahaha https://review.openstack.org/#/c/615831/  - f28 two liner heat-agent15:09
mwhahahassbarnea: ping therve and zaneb for that (trripleo doesn't own it)15:09
*** vinaykns has joined #tripleo15:09
trozetmwhahaha: regarding the comment about the scenario008 trigger broken due to invalid yaml...was that intentional? https://review.openstack.org/#/c/597852/1/zuul.d/layout.yaml15:09
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176915:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]15:10
mwhahahatrozet: no it was not, it was their python3 script, please propose a patch to fix the spacing15:10
trozetmwhahaha: ok thanks15:10
mwhahahatrozet: the comment at the end of the line broke the spacing15:10
mwhahahai caught most of the cases of that but i guess i missed that one15:11
*** aedc has joined #tripleo15:11
openstackgerritMarc Methot proposed openstack/paunch master: Allow to set limit memory on containers  https://review.openstack.org/61590915:12
EmilienMchandankumar: ack15:14
trozetmwhahaha: it was fixed in master by this patch https://review.openstack.org/#/c/596061/2/zuul.d/layout.yaml could we just cherry pick that?15:16
mwhahahatrozet: yes15:16
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/rocky: switch documentation job to new PTI  https://review.openstack.org/61591015:16
*** Vorrtex has joined #tripleo15:16
*** toure|biab is now known as toure15:19
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/queens: switch documentation job to new PTI  https://review.openstack.org/61591115:21
pandafultonj: what's the bluprint that best describes what EDGE does ?15:21
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/queens: switch documentation job to new PTI  https://review.openstack.org/61591115:22
fultonjpanda: https://blueprints.launchpad.net/tripleo/+spec/split-controlplane15:22
fultonjpanda: the dep tree there has a sprint-controlplane-ci blueprint linked15:22
*** ykarel is now known as ykarel|away15:22
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/queens: Change log directory for OpenDaylight  https://review.openstack.org/61245915:23
pandafultonj: I don't see any reference to the word EDGE there :)15:25
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/rocky: Sets ODL OVSDB inactivity probe timer  https://review.openstack.org/61537615:25
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/queens: Sets ODL OVSDB inactivity probe timer  https://review.openstack.org/61537715:26
mschuppertcould I please get reviews for https://review.openstack.org/613971 . is required for 1415:26
fultonjpanda: tripleo's split control plane can be used for edge deployments there's some description here https://specs.openstack.org/openstack/tripleo-specs/specs/rocky/split-controlplane.html#scope-and-phases15:26
*** aedc has quit IRC15:26
fultonji've been talking about split control plane in the tripleo edge squad w/ slagle https://etherpad.openstack.org/p/tripleo-edge-squad-status15:28
*** cdearborn has quit IRC15:30
*** gkadam has quit IRC15:30
beagleschem: seeing this when testing upgrade code "msg": "The task includes an option with an undefined variable. The error was: \'cinder_api_short_bootstrap_node_name\' is undefined\\n\\nThe error appears to have been in \'/var/lib/mistral/5d2ce1a3-5861-41b8-9464-051959f91fdc/Controller/upgrade_tasks.yaml\': line 111, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n-15:31
beagles block:\\n  - name: set is_bootstrap_node fact\\n    ^ here\\n"}',15:31
beagleschem: thoughts? am I missing updates/doing something wrong/other?15:31
chembeagles: hum let me check15:32
*** yprokule has quit IRC15:33
beagleschem: oh maybe I didn't run "upgrade prepare" for this deploy. Is that required just to run the ansible tasks?15:33
chembeagles: yes15:33
beagleschem okay my bad... let me try again15:34
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Enable testing using ansible molecule  https://review.openstack.org/61367215:36
*** itlinux has quit IRC15:38
*** ramishra has quit IRC15:40
evrardjpjaosorior: did you see this going on at the summit?  https://etherpad.openstack.org/p/BER-container-security15:42
*** ykarel|away has quit IRC15:43
evrardjpIt might be interesting for people interested to container security, and maybe sharing practices15:43
*** kopecmartin is now known as kopecmartin|off15:46
openstackgerritJames Slagle proposed openstack/tripleo-common master: Sync state if needed during retrieval  https://review.openstack.org/61116815:46
quiquellmwhahaha: We have to do the puppet py3 for fedora at aodhclient and barbicanclient too15:47
quiquellmwhahaha: http://logs.openstack.org/79/615479/16/check/tripleo-ci-fedora-28-standalone/9c6023b/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-11-06_15_32_4815:47
mwhahahaK15:47
quiquellmwhahaha: How do we new what puppet projects do we have to change ?15:47
naturalbluedoes anyone know why when I run hiera to get any info I am getting back 'nil' as the answer. Does matter what passwoprd or username I request15:48
jaosoriorevrardjp: nice! gotta check it out15:48
evrardjpjaosorior: or send someone :)15:49
openstackgerritJames Slagle proposed openstack/tripleo-common stable/rocky: Delete old tarball from config container on download  https://review.openstack.org/61571315:51
openstackgerritSam Doran proposed openstack/ansible-role-redhat-subscription master: Add resiliency to subscription management role  https://review.openstack.org/61568915:51
openstackgerritCarol Bouchard proposed openstack/tripleo-heat-templates master: CiscoNexus: Update and Remove deprecated vars  https://review.openstack.org/60695815:52
mwhahahaquiquell: it would be the licents listed https://review.openstack.org/#/c/613394/3/puppet/services/openstack-clients.yaml15:53
*** ksambor has quit IRC15:57
quiquellmwhahaha: ack15:57
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044516:00
openstackgerritAlex Schultz proposed openstack/tripleo-repos master: Fix pep8 errors  https://review.openstack.org/61570016:00
openstackgerritAlex Schultz proposed openstack/tripleo-repos master: Add fedora support to tripleo-repos  https://review.openstack.org/61569316:00
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: add zuul.projects and executed playbooks into reproducer  https://review.openstack.org/61357216:02
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: Remove tripleo-ci-centos-7-containerized-undercloud-upgrades from gate  https://review.openstack.org/61592016:04
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Handle LP openvswitch meta-package on upgrade  https://review.openstack.org/60520016:05
openstackgerritEmilien Macchi proposed openstack/paunch master: Remove tripleo-ci-centos-7-containerized-undercloud-upgrades from gate  https://review.openstack.org/61592116:06
*** panda is now known as panda|ko16:06
beaglesmarios, chem, jistr: this is ready for another look https://review.openstack.org/#/c/605200/16:06
mwhahahasshnaidm|rover: so if you're -1 on adding a role for tripleo-repos, what do you propose as a solution for the build containers/images jobs in https://review.openstack.org/#/c/615207/16:07
beaglesI'm trying to test again in the context of an actual upgrade, but having a few unrelated issues - worked as isolated ansible16:07
*** quiquell is now known as quiquell|off16:08
sshnaidm|rovermwhahaha, which build containers job do you mean? In check jobs?16:08
mwhahahasshnaidm|rover: yes16:08
sshnaidm|rovermwhahaha, doesn't repo-setup role do its job now?16:08
mwhahahasshnaidm|rover: nope16:08
mwhahahasshnaidm|rover: https://review.openstack.org/#/c/614392/ and https://review.openstack.org/#/c/610102/16:08
mwhahahasshnaidm|rover: neither use quickstart16:08
mwhahahasshnaidm|rover: we actually don't have container build coverage in CI16:08
mwhahahasshnaidm|rover: i do not want to add quickstart to these jobs because it's too much bloat for just setting up repos16:09
sshnaidm|rovermwhahaha, I see.. I thought we can adopt current stuff from promotion jobs to run on patches16:09
mwhahahasshnaidm|rover: that's way too much complexity16:10
sshnaidm|rovermwhahaha, like we have in build-containers job16:10
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176916:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]16:10
*** ooolpbot has quit IRC16:10
sshnaidm|rovermwhahaha, maybe it's worth then redesign the current build containers job to be both promotion and patch able?16:10
sshnaidm|rovermwhahaha, I think it's the same effort as to create a new job from scratch..16:11
mwhahahadoubt it16:11
mwhahahasince i already created the job from scratch16:11
sshnaidm|rovermwhahaha, you're cheating! :D16:11
mwhahahai wanted something that actually matched the documentation16:12
*** agurenko has quit IRC16:12
mwhahahaand didn't do anything else16:12
sshnaidm|rovermwhahaha, well, if it's only for patches for container job I think tripleo-repos could be fine, but not for upgrades, promotions, etc16:12
mwhahahasshnaidm|rover: yes i do not have any intention on using that for upgrades/promotions at this time16:12
mwhahahasshnaidm|rover: it's only for these types of jobs that small/quick and not deployment related16:13
mwhahahacontainer/image building16:13
mwhahahawhere we need the repos but that's it16:13
sshnaidm|rovermwhahaha, maybe we can still plan a job that could do it both for patches and promotion, later16:14
mwhahahasshnaidm|rover: yes i think it would be a good idea to combine them, but i'm not sure what the existing promotion requirements are16:14
mwhahahain theory this job should be able ot work for promotion if it has credentials16:14
sshnaidm|rovermwhahaha, it's very simple - just build everything with current repos16:15
mwhahahayea thats what this does16:15
sshnaidm|rovermwhahaha, so I suppose for patches we need to build only what was changed16:15
sshnaidm|rovermwhahaha, or you build all?>16:15
mwhahahasshnaidm|rover: well for patches we run the process on only code that affects the build process.16:16
mwhahahawe build all16:16
mwhahahawe should actually have this job on kolla16:16
* mwhahaha is trying to track down why grafana is failing as of yesterday16:16
mwhahahasshnaidm|rover: are you seeing grafana failing or it it just my job because i'm kolla from source16:18
sshnaidm|rovermwhahaha, not sure I understand about grafana, where are the logs?16:18
EmilienMmwhahaha: yes we should propose that job in kolla16:18
mwhahahasshnaidm|rover: http://logs.openstack.org/92/614392/30/check/tripleo-build-containers-centos-7/ed17b24/logs/build-err.log.txt.gz16:19
*** hamzy_ has quit IRC16:19
mwhahahaEmilienM: yea i agree to give us some coverage over there for once16:19
EmilienMonce the job is ready we can propose it16:19
sshnaidm|rovermwhahaha, INFO:kolla.common.utils.grafana:No package grafana available.16:20
mwhahahayea i know, i'm not sure if we provide that in rdo16:20
sshnaidm|rovermwhahaha, I can't know, because last containers jobs were failing on undercloud install, there is a bug with hostnames, so it didn't reach the building containers step16:20
sshnaidm|rovermwhahaha, but I didn't see it before - before yesterday16:21
mwhahahayea it's new as of yesterday16:21
sshnaidm|roverjpena, do you know if we provide grafana in rdo?16:21
mwhahahai had a successful run last week16:21
mwhahahabut my post command was crap :o16:21
sshnaidm|rovermwhahaha, seems like we really need this job :)16:21
mwhahahaindeed :D16:21
jpenasshnaidm|rover: do you mean as a package?16:22
mwhahahajpena: yea16:22
sshnaidm|roverjpena, yep,16:22
sshnaidm|roverjpena, in building containers: INFO:kolla.common.utils.grafana:No package grafana available.16:22
mwhahahahttp://logs.openstack.org/92/614392/24/check/tripleo-build-containers-centos-7/12f91b3/job-output.txt.gz#_2018-11-02_22_56_16_05552216:22
mwhahahawas working on friday16:22
mwhahahaso someone added grafana as a thing over the weekend16:22
* mwhahaha hunts down the offender16:22
sshnaidm|rovermwhahaha, yeah, I discover sometimes some new weird stuff there..16:23
jpenano, we don't provide grafana. Let me check if the opstools repo does16:23
jpenano, it's not there either16:24
mwhahahahuh weird, nothing has merged in kolla16:24
mwhahahawill have to track the down manually i guess16:24
jpenait was built some time ago (https://review.rdoproject.org/grafana/), but never releaased16:24
mwhahahai wonder if it was in deps and got dropped?16:24
sshnaidm|rovermwhahaha, isn't it? https://github.com/openstack/tripleo-common/commit/042a28195fa35db2b86400ae8c03e26cf0a71d5b16:25
mwhahahashouldn't be16:25
mwhahahathat's not the kolla config16:25
mwhahaha(despite poor naming choices)16:26
sshnaidm|rovermwhahaha, merged yesterday.. fits the time16:26
jpenahttp://cbs.centos.org/koji/packageinfo?packageID=2240 has all the builds and tags for grafana in CBS16:26
*** ccamacho has quit IRC16:27
mwhahahasshnaidm|rover: you might be right, let me go look at the kolla-builder16:30
mwhahahacause earlier last week it was not adding grafana to the list of kolla images16:30
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Create a reproducer-quickstart dir  https://review.openstack.org/61563516:31
openstackgerritDouglas Mendizábal proposed openstack/tripleo-heat-templates master: Add template code to configure hsm backends for barbican  https://review.openstack.org/61063416:31
*** mjturek has joined #tripleo16:32
*** udesale has quit IRC16:32
*** aufi has quit IRC16:33
*** panda|ko is now known as panda|off16:34
*** mjturek has quit IRC16:36
*** mjturek has joined #tripleo16:36
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28  https://review.openstack.org/61547916:37
mwhahahamandre: fyi, https://bugs.launchpad.net/tripleo/+bug/180196916:38
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [High,Triaged]16:38
*** quiquell|off is now known as quiquell16:38
quiquellmwhahaha: added the missing puppet projects16:38
quiquellmwhahaha: let's check tomorrow16:38
*** quiquell is now known as quiquell|off16:38
mwhahahaquiquell: k i'll review when i can. the sahara patch seems broken on centos 7 (unrelated to patch)16:38
quiquell|offmwhahaha: has recheck16:39
*** artom has quit IRC16:43
beagleschem: even after prepare having problems with those boostrap node names16:44
*** hamzy_ has joined #tripleo16:44
beagleschem, where do they come from and how are they fed into the ansible-playbook run?16:45
chembeagles: yeah, thats' strange, the name is from the puppet tripleo module16:45
chembeagles: it's as if the bootrap node setting wasn't working properly there16:45
chembeagles: can you open a lp about this, so I can review this later, I need to be off in a few minutes.16:46
beagleschem: ah could be - I see the name appear in a map in overcloud/Controller/overcloud-controller-0   etc16:46
beagleschem: sure16:46
mandremwhahaha: it looks like this container image build job is assuming that all the containers are build from kolla?16:46
mwhahahamandre: i guess?16:46
* beagles concedes that he have simply mangled things past the point of sanity :)16:46
mandremwhahaha: I remember having to add filters to the ansible-role-tripleo-modify-image role to skip images that were not originated from the kolla project, maybe we can take the same approach here?16:48
mwhahahamandre: probably need to16:49
mwhahahamandre: but we might need to backout the openshift stuff until we have a fix16:49
*** artom has joined #tripleo16:51
mandremwhahaha: hmm, if possible I'd like to avoid reverting the openshift images patch :D16:51
mandremwhahaha: https://review.openstack.org/#/c/579918/ <- my commit to filter openshift images16:52
mwhahahamandre: yes but we can't build any containers now, which is kinda critical to, you know, everything16:52
mandremwhahaha: can you point me to the task that builds the containers?16:53
mwhahahamandre: you mean the command? or what16:53
mwhahahamandre: this is the tripleo command, https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/container_image.py#L165 which trails off into tripleo-common fun16:54
mwhahahamandre: this is my attempt to actually have CI to cover teh container build process, https://review.openstack.org/#/c/614392/16:54
mwhahahawhich is what exposed this thing16:54
chemdciabrin: I've let some note in https://review.openstack.org/#/c/609998/, will check tomorrow how to fix this.16:54
chemdciabrin: again16:54
mandremwhahaha: perfect, this is what I was looking for16:55
chembandini: ^16:55
chembye16:55
dciabrinchem, awesome thx a lot!16:55
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: remove non-voting job from gate  https://review.openstack.org/61594416:58
*** chandankumar is now known as chkumar|pto16:59
weshaysshnaidm|rover, rfolco|ruck mwhahaha ^16:59
mandremwhahaha: hey btw, are you planning to make this job available in kolla too?16:59
mwhahahamandre: yes16:59
mandreawesome!16:59
mandremwhahaha: for a quick fix, we may want to add a new attribute to the images in overcloud_containers.yaml.j2 to distinguish the ones coming from kolla from the other ones17:07
mwhahahamandre: or perhaps a build flag17:08
mandremwhahaha: yeah, it's equivalent17:09
mwhahaharather than lock us in to kolla :D at least just indicate this should be part of the build or not.i can propose soemthing later if you want17:09
mandremwhahaha: feel free to assign the bug to me, I can look at it tomorrow if needed17:09
mwhahahamandre: k i'll assign it to you if i don't get something together today17:10
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176917:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196917:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged]17:10
mandremwhahaha: ack, maybe stevebaker wants to get involved too17:10
mwhahahayea i'll poke him later today17:10
*** jfrancoa has quit IRC17:13
*** hkominos_ has joined #tripleo17:13
hkominos_fultonj: ping v217:15
fultonjhkominos_: what's up?17:15
hkominos_fultonj: Hi. I was wondering if you have a couple of mins to help me with an issue that I am facing regarding https://github.com/openstack/tripleo-common/blob/stable/rocky/workbooks/swift_backup.yaml#L4917:15
fultonjhkominos_: sure, what's the issue?17:16
hkominos_fulton: I have this aarch64 opnfv-apex that I am working on and currently I am stuck in in the deployment face because (i think) that workbook is failing. THe logs basically just inform me that set_put_tempurl_failed: FAILED17:17
hkominos_fultonj: So i am wondering why17:18
hkominos_first of all what does that do  exaclty.17:18
fultonjthis workflow generates two URLs you can use w/ swift which don't require authentication17:19
fultonjthe UC runs swift and things about the deployment can be stored there17:19
fultonja PUT url and GET url are used for write and read respectively17:20
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Enable testing using ansible molecule  https://review.openstack.org/61367217:20
fultonjyou can curl directly to them without auth until they expire17:20
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Set default unbound_cache_min_ttl to 0 in the base job  https://review.openstack.org/60805117:22
fultonjhkominos_: if you're on your undercloud you want to find the out why the task failed17:22
hkominos_fultonj: I would like to but there is nothing in heat/zaqar/mistral logs17:22
*** ccamacho has joined #tripleo17:22
hkominos_at least nothing for my non-tripleo Dev eyes17:23
hkominos_fultonj: which container is this supposed to be run against ? I dont have any containers with "swift-rings" suffix17:27
hkominos_?17:27
fultonjhkominos_: the container name is a concatenation https://github.com/openstack/tripleo-common/blob/stable/rocky/workbooks/swift_backup.yaml#L2417:29
fultonjalso, it's good not to confuse the swift contianers with docker containers :)17:29
fultonj(not saying you are, i just like to get that out there)17:30
hkominos_fultonj: Well in the case I was so thx for the heads up17:30
fultonjhkominos_: every time mistral runs it stores what it executed in its execution table17:32
fultonjyou can look at the exection of the workflow and then look at the tasks within the execution to find the failed task17:33
*** ccamacho has quit IRC17:33
*** jpich has quit IRC17:34
hkominos_fulton: Can i somehow run a single task as well ?17:34
hkominos_fulton: Cancel that17:35
hkominos_trozet^ ? Are you able to run that workflow on x86 ?17:36
fultonjhkominos_: https://github.com/fultonj/tripleo-ceph-ansible/blob/master/see-last-task.sh17:36
fultonja variation of ^ might help you17:36
hkominos_fulton: Thx for the tip17:37
fultonjchange the workflow name and find your execution17:37
fultonjby UUID17:37
fultonjthen list the tasks for that execution17:37
fultonjthere's a horrible bug in `mistral execution-list` by the way17:37
fultonjit has an implicit "limit 100" or similar17:38
fultonjmistral execution-list won't list all of the executions, just the first 10017:38
fultonjthe way i get it to show me more is.... delete older executions17:38
fultonjor go directly into the DB17:38
fultonjthe failed task runs a mistral action, which is a chunk of python17:39
hkominos_trozet: ^^17:39
fultonjit sounds like you've tracked it to `action: tripleo.parameters.update`17:40
hkominos_yes17:40
hkominos_I have some logs somewhere17:40
*** thrash is now known as thrash|biab17:41
fultonjhkominos_: version?17:42
hkominos_openstack-tripleo-common-9.4.1-017:44
fultonjqueens, rocky?17:44
hkominos_rocky17:45
hkominos_it has been a rocky start though...17:45
*** dprince has quit IRC17:45
openstackgerritMerged openstack/tripleo-quickstart-extras master: Use dnf and python3 on platforms where these are default  https://review.openstack.org/61049117:46
* fultonj steps out for lunch17:47
*** bogdando has quit IRC17:48
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Enable Scale up/redeploy of a deployment  https://review.openstack.org/60793117:52
*** sshnaidm|rover is now known as sshnaidm|afk17:56
openstackgerritCédric Jeanneret proposed openstack/tripleo-heat-templates master: Move "/run/libvirt" shared directory to a persistent location  https://review.openstack.org/61595617:58
honzawhere can i find haproxy logs on a containerized undercloud?17:59
honzanot exactly sure what /dev/log local0 is supposed to mean17:59
*** derekh has quit IRC18:01
openstackgerritAna Krivokapic proposed openstack/tripleo-validations master: Fix disk space validation  https://review.openstack.org/61595918:02
*** apetrich has quit IRC18:03
*** jpena is now known as jpena|off18:05
*** dtantsur is now known as dtantsur|afk18:06
*** salmankhan has quit IRC18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176918:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196918:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]18:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged]18:10
*** amoralej is now known as amoralej|off18:10
*** thrash|biab is now known as thrash18:12
*** hberaud has quit IRC18:14
*** hberaud has joined #tripleo18:15
*** jtomasek has quit IRC18:18
*** gfidente is now known as gfidente|afk18:21
*** dprince has joined #tripleo18:23
*** rpioso is now known as rpioso|afk18:24
*** tongl has joined #tripleo18:29
*** rfolco|ruck is now known as rfolco|doctor18:30
d0ugalfultonj: mistral execution-list --limit=-118:33
hkominos_!!18:33
openstackhkominos_: Error: "!" is not a valid command.18:33
fultonjd0ugal really!? wow i never new that. thanks18:33
d0ugalfultonj: :)18:33
hkominos_haha!18:33
fultonjabishop: ^18:33
fultonjis that new?18:33
d0ugalfultonj: It might be a pain, but some folks have hundreds of thousands of executions - listing them all by default can bring everything down.18:34
d0ugalfultonj: Nah, it had been there since the limit was added iirc18:34
abishopd0ugal: does that work for tasks ask well?18:36
d0ugalabishop: yup18:36
d0ugalit should be covered in --help18:36
abishopd0ugal: cool, it's been a while since I had to debug a complex workflow (derived-parameters stuff), but this is good to know!18:39
d0ugalabishop: You may also find this useful, harder to use: https://github.com/d0ugal/tripleo-util/blob/master/scripts/wf-report18:48
d0ugalI keep meaning to add some version of it to mistralclient18:49
d0ugalabishop: It lists out all the tasks, action executions and sub workflows (and tasks and actions) for a given execution ID18:49
abishopd0ugal: nice! fultonj ^18:50
tonglmwhahaha: Could you please help look at the cherry-picked patches from master: https://review.openstack.org/#/c/614264/, https://review.openstack.org/#/c/614594/?18:52
*** hamzy_ is now known as hamzy18:53
fultonjhkominos_: try d0ugal's example to help debug your problem18:58
fultonjd0ugal: thanks that's much better than my shell script18:58
*** shardy has quit IRC19:00
d0ugalfultonj: np! Sorry I didn't tell you before.19:00
*** artom has quit IRC19:01
hkominos_fultonj: sure. Thx19:03
honzai'm getting this from keystone on auth AssertionError: PROGRAMMING ERROR: enforcement (`keystone.common.rbac_enforcer.enforcer.RBACKEnforcer.enforce_call(), any ideas?   cc jrist19:08
honzadtrainor: ^^^ this is probably the root cause19:08
*** agopi has quit IRC19:09
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176919:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196919:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged]19:10
jristhonza: zero idea19:10
*** rpioso|afk is now known as rpioso19:15
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Create a reproducer-quickstart dir  https://review.openstack.org/61563519:15
dtrainorsorry honza, context?19:18
honzadtrainor: https://bugs.launchpad.net/tripleo/+bug/180177819:19
openstackLaunchpad bug 1801778 in tripleo "Keystone circular reference on OPTIONS" [High,Triaged]19:19
dtrainorI'm getting an error with what I think is role matching.  A downstream Queens deploy.  The error is:  NoValidHost_Remote: No valid host was found. There are not enough hosts available.  Digging deeper, it looks like a filter is catching it:  'control' does not match 'ceph-storage'19:19
dtrainorthe node is tagged properly, and the Flavor has the same Role assigned to it, so I'm not sure.  Funny that the failure always and only mentions "...does not match 'ceph-storage'", as if this is hard-coded in here somewhere19:20
honzaall the errors today!19:21
dtrainorall of them.19:23
*** rlandy is now known as rlandy|vote19:26
*** cdearborn has joined #tripleo19:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: (Partial) Revert "Use container_cli for post_upgrade_tasks & external_upgrade_tasks"  https://review.openstack.org/61589119:32
*** artom has joined #tripleo19:34
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud  https://review.openstack.org/61585319:34
*** mjturek has quit IRC19:41
*** salmankhan has joined #tripleo19:43
*** mjturek has joined #tripleo19:43
*** dsneddon has joined #tripleo19:46
*** akrivoka has quit IRC19:47
tonybAre we using CentOS 7.5 or 7.6 in the gate these days?19:47
*** salmankhan has quit IRC19:47
*** tongl has quit IRC19:56
*** rlandy|vote is now known as rlandy19:58
weshaytonyb, CentOS7.6. is not out yet20:08
weshayhaven't seen any pre-release rpms either20:08
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176920:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196920:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged]20:10
*** rfolco|doctor is now known as rfolco|off20:16
openstackgerritJohn Fulton proposed openstack/tripleo-quickstart master: WIP/DNM: Add featureset 57 for split controlplane  https://review.openstack.org/61598720:17
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Use ready rendered templates for reproducer  https://review.openstack.org/61463820:17
openstackgerritJohn Fulton proposed openstack-infra/tripleo-ci master: WIP/DNM: Add split-controlplane standalone non-voting job definition  https://review.openstack.org/61598820:19
*** jtomasek has joined #tripleo20:21
*** jtomasek has quit IRC20:24
*** Vorrtex has quit IRC20:31
openstackgerritHarald Jensås proposed openstack/tripleo-docs master: Fix tempest docs - correct options for openstack commands  https://review.openstack.org/61599720:32
*** mjturek has quit IRC20:38
*** mmethot_ is now known as mmethot20:38
*** mjturek has joined #tripleo20:39
*** mjturek has quit IRC20:46
*** mjturek has joined #tripleo20:51
*** mjturek has quit IRC20:54
*** cdearborn_ has joined #tripleo20:55
*** radeks has quit IRC20:55
*** cdearborn has quit IRC20:58
*** tongl has joined #tripleo21:03
*** raildo has quit IRC21:06
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176921:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196921:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged]21:10
*** boazel has joined #tripleo21:12
*** dprince has quit IRC21:13
*** boazel_ has quit IRC21:15
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: DNM install tar-split binary  https://review.openstack.org/61600821:19
stevebakerI'm going to hell for that one21:19
openstackgerritAlan Bishop proposed openstack/puppet-tripleo master: Fix cinder's default RBD backend_host  https://review.openstack.org/61600921:22
*** jcoufal has quit IRC21:25
*** mjturek has joined #tripleo21:26
*** apetrich has joined #tripleo21:29
*** pcaruana has quit IRC21:31
*** mjturek has quit IRC21:31
dpeacockstevebaker: ew.21:32
stevebakerdon't judge me!21:32
dpeacocklol21:33
dpeacockI'm very pro Go FWIW.  Would definitely love to see a lot more of that language in OpenStack.21:33
stevebakeroh, same. I'm enjoying the little bit I'm doing21:35
*** salmankhan has joined #tripleo21:39
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM: Testing  https://review.openstack.org/61601121:39
*** ansmith has quit IRC21:42
*** dsneddon_ has quit IRC21:43
*** salmankhan has quit IRC21:44
*** mjturek has joined #tripleo21:44
*** hamzy has quit IRC21:45
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM: Testing  https://review.openstack.org/61601121:47
mwhahahastevebaker: so bug 1801969, what would be the best way to indicate that we shouldn't build a container with kolla? I'm messing with adding a build: False flag to the overcloud_containers.yaml but wanted to know if there's a better way21:47
openstackbug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged] https://launchpad.net/bugs/180196921:47
stevebakermwhahaha: I've considered 2 options in the past:21:50
stevebakermwhahaha: 1. conditional jinja blocks in overcloud_containers.yaml.j2 which group together images from common projects, so the existing "set" mechanism can be used to get all images, or only kolla images21:51
stevebaker2. a new attribute for each entry in overcloud_containers.yaml.j2 which says where the image is from (kolla, okd etc) then add a feature in all the tooling to filter by that21:53
stevebakermwhahaha: but regarding bug #1801969 can they just add excludes arguments to the prepare call to filter out the non-kolla images?21:54
openstackbug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged] https://launchpad.net/bugs/180196921:54
*** salmankhan has joined #tripleo21:54
stevebakermwhahaha: can you point me at the CI logs for that failure?21:55
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM: Testing  https://review.openstack.org/61601121:55
mwhahahastevebaker: the problem is that it's the default behagiy. I could probably update the image name from regex or whatever but I think we need to indicate what is buildable and what isnt21:56
mwhahahas/behagiy/behavior. Wtf autocorrect21:57
stevebakeryep, we need a whitelist not a blacklist, because images get added upstream21:57
*** lblanchard has quit IRC21:57
mwhahahaSo maybe build: True21:57
mwhahahaInstead of build: False21:57
stevebakerthat was one of the purposes of overcloud_containers.yaml.j2 but its usefulness to the build pipeline has been neglected compared to its usefulness to deployment21:58
mwhahahaDo we use the .j2 for building or just deployment21:58
mwhahahaAnd we use the non-.j2 for building21:59
stevebakermwhahaha: I can't recall, can you point me at the failing CI logs?22:00
mwhahahastevebaker: http://logs.openstack.org/92/614392/30/check/tripleo-build-containers-centos-7/ed17b24/logs/build-err.log.txt.gz22:01
mwhahahaI think it's happening because the container name is just grafana so it doesn't get excluded22:01
mwhahahaAfk for a bit, I'll take another look in a bit22:01
stevebakerok, ping me when you're back22:05
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP implement python based uploader  https://review.openstack.org/61601822:09
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP make python uploader the default  https://review.openstack.org/61601922:09
stevebakerdpeacock: having said that, ^ replaces go with python partly because they didn't design for concurrency, wee!22:10
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176922:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196922:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,Triaged]22:10
tonglIs the third party CI still hosed?22:12
*** mjturek has quit IRC22:12
*** rh-jelabarre has quit IRC22:14
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: DNM install tar-split binary  https://review.openstack.org/61600822:19
mwhahahastevebaker: back22:21
mwhahahatongl: yes22:21
tonglmwhahaha: Thanks Alex! Then I won't recheck anymore on my patches to avoid extra load.22:22
mwhahahatongl: yea if zuul is +1 you're good22:22
tonglmwhahaha: When you have time, could you help look at this patch: https://review.openstack.org/#/c/605056/?22:25
mwhahahatongl: it's currently in merge conflict, what do you need?22:26
mwhahahai think we got rid of the environments/docker.yaml contents22:26
tonglLet me rebase and resolve the conflicts22:27
tonglDo you think we can leave environments/neutron-nsx.yaml and environments/nsx-config.yaml?22:29
tonglas a sample config22:29
mwhahahatongl: yes those should be fine22:29
stevebakermwhahaha: do you want to build all buildable images in that job, or just a representitive sample?22:29
mwhahahastevebaker: all22:29
mwhahahai'm basically running the defaults22:29
mwhahahastevebaker: here's the job definition, https://review.openstack.org/#/c/614392/22:30
mwhahahait's likely i need to sync the overcloud_container.yaml to include whatever build flag22:31
openstackgerritTong Liu proposed openstack/tripleo-heat-templates master: Add config files/templates to integrate nsx plugin with container  https://review.openstack.org/60505622:33
openstackgerritTong Liu proposed openstack/tripleo-heat-templates master: Add config files/templates to integrate nsx plugin with container  https://review.openstack.org/60505622:34
openstackgerritTong Liu proposed openstack/tripleo-heat-templates master: Add config files/templates to integrate nsx plugin with container  https://review.openstack.org/60505622:35
tonglmwhahaha: Resolved conflicts and removed the change in environments/docker.yaml22:36
mwhahahatongl: k22:37
tonglmwhahaha: thanks!22:37
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add flag to indicate container can be built  https://review.openstack.org/61602222:42
mwhahahastevebaker: -^ how about that22:42
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add flag to indicate container can be built  https://review.openstack.org/61602222:43
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add flag to indicate container can be built  https://review.openstack.org/61602222:50
openstackgerritAlex Schultz proposed openstack-infra/tripleo-ci master: TripleO Container build CI job  https://review.openstack.org/61439222:51
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud  https://review.openstack.org/61585323:05
tonybweshay: Well that would make a difference :( ... I really thought I saw some discussion about the drop on centos-devel23:06
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180176923:10
openstackLaunchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged]23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180196923:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz)23:10
tonybstevebaker, bfournie: Any chance y'all can look at https://review.openstack.org/#/c/580829/ ?23:17
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: DNM - Testing no undercloud cleaning  https://review.openstack.org/61160223:22
*** vpickard is now known as vpickard_23:25
*** rfolco|off has quit IRC23:33
*** rpioso is now known as rpioso|afk23:54
stevebakermwhahaha: let me know what you think of my review, also if you want me to make that change23:55
mwhahahastevebaker: sure i'll check it out in a few23:55
*** tzumainn has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!