Thursday, 2020-11-12

*** holser has quit IRC00:05
*** rfolco has joined #tripleo00:21
openstackgerritIan Wienand proposed openstack/diskimage-builder master: WIP: boot test of containerfile image  https://review.opendev.org/72214800:27
openstackgerritMerged openstack/python-tripleoclient stable/ussuri: Handle root as a deployment user better  https://review.opendev.org/76069300:36
*** d34dh0r53 has quit IRC00:42
*** rfolco has quit IRC01:03
openstackgerritSteve Baker proposed openstack/tripleo-ansible master: Reimplement os_tripleo_baremetal_node_introspection  https://review.opendev.org/76244601:12
openstackgerritMerged openstack/tripleo-common stable/train: Add retry to build  https://review.opendev.org/76174601:44
*** Guest80385 has quit IRC02:11
*** Goneri has quit IRC02:15
*** rcernin has quit IRC02:18
openstackgerritIan Wienand proposed openstack/diskimage-builder master: WIP: boot test of containerfile image  https://review.opendev.org/72214802:34
openstackgerritMerged openstack/tripleo-heat-templates stable/train: Add package install for openssl-perl  https://review.opendev.org/75972202:35
openstackgerritMerged openstack/tripleo-common stable/victoria: Drop --layers from buildah  https://review.opendev.org/76234502:43
openstackgerritMerged openstack/os-apply-config master: Bump pyflakes and fixup sphinx  https://review.opendev.org/76195502:43
*** rcernin has joined #tripleo02:50
*** skramaja has joined #tripleo03:03
*** rlandy has quit IRC03:10
*** Lewis has joined #tripleo03:11
*** Lewis is now known as Guest9952603:11
*** psachin has joined #tripleo03:28
openstackgerritKarthik S proposed openstack/os-net-config stable/victoria: Fixing the dependency for sriov_config service  https://review.opendev.org/76245903:35
openstackgerritKarthik S proposed openstack/os-net-config stable/ussuri: Fixing the dependency for sriov_config service  https://review.opendev.org/76246003:35
openstackgerritKarthik S proposed openstack/os-net-config stable/train: Fixing the dependency for sriov_config service  https://review.opendev.org/76246103:35
*** ykarel has joined #tripleo03:40
openstackgerritIan Wienand proposed openstack/diskimage-builder master: WIP: boot test of containerfile image  https://review.opendev.org/72214804:16
*** udesale has joined #tripleo04:27
*** jmasud has joined #tripleo04:29
*** udesale_ has joined #tripleo04:29
openstackgerritMerged openstack/tripleo-common stable/train: Add heat-engine healthcheck  https://review.opendev.org/76220304:30
openstackgerritMerged openstack/tripleo-quickstart-extras master: IPA test - adding all forwarders to IPA server  https://review.opendev.org/75794304:30
openstackgerritMerged openstack/tripleo-common stable/victoria: Add heat-engine healthcheck  https://review.opendev.org/76220104:30
*** udesale has quit IRC04:32
*** gthiemonge has quit IRC04:32
*** gthiemonge has joined #tripleo04:32
*** jmasud has quit IRC05:10
*** udesale_ has quit IRC05:11
*** social has quit IRC05:26
*** evrardjp has quit IRC05:33
*** evrardjp has joined #tripleo05:33
*** jfrancoa has joined #tripleo05:47
*** udesale has joined #tripleo05:48
*** ysandeep|away is now known as ysandeep|ruck05:49
openstackgerritJose Luis Franco proposed openstack/tripleo-validations master: DNM: Test for molecule check_network_gateway.  https://review.opendev.org/76236405:55
*** hjensas has quit IRC05:57
*** hjensas has joined #tripleo05:59
*** rcernin_ has joined #tripleo05:59
*** rcernin has quit IRC06:01
*** cloudnull has quit IRC06:01
*** cloudnull has joined #tripleo06:02
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart-extras master: Allow deploying composable networks without heat  https://review.opendev.org/76235006:05
*** mburned is now known as mburned_out06:23
*** marios has joined #tripleo06:27
openstackgerritJose Luis Franco proposed openstack/tripleo-validations master: Add validation to verify undercloud.conf content.  https://review.opendev.org/76116206:29
ysandeep|ruckmschuppert, ade_lee fyi.. periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria job runs are not hitting Duplicate declaration error anymore since yesterday. https://bugs.launchpad.net/tripleo/+bug/190382806:37
openstackLaunchpad bug 1903828 in tripleo "periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria is failing with Error while evaluating a Resource Statement, Duplicate declaration: Exec[/etc/pki/CA/certs/qemu.pem] is already declared" [Critical,Triaged]06:37
ysandeep|ruckhttps://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria/1bf7f31/job-output.txt06:37
ysandeep|ruckBut, I am not sure with patch fixed the issue.06:37
ysandeep|ruckDo you know which patch fixed the issue? So that we can reference in bug while closing it.06:41
openstackgerritIan Wienand proposed openstack/diskimage-builder master: WIP: boot test of containerfile image  https://review.opendev.org/72214806:43
*** lmiccini has joined #tripleo06:48
*** udesale_ has joined #tripleo06:51
*** udesale has quit IRC06:54
*** ade_lee has quit IRC07:03
*** ade_lee has joined #tripleo07:05
mschuppertysandeep|ruck morning . then I think it previously failed as it were running without https://review.opendev.org/#/c/760522/07:08
ysandeep|ruckmschuppert, i can check and confirm that..07:08
ysandeep|ruckmschuppert, thank you :)07:09
ysandeep|ruckmschuppert, you are right http://paste.openstack.org/show/799939/07:17
mschuppertysandeep|ruck perfect! one less thing. now need to hunt why it complains on missing libvirt-vnc cert.07:19
ysandeep|ruckmschuppert, yes.. and libvirt-vnc issue is also seen on victoria branch.. i have updated the bug with this info07:20
ysandeep|rucki will close https://bugs.launchpad.net/tripleo/+bug/190382807:22
openstackLaunchpad bug 1903828 in tripleo "periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria is failing with Error while evaluating a Resource Statement, Duplicate declaration: Exec[/etc/pki/CA/certs/qemu.pem] is already declared" [Critical,Triaged]07:22
mschuppertysandeep|ruck ack07:24
*** rcernin_ has quit IRC07:26
*** suuuper has joined #tripleo07:26
*** slaweq has joined #tripleo07:29
*** suuuper has quit IRC07:34
mschuppertade_lee owalsh from e.g. https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria/2858887/logs/undercloud/var/log/extra/podman/containers/nova_libvirt/podman_info.log.txt.gz the bind mounts are there and the certs exist on the07:38
mschupperthost07:38
marios[tripleo] stein branch transition to extended maintenance  http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018680.html07:46
*** jmasud has joined #tripleo07:57
*** pcaruana has quit IRC07:59
*** waleedm has joined #tripleo08:01
*** amoralej|off is now known as amoralej08:03
*** jcapitao has joined #tripleo08:09
*** jpena|off is now known as jpena08:09
*** rcernin_ has joined #tripleo08:10
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Ansible - create overcloud composable network  https://review.opendev.org/75173908:11
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Get overcloud domain name from ctlplane  https://review.opendev.org/75836808:11
*** pcaruana has joined #tripleo08:12
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Deployed networks template  https://review.opendev.org/75187608:13
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Network data v2 YAML definition for CI  https://review.opendev.org/76234808:13
*** rpittau|afk is now known as rpittau08:19
*** waleedm_ has joined #tripleo08:20
*** rcernin_ has quit IRC08:22
*** waleedm has quit IRC08:23
*** ysandeep|ruck is now known as ysandeep|lunch08:24
*** ccamacho has joined #tripleo08:25
*** cylopez has joined #tripleo08:27
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade stable/train: DNM: Add a east-west ping test for internal connectivity.  https://review.opendev.org/76236208:30
*** waleedm__ has joined #tripleo08:35
*** waleedm has joined #tripleo08:36
*** jbadiapa has joined #tripleo08:37
*** waleedm_ has quit IRC08:38
*** ccamacho has quit IRC08:38
*** waleedm__ has quit IRC08:40
*** morazi_ has joined #tripleo08:42
*** morazi has quit IRC08:42
*** rcernin_ has joined #tripleo08:45
*** ysandeep|lunch is now known as ysandeep|ruck08:54
owalshmschuppert: ack, but does the path the files are bind mounted to exist?08:57
*** rcernin_ has quit IRC08:57
mschuppertowalsh I'll have a reproducer env soon so I can check in detail08:59
owalshack09:00
*** lucasagomes has joined #tripleo09:03
openstackgerritJose Luis Franco proposed openstack/tripleo-validations master: Add validation to verify undercloud.conf content.  https://review.opendev.org/76116209:07
openstackgerritKarthik S proposed openstack/os-net-config master: For linux bonds over VF, the default value of spoofcheck shall be false  https://review.opendev.org/76248109:13
*** ccamacho has joined #tripleo09:16
*** jpich has joined #tripleo09:18
openstackgerritKarthik S proposed openstack/os-net-config master: For linux bonds over VF, the default value of spoofcheck shall be false  https://review.opendev.org/76248109:20
*** gfidente has joined #tripleo09:21
*** paramite has joined #tripleo09:28
*** paramite|clone has quit IRC09:29
*** yolanda has quit IRC09:29
*** yolanda has joined #tripleo09:30
*** matbu has quit IRC09:31
*** matbu has joined #tripleo09:33
*** waleedm has quit IRC09:38
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/ussuri: Run online migration tasks from external_update_tasks too.  https://review.opendev.org/75972509:41
*** bogdando has joined #tripleo09:43
*** zzzeek has quit IRC09:45
*** zzzeek has joined #tripleo09:46
*** bogdando has quit IRC09:52
*** bogdando has joined #tripleo09:53
mschuppertowalsh found the issue. the issue is that qemu user id in the container does no longer match what we have on the host. that is different from what we have in earlier releases e.g. osp16.09:54
mschuppertmight be related to the new image create process?09:54
mschuppertade_lee ^^ fyi09:58
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: CLI playbook: create overcloud composable network  https://review.opendev.org/75173910:00
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Get overcloud domain name from ctlplane  https://review.opendev.org/75836810:00
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: module_utils - network_data_v2  https://review.opendev.org/76248710:00
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Module: tripleo_composable_network  https://review.opendev.org/76248810:00
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Module: tripleo_network_populate_environment  https://review.opendev.org/76248910:00
owalshmschuppert: yea, sounds like a bug in the new image building10:02
openstackgerritYariv Rachmani proposed openstack/tripleo-upgrade master: [FFWD] Propose a fix for node_upgrade_pre.sh.j2  https://review.opendev.org/76183610:10
mschuppertowalsh you know where the code is which now creates the images?10:12
mschupperthttps://github.com/openstack/tripleo-common/blob/master/container-images/kolla/base/uid_gid_manage.sh#L6610:18
*** zzzeek has quit IRC10:22
*** zzzeek has joined #tripleo10:25
owalshmschuppert: https://github.com/openstack/tripleo-ansible/tree/master/tripleo_ansible/roles/tripleo_container_image_build10:26
*** dtantsur|afk is now known as dtantsur10:29
owalshmschuppert: oh, ack, you found the uids :-)10:33
openstackgerritJose Luis Franco proposed openstack/tripleo-common master: Add missing ComputeNeutronMetadataAgent service into tripleo_containers.yaml.j2.  https://review.opendev.org/76142610:35
Tenguhello there! Does this one rings any bell? AnsibleUndefinedVariable: {% if job is defined and job.build_container_images is defined and job.build_container_images|default(false)|bool or build_container_images|default(false)|bool -%} 127.0.0.1:5001 {%- elif job.consumer_job | default(false) | bool  -%} {{ job.registry_ip_address_branch[release] }}:5001 {%- else -%} docker.io {%- endif -%}: 'dict object' has10:43
Tenguno attribute 'registry_ip_address_branch'10:43
Tenguit's here: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fb6/760905/1/check/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-train/fb62779/job-output.txt10:43
Tenguysandeep|ruck: -^^  maybe? :)10:44
ysandeep|ruckTengu, checking10:44
*** udesale__ has joined #tripleo10:51
ysandeep|ruckTengu, i don't we have bug for this yet.. I will write a bug for this10:52
ysandeep|ruckchandankumar, marios ^^ fyi..10:52
openstackgerritMartin Schuppert proposed openstack/tripleo-common master: Change qemu user id to match previous releases  https://review.opendev.org/76249710:52
chandankumarysandeep|ruck: let me take a look,10:52
Tenguthanks folks :)10:53
mschuppertowalsh https://review.opendev.org/76249710:53
Tengumaybe it's just a glitch in the matrix10:53
*** udesale_ has quit IRC10:54
chandankumarissue is here https://opendev.org/openstack/python-tripleoclient/src/branch/stable/train/zuul.d/layout.yaml#L4210:55
chandankumarit is not dependent on content provider10:55
chandankumarlet me check stable/train patches10:55
*** efoley has joined #tripleo10:57
slaweqhi tripleo core team, can You take a look at my patches https://review.opendev.org/#/q/topic:min-bw-stable/ussuri+(status:open+OR+status:merged) and https://review.opendev.org/#/q/topic:min-bw-stable/ussuri-stable/train+(status:open+OR+status:merged) ?10:58
owalshmschuppert: not sure what the commit message/reno make sense10:58
slaweqthx in advance for reviews :)10:58
owalshs/what/that10:58
chandankumarTengu: proposing a patch to cleanup zuul layout sorry for the issue11:00
ysandeep|ruckchandankumar, this job passed - https://review.opendev.org/#/c/761892/5/zuul.d/layout.yaml for same job - may be we just need a rebase11:01
ysandeep|ruckthis patch* passed for same job11:02
Tenguchandankumar: it happens ;)11:02
Tenguthank you for jumping in!11:02
*** rh-jelabarre has quit IRC11:02
Tenguchandankumar: once you get a patch, lemme know so that I can depends-on if needed.11:02
*** rh-jelabarre has joined #tripleo11:02
chandankumarysandeep|ruck: yes correct11:02
Tenguysandeep|ruck: apparently branch is up-to-date for the change: https://review.opendev.org/#/c/760905/11:03
chandankumarTengu: can you rebase on top of this https://review.opendev.org/#/c/761892/11:03
mschuppertowalsh ok let me rephrase it11:03
Tenguah. that. but it failed anyway.. ?11:03
chandankumarrecheck fun is going on11:03
Tengufine :)11:03
openstackgerritCédric Jeanneret (Tengu) proposed openstack/python-tripleoclient stable/train: Do not require authentication for the validator CLI  https://review.opendev.org/76090511:04
Tengurebased -^11:04
chandankumarTengu: ysandeep|ruck thanks!11:04
Tenguwell, thank you both for the pointer :)11:04
ysandeep|ruckchandankumar, Tengu thanks, and i have still opened bug for tracking https://bugs.launchpad.net/tripleo/+bug/190398011:06
openstackLaunchpad bug 1903980 in tripleo "tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-train job is failing on TASK [undercloud-deploy : Write containers-prepare-parameter.yaml] with error 'dict object' has no attribute 'registry_ip_address_branch'" [Critical,Triaged]11:06
mariosysandeep|ruck: in call will check in bit11:07
ysandeep|ruckmarios, ignore.. sorted thanks11:07
openstackgerritMartin Schuppert proposed openstack/tripleo-common master: Change qemu user id to match previous releases  https://review.opendev.org/76249711:09
openstackgerritMerged openstack/tripleo-ansible stable/ussuri: Add support for rdo openvswitch layered upgrade special treatment.  https://review.opendev.org/76185611:09
mschuppertowalsh updated11:09
owalshmschuppert: LGTM11:11
mschuppertowalsh cool thanks!11:11
mariosysandeep|ruck: thanks11:17
*** marios has quit IRC11:19
bogdando12:26:24 PM - bogdando: hi, I'm running TCIB in a local sandbox, to investigate how the --layers works in buildah... but the build command doesn't work for me with "vanilla" ubi8.211:27
bogdando12:26:35 PM - bogdando: at the least I had to install there delorean11:27
bogdando12:27:02 PM - bogdando: I thought TCIB (including ustream CI jobs) builds out of box, vanilla ubi, no hacks?do I miss anything11:27
bogdando12:27:40 PM - bogdando: anyway, with added tripleo repos, it goes forward to fail the next step - nothing provides dumb-init :(11:27
bogdando12:27:48 PM - bogdando: any help is appreciated11:27
bogdandooh, I got it, that TCIB relies on bind mounted repos from hosts, and more things like /usr/share/tripleo-common11:29
bogdandoso I need all that magic locally as well... okay11:29
bogdando...so what on earth provides dumb-init :)11:30
bogdando(for our CI nodes' repos)11:30
*** rfolco has joined #tripleo11:39
*** jcapitao is now known as jcapitao_lunch11:40
*** jaosorior has quit IRC11:43
*** jaosorior has joined #tripleo11:53
*** jmasud has quit IRC11:54
*** mburned_out is now known as mburned12:14
*** marios has joined #tripleo12:15
openstackgerritMerged openstack/openstack-tempest-skiplist master: Create tox environment to build pdf documentation  https://review.opendev.org/75942712:16
*** tosky has joined #tripleo12:18
*** waleedm has joined #tripleo12:31
*** waleedm_ has joined #tripleo12:32
*** rlandy has joined #tripleo12:32
*** jpena is now known as jpena|lunch12:32
*** holser has joined #tripleo12:34
*** waleedm has quit IRC12:36
*** dmacpher_ has quit IRC12:46
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo stable/victoria: Allow to specify a nic for the VIPs  https://review.opendev.org/76196312:47
*** waleedm_ has quit IRC12:50
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add a debug tasks on  pull_success output  https://review.opendev.org/76240012:53
*** jcapitao_lunch is now known as jcapitao12:54
*** artom has joined #tripleo12:54
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM - force fail on pull  https://review.opendev.org/76241712:55
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Adding Ceph Dashboard to the Edge roles  https://review.opendev.org/76143712:55
rlandymarios: ^^ pls see two patches above12:55
*** ysandeep|ruck is now known as ysandeep|mtg13:01
mariosrlandy: ack will do13:02
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo stable/train: Generic cluster properties support  https://review.opendev.org/76251613:08
openstackgerritRaildo Mascena proposed openstack/puppet-tripleo master: Adding key_size option on the certmonger_certificate function  https://review.opendev.org/75714213:10
*** amoralej is now known as amoralej|lunch13:13
*** jaosorior has quit IRC13:14
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Module: tripleo_composable_network  https://review.opendev.org/76248813:14
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Module: tripleo_network_populate_environment  https://review.opendev.org/76248913:14
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: CLI playbook: create overcloud composable network  https://review.opendev.org/75173913:14
openstackgerritHarald Jensås proposed openstack/tripleo-ansible master: Get overcloud domain name from ctlplane  https://review.opendev.org/75836813:14
*** jpena|lunch is now known as jpena13:29
openstackgerritMartin Mágr proposed openstack/puppet-tripleo master: [WIP] Add posibility to download scripts for sensubility  https://review.opendev.org/75441813:35
openstackgerritMerged openstack/tripleo-ansible master: Remove the need for docker  https://review.opendev.org/76220013:44
*** pleimer has joined #tripleo13:49
*** Goneri has joined #tripleo13:53
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-repos master: Add support for UBI8 as a subcase of RHEL8  https://review.opendev.org/76252013:58
openstackgerritMartin Mágr proposed openstack/puppet-tripleo master: Add posibility to download scripts for sensubility  https://review.opendev.org/75441813:59
*** TrevorV has joined #tripleo14:01
*** waleedm has joined #tripleo14:06
*** amoralej|lunch is now known as amoralej14:10
openstackgerritSergii Golovatiuk proposed openstack/tripleo-heat-templates stable/train: Run online migration tasks from external_update_tasks too.  https://review.opendev.org/76252414:15
mwhahahaweshay|ruck: have we seen less failures since https://review.opendev.org/#/c/761996/1 merged?14:17
weshay|ruckmwhahaha, in planning atm but .. but will run some queries and report back to you14:19
mwhahahak14:19
*** waleedm_ has joined #tripleo14:22
*** d34dh0r53 has joined #tripleo14:23
*** waleedm has quit IRC14:24
openstackgerritMerged openstack/python-tripleoclient stable/victoria: Removed ubi-8 based jobs  https://review.opendev.org/76186414:34
openstackgerritMerged openstack/tripleo-ci master: Install python/pyton3-setuptools based on python version  https://review.opendev.org/76185414:34
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-repos master: Add support for UBI8 as a subcase of Centos8  https://review.opendev.org/76252014:34
*** artom has quit IRC14:37
*** artom has joined #tripleo14:37
openstackgerritMerged openstack/tripleo-heat-templates master: Remove mistral parameters from undercloud.yaml  https://review.opendev.org/76104214:46
openstackgerritMerged openstack/os-net-config master: Replace deprecated UPPER_CONSTRAINTS_FILE variable  https://review.opendev.org/76137314:46
openstackgerritMerged openstack/os-net-config master: Update master for stable/victoria  https://review.opendev.org/76058814:46
openstackgerritMerged openstack/os-net-config master: Add Python3 wallaby unit tests  https://review.opendev.org/76058914:46
*** waleedm has joined #tripleo14:46
*** waleedm_ has quit IRC14:48
*** udesale_ has joined #tripleo14:52
*** udesale__ has quit IRC14:55
*** waleedm_ has joined #tripleo14:55
*** waleedm has quit IRC14:59
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart master: fs001, provision networks prior to heat stack  https://review.opendev.org/76235115:01
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart-extras master: Allow deploying composable networks without heat  https://review.opendev.org/76235015:02
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible stable/victoria: Remove the need for docker  https://review.opendev.org/76253415:03
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible stable/ussuri: Remove the need for docker  https://review.opendev.org/76253515:03
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible stable/train: Use the UBI8 image for testing  https://review.opendev.org/75446015:05
ade_leemschuppert, hows it going?15:07
mschuppertade_lee I have found the root cause for it, check my updates on https://bugs.launchpad.net/tripleo/+bug/1903508 . this is the fix I submitted for it https://review.opendev.org/#/c/762497/15:09
openstackLaunchpad bug 1903508 in tripleo "periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa and featureset039 on master/victoria are failing on tempest test while spawning instance with Error - Cannot load certificate '/etc/pki/libvirt-vnc/server-cert.pem' & key '/etc/pki/libvirt-vnc/server-key.pem': Error while reading file."" [Critical,In progress] - Assigned to Martin Schuppert (mschuppert)15:09
ade_leemschuppert, nice catch on the uid15:09
ade_leemschuppert, ack - just read them15:09
*** yolanda has quit IRC15:11
openstackgerritMartin Schuppert proposed openstack/tripleo-common master: Change qemu user id to match previous releases  https://review.opendev.org/76249715:13
openstackgerritMartin Schuppert proposed openstack/tripleo-common master: Change qemu user id to match previous releases  https://review.opendev.org/76249715:15
openstackgerritMerged openstack/ansible-role-chrony master: Bump pyflakes  https://review.opendev.org/76194115:16
openstackgerritMerged openstack/ansible-role-openstack-operations master: Bump pyflakes and pin ansible-lint  https://review.opendev.org/76195115:20
openstackgerritMerged openstack/tripleo-ipsec master: Skip ansible-lint 106  https://review.opendev.org/76197615:20
openstackgerritMerged openstack/ansible-role-redhat-subscription master: Bump pyflakes, exclude ansible-lint 106  https://review.opendev.org/76195315:21
*** dsneddon has quit IRC15:25
*** udesale has joined #tripleo15:28
*** udesale has quit IRC15:29
*** udesale_ has quit IRC15:30
*** udesale has joined #tripleo15:30
*** udesale has quit IRC15:31
*** udesale has joined #tripleo15:31
*** udesale has quit IRC15:32
*** udesale has joined #tripleo15:33
*** udesale has quit IRC15:34
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: ironic: stop defaulting to iscsi deploy in preparation for its deprecation  https://review.opendev.org/74947515:36
*** udesale has joined #tripleo15:37
*** waleedm_ has quit IRC15:37
*** udesale has quit IRC15:38
*** skramaja has quit IRC15:38
*** udesale has joined #tripleo15:38
*** udesale has quit IRC15:40
*** udesale has joined #tripleo15:40
*** rlandy is now known as rlandy|rover15:40
*** marios is now known as marios|ruck15:40
pleimerIt appears rdo CI is down on, anyway I can still get this merged? https://review.opendev.org/#/c/760739/15:42
zbrcloudnull: you are using testinfra with molecule, right? if so review https://github.com/ansible-community/molecule/pull/2962 to avoid a surprise.15:45
weshay|ruckpleimer, to merge you just need the upstream jobs to +1, and you have that15:47
weshay|ruckyou should get 3rd party to +1 but it's not mandatory15:47
weshay|rucknot the best results atm https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset001&job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset00115:49
*** ysandeep|mtg is now known as ysandeep15:54
*** ysandeep is now known as ysandeep|away15:56
*** ysandeep|away is now known as ysandeep|holiday15:56
*** udesale has quit IRC15:58
*** udesale has joined #tripleo16:01
*** udesale has quit IRC16:01
*** udesale has joined #tripleo16:02
*** pleimer has quit IRC16:07
*** pleimer has joined #tripleo16:07
pleimerweshay: I do not have the permissions to merge it myself16:18
weshay|ruckk16:18
cloudnullzbr does that break the use of inventory files ?16:18
cloudnullif so then I think that will break everyone using testinfra16:19
cloudnullwhich seems like a bad idea16:19
zbrcloudnull: no idea, clearly it does not break the tests.16:20
zbrthat is why i asked, i did had the impression that it may break but I am not sure.16:21
openstackgerritMarios Andreou proposed openstack/tripleo-upgrade master: Add the containers-prepare-parameter to the update prepare  https://review.opendev.org/76254716:21
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart master: fs001, provision networks prior to heat stack  https://review.opendev.org/76235116:24
*** udesale has quit IRC16:27
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-common master: [DNM] testing failover registry  https://review.opendev.org/75984916:34
*** ykarel is now known as ykarel|holiday16:34
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-common master: [DNM] testing failover registry  https://review.opendev.org/75984916:39
*** ykarel|holiday has quit IRC16:48
rlandy|roverweshay|ruck: fs020 was taken out of criteria on master -was that temp?16:49
*** chandankumar is now known as raukadah16:52
weshay|ruckrlandy|rover, so we're covered by the standalone-full-scenario and standalone-full-api16:54
openstackgerritRafael Folco proposed openstack/tripleo-ci master: html build report for tcib container builds  https://review.opendev.org/75512416:55
weshay|ruckrlandy|rover, if the test passes on standalone, the failure are suspect in fs20 imho16:55
rlandy|roverweshay|ruck: k - just didn't know if I should reverse that commented out skip16:55
weshay|ruckrlandy|rover, I use fs020 as additional info, but not criteria .. when it's passing consistently I put it back in as criteria so at least we're alerted and we're not ignoring16:57
*** lucasagomes has quit IRC17:00
*** marios|ruck is now known as marios|out17:01
*** rlandy|rover is now known as rlandy|rover|brb17:03
*** jcapitao has quit IRC17:05
*** tmazur has joined #tripleo17:14
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade stable/train: DNM: Improve coverage of ping test.  https://review.opendev.org/76255217:18
*** amoralej is now known as amoralej|off17:22
*** cfontain_ has joined #tripleo17:22
*** jmasud has joined #tripleo17:24
*** pleimer has quit IRC17:27
*** pleimer has joined #tripleo17:27
*** dtantsur is now known as dtantsur|afk17:34
*** marios|out has quit IRC17:37
*** rlandy|rover|brb is now known as rlandy|rover17:45
*** cfontain_ has quit IRC17:46
*** jpena is now known as jpena|off17:49
*** rpittau is now known as rpittau|afk17:51
*** pleimer has quit IRC17:51
*** lmiccini has quit IRC17:52
*** jmasud has quit IRC17:53
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Fix wait condition for retry  https://review.opendev.org/76255817:54
*** jpich has quit IRC17:55
*** pleimer has joined #tripleo17:57
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Reimplement "Drop --layers from buildah"  https://review.opendev.org/76256118:02
*** jmasud has joined #tripleo18:02
weshay|ruckmwhahaha, when you have a sec..18:10
weshay|ruckhttp://dashboard-ci.tripleo.org/d/3-DYSmOGk/jobs-exploration?orgId=1&var-influxdb_filter=job_name%7C%3D~%7C%2Ftripleo-ci-centos-8-content-provide.*%2F18:10
weshay|ruckthe container-buidlds look pretty solid 92+ pass rate..18:11
mwhahahaup from 80 something?18:11
weshay|ruckadded a new module at the very bottom to track builds and failures at the bottom18:11
weshay|rucknot seeing much of a trend to be honest18:11
* weshay|ruck changes dates.. to see if I can see it much lower18:11
weshay|rucksec18:11
weshay|ruck85%18:12
weshay|ruckhttp://dashboard-ci.tripleo.org/d/3-DYSmOGk/jobs-exploration?orgId=1&var-influxdb_filter=job_name%7C%3D~%7C%2Ftripleo-ci-centos-8-content-provide.*%2F&from=1602612725230&to=160442712523018:12
*** ykarel|holiday has joined #tripleo18:12
weshay|ruckso it's improved18:12
weshay|ruckoh boy.. slow as hell18:14
weshay|ruckgrafana that is18:14
*** pcaruana has quit IRC18:14
*** gfidente is now known as gfidente|afk18:16
*** numans_ is now known as numans18:16
weshay|ruckcan't have nice things18:16
weshay|ruckrestarting18:18
weshay|ruckis vexx going down?18:19
weshay|ruckjebus18:19
weshay|ruckmwhahaha, ok.. it's back.. was my chrome.. anyhoo.. I'll watch it.. but 85 -> 92 is an improvement..18:21
weshay|rucknot sure what to say..18:22
openstackgerritMichele Baldessari proposed openstack/paunch stable/ussuri: Fix inspect images calls to not use the index format keyword  https://review.opendev.org/76256318:23
*** ykarel|holiday has quit IRC18:24
*** gfidente|afk has quit IRC18:27
*** psachin has quit IRC18:28
*** jmasud has quit IRC18:45
*** ykarel|holiday has joined #tripleo18:52
*** bogdando has quit IRC18:59
*** ykarel|holiday has quit IRC19:01
openstackgerritMichele Baldessari proposed openstack/paunch stable/train: Fix inspect images calls to not use the index format keyword  https://review.opendev.org/76256719:03
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add better exception logging for builds  https://review.opendev.org/76257119:17
fultonjmwhahaha: https://bugs.launchpad.net/tripleo/+bug/1903775/comments/119:18
openstackLaunchpad bug 1903775 in tripleo "tripleo-admin user is created on controllers but not computes" [High,Triaged]19:18
mwhahahafun19:18
fultonjRoleNetIpMap IS missing the ctlplane IPs for the computes and ceph-storage according to heat19:18
fultonjso i need to backtrack further to stack creation19:19
mwhahahahjensas might have some thoughts on that19:19
*** amoralej|off is now known as amoralej19:59
openstackgerritMerged openstack/tripleo-ansible stable/victoria: Remove the need for docker  https://review.opendev.org/76253420:01
hjensasfultonj: radez had a similar issue yesterday.20:03
fultonjhjensas: ok, that's good to know20:03
fultonjper comment #3 do you think something wrong with my hostname map?20:03
fultonjhttps://bugs.launchpad.net/tripleo/+bug/190377520:03
openstackLaunchpad bug 1903775 in tripleo "tripleo-admin user is created on controllers but not computes" [High,Triaged]20:03
fultonjwhere in tripleo is the code that converts the DeployedServerPortMap into RoleNetIpMap?20:05
fultonj(assuming that's even what happens)20:05
fultonjhjensas: ^ i assume this because DeployedServerPortMap is pased as input to the heat stack creation and RoleNetIpMap is an attribute of the heat stack20:05
fultonjradez: are you still stuck on this?20:06
radezI'm not, my problem was the overcloud node I was trying to add to the clusted had failed to deploy20:07
radezI delete the nova server record manually instead of through the overcloud node delete command20:07
radezso ooo thought there was supposed to be a node with a port with an ip assigned to it20:08
radezbut there wasn't because I had deleted it.20:08
radezWhen I deleted the overcloud node record I stopped getting that error and was able to successfully deploy in a different env, so I just ditched the env I had the error in20:09
radezfultonj: not sure if that helps you or not20:09
hjensasfultonj: the DeployedServerPortMap is input, does it not have the ctlplane IPs?20:10
hjensasfultonj: comment #3 in the bug looks truncated?20:10
fultonjhjensas: it does have trhe ctrol plane ips20:10
fultonjhjensas: i truncated it to see the hostname in its many forms20:11
fultonjfull output is available however and attached20:11
fultonjcomment #220:11
fultonjhttps://launchpadlibrarian.net/506685163/overcloud-baremetal-deployed-0.yaml20:11
fultonjyou can see DeployedServerPortMap has 192.168.24.0/24 IPs for each node20:12
hjensasfultonj:     oc0-ceph-1-ctlplane:20:12
hjensas      fixed_ips:20:12
hjensas      - ip_address: 192.168.24.2420:12
hjensas^^ looks right.20:13
fultonjhttp://paste.openstack.org/show/799971/20:13
fultonjand i can ping them all ^20:13
fultonjradez: were you using metalsmith?20:13
hjensasfultonj: you have OS::TripleO::DeployedServer::ControlPlanePort: /usr/share/openstack-tripleo-heat-templates/deployed-server/deployed-neutron-port.yaml in the environment?20:14
fultonjradez: you had said nova which made me think you were using heat > nova > ironic20:14
radezfultonj: no just scaling out extra networker nodes thought director20:14
radezie heat, nova, ironic20:14
* hjensas needs to night the kid. I'll check back later.20:14
fultonjok, thanks hjensas20:15
fultonjradez: ok thanks seems a little different in my case20:15
fultonji have environments/deployed-server-environment.yaml20:15
fultonjand20:15
fultonjhttps://launchpadlibrarian.net/506685163/overcloud-baremetal-deployed-0.yaml20:15
fultonjshould show that20:16
fultonjresource_registry: OS::TripleO::DeployedServer::ControlPlanePort: /usr/share/openstack-tripleo-heat-templates/deployed-server/deployed-neutron-port.yaml20:16
fultonjwas included20:16
fultonjthat file exists on my filesystem too20:16
openstackgerritMerged openstack/os-net-config stable/ussuri: Fixing the dependency for sriov_config service  https://review.opendev.org/76246020:23
openstackgerritMerged openstack/tripleo-heat-templates stable/victoria: Properly compute hostname when looking for the ceph-mon container  https://review.opendev.org/76228320:27
openstackgerritMerged openstack/tripleo-image-elements master: Bump pyflakes  https://review.opendev.org/76197420:27
openstackgerritMerged openstack/os-refresh-config master: Bump pyflakes  https://review.opendev.org/76198720:27
openstackgerritMerged openstack/os-collect-config master: Bump pyflakes  https://review.opendev.org/76195620:27
*** d34dh0r53 has quit IRC20:29
*** d34dh0r53 has joined #tripleo20:31
*** rcernin has joined #tripleo20:32
*** TrevorV has quit IRC20:34
*** rcernin has quit IRC20:35
openstackgerritAlex Schultz proposed openstack/ansible-role-container-registry master: Bump pyflakes and fix missing permissions  https://review.opendev.org/76194320:38
*** efoley has quit IRC20:38
openstackgerritSteve Baker proposed openstack/tripleo-docs master: WIP document new baremetal provision config_drive YAML  https://review.opendev.org/76257820:41
*** bandini has quit IRC20:47
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Move cell_v2 discovery off compute hosts  https://review.opendev.org/76257920:48
*** ccamacho has quit IRC20:49
openstackgerritOliver Walsh proposed openstack/puppet-tripleo master: Stop including nova::metadata on computes  https://review.opendev.org/76241520:54
*** jmasud has joined #tripleo20:58
*** mrda has quit IRC20:58
*** jmasud has quit IRC21:01
*** mrda has joined #tripleo21:01
openstackgerritSteve Baker proposed openstack/tripleo-ansible master: Add config_drive support to baremetal provision YAML  https://review.opendev.org/76258021:04
openstackgerritSteve Baker proposed openstack/tripleo-docs master: Document new baremetal provision config_drive YAML  https://review.opendev.org/76257821:04
*** cylopez has left #tripleo21:12
*** artom has quit IRC21:15
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Refactor nova db config  https://review.opendev.org/71855221:18
*** jbadiapa has quit IRC21:30
*** slaweq has quit IRC21:30
hjensasfultonj: did you figure it out?21:32
fultonjhjensas: no21:33
*** slaweq has joined #tripleo21:33
fultonji have a theory that21:33
fultonjfor some reason the DeployedServerPortMap isn't getting translated into a correct RoleNetIpMap21:34
fultonjdoes the hostname map play a role in that translation ?21:35
fultonjhjensas: to answer your previous quesiotn, yes i have a /usr/share/openstack-tripleo-heat-templates/deployed-server/deployed-neutron-port.yaml21:35
hjensasfultonj: ack, it does look like the input is to the stack is correct.21:36
openstackgerritMerged openstack/tripleo-heat-templates stable/train: [train-only] Switch sriov agent in hybrid state  https://review.opendev.org/75495221:36
openstackgerritMerged openstack/tripleo-heat-templates master: Remove old nic config templates  https://review.opendev.org/75964021:36
openstackgerritMerged openstack/tripleo-heat-templates stable/train: [train-only] Introduce hybrid state also for ovn_controller  https://review.opendev.org/75604221:36
openstackgerritMerged openstack/tripleo-quickstart-extras master: Bump linters  https://review.opendev.org/76135321:36
fultonjdo you see anything in the input that doesn't look right?21:36
fultonjis it true that stuff(DeployedServerPortMap, HostnameMap) --> RoleNetIpMap ?21:37
hjensasfultonj: Nope it looks ok, I see composable networks have the _0 suffix, but that I belive is how tripleo-lab does things?21:37
fultonjyes, i'm using tripleo lab21:38
fultonjhttps://github.com/fultonj/victoria/blob/master/standard/deploy.sh#L5221:38
fultonjis how i'm deploying21:38
*** slaweq has quit IRC21:41
fultonjthe Hostname map: novacompute vs compute, cephstorage vs ceph, and overcloud vs oc021:42
fultonjcould that be messing something up?21:42
fultonj(these paramters worked fine on a UC i built 3 weeks ago)21:43
hjensasfultonj: yes, DeployedServerPortMap -> carries the properties for the deployed-neutron-port.yaml, wich is used as OS::TripleO::DeployedServer::ControlPlanePort in deployed-server/deployed-server.yaml21:43
hjensasfultonj: deployed-server/deployed-server.yaml outputs in L83-L86 the same 'networks' structure that a OS::Nova::Server resource would.21:44
hjensasfultonj: and I don't see any changes in those files for quite some time.21:44
fultonjok21:44
*** rcernin has joined #tripleo21:47
openstackgerritAlan Bishop proposed openstack/puppet-tripleo master: Support configuring multiple Netapp cinder backends  https://review.opendev.org/76258421:47
openstackgerritAlan Bishop proposed openstack/tripleo-heat-templates master: Support deploying multiple Cinder Netapp Storage backends  https://review.opendev.org/76258521:47
*** rcernin has quit IRC22:01
hjensasfultonj: I wonder if it's the hostnames not matching. looking at the stack output the hosts value is not right as well.22:04
hjensasfultonj: " oc0-cephstorage-2.ctlplane.mydomain.tld oc0-cephstorage-2.ctlplane"22:04
openstackgerritwes hayutin proposed openstack/python-tripleoclient stable/ussuri: [ussuri] Migrate to content provider jobs/template  https://review.opendev.org/75783622:04
openstackgerritwes hayutin proposed openstack/tripleo-ci master: Adds layouts for victoria upgrade and update jobs  https://review.opendev.org/76032322:05
fultonj[heat-admin@oc0-ceph-0 ~]$ hostname22:05
fultonjoc0-ceph-022:05
*** pleimer has quit IRC22:05
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-specs master: Add spec for the removal of Swift from the Undercloud  https://review.opendev.org/75207822:06
hjensasfultonj: if in your metal-big.yaml set 'hostname_format: %stackname%-ceph-%index%' and hostname_format: %stackname%-compute-%index%' ?22:06
*** Goneri has quit IRC22:07
*** raildo has quit IRC22:07
fultonjhjensas: http://paste.openstack.org/show/799973/22:08
fultonjso try again with ^ ?22:08
hjensasfultonj: yeah, that looks about right. Give it a go.22:09
fultonjwould i also need to change this?22:09
fultonjHostnameMap:22:09
fultonj    overcloud-0-cephstorage-0: oc0-ceph-022:09
fultonj    overcloud-0-novacompute-0: oc0-compute-022:10
hjensasfultonj: ah, yes.22:10
fultonjsimilar removal of "storage" and "nova"?22:10
fultonjoik22:10
hjensasfultonj: if that works, I think the entries in overcloud-baremetal-deployed-0.yaml which is generated by the baremetal provision workflow in TripleO might not do it right.22:10
*** tmazur has quit IRC22:10
fultonjhttp://paste.openstack.org/show/799974/22:11
fultonjright ? ^22:11
hjensasfultonj: let's try it?22:12
fultonjhjensas: ok, thank you22:12
fultonji've got a deploy running now. i'm going to make some dinner and i'll update the bz with any info i find22:13
fultonjthanks for your help hjensas22:13
fultonj(may not be until tomorrow)22:14
*** rcernin has joined #tripleo22:14
hjensasfultonj: ok, enjoy!22:14
*** jmasud has joined #tripleo22:27
*** zbitter has joined #tripleo22:29
*** matbu has quit IRC22:29
*** zaneb has quit IRC22:31
*** zzzeek has quit IRC22:34
*** zzzeek has joined #tripleo22:34
openstackgerritMerged openstack/tripleo-ansible master: module_utils - network_data_v2  https://review.opendev.org/76248722:38
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Deployed networks template  https://review.opendev.org/75187622:42
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart master: fs001, provision networks prior to heat stack  https://review.opendev.org/76235122:43
*** tosky has quit IRC22:44
*** jmasud has quit IRC22:48
openstackgerritAlex Schultz proposed openstack/ansible-role-container-registry master: Bump pyflakes and fix missing permissions and lint  https://review.opendev.org/76194322:48
*** zbitter is now known as zaneb22:49
mwhahahacloudnull, EmilienM: https://review.opendev.org/#/c/762563/22:50
EmilienMok22:51
mwhahahastupid podman22:51
*** jmasud has joined #tripleo22:56
*** jmasud has quit IRC22:58
*** jmasud has joined #tripleo22:59
*** jmasud has quit IRC23:03
fultonjhjensas: that was the fix23:05
mwhahahafultonj: so the issue was bad hostname matching for hostnamemap?23:08
mwhahahawe should probably put a check in tripleoclient for that or somewhere23:08
mwhahahalike if you define values in hostnamemap that cannot be matches against role hostnames23:08
mwhahahafail23:08
fultonjmwhahaha: basically yeah23:08
fultonjhttp://paste.openstack.org/show/799974/23:08
fultonjmade it work23:08
fultonjbut that filed was genereated by 'openstack overcloud node provision'23:09
openstackgerritMerged openstack/tripleo-ansible stable/ussuri: Remove the need for docker  https://review.opendev.org/76253523:09
fultonji'm going to update the bug23:09
mwhahahawait you changed them from cephstorage in both places and it worked?23:09
mwhahahai mean it seems to have been consistent. weird23:09
fultonjthe host didn't get the names specified by ComputeHostnameFormat23:12
fultonjhttp://paste.openstack.org/show/799975/23:12
mwhahahahmm i see23:12
fultonjso i updated the $ROLEHostnameFormat to be what the hosts actually got23:12
*** Goneri has joined #tripleo23:18
fultonjhttps://bugs.launchpad.net/tripleo/+bug/1903775/comments/423:21
openstackLaunchpad bug 1903775 in tripleo "tripleo-admin user is created on controllers but not computes" [High,Triaged]23:21
fultonjhjensas mwhahaha ^23:21
fultonjso we could use this bug to track an improvement like the validation in the client and a change to overcloud node provsion23:21
*** jmasud has joined #tripleo23:22
mwhahahastill odd about the whole hostname thing23:23
*** frenzy_friday has quit IRC23:24
*** jmasud has quit IRC23:27
*** jmasud has joined #tripleo23:30
*** gthiemonge has quit IRC23:45
*** gthiemonge has joined #tripleo23:46
*** jmasud has quit IRC23:54

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!