Tuesday, 2021-01-12

*** gthiemonge has quit IRC00:14
*** gthiemonge has joined #tripleo00:15
*** tosky has quit IRC00:15
*** rlandy has quit IRC00:21
openstackgerritMerged openstack/python-tripleoclient master: Run flake8 using pre-commit  https://review.opendev.org/c/openstack/python-tripleoclient/+/76992100:50
openstackgerritMichele Valsecchi proposed openstack/tripleo-heat-templates stable/victoria: Add SwiftHashPrefix parameter  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76950701:02
*** pleimer has joined #tripleo01:50
*** jmasud has quit IRC01:57
*** pmannidi has quit IRC02:02
*** dwilde has quit IRC02:04
*** dwilde has joined #tripleo02:07
*** dwilde has quit IRC02:07
*** dwilde has joined #tripleo02:08
*** pmannidi has joined #tripleo02:22
openstackgerritDavid Vallee Delisle proposed openstack/tripleo-heat-templates master: Deleting nova-consoleauth services in post-upgrade  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77021302:33
openstackgerritMerged openstack/tripleo-heat-templates master: Serialize shutdown of pacemaker nodes  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/75924202:43
*** dwilde has quit IRC02:44
*** jmasud has joined #tripleo02:48
*** dsneddon has quit IRC02:57
*** skramaja has joined #tripleo03:13
*** rcernin_ has quit IRC03:14
openstackgerritMerged openstack/tripleo-heat-templates stable/train: [Train Only] Make the docker restart in hybrid state idempotent.  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76626203:30
*** psachin has joined #tripleo03:32
*** d34dh0r53 has quit IRC03:32
*** d34dh0r53 has joined #tripleo03:33
openstackgerritchandan kumar proposed openstack/tripleo-quickstart-extras master: Sync pkgs from host to container in standalone upgrade  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76708203:34
*** d34dh0r53 has quit IRC03:35
*** pleimer has quit IRC03:35
*** d34dh0r53 has joined #tripleo03:35
*** jmasud has quit IRC03:35
*** pleimer has joined #tripleo03:45
*** zzzeek has quit IRC04:06
*** zzzeek has joined #tripleo04:07
*** d34dh0r53 has quit IRC04:19
*** d34dh0r53 has joined #tripleo04:20
*** d34dh0r53 has quit IRC04:22
*** d34dh0r53 has joined #tripleo04:22
*** pleimer has quit IRC04:22
*** rcernin_ has joined #tripleo04:25
*** rcernin_ has quit IRC04:25
*** rcernin has joined #tripleo04:25
*** pleimer has joined #tripleo04:30
*** udesale has joined #tripleo04:50
*** pojadhav|away is now known as pojadhav04:51
*** pleimer has quit IRC04:55
*** d34dh0r53 has quit IRC04:59
*** d34dh0r53 has joined #tripleo04:59
*** d34dh0r53 has quit IRC05:01
*** d34dh0r53 has joined #tripleo05:01
*** d34dh0r53 has quit IRC05:03
*** d34dh0r53 has joined #tripleo05:03
*** d34dh0r53 has quit IRC05:03
*** d34dh0r53 has joined #tripleo05:04
*** saneax has joined #tripleo05:09
*** ykarel has joined #tripleo05:23
*** ykarel_ has joined #tripleo05:26
*** ykarel has quit IRC05:29
openstackgerritSaravanan KR proposed openstack/os-net-config master: Exclude DPDK mapped VF device even if not present  https://review.opendev.org/c/openstack/os-net-config/+/77023005:45
*** jmasud has joined #tripleo05:55
*** ratailor has joined #tripleo05:56
*** ykarel_ has quit IRC06:01
openstackgerritMerged openstack/tripleo-validations stable/train: replace max_fds with sysctl call  https://review.opendev.org/c/openstack/tripleo-validations/+/76430106:02
openstackgerritMerged openstack/tripleo-heat-templates master: Use RootStackName parameter instead for consistency  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76688706:02
openstackgerritMerged openstack/tripleo-heat-templates master: Adding an optional startup delay to nova-compute  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76600506:02
openstackgerritMerged openstack/tripleo-common master: TCIB: Add cinder-volume extend_start.sh script  https://review.opendev.org/c/openstack/tripleo-common/+/76965906:03
openstackgerritMerged openstack/os-net-config stable/train: Wire up new tripleo upgrades jobs template  https://review.opendev.org/c/openstack/os-net-config/+/76949406:03
openstackgerritMerged openstack/tripleo-heat-templates stable/train: move c7 container upgrades to nv in train tht  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76998906:03
openstackgerritMerged openstack/tripleo-heat-templates stable/train: Adding key_size option on the certificate creation  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76951306:09
openstackgerritMerged openstack/puppet-tripleo master: Fix manila-share with ceph-nfs backend  https://review.opendev.org/c/openstack/puppet-tripleo/+/76990606:10
*** ksambor has quit IRC06:15
openstackgerritRabi Mishra proposed openstack/python-tripleoclient master: Create overcloudrc using stack environment  https://review.opendev.org/c/openstack/python-tripleoclient/+/77009606:17
*** ykarel has joined #tripleo06:20
*** rcernin has quit IRC06:26
*** rcernin has joined #tripleo06:26
*** ksambor has joined #tripleo06:30
*** hoonetorg has quit IRC06:34
*** marios has joined #tripleo06:44
*** lmiccini has joined #tripleo06:46
*** jmasud has quit IRC06:55
*** jmasud has joined #tripleo06:58
openstackgerritTakashi Kajinami proposed openstack/puppet-tripleo master: Remove useless parameters to manage keystone resources and volume type  https://review.opendev.org/c/openstack/puppet-tripleo/+/76838007:01
*** zzzeek has quit IRC07:02
openstackgerritTakashi Kajinami proposed openstack/puppet-tripleo master: Remove parameters to manage keystone resources and volume type  https://review.opendev.org/c/openstack/puppet-tripleo/+/76838007:03
*** hakhande has joined #tripleo07:04
*** amoralej|off is now known as amoralej07:05
*** zzzeek has joined #tripleo07:07
*** rcernin has quit IRC07:07
*** jcapitao has joined #tripleo07:09
*** ykarel_ has joined #tripleo07:11
*** ykarel has quit IRC07:12
*** mmethot has quit IRC07:29
*** saneax has quit IRC07:37
*** bandini has joined #tripleo07:43
*** jpena|off is now known as jpena07:46
*** ykarel_ has quit IRC07:55
*** ykarel_ has joined #tripleo07:56
*** ratailor has quit IRC07:56
*** ratailor has joined #tripleo07:56
*** ykarel__ has joined #tripleo07:58
*** hjensas has quit IRC07:59
openstackgerritxinliang proposed openstack/diskimage-builder master: Fix building error with element dracut-regenerate  https://review.opendev.org/c/openstack/diskimage-builder/+/77024108:00
*** ykarel__ is now known as ykarel08:01
*** ykarel_ has quit IRC08:01
*** slaweq has joined #tripleo08:03
*** rpittau|afk is now known as rpittau08:08
*** gfidente|afk is now known as gfidente08:13
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart-extras master: Allow deploying composable networks without heat  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76235008:14
openstackgerritHarald Jensås proposed openstack/tripleo-quickstart-extras master: Baremetal deploy network ports support  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76707008:14
*** hakhande_ has joined #tripleo08:20
*** tosky has joined #tripleo08:22
*** hakhande has quit IRC08:24
*** apetrich has joined #tripleo08:27
openstackgerritHarald Jensås proposed openstack/python-tripleoclient master: Add routes and MTU to ctlplane attrs for standalone  https://review.opendev.org/c/openstack/python-tripleoclient/+/76970908:35
*** hjensas has joined #tripleo08:35
openstackgerritSergey Galas' proposed openstack/os-net-config master: basic support for Linux tap devices  https://review.opendev.org/c/openstack/os-net-config/+/76581608:37
*** jfrancoa has quit IRC08:38
*** jfrancoa has joined #tripleo08:41
openstackgerritHarald Jensås proposed openstack/tripleo-common master: Re-factor inventory.py use setdefault pervasively  https://review.opendev.org/c/openstack/tripleo-common/+/76610908:46
openstackgerritHarald Jensås proposed openstack/tripleo-common master: Inventory from neutron resources  https://review.opendev.org/c/openstack/tripleo-common/+/76590208:46
*** xek has joined #tripleo08:53
*** jfrancoa has quit IRC08:54
*** ysandeep is now known as ysandeep|lunch08:55
*** jfrancoa has joined #tripleo08:57
*** lucasagomes has joined #tripleo09:04
*** jlopezgu has joined #tripleo09:08
*** jlarriba has joined #tripleo09:08
openstackgerritMerged openstack/tripleo-validations master: Add tripleo-validations role for component testing  https://review.opendev.org/c/openstack/tripleo-validations/+/75539009:22
openstackgerritmbu proposed openstack/tripleo-quickstart-extras master: add validations playbook for VF component testing  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/75538209:24
openstackgerritSergey Galas' proposed openstack/os-net-config master: basic support for Linux tap devices  https://review.opendev.org/c/openstack/os-net-config/+/76581609:28
openstackgerritMerged openstack/python-tripleoclient master: Make workflow monitoring more resilient  https://review.opendev.org/c/openstack/python-tripleoclient/+/76822509:29
*** jmasud has quit IRC09:29
*** mmethot has joined #tripleo09:30
*** derekh has joined #tripleo09:30
*** amoralej is now known as amoralej|off09:31
openstackgerritArx Cruz proposed openstack/tripleo-quickstart master: Remove tempest_test_whitelist variable  https://review.opendev.org/c/openstack/tripleo-quickstart/+/77012709:35
*** jpich has joined #tripleo09:40
*** dtantsur|afk is now known as dtantsur09:42
*** bogdando has joined #tripleo09:59
openstackgerritMerged openstack/tripleo-common stable/train: remove c7 update/upgrdae jobs  https://review.opendev.org/c/openstack/tripleo-common/+/77016210:02
openstackgerritMerged openstack/python-tripleoclient stable/train: remove c7 update/upgrdae jobs  https://review.opendev.org/c/openstack/python-tripleoclient/+/77015910:02
openstackgerritMerged openstack/paunch stable/train: remove c7 update/upgrdae jobs  https://review.opendev.org/c/openstack/paunch/+/77015710:11
*** adrianc has quit IRC10:12
*** adrianc has joined #tripleo10:13
*** adrianc has quit IRC10:17
*** adrianc has joined #tripleo10:18
*** saneax has joined #tripleo10:30
*** udesale_ has joined #tripleo10:33
*** udesale has quit IRC10:36
openstackgerritEduardo Olivares proposed openstack/tripleo-heat-templates master: Configure OVNCMSOptions=enable-chassis-as-gw within neutron-ovn-sriov.yaml  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76971810:38
openstackgerritSaravanan KR proposed openstack/tripleo-upgrade master: Handle errors on system upgrade  https://review.opendev.org/c/openstack/tripleo-upgrade/+/77005910:43
*** ysandeep|lunch is now known as ysandeep10:43
openstackgerritShnaidman Sagi proposed openstack/tripleo-ansible stable/train: DNM test train  https://review.opendev.org/c/openstack/tripleo-ansible/+/77026310:44
*** hjensas is now known as hjensas|afk10:51
openstackgerritDaniel Bengtsson proposed openstack/tripleo-common stable/rocky: Fix the log path on the host.  https://review.opendev.org/c/openstack/tripleo-common/+/77026610:53
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Revert "Reset sriov_numvfs to 0 before leapp upgrade"  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77018310:58
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates stable/train: Revert "Reset sriov_numvfs to 0 before leapp upgrade"  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77018411:00
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates stable/train: Revert "Reset sriov_numvfs to 0 before leapp upgrade"  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77018411:02
*** rpittau has quit IRC11:06
*** rpittau has joined #tripleo11:07
*** otherwiseguy has quit IRC11:07
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Force json output format for hiera in derive pci whitelist  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77026811:10
*** otherwiseguy has joined #tripleo11:12
*** jpich has quit IRC11:16
*** jpich has joined #tripleo11:17
*** jcapitao is now known as jcapitao_lunch11:22
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Add NovaImageTypeExcludeList parameter  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77022011:39
*** udesale_ has quit IRC11:42
openstackgerritJose Luis Franco proposed openstack/tripleo-ansible master: Do not use fetch with become in transfer_data role  https://review.opendev.org/c/openstack/tripleo-ansible/+/76717711:49
*** jaosorior has quit IRC11:53
*** raildo has joined #tripleo12:14
*** efoley has joined #tripleo12:15
*** udesale has joined #tripleo12:19
*** jlopezgu has quit IRC12:20
*** jpena is now known as jpena|lunch12:25
openstackgerritAdriano Petrich proposed openstack/python-tripleoclient stable/train: Expose --ansible-forks  https://review.opendev.org/c/openstack/python-tripleoclient/+/76585512:37
*** ratailor has quit IRC12:40
*** jcapitao_lunch is now known as jcapitao12:41
openstackgerritmbu proposed openstack/tripleo-quickstart-extras master: add validations playbook for VF component testing  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/75538212:42
*** elGamal has quit IRC12:44
*** rlandy has joined #tripleo12:50
openstackgerritGrzegorz Grasza proposed openstack/puppet-tripleo master: Add authtoken security options  https://review.opendev.org/c/openstack/puppet-tripleo/+/76934912:51
openstackgerritCarlos Gonçalves proposed openstack/tripleo-ansible master: Add tripleo_frr role  https://review.opendev.org/c/openstack/tripleo-ansible/+/76357212:53
openstackgerritCarlos Gonçalves proposed openstack/tripleo-ansible master: tripleo_frr: add BGP TTL security  https://review.opendev.org/c/openstack/tripleo-ansible/+/77030112:53
openstackgerritShnaidman Sagi proposed openstack/tripleo-quickstart-extras master: Add 'until' condition for retries in container push  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77030312:54
*** amoralej|off is now known as amoralej12:54
*** boazel_ has joined #tripleo12:55
*** boazel has quit IRC12:56
*** udesale_ has joined #tripleo13:01
openstackgerritRabi Mishra proposed openstack/tripleo-common master: Drop parameter caching  https://review.opendev.org/c/openstack/tripleo-common/+/77033513:03
*** udesale has quit IRC13:04
*** jpena|lunch is now known as jpena13:21
*** rfolco has joined #tripleo13:41
*** pleimer has joined #tripleo13:43
*** tmazur has joined #tripleo13:48
*** jpich has quit IRC13:56
*** jpich has joined #tripleo13:56
*** rh-jelabarre has quit IRC13:56
sshnaidm|ruckcloudnull, hey13:57
*** rh-jelabarre has joined #tripleo14:03
mwhahahaweshay|ruck, sshnaidm|ruck: so victoria upgrades are failing with the same stuff as ussuri14:05
sshnaidm|ruckthat's sad14:06
mwhahahavery much so14:06
openstackgerrityatin proposed openstack/tripleo-quickstart-extras master: [DNM] test branchless project build in train  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77034714:07
sshnaidm|ruckmwhahaha, we didn't merge something and ported back?14:07
sshnaidm|ruckmaybe not from tripleo14:07
sshnaidm|ruckakahat|rover, did you have a chance to talk with upgrades folks?14:08
mwhahahathat key size change is what we thought it was yesterday14:08
mwhahahasince that's landed back to U14:08
akahat|roversshnaidm|ruck, no.14:09
mwhahahahttps://review.opendev.org/c/openstack/tripleo-heat-templates/+/77017714:09
mwhahahabut the upgrade jobs are still failing ont he revert14:09
mwhahahabut we didn't know if the upgrade jobs were actually testing things correctly14:09
sshnaidm|ruckthey also passed in the original review, which is not clear for me..14:10
mwhahahayes14:10
sshnaidm|ruckmaybe we don't update a patch in a upgrade job and it will work after merge only14:10
*** udesale_ has quit IRC14:10
mwhahahathat's why we weren't sure if the upgrade jobs were actually testing the change14:10
*** udesale has joined #tripleo14:10
sshnaidm|ruckthen we have a problem in ci14:10
sshnaidm|ruckyeah14:11
sshnaidm|ruckakahat|rover, do you know how to check if the patch is used in a job?  ^14:13
akahat|roversshnaidm|ruck, yes.. using patch no.14:14
sshnaidm|ruckakahat|rover, ok, we'll see together after a mtg14:15
akahat|roversshnaidm|ruck, okay14:15
*** efoley has quit IRC14:16
mwhahahalooks like master is broken too, so upgrades is just broke across the board14:31
mwhahahafun14:31
mwhahahawould be nice if we could a debug output of the os_tempest failure14:31
openstackgerritAlan Bishop proposed openstack/tripleo-common stable/victoria: TCIB: Add cinder-volume extend_start.sh script  https://review.opendev.org/c/openstack/tripleo-common/+/77034914:31
*** efoley has joined #tripleo14:32
sshnaidm|ruckmwhahaha, which container better to check for heat-templates version?14:32
sshnaidm|ruckis it installed in one of them?14:32
mwhahahanot on the overcloud14:32
mwhahahait was in mistral on the undercloud until ussuri14:33
openstackgerritMerged openstack/tripleo-common stable/train: Wire up new tripleo upgrades jobs template  https://review.opendev.org/c/openstack/tripleo-common/+/76916814:33
mwhahahabut even then it's only an undercloud thing14:33
ade_leeweshay|ruck, mwhahaha trying to get https://review.opendev.org/c/openstack/tripleo-heat-templates/+/770212 in, but one of the ci jobs keeps timing out14:33
sshnaidm|ruckon undercloud I see the right patch installed, from ussuri..14:35
sshnaidm|ruckand in mistral container as well14:36
mwhahahak then something else is broke14:37
mwhahahai'll do an rpm diff in a few14:37
cloudnullsshnaidm|ruck mwhahaha have we nailed down some changes for the excising of all things docker in train ?14:38
mwhahahai didn't look yesterday, was on my list for today14:38
mwhahahabut i think the upgrades thing might be more important since it's blocking U/V atm14:39
mwhahaha(we removed it from T)14:39
sshnaidm|ruckcloudnull, I'm trying it in https://review.opendev.org/c/openstack/tripleo-ansible/+/770123 but need help with that14:41
sshnaidm|ruckmwhahaha, ^14:41
cloudnullmwhahaha ++14:41
cloudnullsshnaidm|ruck looking14:41
sshnaidm|ruckmostly I copied ussuri molecule configs to train14:41
*** jaosorior has joined #tripleo14:41
sshnaidm|ruckright now it can't find redhat8 trunk repos, because it runs on ubi image..14:42
cloudnullwe need to also copy in the test_dep fixes to make that go14:42
sshnaidm|ruckcloudnull, yeah, I think copied it as well14:42
cloudnullhttps://review.opendev.org/c/openstack/tripleo-ansible/+/72060914:43
cloudnullthis was the review I langed a while back to make that go14:43
mwhahahasshnaidm|ruck: did we recently change how tempest was installed?14:45
* cloudnull wonders if that can just be cherrypciked 14:45
mwhahahasshnaidm|ruck: upgrades are failing when tempest comes from package and the last successful run didn't have a tempest rpm installed14:45
mwhahahahttps://www.diffchecker.com/Bf8uHK8x14:45
mwhahahachandankumar: -^ do you know?14:46
chandankumarmwhahaha: which upgrade job it is?14:47
mwhahahathat's ussuri14:47
sshnaidm|ruckmwhahaha, we added os_tempest, but for c7 only I think, in c8 it always worked14:48
sshnaidm|ruckarxcruz, is it right? ^14:48
mwhahahasorry that's victoria14:48
mwhahahalet me check ussuri14:48
arxcruzsshnaidm|ruck: yes, right it worked on c7 too14:48
mwhahahawhere's that change?14:49
chandankumarsshnaidm|ruck: mwhahaha now ostempest is running every where14:49
arxcruzmwhahaha: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/74650914:49
sshnaidm|ruckmwhahaha, I'm talking about https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/74650914:49
chandankumarmwhahaha: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/74650914:49
sshnaidm|ruckdidn't we have tempest rpm before..?14:50
chandankumarfor standalone upgrade job I think tempest is disabled14:50
mwhahahayea ussii is the same14:50
mwhahahawe didn't use tempest via rpm14:50
chandankumarwe have tempest rpm everywhere in ci14:50
sshnaidm|ruck* Generate clouds.yaml only for overcloud.14:50
mwhahahahttps://www.diffchecker.com/ArXoVEtr14:50
sshnaidm|ruckmaybe this14:50
mwhahahachandankumar: no you weren't doing that previously on multinode14:51
mwhahahaalso 26.0 is being used for stable branches14:51
mwhahahaso yea https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/746509 probably broke the world14:51
sshnaidm|ruckchandankumar, now tempest runs on standalone upgrades and fails14:51
mwhahahathese aren't standalone14:52
mwhahahait's multipnode14:52
mwhahahatripleo-ci-centos-8-scenario000-multinode-oooq-container-upgrades-ussuri14:52
*** TrevorV has joined #tripleo14:52
* mwhahaha reverts14:53
sshnaidm|ruckalso tripleo-ci-centos-8-undercloud-upgrade-ussuri14:53
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Revert "Os_tempest everywhere"  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77018814:53
chandankumarmwhahaha: recent upgrade jobs are failing at this error Ensure private network exists]14:54
sshnaidm|ruckarxcruz, chandankumar any ideas what can be wrong that affects upgrades?14:54
mwhahahachandankumar: which ones? master?14:55
chandankumarkeystoneauth1.exceptions.catalog.EndpointNotFound: internal endpoint for network service in regionOne region not found14:55
chandankumarhttps://f74421733851eefe6f60-4ffbc3bd26f661e6bef765a62d1398b4.ssl.cf5.rackcdn.com/764186/3/check/tripleo-ci-centos-8-scenario000-multinode-oooq-container-upgrades-ussuri/3383cee/job-output.txt14:55
mwhahahakeystoneauth1.exceptions.catalog.EndpointNotFound: internal endpoint for network service in regionOne region not found14:55
chandankumarhttps://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario000-multinode-oooq-container-upgrades-ussuri+14:55
mwhahahasam error everywhere14:55
chandankumargrabbed the recent one14:55
mwhahahait can't talk to the cloud correctly14:55
chandankumarmwhahaha: it is due RegionOne not regionOne14:55
mwhahahaplz revert os_tempest everywehre14:55
mwhahahait should be regionOne14:55
mwhahahaunless we changed it?14:55
chandankumarneed to check what got changed there14:56
arxcruzmwhahaha: not necessary, we just need to skip os_tempest on upgrade jobs14:56
mwhahahahttps://github.com/openstack/tripleo-heat-templates/blob/master/deployment/keystone/keystone-container-puppet.yaml#L9914:56
mwhahahait's regionOne14:56
mwhahahaalways has been14:57
mwhahahaplz revert the os_tempest change, you can fix it later14:57
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018914:58
mwhahahajfyi we didn't execute tempest on upgrade jobs14:59
mwhahahaso that's why this is broken14:59
openstackgerritJiri Podivin proposed openstack/tripleo-quickstart master: Fixes url typo  https://review.opendev.org/c/openstack/tripleo-quickstart/+/77035014:59
mwhahahapreviously we weren't invoking tempest on upgrades14:59
mwhahahahttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f51/769594/1/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-upgrades-victoria/f51330b/job-output.txt14:59
mwhahahaso you enabled tempest where it wasn't running previously15:00
mwhahahait's probably been broken for a long time15:00
mwhahaha(we're can debate that issue later)15:00
mwhahahaweshay|ruck: -^ jfyi15:01
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Do not run tempest in upgrade job  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77035115:01
arxcruzmwhahaha: ^15:01
arxcruzmwhahaha: since tempest never ran before in the upgrade job...15:01
arxcruzi checked last good run and it wasn't running15:01
mwhahahaarxcruz: what about multinode upgrades, i'm not talking about standalone15:02
mwhahahai think standalone was fine, but i'm pasting multinode-upgrade johbs15:02
*** pcaruana has quit IRC15:03
arxcruzmwhahaha: ok, i'll digg into it, let's revert for now15:03
arxcruzsince it's an emergency15:03
mwhahahastandalone seems fine-ish https://zuul.openstack.org/builds?job_name=tripleo-ci-c8-standalone-upgrade&job_name=tripleo-ci-centos-8-standalone-upgrade-victoria15:04
mwhahahamultinode is not https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario000-multinode-oooq-container-upgrades-victoria15:04
mwhahahahttps://review.opendev.org/c/openstack/tripleo-quickstart/+/759830 seems not to have worked15:06
mwhahahabecause it should have prevented tempest15:06
mwhahahaoh i see it15:07
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Fix when condition  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77035715:09
mwhahahaarxcruz, chandankumar, sshnaidm|ruck -^ alternative to the revert15:09
mwhahahachoose your poison15:09
* mwhahaha runs away screaming15:10
*** mwhahaha changes topic to "CI status: RED: stable upgrades LP#1911020, tripleo-ansible train molecule, the rest: ¯\_(ツ)_/¯ | tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs"15:11
openstackgerritArx Cruz proposed openstack/tripleo-quickstart master: Set use_os_tempest to false  https://review.opendev.org/c/openstack/tripleo-quickstart/+/77035915:11
arxcruzmwhahaha: please add a depends on ^15:11
arxcruzthis set featureset050 use_os_tempest to false15:11
mwhahahak15:12
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Fix when condition  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77035715:12
*** pcaruana has joined #tripleo15:12
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Drop python_cmd set_Fact  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76678715:15
*** tmazur has quit IRC15:29
*** tmazur has joined #tripleo15:29
*** lbragstad has quit IRC15:29
openstackgerritBrent Eagles proposed openstack/tripleo-ansible master: WIP: use ansible to generate bind pool configuration  https://review.opendev.org/c/openstack/tripleo-ansible/+/77020515:33
*** lbragstad has joined #tripleo15:34
*** lbragstad has quit IRC15:35
*** lbragstad has joined #tripleo15:35
openstackgerritShnaidman Sagi proposed openstack/tripleo-ansible stable/train: WIP: Move all molecule train jobs to use podman driver  https://review.opendev.org/c/openstack/tripleo-ansible/+/77012315:35
*** jpich has quit IRC15:36
*** jpich has joined #tripleo15:37
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018915:41
cloudnullsshnaidm|ruck ^ I did a cherry-pick from the original docker > podman review . maybe that will go?15:43
sshnaidm|ruckcloudnull, cool,let's see15:44
mwhahahazuul problems15:44
mwhahahafound undefined alias15:44
sshnaidm|ruckcloudnull, need to ensure we don't create new roles with underscore-dash issue15:44
cloudnullyea, fixing the zuul issue now15:46
cloudnulli think the role names are right - https://github.com/openstack/tripleo-ansible/tree/stable/train/tripleo_ansible/roles15:47
cloudnullin train we had both _ and -15:47
mwhahahashould we removed the extra added molecule tests?15:47
mwhahahai saw some additions for tripleo packages in tehre15:47
mwhahahawe can let it ride for now, but just wondering if we need to revisit15:48
cloudnull++15:48
cloudnullI think it makes sense to pair it down as much as possible15:48
cloudnullI also removed all the cent7 molecule tests in that revie w15:48
openstackgerritHarald Jensås proposed openstack/python-tripleoclient master: Add routes and MTU to ctlplane attrs for standalone  https://review.opendev.org/c/openstack/python-tripleoclient/+/76970915:49
cloudnullsince podman and cent7 don't like one another15:49
*** skramaja has quit IRC15:49
*** d34dh0r53 has quit IRC15:51
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018915:52
*** d34dh0r53 has joined #tripleo15:53
*** langdon has quit IRC16:04
*** hakhande_ has quit IRC16:09
openstackgerritDavid Vallee Delisle proposed openstack/tripleo-heat-templates master: Making sure virt-guest-shutdown.target exists  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76834816:12
*** psachin has quit IRC16:13
*** boazel has joined #tripleo16:13
*** lmiccini has quit IRC16:13
*** udesale has quit IRC16:15
*** boazel_ has quit IRC16:16
*** d34dh0r53 has quit IRC16:19
*** d34dh0r53 has joined #tripleo16:19
openstackgerritAlex Schultz proposed openstack/os-apply-config master: Bump hacking to fix ci  https://review.opendev.org/c/openstack/os-apply-config/+/77044916:30
*** jaosorior has quit IRC16:32
*** amoralej is now known as amoralej|off16:33
openstackgerritAlex Schultz proposed openstack/os-apply-config master: Clean up CI  https://review.opendev.org/c/openstack/os-apply-config/+/77044916:33
openstackgerritAlex Schultz proposed openstack/os-apply-config master: Add Python3 wallaby unit tests  https://review.opendev.org/c/openstack/os-apply-config/+/76912416:38
*** jcapitao has quit IRC16:39
*** jaosorior has joined #tripleo16:45
*** sshnaidm|ruck is now known as sshnaidm|afk16:47
*** jaosorior has quit IRC16:51
*** jaosorior has joined #tripleo16:51
openstackgerritSandeep Yadav proposed openstack/tripleo-ansible master: Check name_lower string contains valid pattern  https://review.opendev.org/c/openstack/tripleo-ansible/+/77045316:52
*** openstackgerrit has quit IRC16:55
*** jfrancoa has quit IRC16:57
*** jaosorior has quit IRC16:57
*** jaosorior has joined #tripleo16:57
*** lucasagomes has quit IRC16:58
*** marios is now known as marios|out16:59
ysandeepfolks o/ , looking for some reviews here https://review.opendev.org/c/openstack/python-tripleoclient/+/769867 , py2 removal17:00
*** jpena is now known as jpena|off17:01
*** ysandeep is now known as ysandeep|away17:01
*** marios|out has quit IRC17:05
*** slaweq has quit IRC17:10
*** artom has quit IRC17:10
*** d34dh0r53 has quit IRC17:10
*** jpich has quit IRC17:12
*** jmasud has joined #tripleo17:22
*** d34dh0r53 has joined #tripleo17:27
*** artom has joined #tripleo17:30
*** rpittau is now known as rpittau|afk17:32
mwhahahaweshay|ruck: when did 8.3 hit? was it ~12/9?17:35
ykarelmwhahaha, i think wes is on PTO today, wrt 8.3 yes it was in that week itself17:43
ykarelon 12/817:43
mwhahahak thats probably why the tripleo-ansible tripleo_podman has been broken17:43
* mwhahaha tries to figure out the fix17:43
mwhahahamight be an iptables thing too17:44
mwhahahathough it's probably podman networking stuff17:44
bandinioh gawd17:44
ykarelack17:45
*** dtantsur is now known as dtantsur|afk17:51
*** ykarel has quit IRC18:00
*** d34dh0r53 has quit IRC18:13
*** bogdando has quit IRC18:15
*** gfidente is now known as gfidente|afk18:19
*** d34dh0r53 has joined #tripleo18:31
mwhahahaport binding in podman don't appear to work18:57
*** ianw_pto is now known as ianw18:58
mwhahahaor maybe because it's run not privileged18:59
mwhahaha(╯°□°)╯︵ sʇıɯıʃǝʇɐɹ ɹǝʞɔop19:10
cloudnullhttps://review.opendev.org/c/openstack/tripleo-ansible/+/770189 - mwhahaha sagi - that review has a fairly high rate of success, I'm going to push another revision to fix the linter issues but I think we're close19:31
*** holser has quit IRC19:31
mwhahahacloudnull: i figured out the tripleo_podman problem19:32
mwhahahaif that's failing19:32
cloudnullthat one is failing :D19:32
mwhahahagerritbot appears broken19:32
cloudnullyeah...19:32
mwhahahahttps://review.opendev.org/c/openstack/tripleo-ansible/+/77046519:32
mwhahahayou'll need to pull that in19:32
mwhahaha-p 5000:5000 no longer works when running non-privileged19:33
cloudnulloh , E_MOARSUDO19:33
cloudnull++19:33
mwhahahait should pass master19:33
mwhahahai tested locally and it worked19:33
mwhahaha(molecule testing locally makes me sad)19:33
cloudnullshipit19:33
*** holser has joined #tripleo19:35
*** holser has quit IRC19:39
*** holser has joined #tripleo19:40
*** jmasud has quit IRC19:41
*** holser has quit IRC19:43
*** holser has joined #tripleo19:43
*** slaweq has joined #tripleo19:44
*** dsneddon has joined #tripleo19:48
*** openstackgerrit has joined #tripleo19:49
openstackgerritAlex Schultz proposed openstack/tripleo-ansible master: Switch off of docker.io for testing  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046619:49
mwhahahaoo the bot has returned19:49
mwhahahaweshay|ruck, cloudnull: https://review.opendev.org/c/openstack/tripleo-ansible/+/770465/ has passed (currently pending useless provider job)19:52
openstackgerritAlex Schultz proposed openstack/tripleo-ansible stable/victoria: Fix tripleo_podman molecule jobs  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046919:53
openstackgerritAlex Schultz proposed openstack/tripleo-ansible stable/ussuri: Fix tripleo_podman molecule jobs  https://review.opendev.org/c/openstack/tripleo-ansible/+/77047019:53
*** slaweq has quit IRC20:03
sshnaidm|afkmwhahaha, does adding --privileged work in that case?20:11
mwhahahai don't think so20:11
sshnaidm|afknot sure what is the problem there20:11
openstackgerritAlex Schultz proposed openstack/tripleo-ansible master: Switch off of docker.io for testing  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046620:12
mwhahahai think user network doesn't hook into the host anymore20:12
sshnaidm|afkcloudnull, my patch seems working mostly, maybe we can leave only passing jobs and then work on fixing others: https://zuul.opendev.org/t/openstack/status/change/770123,420:13
sshnaidm|afkmwhahaha, idk, I held a host and run this command locally and it worked, but container from task couldn't accept connections, very weird20:13
mwhahahanetstat didn't show it listening20:13
mwhahahaand there was no ip address info in the container20:14
sshnaidm|afkyep20:14
mwhahahahttps://github.com/containers/podman/issues/804020:14
mwhahahai feel like it's related to that20:14
*** arxcruz has quit IRC20:14
mwhahahahonestly we can just run it as root and move on with life :D20:14
sshnaidm|afkwe don't create a network there though20:15
sshnaidm|afkyeah, agree on that, but really weird behavior20:15
mwhahahahttps://podman.io/getting-started/network#rootless-networking20:18
mwhahahaguess it no longer works20:18
mwhahahadonno how it worked previously20:18
* mwhahaha tests20:20
mwhahahamaybe it's a bug in the module20:21
mwhahahaugh it's ipv6 by default20:25
mwhahahanot ipv420:25
mwhahahathat's teh problem20:25
mwhahahaerrr now it's working20:26
mwhahahaselinux20:29
* mwhahaha screams20:29
sshnaidm|afkselinux where?20:29
mwhahahahttps://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_podman/molecule/login/prepare.yml#L8220:30
mwhahahait's enforcing when we launch it so it doesn't listen20:30
mwhahahawe need to disable selinux first20:30
openstackgerritAlex Schultz proposed openstack/tripleo-ansible master: Fix tripleo_podman molecule jobs  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046520:33
openstackgerritAlex Schultz proposed openstack/tripleo-ansible master: Switch off of docker.io for testing  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046620:33
mwhahahalet's see if that works20:33
*** jmasud has joined #tripleo20:35
*** bandini has quit IRC20:36
*** dsneddon has quit IRC20:37
*** dsneddon has joined #tripleo20:38
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018920:42
johnsomHi neighbors. I'20:45
johnsomI'm (can't type) working on a new container image related to designate. What is the intent with the "ConfigImage" containers? For example, the rsyslog container has an associated ConfigImage container. I'm trying to understand if I need one or not as some have them, some don't, and I see we map in directories for some persistent data, etc.20:47
mwhahahajohnsom: the config image is usually the same image as will be used by the service. it's the image used to invoke puppet in to perform service configuration20:48
mwhahahasshnaidm|afk: so yea it's selinux, moving the disable up fixed it too20:48
mwhahahacloudnull: -^ the fix for tripleo_podman is to move the selinux disable up. we don't need to run it as sudo20:49
openstackgerritAlex Schultz proposed openstack/tripleo-ansible stable/victoria: Fix tripleo_podman molecule jobs  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046920:49
sshnaidm|afkcool, mystery solved :)20:50
johnsommwhahaha Ok, so is it throw away after puppet runs, or just another name for the same container image that the service runs in?20:50
openstackgerritAlex Schultz proposed openstack/tripleo-ansible stable/ussuri: Fix tripleo_podman molecule jobs  https://review.opendev.org/c/openstack/tripleo-ansible/+/77047020:50
mwhahahajohnsom: it's just another name for the other one. because we used to use a different configuration image from some services (neutron plugin stuff)20:50
mwhahahaso we established a pattern to allow a user to specify a different image to run the configuration in vs the service being deployed20:51
johnsommwhahaha Ah, for some kind of override. Ok, I think I follow now. Thank you!20:51
mwhahahajohnsom: there's a mapping file in tripleo-common where we usually populate those values for users. it's invoked either by the ContainerImagePrepare (train+) values or using `openstack overcloud container image prepare` (queens)20:52
mwhahahahttps://opendev.org/openstack/tripleo-common/src/branch/master/container-images/tripleo_containers.yaml.j220:53
mwhahahaso like the mariadb container is shared for several config things20:53
johnsomYeah, I think I follow now.20:54
*** derekh has quit IRC20:56
mwhahahahttps://github.com/openstack/tripleo-heat-templates/blob/master/deployment/experimental/designate/designate-api-container-puppet.yaml#L127-L13220:56
*** TrevorV has quit IRC20:56
*** derekh has joined #tripleo20:56
mwhahahaspecifically the config image is generlaly used in concert with the puppet_config20:56
*** jmasud has quit IRC20:57
mwhahahathe future thinking was also about being able to have a single configuration container image so we could stop shipping puppet bits in the service containers20:57
mwhahahabut we never got around to implementing that20:58
johnsomYeah, I still have some questions about why puppet is needed with the Ansible, but I think that is a whole different conversation over beverages.20:59
mwhahahaansible can't do configuration inside a container20:59
mwhahahawhere we can invoke puppet in a container which allows us to do things that ansible can't20:59
mwhahahaalso we already have invested a lot in openstack configuration in the puppet world that having to duplicate in ansible would be a lot of effort20:59
mwhahahadesignate already has a puppet-designate that works/tested over in puppet-openstack21:00
mwhahahait's not a hard requirement but it's an existing pattern. we have some services which don't have puppet because it would be more effort to bootstrap a puppet module than just write a single template in ansible21:00
mwhahahafor openstack services it would make sense to continue using the existing and well established pattern21:01
mwhahahaansible also has no concept like hieradata which makes role/host overrides a bit tougher in the framework21:01
*** derekh has quit IRC21:05
openstackgerritJames Slagle proposed openstack/tripleo-specs master: Ephemeral Heat for the Overcloud spec  https://review.opendev.org/c/openstack/tripleo-specs/+/76500021:07
openstackgerritAlex Schultz proposed openstack/tripleo-ansible master: Switch off of docker.io for testing  https://review.opendev.org/c/openstack/tripleo-ansible/+/77046621:11
*** hjensas|afk is now known as hjensas21:15
*** rfolco has quit IRC21:25
mwhahahacloudnull: so bad news, tripleo-podman is failing train because the default network is bad21:30
mwhahahacause we need https://review.opendev.org/c/openstack/tripleo-ansible/+/76961821:30
mwhahahamight need to switch that one to sudo for now after all21:30
cloudnull:\21:30
cloudnullhttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fdd/770189/5/check/tripleo-ansible-centos-8-molecule-tripleo-podman/fdd94e0/reports.html21:32
cloudnulllooks like its failing in the train job too21:32
mwhahahaError: error configuring network namespace for container 19e64b8caf434cdcffc0662745ffc5741a8a483006aa879a7429be5d3b90295c: Missing CNI default network21:32
mwhahahaah possibly21:32
cloudnull** the train podman conversion job21:33
mwhahahait's all a mess21:33
*** lbragstad has quit IRC21:35
mwhahahacloudnull: https://review.opendev.org/c/openstack/ansible-role-container-registry/+/770148/ when you have a moment21:36
*** lbragstad has joined #tripleo21:38
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018921:43
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018921:46
*** derekh has joined #tripleo21:50
*** derekh has quit IRC21:50
*** efoley has quit IRC21:53
*** pleimer has quit IRC22:03
*** rh-jelabarre has quit IRC22:04
*** raildo has quit IRC22:05
cloudnull mwhahaha https://7c4af2962e649c78a0b8-8dbbf393fa4fd2548b6eb8c45bf2a7e6.ssl.cf2.rackcdn.com/770189/7/check/tripleo-ansible-centos-8-molecule-tripleo-podman/dc7decf/reports.html still failing even after moving the selinux task and running all the podman commands with sudo22:15
mwhahahameh22:16
cloudnullat this point I think that's the only molecule job failing22:17
cloudnullwhich is the podman login22:17
cloudnullmaybe I just remove that job for now?22:18
mwhahahai'd comment it out for now and fix it later22:18
cloudnull++22:18
mwhahahai'd be ok with that since this has been broken since mid dec22:18
mwhahahai bet the initial podman run is failing22:19
mwhahahabecause no network22:19
openstackgerritKevin Carter proposed openstack/tripleo-ansible stable/train: Convert docker molecule usage to podman  https://review.opendev.org/c/openstack/tripleo-ansible/+/77018922:19
mwhahahabut i don't know why the cni network is missing22:19
mwhahahaoh i think we're still breaking the networking bits22:20
mwhahahai'll poke at train tomorrow22:21
cloudnull++22:21
mwhahahaif core people are still around, https://review.opendev.org/c/openstack/tripleo-ansible/+/77046522:25
* mwhahaha wanders off22:25
openstackgerritMichael Johnson proposed openstack/tripleo-common master: Create a container for Unbound DNS resolver  https://review.opendev.org/c/openstack/tripleo-common/+/77048822:57
supamattthey are all the same22:59
supamatterr wrong window sorry chaps!22:59
*** rcernin has joined #tripleo23:02
*** jmasud has joined #tripleo23:02
*** pleimer has joined #tripleo23:03
*** arxcruz has joined #tripleo23:09
*** rlandy has quit IRC23:19
*** pleimer has quit IRC23:34
openstackgerritMerged openstack/tripleo-ansible master: Expand roles honstname to role mapping  https://review.opendev.org/c/openstack/tripleo-ansible/+/76583723:37
*** jmasud has quit IRC23:50
cloudnullsshnaidm|afk mwhahaha - https://review.opendev.org/c/openstack/tripleo-ansible/+/770189/ - all molecule jobs are passing.23:51
cloudnulljust waiting on provider / functional jobs so with a bit of luck we should be good to go23:52
cloudnulli'm babysitting https://zuul.opendev.org/t/openstack/status#77018923:52
* cloudnull vanishes in a puff of smoke 23:53

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!