Monday, 2021-10-25

opendevreviewIan Wienand proposed openstack/diskimage-builder master: centos-minimal: boot test 9-stream  https://review.opendev.org/c/openstack/diskimage-builder/+/81484400:07
*** pmannidi is now known as pmannidi|brb02:03
*** pmannidi|brb is now known as pmannidi04:12
*** amoralej|off is now known as amoralej06:06
opendevreviewJose Luis Franco proposed openstack/tripleo-ansible stable/wallaby: Add tripleo_dnf_stream Ansible module.  https://review.opendev.org/c/openstack/tripleo-ansible/+/81494806:07
opendevreviewJiri Podivin proposed openstack/tripleo-quickstart-extras master: Override inventory var value if the job defines it's own  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81480106:11
opendevreviewJose Luis Franco proposed openstack/tripleo-ansible stable/wallaby: Add tripleo_dnf_stream Ansible module.  https://review.opendev.org/c/openstack/tripleo-ansible/+/81494806:12
*** bcafarel_ is now known as bcafarel07:01
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Use venv instead of virtualenv on EL9.  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81362407:02
opendevreviewAnanya proposed openstack/tripleo-ci master: Update ci-health url if not present in CIX card desc  https://review.opendev.org/c/openstack/tripleo-ci/+/81397007:03
opendevreviewJiri Podivin proposed openstack/tripleo-ci master: Add oc roles and composable roles to overrides  https://review.opendev.org/c/openstack/tripleo-ci/+/81522207:05
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Use venv instead of virtualenv on EL9.  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81362407:05
opendevreviewAlfredo Moralejo proposed openstack/diskimage-builder master: Add support for CentOS Stream 9 in DIB  https://review.opendev.org/c/openstack/diskimage-builder/+/81139207:40
*** ykarel is now known as ykarel|lunch07:49
opendevreviewGael Chamoulaud proposed openstack/validations-libs master: [WIP] Add new CLI sub command to create community validations  https://review.opendev.org/c/openstack/validations-libs/+/81482908:02
opendevreviewGael Chamoulaud proposed openstack/validations-libs master: [WIP] Add new CLI sub command to create community validations  https://review.opendev.org/c/openstack/validations-libs/+/81482908:04
opendevreviewAnanya proposed openstack/tripleo-ci master: Update ci-health url if not present in CIX card desc  https://review.opendev.org/c/openstack/tripleo-ci/+/81397008:39
*** ykarel|lunch is now known as ykarel09:35
opendevreviewShnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Run TripleO jobs on Ansible 2.11  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/79782309:40
opendevreviewMerged openstack/tripleo-common stable/victoria: Add ipa client to tripleoclient container  https://review.opendev.org/c/openstack/tripleo-common/+/81490909:51
opendevreviewMerged openstack/puppet-pacemaker master: Make sure we wait for the cluster before setting resource_default_op  https://review.opendev.org/c/openstack/puppet-pacemaker/+/81501609:51
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Centos 9 periodic release file  https://review.opendev.org/c/openstack/tripleo-quickstart/+/80691010:02
opendevreviewHarald Jensås proposed openstack/tripleo-docs master: Partial revert "Document ``--skip-nodes-and-networks``"  https://review.opendev.org/c/openstack/tripleo-docs/+/81527410:24
opendevreviewRajesh Tailor proposed openstack/tripleo-heat-templates stable/wallaby: Enable support for Libvirt modular daemons  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81456810:26
opendevreviewHarald Jensås proposed openstack/tripleo-docs master: Partial revert "Document ``--skip-nodes-and-networks``"  https://review.opendev.org/c/openstack/tripleo-docs/+/81527510:26
*** jcapitao is now known as jcapitao_lunch10:27
opendevreviewHarald Jensås proposed openstack/tripleo-docs master: Partial revert "Document ``--skip-nodes-and-networks``"  https://review.opendev.org/c/openstack/tripleo-docs/+/81314110:27
*** rlandy is now known as rlandy|ruck10:32
*** ysandeep is now known as ysandeep|afk10:48
*** jbadiapa_ is now known as jbadiapa11:00
*** dviroel|rover|afk is now known as dviroel|rover11:10
opendevreviewchandan kumar proposed openstack/tripleo-ci master: [EL9] skip collectd container  https://review.opendev.org/c/openstack/tripleo-ci/+/81442811:39
opendevreviewDouglas Viroel proposed openstack/tripleo-quickstart-extras master: Revert "Add default forwarder to multinode-ipa - vexxhost"  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81523411:41
*** ysandeep|afk is now known as ysandeep11:50
*** jcapitao_lunch is now known as jcapitao12:07
*** amoralej is now known as amoralej|lunch12:12
opendevreviewmbu proposed openstack/tripleo-quickstart-extras master: Add validation playbook in ovb.yml and make conditional import  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81529212:32
opendevreviewRadomir Dopieralski proposed openstack/puppet-tripleo stable/wallaby: Revert Horizon's Memecached config to use IP addresses  https://review.opendev.org/c/openstack/puppet-tripleo/+/81524212:42
*** amoralej|lunch is now known as amoralej12:54
ykarelmarios, when you get chance please revisit https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81469912:54
ykareli replied to your questions there12:54
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Use venv instead of virtualenv on EL9.  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81362412:57
*** jpena|off is now known as jpena13:00
mariosykarel: ack calls now but adding to list 13:04
ykarelack Thanks marios 13:05
*** artom_ is now known as artom13:06
cloudnullmornings 13:10
slaglegood morning13:10
rlandy|ruckcloudnull: hello13:16
cloudnullrlandy|ruck 👋13:16
rlandy|ruckwelcome back :)13:16
cloudnullthanks! its good to be back13:16
cloudnullhow are things ?13:16
rlandy|ruckyou know - always fun13:18
cloudnullso ... https://c.tenor.com/fKIG2kiLVPgAAAAC/this-is-fine-its-fine.gif13:24
*** ykarel is now known as ykarel|away13:25
opendevreviewmbu proposed openstack/tripleo-quickstart-extras master: Add validation playbook in ovb.yml and make conditional import  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81529213:35
*** cloudnull2 is now known as cloudnull13:41
opendevreviewJiri Podivin proposed openstack/tripleo-ci master: Add OC roles and composable roles to overrides  https://review.opendev.org/c/openstack/tripleo-ci/+/81522214:09
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Centos 9 periodic release file  https://review.opendev.org/c/openstack/tripleo-quickstart/+/80691014:40
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Centos 9 periodic release file  https://review.opendev.org/c/openstack/tripleo-quickstart/+/80691014:47
sshnaidmade_lee, hi, do you know - what is the status of https://review.opendev.org/c/x/tripleo-ipa/+/771065 ? Do you plan to use modules from community or freeipa?14:51
sshnaidmade_lee, I'm asking because we should change everything to collections now and it's critical for tripleo-ipa to work with tripleo14:52
opendevreviewSandeep Yadav proposed openstack/tripleo-quickstart master: Centos 9 periodic release file  https://review.opendev.org/c/openstack/tripleo-quickstart/+/80691014:53
ade_leesshnaidm, ack -- just got out of meeting - will take a look today15:01
*** amoralej is now known as amoralej|off15:09
*** dviroel|rover is now known as dviroel|rover|lunch15:15
rlandy|rucklbragstad: hey - we're going to merge the fs001 and fs020 rbac job definitions15:23
lbragstadrlandy|ruck sweet - that sounds good15:23
rlandy|rucklbragstad: you can ping when you think we are ready to start adding the jobs to the integration line15:24
rlandy|ruckto run regularly15:24
lbragstadyeah - i need to follow up on all that stuff since i haven't looked at it in a week due to the PTG15:24
rlandy|rucklbragstad: no rush - just want to push forward our side15:25
*** ysandeep is now known as ysandeep|out15:32
opendevreviewAlan Bishop proposed openstack/tripleo-heat-templates master: Clear up confusion on cinder's default volume type  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81533216:01
*** dviroel|rover|lunch is now known as dviroel|rover16:11
opendevreviewMerged openstack/diskimage-builder master: Fix bootloader installation for gentoo  https://review.opendev.org/c/openstack/diskimage-builder/+/81486617:02
*** jpena is now known as jpena|off17:05
opendevreviewFrancesco Pantano proposed openstack/tripleo-heat-templates stable/train: [TRAIN-ONLY] Restart ceph-nfs after the Ceph Upgrade stage  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81252617:08
opendevreviewMerged openstack/tripleo-quickstart master: Enable use of yum-config module in wallaby  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81267217:25
*** atarakt is now known as nhicher18:09
opendevreviewAlex Schultz proposed openstack/python-tripleoclient master: Save needed stack outputs in working_dir  https://review.opendev.org/c/openstack/python-tripleoclient/+/80360918:13
opendevreviewAlex Schultz proposed openstack/python-tripleoclient master: Refactor --stack-only, --setup-only, --config-download-only  https://review.opendev.org/c/openstack/python-tripleoclient/+/80361018:13
opendevreviewAlex Schultz proposed openstack/python-tripleoclient master: Add additional saved stack outputs  https://review.opendev.org/c/openstack/python-tripleoclient/+/81368118:14
opendevreviewMerged openstack/tripleo-heat-templates stable/train: [FFWD][train-only] Ensure we get ovn_controller cert if needed  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/79884318:15
opendevreviewMerged openstack/tripleo-heat-templates stable/wallaby: Do not set rabbitmq SSL CA certs when InternalTLSCAFile is ''  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81482418:16
opendevreviewMerged openstack/tripleo-heat-templates master: Fix TLS-e with custom network names  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81228618:16
opendevreviewMerged openstack/tripleo-ci master: Adds check/gate job definitions for centos9 build image  https://review.opendev.org/c/openstack/tripleo-ci/+/81361818:16
dviroel|roverhi cloudnull o/18:52
cloudnullo/ 18:52
dviroel|rovercloudnull: pinging you about this revert https://review.opendev.org/c/openstack/tripleo-common/+/81465318:53
dviroel|rovercloudnull: we are trying to  investigate this issue here https://bugs.launchpad.net/tripleo/+bug/194740318:54
dviroel|rovercloudnull: this is happenning for a while now https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train18:55
dviroel|rovercloudnull: i started to testproject the revert here: https://review.rdoproject.org/r/c/testproject/+/35132/ (PS17)18:56
dviroel|roverone of the conclusions is that after your fix, the instropect node task start to fails, and when it succeeds, take more than 1h to finish. Before that, was a task that was taking ~25min to complete18:57
opendevreviewAlex Schultz proposed openstack/tripleo-common master: [WIP] task-core file generation  https://review.opendev.org/c/openstack/tripleo-common/+/79872118:57
cloudnullyeah that retry will make it take a lot longer to fail. 18:58
cloudnullbut I dont think its the cause root cause of the introspection issue?18:59
dviroel|rovercloudnull: sounds weird, but this jobs started to fail after the fix got merged19:00
dviroel|rovercloudnull: but i don't know why19:00
cloudnulli cant seem that as related, but I'm happy to +2 that revert 19:01
opendevreviewMerged openstack/python-tripleoclient stable/wallaby: Fix overcloud admin authorize  https://review.opendev.org/c/openstack/python-tripleoclient/+/81518419:01
opendevreviewMerged openstack/tripleo-common stable/ussuri: Add ipa client to tripleoclient container  https://review.opendev.org/c/openstack/tripleo-common/+/81491019:01
mwhahahadviroel|rover: it's likely not related to that change but rather uncovers something else.  19:02
dviroel|rovermwhahaha: agree with you19:03
dviroel|roverone of the first occurence was in the patch: https://review.opendev.org/c/openstack/tripleo-common/+/809148/2#message-0feb9c22078df028151f277b3ea01af8a607fca019:03
mwhahahahistorically ovb can be flakey with introspection due to a number of reasons19:03
mwhahahathat patch will extend the possible time it may take to run because it will retry19:03
dviroel|roverthat is my thought too, looking at the fix19:04
mwhahahait was successful in ussuri https://review.opendev.org/c/openstack/tripleo-common/+/81296019:04
cloudnull++ with a 10 second delay between attempts 19:04
mwhahahaso it seems to be specific to the train env19:05
dviroel|roveryes, train only19:05
mwhahahaare we using a different bmc image or pxe image between the two?19:05
dviroel|roveri don't think so, but i can double check19:07
rlandy|ruckshould be the same bmc image19:09
rlandy|ruckbmc was updated a little while ago - but it's the same image used on all releases19:09
mwhahahait's working?19:10
rlandy|ruckwe just updated the cert19:10
mwhahahahttps://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001&branch=stable%2Ftrain19:10
rlandy|ruckand reuploaded it19:10
mwhahahaseems to be highlighting the flakeyness of introspection19:10
rlandy|ruckon train - yep19:10
rlandy|rucklots of failures19:11
mwhahahathe only thing that might have done is cause it to exceed timeouts, but the revert isn't the solution19:11
rlandy|ruckwe tried looking at reducing concurrency - didn't help19:12
mwhahahawhat i used to see was issues with the bmc reboots19:12
mwhahahabut i don't know if that was resolved in newer ironic or what19:12
rlandy|ruckhttps://trello.com/c/zil37fAT/2148-cixlp1947403tripleociproa-ovb-jobs-are-consistently-failing-on-tripleo-overcloud-node-introspect-introspection-timeout19:12
opendevreviewMerged openstack/tripleo-common stable/train: Add ipa client to tripleoclient container  https://review.opendev.org/c/openstack/tripleo-common/+/81491119:13
opendevreviewMerged openstack/tripleo-heat-templates stable/wallaby: Add InternalApiNetwork entry in ServiceNetMap  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81519819:13
opendevreviewMerged openstack/tripleo-heat-templates stable/wallaby: GaneshaNetwork - fallback -> external -> ctlplane  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81519919:13
rlandy|ruckharold has some comments there19:13
mwhahahaconsole logs are 0 which would make me think problems with the rebooting of nodes19:13
rlandy|ruckTengu reported some issues with provisioning nodes in his lab in train19:14
rlandy|ruckprovisioning is not our issue though19:14
opendevreviewMerged openstack/tripleo-quickstart-extras master: Disable DNSSEC validation on ipa-server install  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81508119:15
mwhahahai recall also having problems with the two commands so i used --provide --all-managable19:15
rlandy|ruckhttps://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master19:15
mwhahahainstead of the split commands19:15
rlandy|ruckright - I remember that19:15
rlandy|rucksomehow fs035 gets the worst of it19:16
rlandy|rucknot sure if ipv6 even comes into the picture yet19:16
mwhahahaall the branches use the same tenant right?19:16
rlandy|ruckyes19:16
Tengurlandy|ruck: sorry, on master, for my issue19:24
rlandy|ruckk - so unrelated then19:25
rlandy|ruckmaster is doing ok for us19:25
TenguI will investigate this deeper over the week. guts say "race condition", especially since second attempt works19:27
Tengui will open an LP with details and logs19:28
mwhahaha2021-10-21 08:51:37.410 9 ERROR mistral.executors.default_executor mistral.exceptions.ActionException: BaremetalIntrospectionAction.wait_for_finish failed: Unable to establish connection to https://192.168.24.2:13050/v1/introspection/c6cbbc32-804a-4a69-a31f-e3d0c52f8ff2: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))19:31
mwhahahafor the record, even on success that workflow fails19:31
mwhahahait just errors differently19:31
mwhahaharlandy|ruck, dviroel|rover: added comment 9 to the LP. looks like ironic is losing it's mind 19:39
rlandy|ruckthanks for the update19:41
dviroel|rovermwhahaha: thanks for adding more info 19:41
opendevreviewmbu proposed openstack/tripleo-quickstart-extras master: Add validation playbook in ovb.yml and make conditional import  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81529219:54
opendevreviewMerged openstack/puppet-tripleo stable/wallaby: Make sure resource_op_defaults are set before bundles  https://review.opendev.org/c/openstack/puppet-tripleo/+/81494019:58
opendevreviewSlawek Kaplonski proposed openstack/tripleo-heat-templates stable/victoria: Added support for Neutron loggings service plugin configuration  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81358220:10
opendevreviewSlawek Kaplonski proposed openstack/tripleo-heat-templates stable/ussuri: Added support for Neutron loggings service plugin configuration  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81358320:11
opendevreviewSlawek Kaplonski proposed openstack/tripleo-heat-templates stable/train: Added support for Neutron loggings service plugin configuration  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81358420:12
*** holser is now known as holser_20:28
*** holser_ is now known as holser20:29
opendevreviewAde Lee proposed openstack/tripleo-heat-templates master: Add environment file for FIPS deployments  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81514120:34
*** dviroel|rover is now known as dviroel|rover|out20:56
opendevreviewMichele Baldessari proposed openstack/puppet-tripleo stable/victoria: Make sure resource_op_defaults are set before bundles  https://review.opendev.org/c/openstack/puppet-tripleo/+/81536121:17
opendevreviewAlex Schultz proposed openstack/tripleo-common master: Fix permissions for deployments  https://review.opendev.org/c/openstack/tripleo-common/+/81538421:27
opendevreviewIan Wienand proposed openstack/diskimage-builder master: [wip] regenerate initramfs with FEDORA-2021-e4843341ca  https://review.opendev.org/c/openstack/diskimage-builder/+/81538521:38
opendevreviewAde Lee proposed openstack/puppet-tripleo master: Add option to configure snmpd auth type  https://review.opendev.org/c/openstack/puppet-tripleo/+/81308721:53
opendevreviewMerged openstack/tripleo-heat-templates master: Adding NovaMkisofsCmd parameter to nova-compute  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81502922:03
opendevreviewAde Lee proposed openstack/tripleo-heat-templates master: Add auth type for snmpd readonly user  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81308922:03
opendevreviewAde Lee proposed openstack/tripleo-heat-templates master: Add environment file for FIPS deployments  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81514122:03
opendevreviewAde Lee proposed openstack/tripleo-heat-templates master: Add environment file for FIPS deployments  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81514122:05
*** holser is now known as holser_22:11
opendevreviewAlex Schultz proposed openstack/tripleo-common master: [WIP] task-core file generation  https://review.opendev.org/c/openstack/tripleo-common/+/79872122:18
opendevreviewIan Wienand proposed openstack/diskimage-builder master: [wip] regenerate initramfs with FEDORA-2021-e4843341ca  https://review.opendev.org/c/openstack/diskimage-builder/+/81538523:00
*** holser_ is now known as holser23:03
opendevreviewDavid Vallee Delisle proposed openstack/os-net-config master: DNM vDPA support for CentOS9  https://review.opendev.org/c/openstack/os-net-config/+/81518623:34
opendevreviewMerged openstack/tripleo-heat-templates stable/victoria: [victoria/ussuri/train] Tune minion defaults  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81432223:34

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!