Monday, 2022-07-18

opendevreviewMerged openstack/tripleo-heat-templates master: Fix missing roles for Octavia services  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/77694200:18
opendevreviewBrendan Shephard proposed openstack/tripleo-heat-templates master: Add handling for OVN Multi-RHEL  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84820804:32
opendevreviewBrendan Shephard proposed openstack/tripleo-common master: Add multi-rhel support for OvnController  https://review.opendev.org/c/openstack/tripleo-common/+/85013104:33
opendevreviewchandan kumar proposed openstack/tripleo-ansible stable/wallaby: Use ubi9-init image as base  https://review.opendev.org/c/openstack/tripleo-ansible/+/84881004:39
opendevreviewBrendan Shephard proposed openstack/tripleo-heat-templates master: Add handling for OVN Multi-RHEL  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84820805:03
opendevreviewManojkatari proposed openstack/tripleo-ansible master: Add tripleo_etcd ansible role  https://review.opendev.org/c/openstack/tripleo-ansible/+/84902405:34
opendevreviewGregory Thiemonge proposed openstack/tripleo-quickstart master: Configure new RBAC type for Octavia tempest  https://review.opendev.org/c/openstack/tripleo-quickstart/+/85011805:42
opendevreviewchandan kumar proposed openstack/tripleo-ci master: Move tripleo-tox-molecule job definition to base-upstream.yaml  https://review.opendev.org/c/openstack/tripleo-ci/+/84985605:42
chkumar|roverTengu: please add +w to this patch https://review.opendev.org/c/openstack/tripleo-ansible/+/848130 +w got lost due to rebase, thanks :-)05:45
Tenguchkumar|rover: done :)06:01
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible stable/wallaby: Run package check for openstack-selinux in privileged mode  https://review.opendev.org/c/openstack/tripleo-ansible/+/84949606:02
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: New role, replacing puppet-auditd  https://review.opendev.org/c/openstack/tripleo-ansible/+/84875806:06
opendevreviewchandan kumar proposed openstack/openstack-tempest-skiplist master: Move octavia LoadBalancerScenarioTest to skip list  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/85013706:08
chkumar|roverTengu: thanks :-)06:08
Tengunp06:11
opendevreviewGregory Thiemonge proposed openstack/tripleo-heat-templates stable/wallaby: Fix missing roles for Octavia services  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/85003306:19
opendevreviewManojkatari proposed openstack/tripleo-heat-templates master: support tripleo_etcd ansible role  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84984406:33
*** amoralej|off is now known as amoralej06:40
Tenguysandeep: heya! iirc you had some issues with IPv6 jobs and the nftables switch? my patch getting the nftables config dir in the logs has merged, care to re-kick a job?06:49
opendevreviewBrendan Shephard proposed openstack/tripleo-heat-templates master: Add handling for OVN Multi-RHEL  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84820806:58
opendevreviewBrendan Shephard proposed openstack/tripleo-heat-templates master: Add handling for OVN Multi-RHEL  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84820807:03
opendevreviewAlfredo Moralejo proposed openstack/tripleo-quickstart master: DNM test rabbitmq update in wallaby and master  https://review.opendev.org/c/openstack/tripleo-quickstart/+/84843907:08
opendevreviewAlfredo Moralejo proposed openstack/tripleo-quickstart master: DNM test rabbitmq update in wallaby and master  https://review.opendev.org/c/openstack/tripleo-quickstart/+/84843907:09
ysandeepTengu, hey o/ I already reran the testproject(with depends-on) when you posted that patch08:04
ysandeepTengu, https://review.rdoproject.org/r/c/testproject/+/31954/36#message-23c03738a4677096e93c463c43d4b4ddef53c80f08:04
ysandeepTengu, we already have the logs: https://logserver.rdoproject.org/54/31954/36/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/847bf87/logs/overcloud-controller-0/etc/nftables/ 08:04
Tenguah, good. lemme check.08:05
Tenguuho. wait. there should be more files.08:05
Tenguo_O08:06
Tenguthose are just the default files.08:06
Tenguhttps://logserver.rdoproject.org/54/31954/36/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/847bf87/logs/undercloud/etc/nftables/08:06
Tengubetter.08:07
Tenguysandeep: we need the UC config, that's where the NAT is set.08:07
Tenguysandeep: https://logserver.rdoproject.org/54/31954/36/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/847bf87/logs/undercloud/etc/nftables/tripleo-rules.nft.gz I don't see anything IPv6 related attached to the "forward" string...08:08
Tenguguess that's a reason.08:08
Tenguall is set for ipv408:08
Tenguysandeep: https://logserver.rdoproject.org/54/31954/36/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/847bf87/logs/undercloud/home/zuul/undercloud-parameter-defaults.yaml.txt.gz no IPv6 either.08:09
Tengumay be the issue?08:10
ysandeepfs035 is not completely IPv6.08:11
ysandeepwe still use ipv4 for ctlplane 08:11
ysandeepand I think for tenant network as well08:11
Tenguhmmm.08:12
ysandeephttps://github.com/openstack/tripleo-heat-templates/blob/master/ci/network_data_v6.yaml#L46-L54 yeah only tenant n/w still have ipv408:13
Tenguysandeep: can you point an iptables job result for that one?08:13
TenguI want to compare the generated iptables rules...08:13
Tenguespecially since we already get the nftables format in the logs :)08:13
Tenguhump.08:14
ysandeepTengu, sure08:14
ysandeepTengu, last green fs035 job: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/a5f1282/ 08:15
Tengugimme a moment to compare - thanks!08:16
Tenguoh. uho.08:16
TenguI'm pretty sure we're, once again, falling in that weird thing where the connection is accepted while it shouldn't, with iptables.08:17
Tengufu..08:17
Tenguthat will be hard to debug.08:17
ysandeepIf logs don't point much and you need to look at live env, please let me know - I can add a env on hold.08:26
ysandeep>> falling in that weird thing where the connection is accepted while it shouldn't, with iptables. ? What's that issue, have you already faced it somewhere else?08:27
Tenguysandeep: yeah, faced it already, when I started to work on nftables. Took about a week to find out the issue, with slaweq 08:28
Tenguysandeep: thing is, with the iptables "way", we're currently dropping only the NEW connections. While, with the nftables thing I've created, we're dropping *all* that doesn't match any rule.08:29
Tenguso not only the new things.08:29
Tenguthat's the "power" of a chain policy vs the "final drop" we're using.08:29
Tenguno more pitfall. but since we never ever bothered, we're seeing some stuff were working because we didn't know it shouldn't :)08:29
Tengufun times ahead!08:30
ysandeepinteresting 08:30
Tenguand this is exactly why I wanted to be able to switch jobs one by one.08:30
Tenguagain08:30
Tengu:)08:30
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987608:33
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987608:34
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987608:35
*** ysandeep is now known as ysandeep|lunch08:38
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987608:38
opendevreviewLuis Tomas Bolivar proposed openstack/tripleo-heat-templates stable/wallaby: Include octavia::driver_agent via puppet-tripleo  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84716908:52
opendevreviewLuis Tomas Bolivar proposed openstack/tripleo-heat-templates stable/wallaby: Configure OVN sb connections for the OVN provider  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84983908:52
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987609:09
opendevreviewMikolaj Ciecierski proposed openstack/tripleo-ansible master: Ensure english locale when running the command for ovs checks.  https://review.opendev.org/c/openstack/tripleo-ansible/+/83267109:09
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987609:33
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987610:01
opendevreviewMarios Andreou proposed openstack/tripleo-quickstart-extras master: Adds toci playbooks and updates for new multinode mixed OS jobs  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/84176410:02
opendevreviewJiri Podivin proposed openstack/validations-common master: Creating bandit CI job and tox env  https://review.opendev.org/c/openstack/validations-common/+/84941710:03
opendevreviewJiri Podivin proposed openstack/validations-common master: Creating bandit CI job and tox env  https://review.opendev.org/c/openstack/validations-common/+/84941710:05
*** ysandeep|lunch is now known as ysandeep10:23
opendevreviewTom Weininger proposed openstack/tripleo-heat-templates stable/wallaby: Add support for log offloading over TCP  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/85014810:26
opendevreviewTom Weininger proposed openstack/tripleo-ansible stable/wallaby: Support amphora log offloading over TCP  https://review.opendev.org/c/openstack/tripleo-ansible/+/85015010:29
*** rlandy|out is now known as rlandy10:30
opendevreviewDamien Ciabrini proposed openstack/puppet-tripleo stable/wallaby: Support mariabackup as a galera SST method  https://review.opendev.org/c/openstack/puppet-tripleo/+/85003510:31
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987610:37
Tenguysandeep: ok, I think I may have found where nftables logs its things - need to add some more parameters to the "log" jump. I'll do some local testing to ensure it's working as expected...11:19
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: [WIP] Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987611:20
ysandeepTengu: great11:20
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987611:20
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987611:20
Tenguysandeep: this means extending a bit the tripleo_nftables to a point it will not be compatible with iptables any more though. Need to find a way to ensure I'm not breaking things at this point... heh..11:21
Tengunote: this is exactly why the iptables-nft doesn't treat the "log" jump.11:21
Tenguthere are more flag, and it would probably have been too complicated in the initial wrapper.11:22
Tengu~> this will allow us to take full advantage of nftables :).11:22
*** dviroel_ is now known as dviroel11:35
Tenguysandeep: found a way to not break iptables support: if a parameter in the rules starts with "nft_", it will be discarded from the tripleo_iptables action plugin.11:50
Tengu#bam.11:50
Tengua way to get proper nftables parameter passthrough.11:50
Tenguneeded as long as we support both - shouldn't be THAT long hopefully.11:50
opendevreviewFernando Royo proposed openstack/puppet-tripleo stable/wallaby: Configure ovn sb connection for the ovn provider  https://review.opendev.org/c/openstack/puppet-tripleo/+/84987612:08
*** amoralej is now known as amoralej|lunch12:15
opendevreviewBrendan Shephard proposed openstack/tripleo-heat-templates master: Add handling for OVN Multi-RHEL  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84820812:23
ysandeepTengu++ nice12:28
opendevreviewBrendan Shephard proposed openstack/tripleo-common master: Add multi-rhel support for OvnController  https://review.opendev.org/c/openstack/tripleo-common/+/85013112:29
opendevreviewchandan kumar proposed openstack/openstack-tempest-skiplist master: [DNM]Remove test_minimum_basic_instance_hard_reboot_after_vol_snap_deletion  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/85015512:30
opendevreviewchandan kumar proposed openstack/tripleo-ci master: [DNM] testing hard_reboot tests  https://review.opendev.org/c/openstack/tripleo-ci/+/85015612:31
opendevreviewJiri Podivin proposed openstack/validations-libs master: DNM Testing if gerrit works properly  https://review.opendev.org/c/openstack/validations-libs/+/85015712:33
opendevreviewyatin proposed openstack/tripleo-heat-templates master: [WIP] Undercloud OVN switch  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/85015812:34
opendevreviewVeronika Fisarova proposed openstack/validations-libs master: DNM testing if gerrit works properly  https://review.opendev.org/c/openstack/validations-libs/+/85015912:35
opendevreviewJohn Fulton proposed openstack/tripleo-heat-templates master: Test Override tripleo_ceph_client_vars in tripleo_run_cephadm role  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84958012:40
opendevreviewVeronika Fisarova proposed openstack/validations-libs master: DNM testing if gerrit works properly  https://review.opendev.org/c/openstack/validations-libs/+/85015912:50
opendevreviewSoniya Murlidhar Vyas proposed openstack/openstack-tempest-skiplist master: Add featureset030 group into tempest-allow  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/84445012:53
opendevreviewMerged openstack/tripleo-ansible stable/wallaby: Run package check for openstack-selinux in privileged mode  https://review.opendev.org/c/openstack/tripleo-ansible/+/84949612:55
*** Guest5324 is now known as rcastillo13:07
*** amoralej|lunch is now known as amoralej13:17
Tenguysandeep: \o/ I have an nftables compatible thing for the logging. It should end in the audit.log. I have to run another test with iptables to ensure I didn't explode its support, but it should be fine... Then I'll be able to push the 2 patches (one for the nft_* drop in tripleo_iptables, and the second one for the nftables logging)13:24
ysandeepawesome, I will test your patches as depends-on once you pushes them13:26
opendevreviewGregory Thiemonge proposed openstack/tripleo-quickstart master: Configure new RBAC type for Octavia tempest  https://review.opendev.org/c/openstack/tripleo-quickstart/+/85011813:27
opendevreviewTakashi Kajinami proposed openstack/python-tripleoclient master: Enable debug in standalone heat  https://review.opendev.org/c/openstack/python-tripleoclient/+/85022013:34
*** dasm|off is now known as dasm|ruck13:43
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: Filter out nftables keys when using iptables  https://review.opendev.org/c/openstack/tripleo-ansible/+/85022113:53
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: Better logging management for nftables rules  https://review.opendev.org/c/openstack/tripleo-ansible/+/85022213:53
Tenguysandeep: -^^13:53
Tenguysandeep: I'll edit your testproject to depends-on that bunch of patches.13:53
ysandeepTengu: ack13:54
Tengudone.13:54
Tengulet's see.13:54
Tenguwe SHOULD get proper logging in the auditd.13:54
Tengufinger crossed.13:54
Tenguiptables isn't broken \o/13:54
Tenguall is fine.13:54
Tenguah, lemme pass those 2 as WIP for now.13:55
opendevreviewchandan kumar proposed openstack/tripleo-ci master: Add load_balancer.RBAC_test_type: "keystone_default_roles"  https://review.opendev.org/c/openstack/tripleo-ci/+/85022514:12
opendevreviewAnanya proposed openstack/tripleo-repos master: Added 17-1 option for releases in get hash  https://review.opendev.org/c/openstack/tripleo-repos/+/85022714:17
dasm|rucko/ we have a backport for octavia roles. It's currently affecting the gate. Can I ask for +W on it? https://review.opendev.org/c/openstack/tripleo-heat-templates/+/85003314:37
Tengudasm|ruck: done14:59
dasm|ruckthanks14:59
Tengunp - simple enough to get a quick review :)14:59
dasm|ruck:)14:59
*** dkehn_ is now known as dkehn15:01
*** dviroel is now known as dviroel|lunch15:08
opendevreviewRabi Mishra proposed openstack/python-tripleoclient stable/train: [train-only] Don't always generate keys for ssh  https://review.opendev.org/c/openstack/python-tripleoclient/+/85023615:09
odyssey4mehi folks - could I get some eyes on https://review.opendev.org/c/openstack/tripleo-common/+/776674 - it's to build an Ansible Execution Environment container... it doesn't affect anything in TripleO but we would like it to start being available for consumption in another initiative related to TripleO15:14
*** ysandeep is now known as ysandeep|out15:17
opendevreviewchandan kumar proposed openstack/openstack-tempest-skiplist master: Remove test_minimum_basic_instance_hard_reboot_after_vol_snap_deletion  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/85015515:17
opendevreviewRabi Mishra proposed openstack/python-tripleoclient stable/train: [train-only] Don't always generate keys for ssh  https://review.opendev.org/c/openstack/python-tripleoclient/+/85023615:22
*** marios is now known as marios|out15:48
opendevreviewAnanya proposed openstack/tripleo-repos master: Added 17-1 option for releases in get hash  https://review.opendev.org/c/openstack/tripleo-repos/+/85022715:48
*** dviroel_ is now known as dviroel16:15
*** amoralej is now known as amoralej|off16:16
opendevreviewyatin proposed openstack/tripleo-heat-templates master: [WIP] Undercloud OVN switch  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/85015816:29
*** rlandy_ is now known as rlandy17:59
opendevreviewMerged openstack/tripleo-upgrade stable/train: Append item to custom_env_files of undercloud.conf  https://review.opendev.org/c/openstack/tripleo-upgrade/+/84982918:03
opendevreviewMerged openstack/tripleo-heat-templates stable/wallaby: Remove the Backup and Restore ReaR Heat templates  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84885018:17
opendevreviewMerged openstack/tripleo-heat-templates master: Use tripleo_kernel standalone role for upgrade_tasks  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83850718:17
opendevreviewMerged openstack/tripleo-heat-templates master: Use tripleo_iscsid standalone ansible role  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83890418:20
opendevreviewMerged openstack/tripleo-heat-templates stable/wallaby: Fix missing roles for Octavia services  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/85003319:03
*** rlandy_ is now known as rlandy19:12
*** tosky_ is now known as tosky19:45
beagleshas anyone ever run into an issue where hostnames on a particular host don't match the ansible inventory/IP addresses19:51
beaglesI've seen two separate deployments, wallaby based, where controller-0 isn't controller-0 or controller-1 isn't controller-1.. basically inventory and hostnames don't agree19:52
beaglesfirst noticed when a bootstrap exec ran on node other than expectedd19:52
opendevreviewMikolaj Ciecierski proposed openstack/tripleo-upgrade master: Add additional tags to update validations tasks  https://review.opendev.org/c/openstack/tripleo-upgrade/+/84930620:20
opendevreviewMikolaj Ciecierski proposed openstack/tripleo-upgrade stable/wallaby: Add additional tags to update validations tasks  https://review.opendev.org/c/openstack/tripleo-upgrade/+/84931420:21
opendevreviewdasm proposed openstack/tripleo-ci master: Add load_balancer.RBAC_test_type: "keystone_default_roles"  https://review.opendev.org/c/openstack/tripleo-ci/+/85022520:35
tonybbeagles: Yup.  I've seen that on an off since queens? (I think).  So you aren't "crazy"21:05
beaglestonyb, that's... interesting :)21:05
tonybbeagles: good luck!21:08
opendevreviewJohn Fulton proposed openstack/tripleo-ansible master: Revert "Set tripleo_ceph_client_vars consistently with THT"  https://review.opendev.org/c/openstack/tripleo-ansible/+/85003721:26
*** dasm|ruck is now known as dasm|off21:27
opendevreviewJohn Fulton proposed openstack/tripleo-heat-templates master: Do not override CephClientConfigVars from CI scenarios  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/84975621:37
*** dviroel is now known as dviroel|out21:39
*** rlandy is now known as rlandy|bbl21:56
opendevreviewMerged openstack/tripleo-ansible stable/wallaby: Rename molecule jobs to centos-stream  https://review.opendev.org/c/openstack/tripleo-ansible/+/84813022:06

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!