| ianw | i've been hard-out with no internet till a few hours ago, technically on pto till 17th but now under home isolation, so won't be far from computer :) | 00:31 |
|---|---|---|
| opendevreview | Brendan Shephard proposed openstack/python-tripleoclient master: Move baremetal Introspect commands from Ansible https://review.opendev.org/c/openstack/python-tripleoclient/+/823925 | 03:31 |
| opendevreview | Brendan Shephard proposed openstack/python-tripleoclient master: Move baremetal Introspect commands from Ansible https://review.opendev.org/c/openstack/python-tripleoclient/+/823925 | 03:35 |
| ramishra | Hmm.. sounds like some pep8 jobs are broken with ansible-lint/rich 11.0.0, FWIW created a PR https://github.com/ansible-community/ansible-lint/pull/1797 | 04:19 |
| opendevreview | Brendan Shephard proposed openstack/python-tripleoclient master: Move baremetal Introspect commands from Ansible https://review.opendev.org/c/openstack/python-tripleoclient/+/823925 | 04:42 |
| *** ysandeep|out is now known as ysandeep | 05:35 | |
| opendevreview | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: Adapt standalone-upgrade to run tripleo operator https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/817552 | 06:46 |
| opendevreview | chandan kumar proposed openstack/tripleo-quickstart-extras master: Inject gating gating repo during image build https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/822897 | 06:53 |
| *** amoralej|off is now known as amoralej | 06:59 | |
| *** ysandeep is now known as ysandeep|afk | 07:05 | |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-ci master: DNM test CI https://review.opendev.org/c/openstack/tripleo-ci/+/823934 | 07:33 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-ci master: Fix ansible-lint by limiting rich module https://review.opendev.org/c/openstack/tripleo-ci/+/823937 | 07:49 |
| opendevreview | Jiri Podivin proposed openstack/validations-common master: DNM - Test of Coverchange job reporting https://review.opendev.org/c/openstack/validations-common/+/805592 | 07:50 |
| opendevreview | Ananya proposed openstack/tripleo-docs master: Updating promotion/CIX/ci-health related info in ruck rover doc https://review.opendev.org/c/openstack/tripleo-docs/+/823399 | 08:16 |
| opendevreview | Gael Chamoulaud proposed openstack/tripleo-validations master: feat(removal): tox-ansible support https://review.opendev.org/c/openstack/tripleo-validations/+/823942 | 08:24 |
| *** ysandeep|afk is now known as ysandeep | 08:36 | |
| *** jpena|off is now known as jpena | 08:51 | |
| opendevreview | mbu proposed openstack/validations-libs master: Move Cliff requirements to 2.16.0 https://review.opendev.org/c/openstack/validations-libs/+/823515 | 09:19 |
| opendevreview | Harald Jensås proposed openstack/python-tripleoclient stable/wallaby: node extract provisioned - use network_config_update https://review.opendev.org/c/openstack/python-tripleoclient/+/823840 | 09:54 |
| opendevreview | mbu proposed openstack/python-tripleoclient master: Exposes the community validation init command https://review.opendev.org/c/openstack/python-tripleoclient/+/823543 | 09:56 |
| opendevreview | Ananya proposed openstack/tripleo-docs master: Updating promotion/CIX/ci-health related info in ruck rover doc https://review.opendev.org/c/openstack/tripleo-docs/+/823399 | 10:00 |
| opendevreview | Harald Jensås proposed openstack/python-tripleoclient stable/wallaby: overcloud node extract provisioned - fixed_ips https://review.opendev.org/c/openstack/python-tripleoclient/+/822416 | 10:01 |
| opendevreview | mbu proposed openstack/python-tripleoclient master: Exposes the community validation init command https://review.opendev.org/c/openstack/python-tripleoclient/+/823543 | 10:09 |
| opendevreview | Merged openstack/tripleo-common stable/wallaby: Remove libcgroup-tools from libvirt container https://review.opendev.org/c/openstack/tripleo-common/+/823608 | 10:28 |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Change ansible_hostname to use ansible_facts https://review.opendev.org/c/openstack/tripleo-validations/+/823843 | 10:36 |
| Tengu | chandankumar: heya! quick question: do we have some CI job deploying an overcloud with master/cs9 ? | 10:37 |
| chandankumar | Tengu: yes | 10:37 |
| chandankumar | Tengu: you can check container multinode job | 10:38 |
| Tengu | chandankumar: what OC image are those jobs using? | 10:38 |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Make sure fact is available to find hostname https://review.opendev.org/c/openstack/tripleo-validations/+/823844 | 10:38 |
| chandankumar | Tengu: oh you mean ovb | 10:38 |
| Tengu | chandankumar: I'm hitting an issue, where the OC node doesn't reboot after the provisionning due to some weird dracut timeout - it's apparently waiting on a disk uuid that just doesn't exist. | 10:38 |
| chandankumar | Tengu: you can use the images from here https://images.rdoproject.org/centos9/master/rdo_trunk/?C=M;O=A | 10:38 |
| chandankumar | Tengu: I have a solution | 10:39 |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Make sure fact is available to find hostname https://review.opendev.org/c/openstack/tripleo-validations/+/823844 | 10:39 |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Make sure fact is available to find hostname https://review.opendev.org/c/openstack/tripleo-validations/+/823844 | 10:39 |
| Tengu | chandankumar: I already usr them - from current-tripleo-ci, because current-tripleo doesn't provide the overcloud-full archive | 10:39 |
| chandankumar | Tengu: you need this boy https://review.opendev.org/c/openstack/diskimage-builder/+/821772 | 10:39 |
| chandankumar | Tengu: may be you can build the image with this patch and test and then consume it | 10:39 |
| chandankumar | let me find a working hash | 10:40 |
| Tengu | hmmm. lemme check if I can just "edit" the image. | 10:40 |
| Tengu | instead of building | 10:40 |
| chandankumar | Tengu: bug link https://bugs.launchpad.net/tripleo/+bug/1954456 | 10:43 |
| Tengu | aha, yep, that's exactly the thing | 10:44 |
| opendevreview | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: Adapt standalone-upgrade to run tripleo operator https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/817552 | 10:45 |
| Tengu | brb | 10:48 |
| *** ysandeep is now known as ysandeep|coffee | 11:08 | |
| *** rlandy|out is now known as rlandy|ruck | 11:16 | |
| opendevreview | Jose Luis Franco proposed openstack/tripleo-ci master: WIP: Add undercloud-ffu-wallaby job. https://review.opendev.org/c/openstack/tripleo-ci/+/822716 | 11:19 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-ci master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-ci/+/823937 | 11:21 |
| *** dviroel|afk is now known as dviroel | 11:22 | |
| opendevreview | Jose Luis Franco proposed openstack/tripleo-heat-templates stable/wallaby: DNM: Test undercloud ffu wallaby upgrade. https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823966 | 11:26 |
| opendevreview | Tom Barron proposed openstack/tripleo-ansible stable/train: Use service_net_map for ganesha_vip https://review.opendev.org/c/openstack/tripleo-ansible/+/822044 | 11:27 |
| opendevreview | Merged openstack/tripleo-heat-templates stable/ussuri: [train,ussuri] Set cinder's __DEFAULT__ vol type description https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823411 | 11:38 |
| bogdando | arxcruz|ruck, rcastillo|rover, rlandy|ruck, chandankumar: hi, do you know if we still have some components promotion issues with glance? https://paste.opendev.org/show/btbbrqkvR9TRQab1rgmf/ <<< that's that I keep getting since the mid of Dec | 11:39 |
| bogdando | with my cs9 reproducer | 11:39 |
| bogdando | in the delorean repo list, there is also validations component with a dead link | 11:39 |
| bogdando | sshnaidm: ^^ | 11:40 |
| bogdando | I hoped that think got resolved by my return from pto, apparently didn't :) | 11:41 |
| bogdando | thing* | 11:41 |
| bogdando | any way to w/a that with overriding a particular component hash? | 11:41 |
| sshnaidm | bogdando, something wrong with your mirrors maybe, mirror01.ord.rax is upstream mirror, not rdo | 11:42 |
| bogdando | mhm | 11:42 |
| bogdando | thanks! | 11:42 |
| sshnaidm | bogdando, components should be in trunk.rdo... | 11:42 |
| bogdando | so what should I use instead of mirror_fqdn: mirror01.ord.rax.opendev.org | 11:43 |
| sshnaidm | bogdando, hash seems wrong, https://trunk.rdoproject.org/centos9-master/component/glance/current-tripleo/delorean.repo | 11:44 |
| rlandy|ruck | bogdando: chandankumar and I are looking at that | 11:44 |
| rlandy|ruck | there was a promotion on 01/06 | 11:44 |
| bogdando | thanks | 11:45 |
| bogdando | so it seems I used a wrong mirrot | 11:45 |
| rlandy|ruck | but glance component promoted after that | 11:45 |
| bogdando | rlandy|ruck: ^^ | 11:45 |
| sshnaidm | bogdando, oops, my bad, the mirror is ok, just wrong hash | 11:45 |
| rlandy|ruck | https://trunk.rdoproject.org/centos9-master/component/glance/current-tripleo/delorean.repo | 11:45 |
| sshnaidm | bogdando, http://mirror01.ord.rax.opendev.org:8080/rdo/centos9-master/component/glance/4e/e7/4ee7799aa7f6a7172e361392ebb8d3da03e0bf7f_dad8929e/ | 11:45 |
| rlandy|ruck | correct | 11:45 |
| bogdando | ugh | 11:45 |
| rlandy|ruck | the content needs to be promoted again | 11:46 |
| bogdando | but I get it automagically evaluated | 11:46 |
| bogdando | how can I fix that? | 11:46 |
| bogdando | tried current-tripleo in launcher job config | 11:46 |
| sshnaidm | bogdando, I see in current-tripleo there is a good hash: http://mirror01.ord.rax.opendev.org:8080/rdo/centos9-master/current-tripleo/delorean.repo | 11:47 |
| sshnaidm | baseurl=https://trunk.rdoproject.org/centos9-master/component/glance/4e/e7/4ee7799aa7f6a7172e361392ebb8d3da03e0bf7f_dad8929e | 11:47 |
| bogdando | so why I end up with a693e066b3a713e0cc76389837593cde472cb9c8... | 11:48 |
| sshnaidm | bogdando, check what you have in /etc/yum.repos.d/ | 11:48 |
| sshnaidm | is there a correct new current-tripleo repos | 11:49 |
| bogdando | sshnaidm: https://paste.opendev.org/show/bFoXiWh4hBsP6vATXxrv/ | 11:49 |
| rlandy|ruck | https://trunk.rdoproject.org/centos9-master/current-tripleo/delorean.repo - still looks like an old hash to me | 11:49 |
| sshnaidm | bogdando, that's old | 11:50 |
| bogdando | yea | 11:50 |
| sshnaidm | bogdando, when you run job it should be changed | 11:50 |
| bogdando | but it keeps the same :/ | 11:50 |
| sshnaidm | bogdando, take a look at repo-setup.log or something like this | 11:50 |
| bogdando | DLRN_PATH_TAG=current-tripleo/9d/dc/9ddc1659709b160e1237982101c11685 | 11:51 |
| bogdando | then curl --silent https://trunk.rdoproject.org/centos9-master/current/1d/89/1d89605bae05efc739334230ede23341/delorean.repo | 11:51 |
| bogdando | that's from DLRN_PATH_TAG_NEWEST=current/1d/89/1d89605bae05efc739334230ede23341 | 11:52 |
| bogdando | so how I get those wrong numbers | 11:52 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-ci master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-ci/+/823937 | 11:53 |
| *** ysandeep|coffee is now known as ysandeep | 11:54 | |
| sshnaidm | bogdando, need a full log | 11:55 |
| opendevreview | amolkahat proposed openstack/tripleo-ci master: [DNM][WIP] Use release file for C8 https://review.opendev.org/c/openstack/tripleo-ci/+/823688 | 11:55 |
| sshnaidm | bogdando, do you use content-provider? | 11:56 |
| bogdando | I'll respin with dea59c8788a0ae58fda5b100cd92844a | 11:56 |
| bogdando | no | 11:56 |
| bogdando | just a standalone job | 11:57 |
| *** akahat_ is now known as akahat | 12:02 | |
| sshnaidm | bogdando, it's done somewhere here: https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/tasks/get-dlrn-hash.yml#L78 | 12:03 |
| bogdando | that already operates by "resolved" current-tripleo into a hash | 12:04 |
| bogdando | the problem is why I get that resolved wrong | 12:05 |
| bogdando | ok, dea59c8788a0ae58fda5b100cd92844a failed to be lookked up in current (but it exists in current-tripleo) | 12:08 |
| bogdando | sigh | 12:08 |
| bogdando | I'll pick some random from https://trunk.rdoproject.org/centos9-master/current/ | 12:08 |
| ykarel | bogdando, one possible reason for wrong component repos is mirror you used served stale or cached content for delorean.repo | 12:08 |
| bogdando | no, I think the problem is in decoding current/current-tripleo vs given dlrn hashes in ansible vars | 12:09 |
| ykarel | full job output.txt and repo_setup output may have more details | 12:09 |
| bogdando | e.g. curl --silent https://trunk.rdoproject.org/centos9-master/current/de/a5/dea59c8788a0ae58fda5b100cd92844a/delorean.repo | 12:09 |
| bogdando | but the right path should be in current-tripleo | 12:10 |
| ykarel | yes you can't use random | 12:10 |
| bogdando | there should be fallback paths in decoding hashes given in words not hashids | 12:10 |
| ykarel | need to put the correct one or let deployment identify it based on tags | 12:10 |
| bogdando | well the latter didn't work for a reason | 12:11 |
| bogdando | ok nvm | 12:11 |
| bogdando | I'll pick whatever deploys for me | 12:11 |
| *** jcapitao is now known as jcapitao_lunch | 12:18 | |
| opendevreview | Arx Cruz proposed openstack/openstack-tempest-skiplist master: Adding standalone job to skip test_trunk_subport_lifecycle https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/823974 | 12:19 |
| opendevreview | Merged openstack/python-tripleoclient master: Handle missing ipv6 boolean in network_data.yaml for Ceph https://review.opendev.org/c/openstack/python-tripleoclient/+/823582 | 12:20 |
| Tengu | chandankumar: hmm, for the DIB cs9 workaround, it fails on opensuse, apparently due to some package installation? https://2ee5b06a468e56eeed6a-992fbb76369d370eec805eb398c9de6e.ssl.cf5.rackcdn.com/821772/16/check/dib-functests/31e0c0a/logs/opensuse_build-succeeds.FAIL.log "install-packages failed with returncode 106 | 12:22 |
| Tengu | " | 12:22 |
| Tengu | does it cound as the actual failure, or are there some other things to check? | 12:22 |
| opendevreview | chandan kumar proposed openstack/tripleo-quickstart master: Pull DLRN from github for CS9 https://review.opendev.org/c/openstack/tripleo-quickstart/+/823975 | 12:23 |
| Tengu | zypper exit code 106: Some repository had to be disabled temporarily because it failed to refresh | 12:24 |
| Tengu | fun. | 12:24 |
| Tengu | Repository 'openSUSE-Leap-15.1-Cloud' is invalid. | 12:24 |
| chandankumar | Tengu: not sure, may be steve baker or Ian (on pto) knows about suse repos | 12:25 |
| Tengu | "yay". so the patch is blocked. | 12:25 |
| chandankumar | yes kind of | 12:27 |
| Tengu | guess I'll check how to actually build the images in order to move on, with the patched DIB thing. | 12:27 |
| Tengu | ah. there's tripleo_overcloud_image_build in tripleo-operator-ansible. | 12:28 |
| chandankumar | Tengu: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/oooci-build-images use this to build the images | 12:28 |
| Tengu | hmm. guess I can make a cs-9 container to build it on-demand and expose the images on some local http server. | 12:29 |
| Tengu | chandankumar: thanks for the link, I'll check it out! | 12:30 |
| opendevreview | Merged openstack/openstack-tempest-skiplist master: Adding standalone job to skip test_trunk_subport_lifecycle https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/823974 | 12:37 |
| opendevreview | amolkahat proposed openstack/tripleo-quickstart master: Revert "Exclude libvirt/qemu from AppStream repo" https://review.opendev.org/c/openstack/tripleo-quickstart/+/823465 | 12:42 |
| opendevreview | chandan kumar proposed openstack/tripleo-ci master: [WIP] Make cs9 jobs voting https://review.opendev.org/c/openstack/tripleo-ci/+/823980 | 12:59 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-ci master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-ci/+/823937 | 13:00 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 13:02 |
| *** amoralej is now known as amoralej|lunch | 13:04 | |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/ansible-role-collect-logs master: Fix ansible-lint failure https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/823982 | 13:05 |
| opendevreview | John Fulton proposed openstack/python-tripleoclient stable/wallaby: Handle missing ipv6 boolean in network_data.yaml for Ceph https://review.opendev.org/c/openstack/python-tripleoclient/+/823845 | 13:07 |
| opendevreview | Merged openstack/tripleo-common master: Don't enable module stream in rhel9 https://review.opendev.org/c/openstack/tripleo-common/+/823820 | 13:13 |
| opendevreview | Sandeep Yadav proposed openstack/tripleo-common stable/wallaby: Don't enable module stream in rhel9 https://review.opendev.org/c/openstack/tripleo-common/+/823846 | 13:14 |
| *** pojadhav is now known as pojadhav|dr_appt | 13:17 | |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Validation actions refactor https://review.opendev.org/c/openstack/validations-libs/+/802901 | 13:26 |
| bogdando | sshnaidm, rlandy|ruck: I've made it deploying with | 13:27 |
| bogdando | dlrn_hash_tag: dea59c8788a0ae58fda5b100cd92844a # looked up in current-tripleo/ | 13:27 |
| bogdando | dlrn_hash_tag_newest: fd09e9932887deea81042c762492e506 # lookup up in current/ | 13:27 |
| bogdando | so TIL dlrn_hash_tag_newest magic a bit better | 13:28 |
| sshnaidm | bogdando, did it take the latest now? | 13:28 |
| bogdando | but dlrn_hash_tag: current-tripleo didn't show well tho | 13:28 |
| bogdando | well, it took what I specified | 13:29 |
| bogdando | iiuc, in CI we only use hash ids, perhaps at some point the tags like current-tripleo resolve got regressed?.. | 13:31 |
| cloudnull | mornings | 13:36 |
| rlandy|ruck | bogdando: yeah - we looked at it - the promoted glance is old | 13:44 |
| rlandy|ruck | we need to promote again | 13:44 |
| opendevreview | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Honor custom mirror value in release configs https://review.opendev.org/c/openstack/tripleo-quickstart/+/823987 | 13:45 |
| bogdando | rlandy|ruck, sshnaidm, ykarel: jfyi https://review.opendev.org/c/openstack/tripleo-quickstart/+/823987 | 13:45 |
| bogdando | that could bring some part of the failure into my case | 13:45 |
| bogdando | with that patch it should be better I think | 13:45 |
| ykarel | iirc that was done to avoid some mirror issues in ci, so ^ may re introduce that | 13:47 |
| ykarel | git history may confirm that | 13:47 |
| sshnaidm | ykarel, probably was related to mirrors sync problem | 13:47 |
| sshnaidm | bogdando, does trunk.rdo... fail for you? | 13:48 |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Validation actions refactor https://review.opendev.org/c/openstack/validations-libs/+/802901 | 13:48 |
| bogdando | no, but in case we want a custom mirror. we should not query the default one | 13:48 |
| bogdando | see some of the release files do use that env var, while others don't | 13:48 |
| bogdando | well, if there is problems in sync mirrors, we should not hide that? | 13:49 |
| bogdando | instead switch all to the most recent one? | 13:49 |
| bogdando | I found that difference confusing | 13:49 |
| opendevreview | Brent Eagles proposed openstack/tripleo-ansible master: Add logging configuration for designate bind https://review.opendev.org/c/openstack/tripleo-ansible/+/823989 | 13:50 |
| ykarel | sshnaidm, yes that's what i recall | 13:50 |
| opendevreview | Brent Eagles proposed openstack/tripleo-heat-templates master: Configure logging for designate bind backend https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823993 | 13:58 |
| *** amoralej|lunch is now known as amoralej | 14:01 | |
| Tengu | chandankumar: hmm. is there a list of the needed repositories for the role oooci-build-images? apparently I need to git clone (or install) «some» packages that aren't listed in the role | 14:04 |
| opendevreview | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix cloud-init detection rules https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823994 | 14:04 |
| chandankumar | Tengu: yes, https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/oooci-build-images/tasks/pre.yaml#L72 | 14:05 |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Remove domain equivalence check https://review.opendev.org/c/openstack/tripleo-validations/+/823995 | 14:06 |
| *** jcapitao_lunch is now known as jcapitao | 14:06 | |
| Tengu | weird... I don't see that openstack_git_root path created... | 14:06 |
| Tengu | oh. ok. the "main" doesn't include the "pre"...... bleh. | 14:11 |
| Tengu | chandankumar: :] where's that "ensure-pip" role defined? | 14:14 |
| chandankumar | Tengu: it comes from https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-pip | 14:15 |
| Tengu | thanks! | 14:15 |
| opendevreview | Soniya Murlidhar Vyas proposed openstack/tripleo-quickstart-extras master: Need to add netstat logging in the background https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654 | 14:15 |
| opendevreview | Brent Eagles proposed openstack/tripleo-heat-templates stable/wallaby: Run designate-api under apache https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823537 | 14:20 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 14:24 |
| opendevreview | Brent Eagles proposed openstack/tripleo-heat-templates stable/wallaby: Run designate-api under apache https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823537 | 14:25 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-ci master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-ci/+/823937 | 14:26 |
| *** rlandy|ruck is now known as rlandy|ruck|brb | 14:36 | |
| opendevreview | Merged openstack/ansible-role-collect-logs master: Fix ansible-lint failure https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/823982 | 14:39 |
| opendevreview | Alan Bishop proposed openstack/tripleo-heat-templates stable/train: [train,ussuri] Set cinder's __DEFAULT__ vol type description https://review.opendev.org/c/openstack/tripleo-heat-templates/+/824001 | 14:39 |
| opendevreview | chandan kumar proposed openstack/diskimage-builder master: Remove opensuse/build-succeeds funtests https://review.opendev.org/c/openstack/diskimage-builder/+/824002 | 14:40 |
| opendevreview | Jiri Podivin proposed openstack/tripleo-ci master: WIP/DNM - Add fips jobs https://review.opendev.org/c/openstack/tripleo-ci/+/808215 | 14:48 |
| Tengu | fultonj: heya! fyi, I can deploy a master/cs9 undercloud. the OC has an issue, but due to the oc-full images that aren't working yet. But that's about it :). tripleo-lab knows cs9/master | 14:50 |
| fultonj | Tengu: ack, thanks for the update | 14:52 |
| fultonj | fmount[m]: fyi ^ | 14:52 |
| fmount | wow Tengu ++ great !! | 14:54 |
| Tengu | fmount, fultonj note you have to switch to the "centos-9" branch for now. | 14:54 |
| Tengu | still under test, I'm making a container in order to build the OC images with some workaround/patches. | 14:55 |
| *** ykarel is now known as ykarel|away | 14:56 | |
| *** rlandy|ruck|brb is now known as rlandy|ruck | 14:59 | |
| *** dviroel is now known as dviroel|lunch | 15:02 | |
| opendevreview | Jiri Podivin proposed openstack/validations-common master: removing callbacks from validations-common https://review.opendev.org/c/openstack/validations-common/+/820562 | 15:04 |
| opendevreview | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/wallaby: Fix killscript regex match for libpod-conmon-* https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823847 | 15:19 |
| opendevreview | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/ussuri: Fix killscript regex match for libpod-conmon-* https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823848 | 15:19 |
| opendevreview | Bogdan Dobrelya proposed openstack/tripleo-heat-templates stable/victoria: Fix killscript regex match for libpod-conmon-* https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823849 | 15:20 |
| opendevreview | Lukas Bezdicka proposed openstack/tripleo-heat-templates stable/train: [train-only][ffwd] All in one upgrade option https://review.opendev.org/c/openstack/tripleo-heat-templates/+/817944 | 15:26 |
| opendevreview | Lukas Bezdicka proposed openstack/tripleo-heat-templates stable/train: [train-only][ffwd] Resolve OVNDB data loss during upgrade https://review.opendev.org/c/openstack/tripleo-heat-templates/+/820023 | 15:26 |
| bogdando | sshnaidm: something for tomorrow to look into, https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823994 seems needed as we disable cloud-init in libvirt reproducer | 15:30 |
| bogdando | but... now I have dlrn failing to build tht | 15:30 |
| bogdando | :] https://paste.opendev.org/show/bQp4bXO6LQS8maEMilh1/ | 15:31 |
| chandankumar | bogdando: we have a bug for that | 15:32 |
| bogdando | chandankumar: ah, good to know | 15:33 |
| bogdando | thanks | 15:33 |
| sshnaidm | bogdando, yaaa.. I had the same problem in Ci jobs | 15:33 |
| chandankumar | bogdando: https://bugs.launchpad.net/tripleo/+bug/1956946 | 15:33 |
| bogdando | nice to see I'm now hitting the same wall as you folks! | 15:34 |
| bogdando | that was a fast catch up after pto | 15:34 |
| bogdando | will try with that an extra dependency | 15:36 |
| opendevreview | Juan Larriba proposed openstack/tripleo-ansible master: The NFS directory parameter is hardcoded instead of re-used https://review.opendev.org/c/openstack/tripleo-ansible/+/824006 | 15:47 |
| *** dviroel|lunch is now known as dviroel | 16:07 | |
| *** amoralej is now known as amoralej|off | 16:12 | |
| opendevreview | chandan kumar proposed openstack/diskimage-builder master: Remove opensuse related funtests https://review.opendev.org/c/openstack/diskimage-builder/+/824002 | 16:16 |
| opendevreview | David Peacock proposed openstack/validations-common master: add pre-update and pre-upgrade groups to ntp playbook https://review.opendev.org/c/openstack/validations-common/+/824007 | 16:16 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 16:16 |
| *** ysandeep is now known as ysandeep|out | 16:17 | |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 17:02 |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Remove Docker references https://review.opendev.org/c/openstack/tripleo-validations/+/824035 | 17:10 |
| opendevreview | David Hill proposed openstack/tripleo-ansible master: Added all Controller* hosts to prepare-overcloud-backup.yaml https://review.opendev.org/c/openstack/tripleo-ansible/+/823740 | 17:12 |
| opendevreview | David Hill proposed openstack/tripleo-ansible master: Added all Controller* hosts to prepare-overcloud-backup.yaml https://review.opendev.org/c/openstack/tripleo-ansible/+/823740 | 17:14 |
| opendevreview | Douglas Viroel proposed openstack/tripleo-quickstart master: Fix ansible-lint failures https://review.opendev.org/c/openstack/tripleo-quickstart/+/824037 | 17:19 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 17:28 |
| opendevreview | mbu proposed openstack/validations-libs master: Move Cliff requirements to 2.16.0 https://review.opendev.org/c/openstack/validations-libs/+/823515 | 17:29 |
| opendevreview | David Hill proposed openstack/tripleo-ansible master: Added controller_group_name to override group name https://review.opendev.org/c/openstack/tripleo-ansible/+/823740 | 17:51 |
| opendevreview | David Hill proposed openstack/tripleo-ansible master: Added controller_group_name to override group name https://review.opendev.org/c/openstack/tripleo-ansible/+/823740 | 17:52 |
| *** jpena is now known as jpena|off | 17:53 | |
| *** odyssey4me is now known as Guest135 | 17:59 | |
| opendevreview | Andre Aranha proposed openstack/tripleo-validations stable/train: Add undercloud-ipa-server-check task https://review.opendev.org/c/openstack/tripleo-validations/+/824046 | 18:28 |
| opendevreview | Douglas Viroel proposed openstack/tripleo-quickstart master: Fix ansible-lint and tox-molecule failures https://review.opendev.org/c/openstack/tripleo-quickstart/+/824037 | 18:33 |
| opendevreview | Alex Schultz proposed openstack/python-tripleoclient master: Use task-core for deployments https://review.opendev.org/c/openstack/python-tripleoclient/+/821624 | 18:55 |
| opendevreview | Alex Schultz proposed openstack/python-tripleoclient master: Configure directord components during deploy https://review.opendev.org/c/openstack/python-tripleoclient/+/821876 | 18:55 |
| mwhahaha | rlandy|ruck, ade_lee: where is the upstream dns server config in the ipa job config. the job failed again on my ntp validation and i'm seeing some query bits in the query log | 19:19 |
| mwhahaha | https://9198d3054a9675b9d3ad-d6410d4969488abef51ee81e70d49bbc.ssl.cf1.rackcdn.com/822948/2/gate/tripleo-ci-centos-8-standalone-on-multinode-ipa/b308df2/logs/10.208.226.207/var/named/data/query_errors.log | 19:21 |
| mwhahaha | specifically a bunch of REFUSED | 19:21 |
| *** artom__ is now known as artom | 19:25 | |
| rlandy|ruck | mwhahaha: are you hitting https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/standalone/tasks/containers.yml#L37? | 19:31 |
| mwhahaha | rlandy|ruck: what do you mean? the undercloud is configured with the ipa server but the ntp hosts aren't resolving | 19:31 |
| mwhahaha | s/undercloud/standalone | 19:32 |
| mwhahaha | so i'm wondering if the ipa node has a bad upstream dns config | 19:32 |
| mwhahaha | but i don't know where to go looking in ipa | 19:32 |
| mwhahaha | that node is configured to use itself https://9198d3054a9675b9d3ad-d6410d4969488abef51ee81e70d49bbc.ssl.cf1.rackcdn.com/822948/2/gate/tripleo-ci-centos-8-standalone-on-multinode-ipa/b308df2/logs/10.208.226.207/etc/resolv.conf | 19:33 |
| mwhahaha | but looks like it's failing to query | 19:33 |
| mwhahaha | since it's green sometimes, i'm wondering if it's region specific | 19:33 |
| rlandy|ruck | depends on the provider https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/ipa-multinode/tasks/ipaserver-subnode-install.yml#L71 | 19:34 |
| rlandy|ruck | some use unbound | 19:34 |
| rlandy|ruck | possible that it is provider specific | 19:36 |
| opendevreview | Rafael Castillo proposed openstack/tripleo-quickstart master: Allow overriding the container registry namespace for Ussuri https://review.opendev.org/c/openstack/tripleo-quickstart/+/824056 | 19:37 |
| mwhahaha | rlandy|ruck: so unbound is configured in IPA node but is not running | 19:39 |
| mwhahaha | unbound.service loaded inactive dead Unbound recursive Domain Name Server | 19:39 |
| mwhahaha | so if IPA is configured to use it (/etc/resolv.conf poing to localhost), it's not working | 19:39 |
| mwhahaha | unbound gets turned off | 19:41 |
| mwhahaha | but freeipa gets configured with the unbound forwarder? | 19:42 |
| mwhahaha | https://paste.opendev.org/show/812005/ | 19:42 |
| mwhahaha | that seems wrong? | 19:42 |
| rlandy|ruck | ade_lee: ^^ we used the forwarder that was accessible per provider | 19:44 |
| rlandy|ruck | the ones that used unbound before - we pulled the forwarders from there | 19:44 |
| rlandy|ruck | the defaults did not work on all providers | 19:44 |
| rlandy|ruck | the ipa server install and confuguration is on subnode-1 | 19:45 |
| rlandy|ruck | the second node in the job | 19:45 |
| mwhahaha | so who configures unbound.conf? is that a zuul thing? | 19:46 |
| mwhahaha | it's configured with 1.0.0.1/8.8.8.8 | 19:46 |
| mwhahaha | i wonder if that doesn't work in rax ord (where that job ran) | 19:46 |
| cloudnull | the 10.x space on rax is the internal "private" network in RAX | 19:47 |
| cloudnull | so potentially ? | 19:47 |
| mwhahaha | 1.0.0.1 not 10.x :D | 19:48 |
| rlandy|ruck | we don't configure unbound | 19:48 |
| * cloudnull oh I miss read that | 19:48 | |
| rlandy|ruck | some clouds don't use it | 19:48 |
| rlandy|ruck | however zuul sets up per cloud provider | 19:48 |
| mwhahaha | ok so the ipa forwarders are getting configured to what matches the unbound conf. i wonder why it's refusing on lookups | 19:53 |
| ade_lee | mwhahaha, rlandy|ruck reading up .. | 19:53 |
| ade_lee | mwhahaha, rlandy|ruck so we're in a case where unbound was configured by zuul, and so we configured ipa to use it as a forwarder, but its not working? | 19:58 |
| mwhahaha | yes that seems to be the situation | 19:58 |
| mwhahaha | https://9198d3054a9675b9d3ad-d6410d4969488abef51ee81e70d49bbc.ssl.cf1.rackcdn.com/822948/2/gate/tripleo-ci-centos-8-standalone-on-multinode-ipa/b308df2/logs/10.208.226.207/var/named/data/security.log | 19:58 |
| mwhahaha | so it gets picked up because the ntp servers never resolve and ntp validation fails | 19:59 |
| mwhahaha | looking on the ipa node, the lookup appear to be failing in named for some reason | 19:59 |
| mwhahaha | i actually dont' see the forwarders in /etc/named.conf | 20:01 |
| mwhahaha | do they go somewhere else? | 20:01 |
| mwhahaha | https://9198d3054a9675b9d3ad-d6410d4969488abef51ee81e70d49bbc.ssl.cf1.rackcdn.com/822948/2/gate/tripleo-ci-centos-8-standalone-on-multinode-ipa/b308df2/logs/10.208.226.207/etc/named.conf | 20:01 |
| ade_lee | mwhahaha, I think they actually go into db entries iirc | 20:02 |
| mwhahaha | the finest in black magic | 20:03 |
| ade_lee | I do see the forwarders being added in the ipa srever install .. | 20:04 |
| mwhahaha | yea i saw that | 20:05 |
| ade_lee | 1.0.0.1, 8.8.8.8 and 1.1.1.1 | 20:05 |
| ade_lee | mwhahaha, is this failing consistently on a particular cloud? | 20:06 |
| mwhahaha | i've seen it on rax, haven't been tracking it | 20:06 |
| ade_lee | mwhahaha, checking with ipa folks .. | 20:11 |
| mwhahaha | k | 20:12 |
| ade_lee | rlandy|ruck, do we know if this is failing like this elsewhere or on other rax nodes? | 20:27 |
| * rlandy|ruck reads back | 20:27 | |
| mwhahaha | ade_lee: so my change to re-enable ntp sync check hasn't landed yet so we don't hit it live in ci | 20:28 |
| mwhahaha | https://review.opendev.org/c/openstack/tripleo-heat-templates/+/822948 | 20:28 |
| mwhahaha | i mean we might be hitting this issue but it'd be hard to track because no failures | 20:28 |
| rlandy|ruck | ade_lee: as of now, https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa does not look too bad | 20:28 |
| mwhahaha | yea cause my patch hasn't landed yet | 20:28 |
| mwhahaha | if it lands it'll trigger a bunch i'm sure | 20:29 |
| rlandy|ruck | is rax the only problematic provider? | 20:29 |
| mwhahaha | thats' where it failed in gate | 20:30 |
| mwhahaha | let me load up some of the jobs and spot check | 20:30 |
| mwhahaha | looks like i'd fail in rax iad https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e04/823849/1/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/e0476a4/logs/undercloud/var/log/extra/chrony.txt | 20:31 |
| mwhahaha | or whatever inmotion-iad3 is | 20:32 |
| mwhahaha | is that rax? | 20:32 |
| rlandy|ruck | if it's more than one cloud provider, we can't blame them | 20:34 |
| mwhahaha | airship-kna1 was fine https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b72/809360/17/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/b72d303/logs/undercloud/var/log/extra/chrony.txt | 20:34 |
| mwhahaha | iweb-mtl01 is fine https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b69/822992/2/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/b6946df/logs/undercloud/var/log/extra/chrony.txt | 20:34 |
| ade_lee | rlandy|ruck, mwhahaha can we run with the patch and hold the node if it fails (on rax?) | 20:34 |
| rlandy|ruck | ade_lee: can't hold nodes on upstream | 20:35 |
| rlandy|ruck | only rdo | 20:35 |
| rlandy|ruck | which is always vexxhost | 20:35 |
| rlandy|ruck | we'd need to see what's similar about these providers | 20:35 |
| rlandy|ruck | are they the ones that use unbound? etc. | 20:35 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 20:36 |
| mwhahaha | airship-kna1 uses unbound and it was fine | 20:36 |
| mwhahaha | i thought they all used unbound now | 20:37 |
| johnsom | Unbound is widely used | 20:37 |
| ade_lee | are there other successful runs on rax (not with ipa)? | 20:37 |
| mwhahaha | yea it's just an ipa job thing | 20:37 |
| ade_lee | can we check those and see what the unbound/ forwarder config is? | 20:38 |
| mwhahaha | they are the same | 20:38 |
| rlandy|ruck | https://b6153a8c6a54f26baa54-e4b54cf82b19c70bded9dbfe71e9b8f5.ssl.cf1.rackcdn.com/823848/1/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/cfd0f66/job-output.txt | 20:38 |
| mwhahaha | the difference here is the standalone node is being configured to use teh IPA server which fails to resolve the ntp hosts | 20:38 |
| rlandy|ruck | ^^ ipa on raxx success | 20:38 |
| rlandy|ruck | ade_lee: we went though a stage of adding some servers with their addresses | 20:40 |
| mwhahaha | no resolver errors on that run https://b6153a8c6a54f26baa54-e4b54cf82b19c70bded9dbfe71e9b8f5.ssl.cf1.rackcdn.com/823848/1/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/cfd0f66/logs/10.209.130.78/var/named/data/query_errors.log | 20:40 |
| rlandy|ruck | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/ipa-multinode/tasks/ipaserver-undercloud-setup.yml#L231 | 20:41 |
| mwhahaha | rlandy|ruck: so that's rax-iad | 20:42 |
| mwhahaha | it's failing in rax-ord | 20:42 |
| mwhahaha | https://b6153a8c6a54f26baa54-e4b54cf82b19c70bded9dbfe71e9b8f5.ssl.cf1.rackcdn.com/823848/1/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/cfd0f66/zuul-info/zuul-info.primary.txt | 20:42 |
| mwhahaha | cloudnull: do you recall any issues with dns in rax-ord? | 20:42 |
| cloudnull | no , none that I know of | 20:42 |
| mwhahaha | me neither | 20:43 |
| mwhahaha | though 1.0.0.1/8.8.8.8 do have rate limits so maybe ord's hitting that | 20:43 |
| mwhahaha | would be nice if they used provider dns servers rather than cloudflare/google | 20:43 |
| rlandy|ruck | we defaulted these servers: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/822858/2/roles/standalone/templates/standalone_config.yaml.j2 | 20:45 |
| rlandy|ruck | and git some objections to those defaults | 20:45 |
| mwhahaha | those are the default if not specified | 20:45 |
| mwhahaha | i don't think in this case that's the issue | 20:45 |
| mwhahaha | it seems more to be the resolution of those hosts | 20:45 |
| mwhahaha | even if not configured there it'd be those values | 20:46 |
| ade_lee | mwhahaha, you'd think if it were hitting rate limits, it would be doing so in the non-ipa jobs too .. | 20:52 |
| ade_lee | mwhahaha, when a patch is tested, do all the jobs end up on one cloud ? | 20:54 |
| rlandy|ruck | per run, yes - on different run, no | 20:54 |
| ade_lee | hmm .. | 20:55 |
| opendevreview | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: Add entries to /etc/hosts - pool.ntp.org https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/824061 | 20:57 |
| ade_lee | I suppose one way to test the rate limit theory would be to only run the ipa job and see if it passes .. | 20:58 |
| cloudnull | mwhahaha I do know that infra did some tomfoolery with their unbound config a while back to better support rax cloud environments. Through that was a long while ago, so I have no idea if that is still in effect. | 21:08 |
| opendevreview | Merged openstack/diskimage-builder master: Remove opensuse related funtests https://review.opendev.org/c/openstack/diskimage-builder/+/824002 | 21:09 |
| *** dviroel is now known as dviroel|out | 21:28 | |
| ade_lee | mwhahaha, if we want to maybe see why its failing -- we could try running rndc querylog on and/or rndc trace <level> | 21:37 |
| mwhahaha | probably wouldn't hurt to throw something like that into the collect-logs role | 21:38 |
| mwhahaha | also can we increase the ipa server debug level? | 21:38 |
| mwhahaha | i asked one of my friends who is a qe in that org and he recommended trying that | 21:39 |
| rlandy|ruck | ok by me to increase the debug level | 21:42 |
| mwhahaha | looks like a server.conf & restart | 21:42 |
| mwhahaha | let me see if i can propose something | 21:42 |
| opendevreview | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Enable IPA debug logging https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/824067 | 21:47 |
| mwhahaha | ade_lee: -^ does that do it or do i also need to restart httpd instead of ipa | 21:48 |
| rlandy|ruck | we'll have to rebase on the linters change | 21:49 |
| mwhahaha | which review? | 21:49 |
| mwhahaha | i thought they released a fix | 21:49 |
| rlandy|ruck | https://review.opendev.org/q/topic:%22a-lint%22+(status:open%20OR%20status:merged) | 21:49 |
| rlandy|ruck | they did but we had those repos pinned | 21:50 |
| mwhahaha | k | 21:50 |
| opendevreview | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Enable IPA debug logging https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/824067 | 21:50 |
| *** rlandy|ruck is now known as rlandy|dentist | 21:52 | |
| opendevreview | Alex Schultz proposed openstack/tripleo-heat-templates master: [DNM] Troubleshooting ntp & ipa stuff https://review.opendev.org/c/openstack/tripleo-heat-templates/+/824069 | 22:01 |
| ade_lee | mwhahaha, I think you still need to do the rndc commands -- I doubt the debug you set will get all the dns info | 22:14 |
| mwhahaha | ok | 22:15 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/823981 | 22:28 |
| opendevreview | Merged openstack/python-tripleoclient stable/train: Passes playbook to the validation action as a list https://review.opendev.org/c/openstack/python-tripleoclient/+/823760 | 23:29 |
| opendevreview | Merged openstack/tripleo-common stable/wallaby: Don't enable module stream in rhel9 https://review.opendev.org/c/openstack/tripleo-common/+/823846 | 23:29 |
| opendevreview | Merged openstack/tripleo-ansible stable/train: Use service_net_map for ganesha_vip https://review.opendev.org/c/openstack/tripleo-ansible/+/822044 | 23:29 |
| opendevreview | Merged openstack/python-tripleoclient stable/wallaby: Handle missing ipv6 boolean in network_data.yaml for Ceph https://review.opendev.org/c/openstack/python-tripleoclient/+/823845 | 23:29 |
| opendevreview | Merged openstack/python-tripleoclient master: Make deployed ceph baremetal file optional https://review.opendev.org/c/openstack/python-tripleoclient/+/822726 | 23:29 |
| opendevreview | Merged openstack/tripleo-ci master: Fix ansible-lint failure https://review.opendev.org/c/openstack/tripleo-ci/+/823937 | 23:29 |
| opendevreview | Tom Barron proposed openstack/tripleo-heat-templates stable/train: Remove ganesha_vip extra config workaround https://review.opendev.org/c/openstack/tripleo-heat-templates/+/823750 | 23:41 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!