| openstackgerrit | Merged openstack/python-tripleoclient master: Clean unit tests from OS::TripleO::Hosts::SoftwareConfig https://review.opendev.org/714677 | 00:02 |
|---|---|---|
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: HA: check before restarting resource on stack update https://review.opendev.org/714634 | 00:02 |
| *** tosky has quit IRC | 00:32 | |
| *** lbragstad has quit IRC | 00:37 | |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/train: Missed variable redefinition during cleanup. https://review.opendev.org/714587 | 00:42 |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: container_update: Retry removing containers. https://review.opendev.org/704655 | 00:42 |
| *** Goneri has quit IRC | 00:53 | |
| openstackgerrit | MaAoyu proposed openstack/tripleo-common master: Don't write the json string in binary mode. https://review.opendev.org/714799 | 01:02 |
| *** cdearborn has quit IRC | 01:05 | |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Replace deprecated gnocchi::storage::coordination_url https://review.opendev.org/714296 | 01:19 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Replace deprecated gnocchi::storage::coordination_url https://review.opendev.org/714296 | 01:20 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Replace deprecated gnocchi::storage::coordination_url https://review.opendev.org/714296 | 01:21 |
| *** hjensas_ has joined #tripleo | 01:26 | |
| *** hjensas has quit IRC | 01:30 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates master: update vbmc_setup yaml for python3 packages https://review.opendev.org/714807 | 01:59 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Replace deprecated gnocchi::storage::coordination_url https://review.opendev.org/714296 | 02:00 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Use CentOS8 to run unit test jobs https://review.opendev.org/714809 | 02:04 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates master: remove py27 tox env for tht https://review.opendev.org/714810 | 02:05 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add unit test job on CentOS8 https://review.opendev.org/714809 | 02:08 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add unit test job on CentOS8 https://review.opendev.org/714809 | 02:09 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add unit test job on CentOS8 https://review.opendev.org/714809 | 02:10 |
| *** jdandrea has quit IRC | 02:16 | |
| *** ianw has quit IRC | 02:30 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 | 02:34 |
| *** jcoufal has joined #tripleo | 02:36 | |
| *** ianw has joined #tripleo | 02:36 | |
| *** ianw has quit IRC | 02:37 | |
| *** hjensas_ has quit IRC | 02:37 | |
| *** ianw has joined #tripleo | 02:40 | |
| openstackgerrit | Alan Bishop proposed openstack/tripleo-heat-templates stable/train: DCN/Edge: Handle ipv6 address for local glance endpoint https://review.opendev.org/714815 | 02:45 |
| *** apetrich has quit IRC | 03:09 | |
| *** ysandeep|away is now known as ysandeep | 03:12 | |
| *** saneax has joined #tripleo | 03:19 | |
| *** cfontain_ has joined #tripleo | 03:20 | |
| *** ekultails has quit IRC | 03:32 | |
| *** psachin has joined #tripleo | 03:33 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: Wait for ironic-neutron-agent bridge_mappings before provide https://review.opendev.org/714823 | 03:36 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Switch to ansible for overcloud node (un)provision https://review.opendev.org/708749 | 03:36 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: overcloud deploy, use ansible for bm provision https://review.opendev.org/712221 | 03:36 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: overcloud node delete, use ansible for unprovision https://review.opendev.org/712222 | 03:36 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Remove unused baremetal provision workflow code https://review.opendev.org/712223 | 03:36 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] Add Fedora 31 support and test jobs https://review.opendev.org/708416 | 03:55 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] Add Fedora 31 support and test jobs https://review.opendev.org/708416 | 03:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] Add Fedora 31 support and test jobs https://review.opendev.org/708416 | 04:02 |
| *** dmsimard|off has joined #tripleo | 04:18 | |
| *** ykarel|away is now known as ykarel | 04:26 | |
| *** udesale has joined #tripleo | 04:51 | |
| *** ratailor has joined #tripleo | 05:00 | |
| *** bdodd has quit IRC | 05:01 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: centos 8 image build: fix mirror https://review.opendev.org/714836 | 05:07 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Use tripleo_common library directly for package_update https://review.opendev.org/706818 | 05:26 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Create *-messages container when setting deployment status https://review.opendev.org/714567 | 05:26 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Remove usage of swifthelper.SwiftTempUrlAction https://review.opendev.org/714640 | 05:26 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Remove CreateContinerAction usage https://review.opendev.org/714838 | 05:26 |
| *** ysandeep is now known as ysandeep| | 05:28 | |
| *** ysandeep| is now known as ysandeep | 05:31 | |
| *** evrardjp has quit IRC | 05:36 | |
| *** evrardjp has joined #tripleo | 05:36 | |
| *** shyamb has joined #tripleo | 05:46 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: centos 8 image build: fix mirror https://review.opendev.org/714836 | 05:55 |
| *** abdysn has joined #tripleo | 06:01 | |
| shyamb | owalsh: ratailor: I will open a BZ for this, we can take it from there | 06:06 |
| *** jmasud has quit IRC | 06:06 | |
| *** xek_ has joined #tripleo | 06:07 | |
| *** jmasud has joined #tripleo | 06:07 | |
| *** marios has joined #tripleo | 06:08 | |
| *** shyam89 has joined #tripleo | 06:18 | |
| *** shyamb has quit IRC | 06:21 | |
| shyam89 | owalsh: ratailor: here is the BZ for my issue: https://bugzilla.redhat.com/show_bug.cgi?id=1816918 | 06:25 |
| openstack | bugzilla.redhat.com bug 1816918 in puppet-tripleo "Error: relabel failed "/var/lib/nova": operation not supported" [Unspecified,New] - Assigned to rhos-maint | 06:25 |
| *** waleedm has joined #tripleo | 06:25 | |
| *** cfontain_ has quit IRC | 06:25 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible master: Reduce tasks for updating plan parameters https://review.opendev.org/714840 | 06:31 |
| *** shyam89 has quit IRC | 06:35 | |
| *** lmiccini has joined #tripleo | 06:43 | |
| *** saneax has quit IRC | 06:46 | |
| *** bandini has joined #tripleo | 06:56 | |
| *** cfontain has joined #tripleo | 06:58 | |
| *** shyamb has joined #tripleo | 06:59 | |
| *** waleedm_ has joined #tripleo | 07:04 | |
| *** waleedm has quit IRC | 07:05 | |
| *** saneax has joined #tripleo | 07:06 | |
| *** jfrancoa has joined #tripleo | 07:06 | |
| *** shyam89 has joined #tripleo | 07:07 | |
| *** shyamb has quit IRC | 07:09 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-heat-templates master: Remove CephAnsibleSkipTags override https://review.opendev.org/714104 | 07:09 |
| *** xek_ has quit IRC | 07:14 | |
| *** xek_ has joined #tripleo | 07:14 | |
| *** bogdando has joined #tripleo | 07:18 | |
| *** shyam89 has quit IRC | 07:27 | |
| *** shyamb has joined #tripleo | 07:27 | |
| *** apetrich has joined #tripleo | 07:27 | |
| ramishra | Looks like some race when doing rsync, seen this failure few times in the last few days https://f2812ca26965a251a9f1-8c8e1caf92c572d2c7c7d35e44c895e7.ssl.cf2.rackcdn.com/714317/4/check/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/3fdf6bf/logs/undercloud/var/log/containers/stdouts/container-puppet-ironic_inspector.log | 07:30 |
| ramishra | we should probably ignore the errors? EmilienM, bogdando ^^^ | 07:31 |
| bogdando | ramishra: it's hard to say what is undionly.kpxe20200325-17-69r029 but looks like something highly ephemeral and temporary | 07:34 |
| *** pojadhav is now known as pojadhav|lunch | 07:34 | |
| ramishra | bogdando: two different containers are probably trying to sync the files | 07:35 |
| ramishra | may be something to do with --delete-after | 07:35 |
| ramishra | I don't know a lot about rsync | 07:35 |
| ramishra | https://f2812ca26965a251a9f1-8c8e1caf92c572d2c7c7d35e44c895e7.ssl.cf2.rackcdn.com/714317/4/check/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/3fdf6bf/logs/undercloud/var/log/containers/stdouts/container-puppet-ironic.log | 07:36 |
| bogdando | hm, that can only be a two ironic containers | 07:36 |
| ramishra | id the other container | 07:36 |
| bogdando | interesting | 07:36 |
| bogdando | ramishra: I'll look into that. flock may help here | 07:37 |
| ramishra | bogdando: thanks! I can create a bug if you want with the details later | 07:38 |
| bogdando | yes please | 07:38 |
| ramishra | bogdando: sure after lunch:) | 07:38 |
| *** shyam89 has joined #tripleo | 07:39 | |
| *** shyamb has quit IRC | 07:42 | |
| *** jschlueter has quit IRC | 07:43 | |
| *** jbadiapa has joined #tripleo | 07:43 | |
| *** jschlueter has joined #tripleo | 07:43 | |
| *** dpawlik has joined #tripleo | 07:47 | |
| *** jschlueter has quit IRC | 07:48 | |
| *** jschlueter has joined #tripleo | 07:50 | |
| *** shyam89 has quit IRC | 07:55 | |
| *** shyamb has joined #tripleo | 07:56 | |
| *** slaweq has joined #tripleo | 07:59 | |
| *** matbu has joined #tripleo | 07:59 | |
| *** abishop_ has quit IRC | 08:04 | |
| *** jcapitao has joined #tripleo | 08:05 | |
| openstackgerrit | Martin Kopec proposed openstack/ansible-role-collect-logs master: WIP: Extend infrared molecule tests https://review.opendev.org/705552 | 08:08 |
| *** chem has joined #tripleo | 08:09 | |
| *** waleedm_ has quit IRC | 08:10 | |
| *** waleedm has joined #tripleo | 08:10 | |
| *** alexmcleod has joined #tripleo | 08:17 | |
| *** tkajinam has quit IRC | 08:19 | |
| *** amoralej|off is now known as amoralej | 08:22 | |
| *** tosky has joined #tripleo | 08:24 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-ansible master: Do not race when truncating the jobs file for sync https://review.opendev.org/714888 | 08:27 |
| *** abishop has joined #tripleo | 08:27 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-ansible master: Do not race when truncating the jobs file for sync https://review.opendev.org/714888 | 08:28 |
| *** dtantsur|afk is now known as dtantsur | 08:29 | |
| ratailor | shyamb, looks like this is also similar issue https://bugzilla.redhat.com/show_bug.cgi?id=1813941 | 08:29 |
| openstack | bugzilla.redhat.com bug 1813941 in openstack-tripleo-heat-templates "overcloud update with NFS volume attached to instance fails" [Unspecified,New] - Assigned to pkopec | 08:29 |
| *** rpittau|afk is now known as rpittau | 08:31 | |
| ramishra | bogdando: https://bugs.launchpad.net/tripleo/+bug/1868934 | 08:33 |
| openstack | Launchpad bug 1868934 in tripleo "Intermittent failures for container-puppet-ironic_inspector with rsync error" [Medium,Triaged] | 08:33 |
| *** pkopec has joined #tripleo | 08:34 | |
| *** shyam89 has joined #tripleo | 08:36 | |
| *** ykarel is now known as ykarel|lunch | 08:39 | |
| *** shyamb has quit IRC | 08:39 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Use tripleo-operator-ansible for standalone https://review.opendev.org/705775 | 08:39 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-ansible master: Do not race with locks for the sync/wrapper script https://review.opendev.org/714888 | 08:41 |
| *** slaweq has quit IRC | 08:41 | |
| *** jpena|off is now known as jpena | 08:54 | |
| *** jmasud has quit IRC | 08:54 | |
| *** jmasud has joined #tripleo | 08:56 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Do not race when applying puppet/rsync in containers https://review.opendev.org/714896 | 09:02 |
| *** jpich has joined #tripleo | 09:03 | |
| *** pojadhav|lunch is now known as pojadhav | 09:04 | |
| *** cylopez has joined #tripleo | 09:04 | |
| *** suuuper has joined #tripleo | 09:04 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Do not race when applying puppet/rsync in containers https://review.opendev.org/714896 | 09:06 |
| *** lucasagomes has joined #tripleo | 09:09 | |
| *** shyam89 has quit IRC | 09:11 | |
| *** shyamb has joined #tripleo | 09:12 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:15 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR: Pacemaker config backup https://review.opendev.org/713897 | 09:16 |
| *** slaweq has joined #tripleo | 09:18 | |
| *** shyamb has quit IRC | 09:18 | |
| openstackgerrit | Flavio Fernandes proposed openstack/puppet-tripleo stable/train: Add Octavia OVN Provider configuration (1 of 2) https://review.opendev.org/705821 | 09:20 |
| *** d0ugal has quit IRC | 09:25 | |
| *** d0ugal has joined #tripleo | 09:25 | |
| openstackgerrit | Flavio Fernandes proposed openstack/puppet-tripleo stable/train: DNM: Please ignore https://review.opendev.org/705821 | 09:27 |
| *** jbadiapa has quit IRC | 09:27 | |
| openstackgerrit | Flavio Fernandes proposed openstack/puppet-tripleo stable/train: DNM: Please ignore https://review.opendev.org/705821 | 09:34 |
| *** dmacpher has joined #tripleo | 09:35 | |
| openstackgerrit | Riccardo Pittau proposed openstack/diskimage-builder master: Use rm instead of rmdir and force it https://review.opendev.org/714903 | 09:35 |
| openstackgerrit | Flavio Fernandes proposed openstack/puppet-tripleo stable/train: Add Octavia OVN Provider configuration (1 of 2) https://review.opendev.org/705821 | 09:37 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Do not race when applying puppet/rsync in containers https://review.opendev.org/714896 | 09:37 |
| *** ykarel|lunch is now known as ykarel | 09:38 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-ci master: Add tempest_tempest_conf_overrides from featureset into jobs vars https://review.opendev.org/714905 | 09:44 |
| marios | chandankumar: sshnaidm: fyi thing we discussed yesterday ^^ (filed a bug for that https://bugs.launchpad.net/tripleo/+bug/1868938 ) | 09:44 |
| openstack | Launchpad bug 1868938 in tripleo "tempest_tempest_conf_overrides ignored in the featureset if set in job featureset_override" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 09:44 |
| marios | chandankumar: sshnaidm: going to post same for rdo-jobs now | 09:45 |
| chandankumar | marios, Ah i forgot about that totally , adding there | 09:45 |
| *** cylopez has quit IRC | 09:45 | |
| marios | chandankumar: ack... that is a new bug not the one we were using yesterday (I referenced that in the description) | 09:45 |
| marios | chandankumar: i already added note for your stuff at https://bugs.launchpad.net/tripleo/+bug/1867807/comments/7 | 09:46 |
| openstack | Launchpad bug 1867807 in tripleo "periodic- centos-8-ovb-3ctlr_1comp-featureset001-baremetal-master fails to download tempest cirros image" [Critical,Fix released] | 09:46 |
| marios | chandankumar: i think we need both things... even if we merge your one, we still want to ensure all the overrides are present and ultimately remove it from the fs so i filed the new bug | 09:47 |
| *** dmacpher has quit IRC | 09:47 | |
| sshnaidm | marios, why not to add image.image_path to featureset itself? What is overridden there? | 09:48 |
| *** dmacpher has joined #tripleo | 09:49 | |
| sshnaidm | marios, if we set option in all featureset_overrides, it seems like a new default | 09:49 |
| *** shyamb has joined #tripleo | 09:49 | |
| marios | sshnaidm: it is in the featureset already https://github.com/openstack/tripleo-quickstart/blob/16beb60b8f6f6dc7233dd3a8e2e839ec48ee8a64/config/general_config/featureset052.yml#L67 | 09:49 |
| marios | sshnaidm: it just gets overridden if you have the same override in the job (not merged overrridden) | 09:49 |
| marios | rrr | 09:49 |
| marios | sshnaidm: for this one i think we just need to remove it from the FS alltogether otherwise like now we think we are using FS values but they are just being ignored | 09:50 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-ci master: Run standalone scenario jobs at tht podman changes https://review.opendev.org/636323 | 09:50 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Do not race when applying puppet/rsync in containers https://review.opendev.org/714896 | 09:51 |
| sshnaidm | oh, it's tempest_tempest_conf_overrides actually.. | 09:51 |
| sshnaidm | omg | 09:51 |
| shyamb | ratailor: Yes, looks similar | 09:51 |
| marios | sshnaidm: yeah ... but luckily its 'only' featureset 52 | 09:51 |
| marios | sshnaidm: (which makes sense... we mainly abused the featurest_override for fs52 cos all the standalones are sharing it) | 09:51 |
| sshnaidm | and is there a better place for image.image_path than tempest_tempest_conf_overrides? <-- chandankumar | 09:52 |
| sshnaidm | maybe it can be just a default for tempest | 09:52 |
| shyamb | ratailor: logged recently | 09:52 |
| marios | sshnaidm: yeah he has that already | 09:52 |
| marios | sshnaidm: there https://review.opendev.org/#/q/topic:tripleocirros+(status:open+OR+status:merged | 09:53 |
| marios | sshnaidm: but as i said we should do that ^^^ but also fix our side, like remove it from the featureset since it is dangerous like that we think it is used but it is not | 09:53 |
| shyamb | ratailor: I need to monitor it | 09:53 |
| sshnaidm | marios, agree | 09:54 |
| shyamb | owalsh: Hi... we found this similar issue https://bugzilla.redhat.com/show_bug.cgi?id=1813941 | 09:54 |
| openstack | bugzilla.redhat.com bug 1813941 in openstack-tripleo-heat-templates "overcloud update with NFS volume attached to instance fails" [Unspecified,New] - Assigned to pkopec | 09:54 |
| shyamb | It's in NEW state | 09:54 |
| sshnaidm | marios, but if we have chandankumar patches merged, we don't need this in featureset_override, right? | 09:54 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: WIP Allow overlap-zone https://review.opendev.org/714908 | 09:54 |
| marios | sshnaidm: right but we need th other one 'dynamic credentials' | 09:54 |
| shyamb | Can we push it? | 09:54 |
| marios | sshnaidm: so adding both for now so we can remove it from the fs | 09:54 |
| marios | sshnaidm: but i can update it before it merges etc depending on progress from chandankumar one | 09:55 |
| sshnaidm | marios, and can be dynamic creadentials as default too..? | 09:55 |
| chandankumar | marios, for dynamic credentials, we can make it permanment in tempestconf | 09:55 |
| marios | sshnaidm: don't know ... i mean that one might not be as 'general' as the image location | 09:55 |
| sshnaidm | marios, chandankumar if we can set in tempestconf, would be great | 09:56 |
| *** Abdallahyas has joined #tripleo | 09:56 | |
| *** abdysn has quit IRC | 09:57 | |
| marios | chandankumar: sshnaidm: ok... so i wasn't sure about that one i thought it was job specific (if we set it on tempest conf, then it get applied to all the things, not just standalone/fs52 ... is that ok? | 09:57 |
| chandankumar | marios, yes it get applied to all jobs | 09:58 |
| marios | chandankumar: ? | 09:58 |
| marios | chandankumar: so is that ok? | 09:58 |
| marios | chandankumar: didn't think that was as general as the image location... | 09:58 |
| chandankumar | marios, in tempestconf it is set to false | 09:59 |
| chandankumar | let me check in tempest what is the default value | 09:59 |
| marios | chandankumar: going to post rdo-jobs change too now we can discuss after... | 10:00 |
| chandankumar | ok | 10:00 |
| marios | chandankumar: i mean once the change is posted and you have time to check | 10:00 |
| *** abdysn has joined #tripleo | 10:00 | |
| *** Abdallahyas has quit IRC | 10:01 | |
| chandankumar | marios, https://opendev.org/openstack/tempest/src/branch/master/tempest/config.py#L59 it is set to true by default | 10:01 |
| chandankumar | marios, I checked the code, https://opendev.org/openstack/python-tempestconf/src/branch/master/config_tempest/main.py#L186 | 10:02 |
| chandankumar | marios, basically it is already true, that is not needed | 10:02 |
| chandankumar | https://opendev.org/openstack/python-tempestconf/src/branch/master/config_tempest/main.py#L198 | 10:03 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart master: Remove tempest_tempest_conf_overrides from featureset 52 https://review.opendev.org/714909 | 10:03 |
| marios | chandankumar: ack ok cool, then ^^ | 10:03 |
| marios | chandankumar: dropping them from the featureset ^^ sshnaidm | 10:03 |
| chandankumar | kopecmartin, ^^ | 10:03 |
| *** florianf has joined #tripleo | 10:03 | |
| sshnaidm | marios, ack | 10:04 |
| marios | chandankumar: can you please go -1 @ https://review.opendev.org/#/c/714905/ and point to your fixes instead | 10:05 |
| marios | chandankumar: and please add some info into the bug | 10:05 |
| ratailor | shyamb, if you are using nfs share then it should not execute https://github.com/shyam-biradar/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover-osp16.yaml#L176 | 10:05 |
| ratailor | shyamb, as per https://github.com/openstack/tripleo-heat-templates/commit/b56c521e01d0a4b42f44f2d9d03f524a4dc60475 | 10:05 |
| marios | chandankumar: i will not post rdo-jobs change then, leaving it to you? (your changes at tempestconf need update) | 10:06 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-ci master: Add Octavia standalone upgrade CI job https://review.opendev.org/710473 | 10:06 |
| shyamb | ratailor: let me check | 10:06 |
| ratailor | shyamb, what do you think ? | 10:06 |
| *** shyam89 has joined #tripleo | 10:08 | |
| shyam89 | hi | 10:08 |
| shyam89 | ratailor: that flag is of openstack | 10:09 |
| shyam89 | nova nfs backend | 10:09 |
| shyam89 | So, if user does not configures NFS as nova backend then it executes the second statment | 10:10 |
| shyam89 | mounting the "/var/lib/nova" in shared,z mode | 10:10 |
| shyam89 | which is happening on my lab | 10:10 |
| shyam89 | what if we do not use z flag? | 10:10 |
| shyam89 | and always mount "/var/lib/nova" in shared mode? | 10:11 |
| marios | chandankumar: https://bugs.launchpad.net/tripleo/+bug/1868938/comments/3 | 10:11 |
| openstack | Launchpad bug 1868938 in tripleo "tempest_tempest_conf_overrides ignored in the featureset if set in job featureset_override" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 10:11 |
| *** shyamb has quit IRC | 10:11 | |
| *** cylopez has joined #tripleo | 10:11 | |
| ratailor | shyam89, it was the case earlier, https://bugs.launchpad.net/tripleo/+bug/1835503 to resolve this issue we had to add z flag when nova nfs is not used. | 10:14 |
| openstack | Launchpad bug 1835503 in tripleo "Deployment with NovaNfsEnabled: True to enabled NFS share for nova ephemeral storage fails when podman is used" [Undecided,Fix released] - Assigned to Martin Schuppert (mschuppert) | 10:14 |
| ratailor | shyam89, but in BZ descrition it is mentioned that is uses NFS share under /var/lib/nova . | 10:15 |
| ratailor | shyam89, you also mentioned that it is working for first time deployment, do you mean that it true for OSP 16 as well ? | 10:16 |
| *** rcernin has quit IRC | 10:17 | |
| *** sshnaidm has quit IRC | 10:18 | |
| ratailor | shyam89, to me it looks like the problem is here if NovaNfsShare is true, https://github.com/shyam-biradar/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover-osp16.yaml#L87-L92 | 10:18 |
| *** sshnaidm has joined #tripleo | 10:19 | |
| shyam89 | ratailor: okay | 10:23 |
| shyam89 | We mount our NFS share | 10:23 |
| ratailor | shyam89, you should modify the condition like below or add the NovaNfsEnabled as role_specific parameter. | 10:23 |
| ratailor | nova_nfs_enabled: | 10:23 |
| ratailor | - equals: [{get_param: NovaNfsEnabled}, true] | 10:23 |
| shyam89 | but that does not sets the flag nova_nfs_backend | 10:24 |
| shyam89 | okay | 10:25 |
| ratailor | you could try modifying the condition as above and try if that works or not. | 10:25 |
| shyam89 | ratailor: Can you please explain more about this change? | 10:26 |
| shyam89 | What we are changing here? | 10:26 |
| shyam89 | I mean, why we are changing this | 10:26 |
| shyam89 | just to understand it | 10:27 |
| ratailor | because we don't define NovaNfsEnabled as role_specific parameter here https://github.com/shyam-biradar/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover-osp16.yaml | 10:27 |
| ratailor | but here we have defined it as role_specific https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/nova/nova-compute-container-puppet.yaml#L93-L98 | 10:28 |
| shyam89 | okay | 10:29 |
| *** holser has joined #tripleo | 10:29 | |
| shyam89 | Let me try that | 10:29 |
| ratailor | to me it looks like that it should not try to use z flag if NFS is enabled, which it is doing and it might be because of the condition which sets nova_nfs_enabled to false instead of true. | 10:30 |
| openstackgerrit | Riccardo Pittau proposed openstack/diskimage-builder master: [WIP] Run hooks with sudo https://review.opendev.org/714913 | 10:33 |
| Madkiss | is the ctlplane n etwork gonna be present on every deployed node automatically? | 10:42 |
| owalsh | ratailor, shyam89: don't think setting NovaNfsEnabled is a solution - that triggers lots of other stuff, e.g tries to mount the share | 10:44 |
| owalsh | ratailor: also need to do something with cinder is NFS backed for BZ1813941... | 10:44 |
| shyam89 | owalsh: okay | 10:45 |
| *** lbragstad has joined #tripleo | 10:45 | |
| owalsh | maybe we can drop the podman/docker selinux relabel and do it ourselves in nova_statedir_ownership.py | 10:46 |
| owalsh | ...skipping any NFS mounts we find | 10:46 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Use global hosts entries to configure /etc/hosts https://review.opendev.org/714611 | 10:46 |
| owalsh | shyam89: BTW ran a quick test with NFS selinux mount options. chcon always returns operation not supported for me | 10:47 |
| owalsh | which makes sense I guess as the NFS mount is faking selinux labels | 10:48 |
| shyam89 | owalsh: okay | 10:48 |
| owalsh | AFAICT it's *possible* to do real selinux labelling with NFS 4.x using xattrs but I don't think we can assume the server supports it | 10:50 |
| owalsh | pkopec: FYI ^ | 10:57 |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates master: Add Octavia OVN Provider configuration https://review.opendev.org/707695 | 10:58 |
| * pkopec looks | 10:58 | |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates stable/train: Add Octavia OVN Provider configuration https://review.opendev.org/714916 | 10:59 |
| owalsh | pkopec: for all of these NFS selinux issue I'm thinking maybe we can do the relabelling in nova_statedir_ownership.py instead, WDYT? | 10:59 |
| pkopec | owalsh, give me few min to check a few things | 11:01 |
| pkopec | owalsh, I was almost sure that we solved this issue in https://review.opendev.org/#/c/669317/ | 11:09 |
| pkopec | in logs i can see | 11:10 |
| pkopec | in logfile {\"TRIPLEO_DEPLOY_IDENTIFIER\": \"1584020954\", \"__OS_DEBUG\": \"false\"}, \"image\": \"tpavcpsat.vici.verizon.com:5000/corona-openstack_containers-nova-compute:16.0-83\", \"net\": \"none\", \"privileged\": false, \"user\": \"root\", \"volumes\": [\"/var/lib/nova:/var/lib/nova:shared,z\", \"/var/lib/container-config-scripts/:/container-config-scripts/:z\"]}', '--conmon-pidfile=/var/run/nova_statedir_owner.pid', '--log-driver', | 11:10 |
| pkopec | 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/nova_statedir_owner.log', '--env=TRIPLEO_DEPLOY_IDENTIFIER=1584020954', '--env=__OS_DEBUG=false', '--net=none', '--privileged=false', '--user=root', '--volume=/var/lib/nova:/var/lib/nova:shared,z' | 11:10 |
| owalsh | pkopec: for BZ1813941 we would need to do the same for CinderEnableNfsBackend | 11:10 |
| pkopec | owalsh, oh right, that's cinder, sorry | 11:10 |
| owalsh | pkopec: and for BZ1816918 neither NFS param is set - it's trilio that is mounting nfs exports in /var/lib/nova | 11:11 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Set tempest_images path for periodic https://review.opendev.org/714609 | 11:13 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Set tempest_tempestconf_image path for periodic https://review.opendev.org/714609 | 11:14 |
| pkopec | owalsh, i think maybe trilio should set the nfs param by default, that would work right? | 11:15 |
| owalsh | pkopec: no, we'll try to mount the share etc... and IIRC glance etc... does different things when nova is NFS backed | 11:16 |
| pkopec | owalsh, i'll need to check that because I don't know atm, sorry | 11:17 |
| *** jcapitao is now known as jcapitao_lunch | 11:18 | |
| pkopec | owalsh, but since you are probably right, what we would need to do for BZ1816918? | 11:18 |
| owalsh | pkopec: think we can manage the relabelling in nova_statedir_ownership.py instead of mounting with the z flag | 11:19 |
| chandankumar | marios, commented on both https://bugs.launchpad.net/tripleo/+bug/1868938 and https://bugs.launchpad.net/tripleo/+bug/1867807/ let me know anything more needed there | 11:21 |
| openstack | Launchpad bug 1868938 in tripleo "tempest_tempest_conf_overrides ignored in the featureset if set in job featureset_override" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 11:21 |
| openstack | Launchpad bug 1867807 in tripleo "periodic- centos-8-ovb-3ctlr_1comp-featureset001-baremetal-master fails to download tempest cirros image" [Critical,Fix released] | 11:21 |
| marios | thanks chandankumar | 11:22 |
| *** tosky is now known as tosky_ | 11:24 | |
| *** yolanda has quit IRC | 11:27 | |
| *** raildo has joined #tripleo | 11:31 | |
| *** yolanda has joined #tripleo | 11:32 | |
| cloudnull | mornings | 11:33 |
| *** sshnaidm has quit IRC | 11:34 | |
| *** sshnaidm has joined #tripleo | 11:34 | |
| *** jcoufal has joined #tripleo | 11:36 | |
| *** jpich has quit IRC | 11:39 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add dynamic concurrency values https://review.opendev.org/714735 | 11:39 |
| *** jpich has joined #tripleo | 11:39 | |
| *** tosky_ is now known as tosky | 11:40 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Dynamically adjust the retires value to fit the deployment https://review.opendev.org/714736 | 11:40 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Normalize how we run validations https://review.opendev.org/714769 | 11:40 |
| *** rpittau is now known as rpittau|bbl | 11:42 | |
| *** gfidente|pto is now known as gfidente | 11:46 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Remove invalid mistral envocation and references https://review.opendev.org/711562 | 11:47 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Remove invalid mistral envocation and references https://review.opendev.org/711562 | 11:48 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Adds centos-8 testing to install-deps https://review.opendev.org/669223 | 11:51 |
| *** jcoufal has quit IRC | 11:52 | |
| *** holser__ has joined #tripleo | 11:52 | |
| *** holser has quit IRC | 11:52 | |
| *** shyam89 has quit IRC | 11:55 | |
| *** rh-jelabarre has joined #tripleo | 11:56 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Use correct novajoin package for centos8 featureset 39 freeipa https://review.opendev.org/714065 | 11:56 |
| *** ekultails has joined #tripleo | 11:58 | |
| *** sshnaidm is now known as sshnaidm|afk | 12:08 | |
| fultonj | cloudnull: would you mind reviewing https://review.opendev.org/#/c/714768 ? | 12:08 |
| cloudnull | fultonj looking now | 12:10 |
| *** waleedm_ has joined #tripleo | 12:13 | |
| cloudnull | fultonj done | 12:14 |
| cloudnull | nicely done | 12:15 |
| fultonj | cloudnull: thanks | 12:15 |
| *** waleedm has quit IRC | 12:15 | |
| fultonj | chandankumar: thanks for working with me on scenario001-multinode-containers.yaml looks like it's working now (after lots of changes) | 12:17 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Fix population of username, public keys https://review.opendev.org/714778 | 12:19 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Wait for ironic-neutron-agent bridge_mappings before provide https://review.opendev.org/714823 | 12:20 |
| *** waleedm_ has quit IRC | 12:23 | |
| *** shyamb has joined #tripleo | 12:23 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:24 | |
| *** waleedm has joined #tripleo | 12:24 | |
| *** rlandy has joined #tripleo | 12:24 | |
| cloudnull | gchamoul - re: https://review.opendev.org/#/c/714769/3/tripleo_ansible/playbooks/cli-baremetal-clean.yaml@41 should I just delete that task ? | 12:25 |
| openstackgerrit | Merged openstack/tripleo-ci master: lint: enable black https://review.opendev.org/703980 | 12:25 |
| gchamoul | cloudnull: yes because I never heard about pre-cleaning validations group | 12:26 |
| cloudnull | ++ | 12:26 |
| cloudnull | gchamoul do the rest of those validation groups look right ? | 12:27 |
| *** jcoufal has joined #tripleo | 12:28 | |
| gchamoul | cloudnull: didn't have a look at everything but all the official groups are referenced in this file https://opendev.org/openstack/tripleo-validations/src/branch/master/groups.yaml | 12:28 |
| cloudnull | I think the only other two were "pre-introspection" and "pre-deployment" | 12:28 |
| gchamoul | cloudnull: both are good! yes | 12:29 |
| cloudnull | cool | 12:29 |
| *** jcapitao_lunch is now known as jcapitao | 12:29 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Normalize how we run validations https://review.opendev.org/714769 | 12:29 |
| *** shyamb has quit IRC | 12:29 | |
| *** jpena is now known as jpena|lunch | 12:29 | |
| cloudnull | gchamoul thanks for the review on that :D | 12:30 |
| *** rfolco has joined #tripleo | 12:30 | |
| *** ansmith_ has quit IRC | 12:31 | |
| *** ansmith has joined #tripleo | 12:31 | |
| *** ratailor has quit IRC | 12:32 | |
| *** udesale_ has joined #tripleo | 12:42 | |
| *** waleedm_ has joined #tripleo | 12:42 | |
| *** udesale has quit IRC | 12:45 | |
| gchamoul | cloudnull: you're welcome | 12:45 |
| *** waleedm has quit IRC | 12:46 | |
| *** ramishra has quit IRC | 12:47 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Less is More - hosts entries https://review.opendev.org/714409 | 12:53 |
| *** jmasud has quit IRC | 12:53 | |
| *** Goneri has joined #tripleo | 12:55 | |
| *** jmasud has joined #tripleo | 12:56 | |
| *** ramishra has joined #tripleo | 12:58 | |
| *** rpittau|bbl is now known as rpittau | 13:00 | |
| ramishra | cloudnull: Hey! I've left some comments on https://review.opendev.org/#/c/714769/, I think that service enabled/hiera ensures that the package is installed unless things have changed now | 13:04 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/train: Do not race when applying puppet/rsync in containers https://review.opendev.org/714937 | 13:05 |
| *** holser__ has quit IRC | 13:06 | |
| bogdando | ramishra, EmilienM: I'm not sure how https://bugs.launchpad.net/tripleo/+bug/1868934 happens in master | 13:09 |
| openstack | Launchpad bug 1868934 in tripleo "Intermittent failures for container-puppet-ironic_inspector with rsync error" [High,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 13:09 |
| bogdando | given that the new code should make races not possible | 13:09 |
| bogdando | https://github.com/openstack/tripleo-ansible/blob/a70139e5b985a25a2a82edfd8df1d93e54c2f872/tripleo_ansible/ansible_plugins/modules/container_puppet_config.py#L182-L185 | 13:09 |
| bogdando | also not sure what to do with the master in tht and its container-puppet.py, shall we removed it maybe? | 13:10 |
| EmilienM | don't trust the comments, only the code ;-) | 13:11 |
| bogdando | EmilienM: could you please take over that bug for the master? I don't know that new code | 13:11 |
| EmilienM | container-puppet.py will be removed once we remove docker support completely | 13:11 |
| bogdando | not sure where config_volume now lives and how to add a bind mount for locks :) | 13:12 |
| EmilienM | bogdando: sure, I'll take a look this week. I'm debugging another thing this morning but I'll look at it next | 13:12 |
| *** amoralej is now known as amoralej|lunch | 13:12 | |
| bogdando | EmilienM: will we do so as well for Train? | 13:12 |
| bogdando | or backporting my patch for T is ok | 13:13 |
| ramishra | EmilienM: lol, I would add don't trust code untill tested and works:) | 13:14 |
| cloudnull | thanks ramishra | 13:14 |
| EmilienM | bogdando: only when paunch is disabled | 13:16 |
| EmilienM | ramishra: true; races are hard to test though | 13:16 |
| EmilienM | I'll take a look at the bug a bit later; however the new module isn't that different from the container-puppet.py script | 13:17 |
| bogdando | well it looks different | 13:17 |
| EmilienM | it's basically running the same things (better of course) on the host | 13:17 |
| bogdando | as I said was not easy to figure out where config_volume now lives and how to add a bind mount for locks | 13:17 |
| bogdando | :) | 13:17 |
| EmilienM | bogdando: https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/container-puppet.py#L459 | 13:17 |
| EmilienM | the comment that you pointed out comes from here ^ | 13:18 |
| bogdando | so any nova_* uses nova? | 13:18 |
| cloudnull | anyone around mind giving - https://review.opendev.org/#/c/714598/ - a look | 13:18 |
| EmilienM | I think the ansible module is safer as it merges the data in the config volume | 13:18 |
| bogdando | config container | 13:19 |
| EmilienM | but again, I can't look deeply now, I'll look a bit later | 13:19 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/train: [Train only] Do not race when applying puppet/rsync in containers https://review.opendev.org/714937 | 13:21 |
| openstackgerrit | Gregory Thiemonge proposed openstack/tripleo-ansible master: Use native modules for octavia certs creation https://review.opendev.org/714396 | 13:22 |
| openstackgerrit | Gregory Thiemonge proposed openstack/tripleo-ansible master: Scan existing controllers on update to pick up existing CA/private key pairs https://review.opendev.org/692599 | 13:22 |
| ramishra | cloudnull: I'm not convinced with the assumption that the packages would be installed somehow without that service enabled:) | 13:23 |
| cloudnull | I still think its better to fail than to silently skip, maybe we check both and add a fail task ? | 13:24 |
| *** jbadiapa has joined #tripleo | 13:25 | |
| mwhahaha | which service? | 13:26 |
| ramishra | cloudnull: this was checked earlier and IMO we can't just get rid of that.. If we plan that then we should first get rid of that service. I really don't know who and how the packages would be instale dthen | 13:26 |
| ramishra | mwhahaha: https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/validations/tripleo-validations-baremetal-ansible.yaml#L39-L44 | 13:26 |
| cloudnull | mwhahaha talking about - https://review.opendev.org/#/c/714769/4/tripleo_ansible/playbooks/cli-deploy-deployment-plan.yaml | 13:27 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/stein: Do not race when applying puppet/rsync in containers https://review.opendev.org/714940 | 13:28 |
| *** ysandeep is now known as ysandeep|away | 13:31 | |
| *** jpena|lunch is now known as jpena | 13:32 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Less is More - hosts entries https://review.opendev.org/714409 | 13:32 |
| mwhahaha | tripleo_validations_enabled is set by the undercloud install, no? | 13:33 |
| ramishra | mwhahaha: yes | 13:33 |
| mwhahaha | if tripleo-validations isn't a dep of like the tripleoclient, it likely won't already be installed. this might also just be legacy crap from before we integrated validations more | 13:34 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Fix build-images for centos8 jobs set tripleoclient and image-yaml https://review.opendev.org/714627 | 13:35 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/queens: Do not race when applying puppet/rsync in containers https://review.opendev.org/714942 | 13:37 |
| ramishra | mwhahaha: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_1d8/713773/2/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/1d83fea/logs/undercloud/etc/puppet/hieradata/all_nodes.json | 13:38 |
| cloudnull | ramishra mwhahaha https://github.com/rdo-packages/tripleoclient-distgit/blob/rpm-master/python-tripleoclient.spec#L125 it looks like validations is a dep ? | 13:38 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/stein: Do not race when applying puppet/rsync in containers https://review.opendev.org/714940 | 13:38 |
| ramishra | vs | 13:38 |
| ramishra | https://82aeb71d4c2ef828072a-87e6ec00760ed1d0ee3af698b2b27828.ssl.cf5.rackcdn.com/713773/2/check/tripleo-ci-centos-8-containers-multinode/e2ca8c9/logs/undercloud/etc/puppet/hieradata/all_nodes.json | 13:38 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/queens: Do not race when applying puppet/rsync in containers https://review.opendev.org/714942 | 13:40 |
| *** holser has joined #tripleo | 13:44 | |
| cloudnull | mwhahaha ramishra - https://github.com/rdo-packages/tripleoclient-distgit/blob/rpm-master/python-tripleoclient.spec#L124-L125 given the package `openstack-tripleo-validations` is a requirement we shouldn't need to run that heira command, right ? | 13:44 |
| ramishra | cloudnull: yeah, it's probably all legacy crap then, not relevant to undercloud | 13:44 |
| mwhahaha | cloudnull: we would need the command because you can disable validations via udnercloud.conf | 13:45 |
| mwhahaha | i think that's what sets that hieradata | 13:45 |
| * mwhahaha would have to go back in history | 13:45 | |
| cloudnull | so should we run the hiera command before running any of those validations within the cli-* playbooks ? | 13:46 |
| mwhahaha | likely | 13:46 |
| mwhahaha | but i'd have to check the history on that | 13:47 |
| mwhahaha | of course we might have lost this specific key in the transition to non-instack-undercloud | 13:48 |
| * mwhahaha goes looking | 13:48 | |
| *** TrevorV has joined #tripleo | 13:49 | |
| mwhahaha | so we rely on that in extraconfig/post_Deploy/undercloud_post.py | 13:49 |
| mwhahaha | if it's disabled, we skip setting up the ssh environment for mistral | 13:50 |
| mwhahaha | if we no longer require that with the conversion to no mistral we can drop it | 13:50 |
| mwhahaha | that's why it was in the cli | 13:50 |
| mwhahaha | because if it wasn't installed, the validations would fail | 13:50 |
| mwhahaha | https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/post_deploy/undercloud_post.py#L186-L189 | 13:51 |
| cloudnull | so... we can drop it? | 13:54 |
| * cloudnull hides behind a table | 13:54 | |
| *** holser has quit IRC | 13:55 | |
| * mwhahaha shrugs | 13:55 | |
| *** dwilde has joined #tripleo | 13:55 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Normalize how we run validations https://review.opendev.org/714769 | 13:58 |
| *** jdandrea has joined #tripleo | 13:59 | |
| *** jdandrea has quit IRC | 13:59 | |
| *** jdandrea has joined #tripleo | 14:00 | |
| cloudnull | ramishra mwhahaha https://review.opendev.org/#/c/714769/5/tripleo_ansible/playbooks/cli-baremetal-introspect.yaml - thoughts ? -cc Tengu gchamoul | 14:00 |
| *** amoralej|lunch is now known as amoralej | 14:00 | |
| *** holser has joined #tripleo | 14:01 | |
| Tengu | cloudnull: so right now the validator subcommand doesn't support the --os-cloud thing iirc | 14:03 |
| Tengu | cloudnull: we're currently working as fast as possible in order to get the packaging done, and this kind of issues should be solved with this kind of change.... | 14:04 |
| cloudnull | it does not , i kept it in the command because all other openstack commands use it but we're sourcing the stackrc file for now. | 14:04 |
| Tengu | oh, I didn't see it. ok | 14:05 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Update image URL links for CentOS 8 https://review.opendev.org/714790 | 14:05 |
| *** holser has quit IRC | 14:05 | |
| *** holser has joined #tripleo | 14:05 | |
| Tengu | cloudnull: so on the other parts - this sounds like a good thing. Also ensuring that config matches the options, I like it | 14:06 |
| cloudnull | ++ | 14:06 |
| Tengu | cloudnull: aaand apparently there are some pre-introspection thngs that might be uneeded - this is another story. | 14:07 |
| *** pojadhav is now known as pojadhav|away | 14:08 | |
| Tengu | cloudnull: guess having the operator would make that thing even easier right? | 14:08 |
| openstackgerrit | Luke Short proposed openstack/tripleo-docs master: Add good use cases for manually running config-download playbooks. https://review.opendev.org/714196 | 14:10 |
| *** aedc has joined #tripleo | 14:13 | |
| *** aedc has quit IRC | 14:13 | |
| sshnaidm | cloudnull, sorry, my webcam and headset had a fight for a mic, and nobody won | 14:13 |
| cloudnull | Tengu yeah. if we could use tripleo-ansible-operator for some of those commands that would be great. | 14:13 |
| Tengu | cloudnull: ok. maybe I can give a try, if I have some help in order to avid doing bad things ^^'. | 14:14 |
| * cloudnull doesn't know if we could do that quite yet ? | 14:14 | |
| cloudnull | sshnaidm no worries. | 14:14 |
| Tengu | cloudnull: for now, we (the squad) have no real time to work on the operator for the validations :( | 14:14 |
| Tengu | with the deadline and all for 16.1 | 14:14 |
| openstackgerrit | Zhuo Zhen proposed openstack/tripleo-image-elements master: Install python3-dev package https://review.opendev.org/714952 | 14:15 |
| cloudnull | Tengu ++ its something we can iterate on later. | 14:15 |
| Tengu | but maybe I can get some time whil gchamoul and matbu are working hard on the python libs and tripleoclient changes | 14:15 |
| Tengu | cloudnull: maybe providing a quick wrapper would already be nice? | 14:15 |
| cloudnull | that would be great too. its just two commands for the cli-* playbooks, so not a massive priority, nothing gchamoul or matbu should breakaway for. | 14:16 |
| Tengu | cloudnull: cool. I'll check if I can slip that in my tasks | 14:17 |
| cloudnull | ++ | 14:17 |
| Tengu | cloudnull: stupid question.... is there some kind of helper in order to create the basic tree for a new operator? I don't see any in the / | 14:20 |
| Tengu | and doc doesn't help XD | 14:21 |
| *** bdodd has joined #tripleo | 14:24 | |
| *** psachin has quit IRC | 14:27 | |
| slaweq | ykarel: hi | 14:33 |
| slaweq | ykarel: yesterday my issue with standalone was indeed due to hiera installed from epel | 14:33 |
| ykarel | slaweq, hi | 14:33 |
| slaweq | ykarel: but now it's failing for me on different stage, error is like http://paste.openstack.org/show/791132/ | 14:33 |
| ykarel | lookin | 14:33 |
| slaweq | do You maybe know what can be wrong with this? | 14:34 |
| *** hjensas has joined #tripleo | 14:35 | |
| ykarel | slaweq, looks like u don't have https://review.opendev.org/#/c/714234/ | 14:37 |
| ykarel | ^^ should help in figurring out what containers failed | 14:38 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Migrate molecule jobs from centos 7 to 8 https://review.opendev.org/712402 | 14:42 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Convert all of the fedora28 container usage to CentOS8 https://review.opendev.org/712509 | 14:42 |
| rlandy | cloudnull: hi - are you working on the cli-baremetal-introspect playbook? | 14:42 |
| slaweq | ykarel: I have this patch already in /usr/share/ansible/ on my vm | 14:44 |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates stable/train: Allow disabling the octavia provider https://review.opendev.org/713978 | 14:45 |
| Tengu | hjensas: heya! I just restaured https://review.opendev.org/714041 - feel free to take over if needed. | 14:46 |
| Tengu | *restored | 14:46 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-ansible master: Rework how we push the content in the file https://review.opendev.org/714041 | 14:47 |
| ykarel | slaweq, sorry i shared wrong patch, i checked again and i see that task has failed_when: false | 14:48 |
| ykarel | slaweq, can u share more logs | 14:48 |
| hjensas | Tengu: if the Extra entry is fixed with that, it's ok with me. One nit, is that I would like to have empty lines beetween entries for each node. | 14:49 |
| Tengu | hjensas: ah, between blocks? | 14:50 |
| hjensas | Tengu: yes, it makes it easier to read for the human eye. Totally a nit. :) | 14:51 |
| Tengu | hjensas: hmmm... not really easy in fact, unless I split the for in three.... But that would probably be less efficient? | 14:52 |
| Tengu | or are you talking about the "tripleo_hosts_entries_block" itself? | 14:52 |
| hjensas | Tengu: drop the - on L41 ? | 14:53 |
| Tengu | that was the one then :). either 41 or 45 in fact. | 14:53 |
| cloudnull | rlandy kinda, what's up? | 14:53 |
| hjensas | Tengu: or L34 - I'm not sure. | 14:54 |
| rlandy | cloudnull: sshnaidm is fixing the issue - thanks | 14:54 |
| Tengu | hjensas: I wasn't that happy with all those blank lines in fact - that's why I added the "-" everywhere | 14:54 |
| Tengu | hjensas: I'd rather push a #comment in fact. | 14:54 |
| cloudnull | Tengu idk of a helper in tripleo-operator-ansible, maybe mwhahaha has one ? | 14:55 |
| Tengu | that would even be more helpful... ? | 14:55 |
| mwhahaha | que? | 14:55 |
| Tengu | cloudnull: discussed with mwhahaha already - rsync is the helper, mostly :). Makes sense, since we won't add that many new commands once we're up-to-date with the current CLI | 14:55 |
| hjensas | Tengu: yeah, maby '\n# {{ host }}' | 14:55 |
| *** udesale_ has quit IRC | 14:57 | |
| Tengu | hjensas: and something with "network" as well I guess? That would help finding issues, even better than "host" | 14:58 |
| Tengu | lemme try to get something and push it back. | 14:58 |
| slaweq | ykarel: but do You need more logs from stdout? or from some files (where should I look exactly?) | 14:58 |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates stable/train: Add Octavia OVN Provider configuration https://review.opendev.org/714916 | 14:59 |
| hjensas | Tengu: we used to have something like this: http://paste.openstack.org/show/791137/ | 14:59 |
| Tengu | hjensas: hmm, there were a lot of other blank lines lately | 15:00 |
| ykarel | slaweq, stdout as first place to see | 15:00 |
| ykarel | as per tasks it should display failed containers | 15:00 |
| Tengu | but seing that..... droping the "-" line 34 or 49 should be enough to get it, imho. | 15:01 |
| hjensas | Tengu: ok, I fully agree that more than that is bad. :) | 15:01 |
| ykarel | if not we can use podman commands | 15:01 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-ansible master: Rework how we push the content in the file https://review.opendev.org/714041 | 15:01 |
| Tengu | hjensas: let's see how it outputs its content there -^^ | 15:01 |
| hjensas | Tengu: thanks. | 15:02 |
| slaweq | ykarel: ok, give me few minutes :) | 15:03 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Disable fact gathering for create_admin https://review.opendev.org/714966 | 15:05 |
| *** beagles is now known as beagles_mtg | 15:06 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Disable fact gathering for create_admin https://review.opendev.org/714966 | 15:08 |
| *** abdysn has quit IRC | 15:09 | |
| hjensas | slagle: is there an easy way to get the undercloud install to run ansible with the '--step' option? | 15:11 |
| mwhahaha | no | 15:11 |
| mwhahaha | you could extract the playbooks after the fact and run it manually though | 15:11 |
| mwhahaha | but it's not a native thing | 15:12 |
| hjensas | mwhahaha: so I can get it to do "stack-only" then config-download and run it manually? | 15:12 |
| mwhahaha | no but if you ctl+c the install at execution time, you can do it | 15:12 |
| mwhahaha | the undercloud install command doesn't support extra fance bits, but you can hax it | 15:13 |
| slaweq | ykarel: I need to try it again to get more logs, I will probably get back to You tomorrow :) | 15:13 |
| slaweq | ykarel: thx a lot for all help | 15:13 |
| mwhahaha | hjensas: that being said, why are you trying to step the install? | 15:13 |
| slagle | hjensas: i think you can use --dry-run. then run the shown tripleo deploy command with --output-only | 15:14 |
| hjensas | mwhahaha: the inspection issue on RHEL 8.2. I can't reproduce it on anything where undercloud install did'nt run first. I tried pulling in bit's like the sysctl settings, firewall, ovs bridges+ports+flows etc. | 15:14 |
| slagle | then run ansible-playbook | 15:15 |
| mwhahaha | if you set cleanup to false (or whatever the option is) it won't tar up the playbooks. you can just let the install run and ctl+c at the start of the ansible run | 15:16 |
| hjensas | mwhahaha: basically something get's jammed and any process listening on udp:67 does not see any traffic. I even tried stopping all tripleo_ services, killed neutron dhcp container, deleted ovs bridges and set ip on the physical interface. i.e I want to step trough, to figure out when it breaks. | 15:16 |
| mwhahaha | fun | 15:17 |
| hjensas | mwhahaha: yeah .. in comic books I'd be saying ''')(&(¤%/(#¤&/#%"¤#&''' atm. :) | 15:17 |
| mwhahaha | yea when i was troubleshooting it, i saw the traffic in the container on the interface but dnsmasq didn't pick it up | 15:17 |
| *** jcoufal has quit IRC | 15:18 | |
| hjensas | thanks slagle, i'll look into that. | 15:19 |
| ykarel | slaweq, ack, and i hope u have disabled epel completely, as if ansible also used from it u can see other issues | 15:21 |
| slaweq | yes, I disabled it completly | 15:21 |
| ykarel | slaweq, ack | 15:21 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Add rabbitmq_erlang_cookie tag for rabbitmq config https://review.opendev.org/714969 | 15:22 |
| ykarel | slaweq, u destroyed the environment? | 15:22 |
| ykarel | if not can u check ansible version | 15:22 |
| slaweq | ykarel: http://paste.openstack.org/show/791141/ | 15:22 |
| ykarel | slaweq, ack version is good | 15:23 |
| ykarel | ok collect more logs, that would help | 15:23 |
| Tengu | hjensas: may it be related to network namespace being nuked at some point? | 15:23 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Add rabbitmq_erlang_cookie tag for rabbitmq config https://review.opendev.org/714969 | 15:23 |
| Tengu | hjensas: I recently learnt that namespace can get nuked whild relevant container are still running in it.... I'm wondering if this doesn't create some issues... | 15:24 |
| Tengu | *while | 15:24 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Add rabbitmq_erlang_cookie tag for rabbitmq config https://review.opendev.org/714969 | 15:24 |
| *** ykarel is now known as ykarel|away | 15:25 | |
| *** bogdando has quit IRC | 15:29 | |
| *** pkopec has quit IRC | 15:30 | |
| *** jcoufal has joined #tripleo | 15:30 | |
| *** arxcruz|rover is now known as arxcruz | 15:30 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Refactorize test_tht_ansible_syntax to welcome new roles/modules https://review.opendev.org/713669 | 15:37 |
| *** dmacpher has quit IRC | 15:38 | |
| *** dmacpher has joined #tripleo | 15:39 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Refactorize test_tht_ansible_syntax to welcome new roles/modules https://review.opendev.org/713669 | 15:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 | 15:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Deprecate Keepalived service https://review.opendev.org/657067 | 15:39 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Convert all of the fedora28 container usage to CentOS8 https://review.opendev.org/712509 | 15:44 |
| *** pojadhav|away is now known as pojadhav | 15:53 | |
| *** pkopec has joined #tripleo | 15:54 | |
| slaweq | ykarel|away: https://transfer.sh/MfsMo/tripleo_standalone.log <-- here is full log from my attempt to deploy standalone on Centos 8, please check if You will have few minutes :) | 16:05 |
| *** florianf has quit IRC | 16:07 | |
| *** jcapitao is now known as jcapitao_afk | 16:14 | |
| *** waleedm_ has quit IRC | 16:14 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/tripleo-heat-templates master: Update octavia playbooks parameters https://review.opendev.org/714982 | 16:14 |
| openstackgerrit | Gregory Thiemonge proposed openstack/tripleo-ansible master: Scan existing controllers on update to pick up existing certs/private keys https://review.opendev.org/692599 | 16:16 |
| *** beagles_mtg is now known as beagles | 16:21 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Improve deployer UX by limiting failure output potential https://review.opendev.org/714985 | 16:26 |
| *** weshay|ruck changes topic to "CI status: GREENISH | community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs" | 16:27 | |
| weshay|ruck | anyone see 500 ise errors tripleo-common image prepare on the auth | 16:28 |
| weshay|ruck | https://bugs.launchpad.net/tripleo/+bug/1869040 | 16:28 |
| openstack | Launchpad bug 1869040 in tripleo "requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://auth.docker.io/token?s" [Critical,Triaged] | 16:28 |
| weshay|ruck | could just be a hiccup.. but wanted to check | 16:28 |
| rlandy | bfournie: hi - if introspection hangs after the first node boot, what response is it waiting for? | 16:34 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Add new rhsm_overcloud_env parameter to tripleo-upgrade. https://review.opendev.org/714989 | 16:37 |
| *** jcoufal has quit IRC | 16:37 | |
| *** jbadiapa has quit IRC | 16:40 | |
| openstackgerrit | Damien Ciabrini proposed openstack/puppet-tripleo master: Support for mariadb's ed25519 authentication https://review.opendev.org/712769 | 16:46 |
| *** jcapitao_afk is now known as jcapitao | 16:50 | |
| *** marios is now known as marios|out | 16:53 | |
| *** jmasud has quit IRC | 16:53 | |
| *** jcoufal has joined #tripleo | 16:56 | |
| *** jmasud has joined #tripleo | 16:56 | |
| *** rfolco is now known as rfolco|bbl | 16:58 | |
| *** marios|out has quit IRC | 16:58 | |
| bfournie | rlandy: is this with 16.1? that is the issue we are debugging with https://bugzilla.redhat.com/show_bug.cgi?id=1814616. The inspector dnsmasq is not receiving dhcp packets from the node | 17:01 |
| openstack | bugzilla.redhat.com bug 1814616 in rhosp-openvswitch "ironic inspector dnsmasq is not receiving DHCP requests" [High,New] - Assigned to rhos-maint | 17:01 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Use tripleo_common library directly for package_update https://review.opendev.org/706818 | 17:02 |
| rlandy | bfournie: I am running RDO master on centos 8 | 17:02 |
| rlandy | checking bug | 17:03 |
| *** dtantsur is now known as dtantsur|afk | 17:06 | |
| rlandy | bfournie: OVB is working but real baremetal is not | 17:06 |
| *** lucasagomes has quit IRC | 17:07 | |
| rlandy | bfournie: I have the environment up if there is any info to check | 17:09 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Ensure that the playbooks and roles can display debug data when enabled https://review.opendev.org/715006 | 17:11 |
| *** weshay|ruck changes topic to "CI status: Upstream GREENISH, 3rd party Registry is migrating to vexxhost | community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs" | 17:11 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Ensure that the playbooks and roles can display debug data when enabled https://review.opendev.org/715006 | 17:13 |
| *** rpittau is now known as rpittau|afk | 17:13 | |
| *** derekh has joined #tripleo | 17:14 | |
| *** jcoufal has quit IRC | 17:16 | |
| bfournie | rlandy: sure I can check but just in meeting right now | 17:18 |
| rlandy | k | 17:18 |
| *** pkopec has quit IRC | 17:25 | |
| *** ed_b has joined #tripleo | 17:30 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Add new rhsm_overcloud_env parameter to tripleo-upgrade. https://review.opendev.org/714989 | 17:31 |
| *** jcoufal has joined #tripleo | 17:32 | |
| *** evrardjp has quit IRC | 17:36 | |
| *** evrardjp has joined #tripleo | 17:36 | |
| openstackgerrit | Merged openstack/ansible-role-redhat-subscription master: Improve support for HTTP proxy https://review.opendev.org/714524 | 17:39 |
| *** dpaterson has joined #tripleo | 17:43 | |
| *** jpich has quit IRC | 17:44 | |
| *** waleedm_ has joined #tripleo | 17:44 | |
| *** dmacpher has quit IRC | 17:44 | |
| *** dmacpher has joined #tripleo | 17:46 | |
| *** dmacpher has quit IRC | 17:46 | |
| *** dmacpher has joined #tripleo | 17:46 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Tolerate NFS exports in /var/lib/nova when selinux relabelling https://review.opendev.org/715015 | 17:57 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-common master: Fix placement dependencies to work with mysql's ed25519 https://review.opendev.org/715016 | 17:58 |
| *** jpena is now known as jpena|off | 18:00 | |
| *** lmiccini has quit IRC | 18:07 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Tolerate NFS exports in /var/lib/nova when selinux relabelling https://review.opendev.org/715015 | 18:08 |
| bfournie | hi rlandy: I can check out the logs if you have link | 18:13 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Tolerate NFS exports in /var/lib/nova when selinux relabelling https://review.opendev.org/715015 | 18:14 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/tripleo-ansible stable/train: Force container deletion if namespace does not exist in service_kill https://review.opendev.org/715019 | 18:15 |
| *** waleedm__ has joined #tripleo | 18:22 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Configure main registry as trunk.registry.rdoproject.org https://review.opendev.org/715021 | 18:24 |
| sshnaidm | weshay|ruck, ^^ | 18:24 |
| *** waleedm_ has quit IRC | 18:25 | |
| weshay|ruck | sshnaidm++ | 18:26 |
| *** bandini has quit IRC | 18:30 | |
| *** alexmcleod has quit IRC | 18:34 | |
| *** irclogbot_1 has quit IRC | 18:37 | |
| *** jcoufal has quit IRC | 18:37 | |
| *** rfolco|bbl is now known as rfolco | 18:39 | |
| *** jcoufal has joined #tripleo | 18:42 | |
| *** jcapitao has quit IRC | 18:44 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Create *-messages container when setting deployment status https://review.opendev.org/714567 | 18:46 |
| *** rh-jelabarre has quit IRC | 18:52 | |
| *** rh-jelabarre has joined #tripleo | 18:52 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-upgrade master: FFWD2 Overcloud upgrade automation. https://review.opendev.org/713528 | 18:54 |
| cloudnull | anyone mind giving https://review.opendev.org/#/c/714640 and https://review.opendev.org/#/c/714838 a look | 18:55 |
| *** lbragstad has quit IRC | 18:57 | |
| cloudnull | this series could do with a couple reviews as well - https://review.opendev.org/#/c/714769 | 18:58 |
| *** amoralej is now known as amoralej|off | 18:59 | |
| rlandy | bfournie: hi - here is the link to the undercloud logs ... https://sf.hosted.upshift.rdu2.redhat.com/logs/40/195040/6/check/periodic-tripleo-ci-centos-8-bm_envA-3ctlr_1comp-featureset001-master/9f95481/logs/undercloud/ | 19:00 |
| rlandy | bfournie: what's interesting is that OVB passes introspection | 19:01 |
| rlandy | with the same versions of ironic and OVS | 19:01 |
| bfournie | rlandy: thanks, I am in another meeting but will look right after | 19:01 |
| *** irclogbot_1 has joined #tripleo | 19:03 | |
| Madkiss | If I want the control plane network interface on my nodes to be a bond interface, can it be an ovs-bridge or does it have to be a linux bridge? | 19:06 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Refactorize test_tht_ansible_syntax to welcome new roles/modules https://review.opendev.org/713669 | 19:20 |
| *** waleedm__ has quit IRC | 19:23 | |
| *** Trevor_V has joined #tripleo | 19:28 | |
| *** TrevorV has quit IRC | 19:32 | |
| *** irclogbot_1 has quit IRC | 19:37 | |
| mwhahaha | weshay|ruck: btw there was https://review.opendev.org/#/c/518374/ to get dstat as a top level zuul role to replace our hacks in quickstart | 19:37 |
| beagles | is there an example of how to load non-default configuration files or set a config-dir for a wsgi based service somewhere? | 19:37 |
| mwhahaha | i fixed it up but we might look to switch to it | 19:37 |
| mwhahaha | beagles: no it picks up the default file i believe | 19:38 |
| mwhahaha | beagles: it uses the oslo.config defaults | 19:38 |
| beagles | interestin | 19:38 |
| mwhahaha | we've hit this downstream when we forget to patch the dist/ path in | 19:38 |
| mwhahaha | breaks stuff every release | 19:38 |
| mwhahaha | always fun | 19:38 |
| beagles | :) | 19:39 |
| weshay|ruck | mwhahaha, that great.. I can't force the issue and still have push back on the dstat, atop thing | 19:40 |
| * weshay|ruck gets folks on the review you pointed out | 19:40 | |
| mwhahaha | weshay|ruck: sure just fyi, it poped back up on my radar | 19:40 |
| weshay|ruck | thanks for pointing it out | 19:40 |
| *** irclogbot_1 has joined #tripleo | 19:41 | |
| mwhahaha | beagles: https://github.com/rdo-packages/oslo-config-distgit/blob/rpm-master/0001-add-usr-share-project-dist.conf-to-the-default-confi.patch | 19:42 |
| *** irclogbot_1 has quit IRC | 19:42 | |
| beagles | mwhahaha: thanks! I was just looking for it :) | 19:42 |
| mwhahaha | breaks heat-api and a few other services if it's missing | 19:43 |
| mwhahaha | but yea, config file locations are special | 19:43 |
| *** hjensas has quit IRC | 19:46 | |
| *** irclogbot_2 has joined #tripleo | 19:46 | |
| *** hjensas has joined #tripleo | 19:50 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo master: rabbitmq_bundle: remove file resources which manage the cookie https://review.opendev.org/715047 | 19:52 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 | 19:52 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 | 19:52 |
| *** jcoufal has quit IRC | 19:53 | |
| *** irclogbot_2 has quit IRC | 20:00 | |
| bfournie | rlandy: that looks like the same problem we are seeing with osp-16.1 CI - the inspector dnsmasq isn't receiving any dhcp packets as shown in https://sf.hosted.upshift.rdu2.redhat.com/logs/40/195040/6/check/periodic-tripleo-ci-centos-8-bm_envA-3ctlr_1comp-featureset001-master/9f95481/logs/undercloud/var/log/containers/ironic-inspector/dnsmasq.log. We found the root cause and should have a fix shortly - https://bugzilla.redha | 20:03 |
| bfournie | ug.cgi?id=1814616 | 20:03 |
| rlandy | bfournie: do you know why OVB would not hit this? | 20:04 |
| *** irclogbot_0 has joined #tripleo | 20:05 | |
| EmilienM | weshay|ruck: nice joke | 20:06 |
| EmilienM | and pretty accurate (re: lunch) | 20:06 |
| weshay|ruck | :) | 20:08 |
| weshay|ruck | it was an easy target | 20:08 |
| *** irclogbot_0 has quit IRC | 20:12 | |
| bfournie | rlandy: well this is due to nffiltering of all udp packets, i'm not sure why ovb would be different, do you have a url? I can look at the dnsmasq.log | 20:13 |
| rlandy | bfournie: http://paste.openstack.org/show/791161/ is from the undercloud where we do see the hang | 20:14 |
| rlandy | doesn't seem to have the include from the bug | 20:15 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: Wait for ironic-neutron-agent bridge_mappings before provide https://review.opendev.org/714823 | 20:15 |
| rlandy | bfournie: either way, I am happy to try this fix | 20:15 |
| rlandy | if it goes upstream, I can depends-on | 20:16 |
| bfournie | rlandy: yeah you're right, not same nftables as problem, but dnsmasq log has same symptom | 20:17 |
| rlandy | bfournie: I'll try the fix - maybe it's related | 20:17 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/train: WIP [train-only] purge /etc/nftables/* https://review.opendev.org/715056 | 20:18 |
| *** irclogbot_2 has joined #tripleo | 20:22 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/train: WIP [train-only] purge /etc/nftables/* https://review.opendev.org/715056 | 20:39 |
| *** tosky has quit IRC | 20:40 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: udpate skip list for master, security_groups basic_ops https://review.opendev.org/715060 | 20:40 |
| rlandy | bfournie: I found something ... | 20:41 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/40/195040/6/check/periodic-tripleo-ci-centos-8-bm_envA-3ctlr_1comp-featureset001-master/9f95481/logs/undercloud/var/log/extra/podman/containers/ironic_inspector_dnsmasq/log/anaconda/ifcfg.log | 20:41 |
| rlandy | the ens3 interface does not exist | 20:42 |
| rlandy | we changed those for consistent naming ... | 20:42 |
| rlandy | as in https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-disabling_consistent_network_device_naming | 20:43 |
| mwhahaha | ens3 isn't that the bad infterface in the qcow images | 20:43 |
| mwhahaha | we fixed that in overcloud-full | 20:43 |
| mwhahaha | is it sneaking into however you're creating the undercloud? | 20:44 |
| mwhahaha | you need to clear it if the undercloud is launched from a centos8 image | 20:44 |
| rlandy | I don't see it in /etc/sysconfig/network-scripts | 20:44 |
| mwhahaha | might be in the container? | 20:44 |
| rlandy | and the same log shows up in ovb | 20:45 |
| rlandy | but that passes introspection just fine | 20:45 |
| mwhahaha | maybe ipa needs to remove it too? | 20:45 |
| rlandy | baremetal is not getting dhcp packets | 20:45 |
| mwhahaha | oh did you see the nftables cause? | 20:45 |
| mwhahaha | for 8.2 | 20:45 |
| rlandy | I did but this is a master deployment | 20:46 |
| rlandy | and nftables was never enabled | 20:46 |
| rlandy | so centos 8 master on baremetal | 20:46 |
| mwhahaha | yes but what version of centos is it | 20:46 |
| mwhahaha | is it 8.2? | 20:46 |
| rlandy | (undercloud) [zuul@undercloud ~]$ cat /etc/redhat-release | 20:47 |
| rlandy | CentOS Linux release 8.1.1911 (Core) | 20:47 |
| mwhahaha | k | 20:47 |
| *** ericyoung has quit IRC | 20:47 | |
| rlandy | interesting enough ... ovb never has this issue ... and https://logserver.rdoproject.org/69/714969/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/a556714/logs/undercloud/var/log/extra/podman/containers/ironic_inspector_dnsmasq/log/anaconda/ifcfg.log.txt.gz | 20:48 |
| mwhahaha | if it's not seeing dhcp packets that seems like the nftables thing | 20:49 |
| mwhahaha | mtu maybe? | 20:49 |
| weshay|ruck | should be at 1350 | 20:49 |
| *** ericyoung has joined #tripleo | 20:49 | |
| weshay|ruck | rlandy, ah.. | 20:50 |
| weshay|ruck | https://sf.hosted.upshift.rdu2.redhat.com/logs/40/195040/6/check/periodic-tripleo-ci-centos-8-bm_envA-3ctlr_1comp-featureset001-master/9f95481/logs/undercloud/var/log/extra/network.txt | 20:50 |
| weshay|ruck | I see 1500 | 20:50 |
| weshay|ruck | well.. on the undercloud | 20:50 |
| rlandy | we stopped dropping mtus on baremetal but I can try that again | 20:50 |
| *** tosky has joined #tripleo | 20:54 | |
| *** jmasud has quit IRC | 20:54 | |
| *** jmasud has joined #tripleo | 20:56 | |
| *** rfolco has quit IRC | 20:58 | |
| *** raildo_ has joined #tripleo | 20:58 | |
| *** raildo has quit IRC | 21:00 | |
| bfournie | rlandy, mwhahaha: yeah there are no dhcp packets in dnsmasq.log but it doesn't look like the nftables issue based on nftables.conf | 21:06 |
| mwhahaha | yea i noticed | 21:06 |
| mwhahaha | is there any dhcp in the neutron dnsmasq? | 21:06 |
| rlandy | trying to put mtus back to 1350 | 21:06 |
| mwhahaha | cause it should show that it didn't match | 21:06 |
| bfournie | mwhahaha: looking | 21:07 |
| *** jmasud has quit IRC | 21:10 | |
| bfournie | mwhahaha: no I don't see neutron logging any dhcp packets either | 21:12 |
| mwhahaha | is the provisioning interface on the bm broken? | 21:13 |
| mwhahaha | or the network? | 21:13 |
| rlandy | so that may be the new nmcli bridge connections | 21:15 |
| bfournie | mwhahaha: br-ctlplane is up | 21:16 |
| bfournie | https://sf.hosted.upshift.rdu2.redhat.com/logs/40/195040/6/check/periodic-tripleo-ci-centos-8-bm_envA-3ctlr_1comp-featureset001-master/9f95481/logs/undercloud/var/log/extra/network.txt | 21:16 |
| mwhahaha | br-ctlplane is always up :D | 21:17 |
| *** xek_ has quit IRC | 21:17 | |
| mwhahaha | i wodner if vlan107 is bad | 21:17 |
| rlandy | so we have a virt undercloud on the undercloud baremetal box | 21:17 |
| rlandy | it makes reprovisioning easier | 21:17 |
| * mwhahaha shrugs | 21:17 | |
| mwhahaha | sounds messy | 21:17 |
| * mwhahaha runs away | 21:17 | |
| bfournie | ah :-) | 21:17 |
| rlandy | pre centos8, controlled those interfaces with network service | 21:18 |
| rlandy | now they are NM controlled | 21:18 |
| rlandy | so ... | 21:18 |
| bfournie | weird - Object "iptables-save" is unknown, try "ip help". https://sf.hosted.upshift.rdu2.redhat.com/logs/40/195040/6/check/periodic-tripleo-ci-centos-8-bm_envA-3ctlr_1comp-featureset001-master/9f95481/logs/undercloud/var/log/extra/network-netns | 21:18 |
| mwhahaha | don't forget to disable firewalld | 21:18 |
| mwhahaha | or fix it | 21:18 |
| rlandy | if you look at the the baremetal box itself, | 21:18 |
| stevebaker | EmilienM: hey do you have a moment to talk about node delete and the scale-down playbook? | 21:19 |
| weshay|ruck | mwhahaha, meh.. it's the better way.. we dicussed the undercloud and vbmc on the undercloud previously | 21:19 |
| rlandy | http://pastebin.test.redhat.com/848411 | 21:19 |
| weshay|ruck | as a way to avoid re-provisioning .. works fine | 21:19 |
| rlandy | the provisioning interface is on eth3 | 21:20 |
| mwhahaha | weshay|ruck: i'm sure it's better, just hard to troubleshoot for those who have no idea what the config is | 21:20 |
| mwhahaha | stevebaker: EmilienM's afk, sup? | 21:20 |
| rlandy | added to bridge brovc | 21:20 |
| weshay|ruck | mwhahaha, k.. understood /me chats w/ rlandy | 21:21 |
| weshay|ruck | anyone want to experiment w/ review priority?? in tripleo repos? https://review.opendev.org/#/c/715069/ | 21:21 |
| weshay|ruck | rlandy, want to chat? | 21:21 |
| rlandy | yeah | 21:22 |
| openstackgerrit | Luke Short proposed openstack/tripleo-common master: Expose --limit, --skip-tags, and --tags in config-download. https://review.opendev.org/692420 | 21:24 |
| sshnaidm | cloudnull, is there a reason behind using async? https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/playbooks/cli-baremetal-introspect.yaml#L66 | 21:25 |
| openstackgerrit | Luke Short proposed openstack/python-tripleoclient master: Expose --limit, --skip-tags, and --tags on the CLI. https://review.opendev.org/692421 | 21:25 |
| sshnaidm | cloudnull, I wonder if we can get rid off it, using just node timeouts config | 21:25 |
| stevebaker | mwhahaha: when specifying node delete --baremetal-deployment, there is code to map the host names to the deployed-server heat resource[1]. These heat stack/resource IDs are then passed to the scale_down. However this fails with this message. The error kind of implies I can just pass the hostname, even for deployed-server http://paste.openstack.org/show/791162/ | 21:25 |
| stevebaker | mwhahaha: [1] https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleoclient/v1/overcloud_node.py#L117 | 21:25 |
| cloudnull | sshnaidm sure . that makes sense | 21:25 |
| sshnaidm | cloudnull, ok, submitting a patch.. | 21:26 |
| mwhahaha | stevebaker: yea i think we need to fix the uuid-to-name stuff for the delete. | 21:26 |
| mwhahaha | or vice versa | 21:26 |
| stevebaker | mwhahaha: so I should be passing in heat resource ID for deployed-server? | 21:27 |
| mwhahaha | i mean we can change it, the issue is that it needs to know where to run the playbook | 21:27 |
| mwhahaha | stevebaker: i vaguely recall EmilienM looking into it previously because of the uuid thing | 21:28 |
| stevebaker | yeah, same | 21:28 |
| Madkiss | mwhahaha: Are tagged VLANs for Controplane still not supported? :) | 21:28 |
| mwhahaha | Madkiss: yes i believe they are still not supported | 21:29 |
| Madkiss | thank you | 21:29 |
| cloudnull | sshnaidm i think at one point we were looping over the nodes so async made sense. with the module now processing nodes concurrently it seems that doesn't make sense any longer. | 21:29 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Remove async from introspection https://review.opendev.org/715073 | 21:29 |
| Madkiss | Am I correct to assume that I do not have to configure a Controlplane port per role because every node will have one anyway? | 21:29 |
| sshnaidm | cloudnull, ack | 21:30 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Correct wrong template reference. https://review.opendev.org/714725 | 21:30 |
| sshnaidm | cloudnull, https://review.opendev.org/#/c/715073/ | 21:30 |
| sshnaidm | rlandy, ^^ should fix this ansible failure at least | 21:30 |
| rlandy | k - thanks | 21:30 |
| sshnaidm | rlandy, if it's possible worth to check it with bm | 21:31 |
| rlandy | sshnaidm: yeah - after I get off the call | 21:32 |
| sshnaidm | cool | 21:32 |
| cloudnull | sshnaidm - https://review.opendev.org/#/c/715073/ i think the var needs to be adjusted. | 21:32 |
| ekultails | stevebaker, mwhahaha: I've been looking into the `node delete` stuff a lot lately as it's related to some of the --limit stuff I've been working on. The existing code *should* technically work with hostnames or UUIDs. | 21:32 |
| sshnaidm | cloudnull, yup | 21:32 |
| stevebaker | ekultails: for deployed-server? (when there is no nova) | 21:33 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Remove async from introspection https://review.opendev.org/715073 | 21:34 |
| mwhahaha | yea i think we need a uuid to hostname thing where before we could pull it out of the stack | 21:34 |
| *** derekh has quit IRC | 21:34 | |
| mwhahaha | stevebaker: oh he did openstack overcloud node delete --baremetal-deployment | 21:35 |
| mwhahaha | you have to use that | 21:35 |
| mwhahaha | https://review.opendev.org/#/c/701296/ | 21:35 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Remove async from introspection https://review.opendev.org/715073 | 21:35 |
| mwhahaha | or is that what you're poking at? | 21:35 |
| *** dpawlik has quit IRC | 21:35 | |
| ekultails | stevebaker: I haven't looked into the novaless use-case for that. I'm not sure if we really even need `node delete` for novaless. | 21:35 |
| mwhahaha | yes we do | 21:36 |
| mwhahaha | to run scale down actions (specifically this) additionally to clean up the stack representation | 21:36 |
| ekultails | Okay, so we do still do some cleanup bits with the stack. | 21:37 |
| mwhahaha | yea | 21:37 |
| mwhahaha | so i think for deployed server you pass the hostname in | 21:37 |
| mwhahaha | it looks like the --baremetal-deployment wants the deployment yaml | 21:38 |
| mwhahaha | and it'd try to figure out the systems being removed | 21:38 |
| cloudnull | sshnaidm ++ looks good ! | 21:39 |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo master: Support for PowerMax Volume Config https://review.opendev.org/712184 | 21:39 |
| *** derekh has joined #tripleo | 21:42 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/train: WIP [train-only] purge /etc/nftables/* https://review.opendev.org/715056 | 21:43 |
| EmilienM | stevebaker: hi I'm back | 21:43 |
| stevebaker | EmilienM: hey | 21:43 |
| EmilienM | stevebaker: if you have a moment, let's talk about it yeah | 21:43 |
| *** derekh has quit IRC | 21:44 | |
| stevebaker | EmilienM: should I try just passing in the hostname for deployed-server? | 21:44 |
| EmilienM | stevebaker: it would be so much easier imo | 21:44 |
| EmilienM | and less error prone | 21:44 |
| stevebaker | it would be nice to delete the ~60 lines of _translate_nodes_to_resource https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleoclient/v1/overcloud_node.py#L117 | 21:45 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: Introduce environments/disable-swift.yaml https://review.opendev.org/714688 | 21:46 |
| *** suuuper has quit IRC | 21:49 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/train: WIP [train-only] purge /etc/nftables/* https://review.opendev.org/715056 | 21:52 |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-redhat-subscription master: Fix ansible_pkg_mgr https://review.opendev.org/715079 | 21:53 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/train: WIP [train-only] purge /etc/nftables/* https://review.opendev.org/715056 | 21:53 |
| stevebaker | EmilienM: when I pass in the hostname instead I get http://paste.openstack.org/show/791164/ | 21:57 |
| EmilienM | stevebaker: are you using deployed server (metalsmith I guess?) | 21:59 |
| *** dking_desktop has quit IRC | 22:01 | |
| stevebaker | EmilienM: yes, node delete --baremetal-deployment. Basically I'm trying to validate that this ansible switchover works, but the underlying scale-down doesn't seem to support deployed-server, so I can't https://review.opendev.org/#/c/712222/ | 22:01 |
| *** jonmasters has joined #tripleo | 22:02 | |
| *** jdandrea has quit IRC | 22:02 | |
| EmilienM | we would make it backward compatible if nova is used; we have customers who have scripts doing node delete using the uuid | 22:03 |
| EmilienM | i would suggest that, if nova is used, we don't change anything for now; and if we do node delete --baremetal-deployment, we use the hostname; which will lookup the resource id | 22:04 |
| stevebaker | EmilienM: ok, that suggests I should delete _translate_nodes_to_resources[1] and pass the hostname, and I'll leave it to DF to handle hostnames in the deployed-server case? [1] _translate_nodes_to_resources | 22:06 |
| stevebaker | [1] https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleoclient/v1/overcloud_node.py#L117 | 22:06 |
| EmilienM | stevebaker: at the end what we want is 1) the resources deleted in heat if nova is deployed, so baremetal nodes are removed | 22:06 |
| EmilienM | if nova isn't deployed, you run the unprovision playbook | 22:06 |
| EmilienM | and 2) give the hostname corresponding to uuid if nova is used, or use straight hostname to call ansible scale down tasks for --limit | 22:06 |
| Madkiss | What is the name of the file again where I can configure what task is supposed to use what network? | 22:07 |
| stevebaker | EmilienM: ok, I think I should raise a bug to track support for specifying hostname when scaling down for deployed server | 22:08 |
| EmilienM | stevebaker: so what you're saying is we could return hostnames in _translate_nodes_to_resources()? | 22:08 |
| EmilienM | I'm sure we can | 22:08 |
| EmilienM | it's in the heat stack resource | 22:09 |
| openstackgerrit | Merged openstack/ansible-role-redhat-subscription master: Fix ansible_pkg_mgr https://review.opendev.org/715079 | 22:09 |
| stevebaker | EmilienM: _translate_nodes_to_resources wouldn't be needed at all, _nodes_to_delete returns the hostnames already | 22:09 |
| EmilienM | oh ok | 22:10 |
| EmilienM | i'll probably ask Rabi if he can help on this one | 22:11 |
| EmilienM | stevebaker: let me know when you have the bug report | 22:11 |
| stevebaker | EmilienM: but the scale_down playbook would need some equivalent logic to discover heat resource ID from hostname. It should be possible for this to work for deployed-server and nova cases, so it could replace whatever nova hostname->id lookup happens | 22:12 |
| *** rh-jelabarre has quit IRC | 22:16 | |
| *** jmasud has joined #tripleo | 22:29 | |
| stevebaker | EmilienM, ekultails: I've raised https://bugs.launchpad.net/tripleo/+bug/1869089 | 22:29 |
| openstack | Launchpad bug 1869089 in tripleo "overcloud node delete needs to support deployed-server delete by hostname" [High,Triaged] | 22:29 |
| ade_lee | stevebaker, EmilienM ah -- I was wondering about this exact question .. | 22:30 |
| ade_lee | ie. when will I be able to run scale_tasks for pre-provisioned nodes .. | 22:31 |
| *** raildo_ has quit IRC | 22:34 | |
| *** slaweq has quit IRC | 22:34 | |
| *** Trevor_V has quit IRC | 22:49 | |
| *** tkajinam has joined #tripleo | 22:52 | |
| EmilienM | stevebaker: thx | 22:52 |
| EmilienM | stevebaker: will look with Rabi tomorrow | 22:53 |
| *** rcernin has joined #tripleo | 22:56 | |
| *** jfrancoa has quit IRC | 23:13 | |
| *** lbragstad has joined #tripleo | 23:25 | |
| *** chem has quit IRC | 23:26 | |
| *** chem has joined #tripleo | 23:27 | |
| openstackgerrit | Merged openstack/tripleo-ansible stable/train: Force container deletion if namespace does not exist in service_kill https://review.opendev.org/715019 | 23:50 |
| *** rlandy has quit IRC | 23:51 | |
| *** jdandrea has joined #tripleo | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!